Can't scrape information going back to the number of days I specify
Question:
I want to grab the posting date and title from this webpage using a python script that I’ve already created. The script is working errorlessly.
Now I wish to add some logic to the script so that it will only fetch stuff up to the date equivalent to the days I enter into days_to_go_back. For example, if I enter 2 right next to days_to_go_back, the script should scrape content up to October 6.
import requests
from bs4 import BeautifulSoup
link = 'https://atlanta.craigslist.org/search/lab'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
days_to_go_back = 2
with requests.Session() as s:
s.headers.update(headers)
res = s.get(link)
soup = BeautifulSoup(res.text,"lxml")
for item in soup.select("[id^='search-results'] li[class*='result']"):
date = ' '.join(item.select_one("time.result-date").text.split())
title = item.select_one("h3.result-heading > a").get_text(strip=True)
print(date,title)
Current output:
Oct 7 General labor crew member needed
Oct 7 General Labor Demo
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 Craftsman Wanted
Oct 7 Home Cleaners Wanted! Make over $145+ for 1bd, $165 for 2bd cleans.
Oct 7 Sign Company In Search of Installers and Fabricators
Oct 7 ***PILE DRIVERS-DRILLERS-TRAINEES***
Oct 7 Warehouse Package Handler
Oct 7 Workers Needed for Local Fair (Cash Paid Daily)
Oct 7 Warehouse Team Member - Multiple Shifts Available
Oct 7 Construction
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Amazon Warehouse Assistant - Immediate Hire
Oct 7 Amazon Picker - Overnight Shifts Available
Oct 7 Warehouse Team Member - With Sign-On Bonus!
Oct 7 Amazon Warehouse Assistant
Oct 7 Need a Crew of 4+
Oct 6 Helper and Construction
Oct 6 Truck Driver/El Camionero
Oct 6 Workers Needed for Local Fair (Cash Paid Daily)
Oct 6 Roofing Crews Needed! ($2,000 Sign On Bonus - Start Today!)
Oct 6 Available Immediately General Labor/Driver with Truck or Van Positions
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 6 $20-25/hr. SIMPLE VALET TRASH COLLECTOR-Camp Creek/East Point 30331
Oct 6 Asphalt workers labors
Oct 6 CDL-A Company Drivers - $5K SIGN-ON BONUS!
Oct 6 EXPERIENCED HANDYMAN ***2 immediate openings
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR
Oct 5 Georgia Canvasser
Oct 5 CLEANING JOBS AVAILABLE $22+/hr, Paid Daily
Oct 5 Farm Help Animal Care
Oct 5 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 5 Maintenance/Custodial
Oct 5 Hiring House Keeper For Airbnb
Oct 5 Warehouse Material Handler: $15.92/hr,M-F/Full Time/Day Shift/Benefits
Oct 5 Laborers, Forman, and Drivers
Oct 5 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 5 CANVASS POSITION (MAKE MONEY THIS WEEK)
Oct 5 Hiring full Auto Detailer
Oct 5 Amazon Packer - Part-time Night Shifts Available
Oct 5 Warehouse Team Member Sun - Wed (06:30PM - 05:00AM)
Oct 5 Amazon Warehouse Assistant - Immediate Hire
Oct 5 EC3 Construction Is Now Hiring Framers
Oct 5 Detailer Wanted $16.00hr. plus 250.00 BONUS$$$
Oct 5 FREE $10 CASH FOR EVERYONE GET IT TODAY! SUPPLIES LIMITED HURRY
Oct 5 Now Hiring: Diesel technicians
Oct 5 Warehouse Associate
Oct 5 Delivery Driver Needed
Oct 5 Drivers Helper - Weekly Pay
Oct 5 Landscape Crew Member
Oct 5 Seasonal Laborers
Oct 5 Seasonal Laborers
Oct 5 WINDOW AND DOOR INSTALLERS NEEDED
Oct 5 Commercial Roofers and Helpers
Oct 5 Lyft Driver (part-time / full-time)
Oct 5 Lyft Driver (part-time / full-time)
Oct 5 Warehouse Associate/Driver
Oct 5 Grand Opening! Now Hiring! Immediate work!
Oct 5 C.D.L Drivers wanted
Oct 5 SET UP Workers Needed for Local Fair (Cash Paid Daily)
Oct 5 Workers Needed for Local Fair (Cash Paid Daily)
Oct 4 Distribution Center General Worker Position
Oct 4 Distribution Center General Worker Position
Oct 4 Distribution Center General Worker Position
Oct 4 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 4 Warehouse Worker
Oct 4 House Cleaner Needed
Oct 4 Janitor Job offer
Oct 4 HIRING TIRE TECHNICIANS
Oct 4 CDL-A Company Drivers - $5K SIGN-ON BONUS!
Oct 4 Part Time Groundskeeper/Cleaner
Oct 4 Driver and helper
Oct 4 Full-Time Community Trash Collector
Oct 4 * Deliver with DoorDash *
Oct 4 Experienced maintenance men
Oct 4 Maintenance Manager for Luxury Apts!
Oct 4 Maintenance Manager for Luxury Apts!
Oct 4 Maintenance Tech II for Luxury Apts!
Oct 4 Maintenance Tech II for Luxury Apts!
Oct 4 Maintenance Tech I for Luxury Apts!
Oct 4 Stucco Repair Help
Oct 4 Lyft Driver (part-time / full-time)
Oct 4 Lyft Driver (part-time / full-time)
Oct 4 Workers Needed for Local Fair (Cash Paid Daily)
Oct 4 Warehouse Team Member - Multiple Shifts Available
Oct 4 Tidal Wave Lawrenceville ǀ Car Wash Attendant
Oct 4 Tidal Wave Kennesaw ǀ Car Wash Attendant
Oct 4 Tidal Wave Gainesville ǀ Car Wash Attendant
Oct 4 Tidal Wave Dacula ǀ Car Wash Attendant
Oct 4 Tidal Wave Canton ǀ Car Wash Attendant
Oct 4 Heavy equipment operator
Oct 4 **Full Time Building Cleaner (pressure washing/brush scrubbing) Needed**
Oct 4 Amazon Picker - Up to $16.5/hr
Oct 4 Warehouse Laborer - Immediate Hire
Oct 4 Warehouse Team Member - Split Shifts (20hr/week)
Oct 4 Warehouse Team Member - Morning Shifts Available
Oct 4 Warehouse Team Member - Multiple Shifts Available
Expected output (up to oct 6):
Oct 7 General labor crew member needed
Oct 7 General Labor Demo
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 Craftsman Wanted
Oct 7 Home Cleaners Wanted! Make over $145+ for 1bd, $165 for 2bd cleans.
Oct 7 Sign Company In Search of Installers and Fabricators
Oct 7 ***PILE DRIVERS-DRILLERS-TRAINEES***
Oct 7 Warehouse Package Handler
Oct 7 Workers Needed for Local Fair (Cash Paid Daily)
Oct 7 Warehouse Team Member - Multiple Shifts Available
Oct 7 Construction
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Amazon Warehouse Assistant - Immediate Hire
Oct 7 Amazon Picker - Overnight Shifts Available
Oct 7 Warehouse Team Member - With Sign-On Bonus!
Oct 7 Amazon Warehouse Assistant
Oct 7 Need a Crew of 4+
Oct 6 Helper and Construction
Oct 6 Truck Driver/El Camionero
Oct 6 Workers Needed for Local Fair (Cash Paid Daily)
Oct 6 Roofing Crews Needed! ($2,000 Sign On Bonus - Start Today!)
Oct 6 Available Immediately General Labor/Driver with Truck or Van Positions
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 6 $20-25/hr. SIMPLE VALET TRASH COLLECTOR-Camp Creek/East Point 30331
Oct 6 Asphalt workers labors
Oct 6 CDL-A Company Drivers - $5K SIGN-ON BONUS!
Oct 6 EXPERIENCED HANDYMAN ***2 immediate openings
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR
Answers:
Try:
import requests
from datetime import datetime
from bs4 import BeautifulSoup
link = "https://atlanta.craigslist.org/search/lab"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
}
days_to_go_back = 2
with requests.Session() as s:
s.headers.update(headers)
res = s.get(link)
soup = BeautifulSoup(res.text, "lxml")
now = datetime.now()
for item in soup.select("[id^='search-results'] li[class*='result']"):
date = " ".join(item.select_one("time.result-date").text.split())
title = item.select_one("h3.result-heading > a").get_text(strip=True)
d = datetime.strptime(item.time["datetime"][:10], "%Y-%m-%d")
if (now - d).days < days_to_go_back:
print(date, title)
Prints:
...
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR
I want to grab the posting date and title from this webpage using a python script that I’ve already created. The script is working errorlessly.
Now I wish to add some logic to the script so that it will only fetch stuff up to the date equivalent to the days I enter into days_to_go_back. For example, if I enter 2 right next to days_to_go_back, the script should scrape content up to October 6.
import requests
from bs4 import BeautifulSoup
link = 'https://atlanta.craigslist.org/search/lab'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
days_to_go_back = 2
with requests.Session() as s:
s.headers.update(headers)
res = s.get(link)
soup = BeautifulSoup(res.text,"lxml")
for item in soup.select("[id^='search-results'] li[class*='result']"):
date = ' '.join(item.select_one("time.result-date").text.split())
title = item.select_one("h3.result-heading > a").get_text(strip=True)
print(date,title)
Current output:
Oct 7 General labor crew member needed
Oct 7 General Labor Demo
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 Craftsman Wanted
Oct 7 Home Cleaners Wanted! Make over $145+ for 1bd, $165 for 2bd cleans.
Oct 7 Sign Company In Search of Installers and Fabricators
Oct 7 ***PILE DRIVERS-DRILLERS-TRAINEES***
Oct 7 Warehouse Package Handler
Oct 7 Workers Needed for Local Fair (Cash Paid Daily)
Oct 7 Warehouse Team Member - Multiple Shifts Available
Oct 7 Construction
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Amazon Warehouse Assistant - Immediate Hire
Oct 7 Amazon Picker - Overnight Shifts Available
Oct 7 Warehouse Team Member - With Sign-On Bonus!
Oct 7 Amazon Warehouse Assistant
Oct 7 Need a Crew of 4+
Oct 6 Helper and Construction
Oct 6 Truck Driver/El Camionero
Oct 6 Workers Needed for Local Fair (Cash Paid Daily)
Oct 6 Roofing Crews Needed! ($2,000 Sign On Bonus - Start Today!)
Oct 6 Available Immediately General Labor/Driver with Truck or Van Positions
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 6 $20-25/hr. SIMPLE VALET TRASH COLLECTOR-Camp Creek/East Point 30331
Oct 6 Asphalt workers labors
Oct 6 CDL-A Company Drivers - $5K SIGN-ON BONUS!
Oct 6 EXPERIENCED HANDYMAN ***2 immediate openings
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR
Oct 5 Georgia Canvasser
Oct 5 CLEANING JOBS AVAILABLE $22+/hr, Paid Daily
Oct 5 Farm Help Animal Care
Oct 5 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 5 Maintenance/Custodial
Oct 5 Hiring House Keeper For Airbnb
Oct 5 Warehouse Material Handler: $15.92/hr,M-F/Full Time/Day Shift/Benefits
Oct 5 Laborers, Forman, and Drivers
Oct 5 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 5 CANVASS POSITION (MAKE MONEY THIS WEEK)
Oct 5 Hiring full Auto Detailer
Oct 5 Amazon Packer - Part-time Night Shifts Available
Oct 5 Warehouse Team Member Sun - Wed (06:30PM - 05:00AM)
Oct 5 Amazon Warehouse Assistant - Immediate Hire
Oct 5 EC3 Construction Is Now Hiring Framers
Oct 5 Detailer Wanted $16.00hr. plus 250.00 BONUS$$$
Oct 5 FREE $10 CASH FOR EVERYONE GET IT TODAY! SUPPLIES LIMITED HURRY
Oct 5 Now Hiring: Diesel technicians
Oct 5 Warehouse Associate
Oct 5 Delivery Driver Needed
Oct 5 Drivers Helper - Weekly Pay
Oct 5 Landscape Crew Member
Oct 5 Seasonal Laborers
Oct 5 Seasonal Laborers
Oct 5 WINDOW AND DOOR INSTALLERS NEEDED
Oct 5 Commercial Roofers and Helpers
Oct 5 Lyft Driver (part-time / full-time)
Oct 5 Lyft Driver (part-time / full-time)
Oct 5 Warehouse Associate/Driver
Oct 5 Grand Opening! Now Hiring! Immediate work!
Oct 5 C.D.L Drivers wanted
Oct 5 SET UP Workers Needed for Local Fair (Cash Paid Daily)
Oct 5 Workers Needed for Local Fair (Cash Paid Daily)
Oct 4 Distribution Center General Worker Position
Oct 4 Distribution Center General Worker Position
Oct 4 Distribution Center General Worker Position
Oct 4 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 4 Warehouse Worker
Oct 4 House Cleaner Needed
Oct 4 Janitor Job offer
Oct 4 HIRING TIRE TECHNICIANS
Oct 4 CDL-A Company Drivers - $5K SIGN-ON BONUS!
Oct 4 Part Time Groundskeeper/Cleaner
Oct 4 Driver and helper
Oct 4 Full-Time Community Trash Collector
Oct 4 * Deliver with DoorDash *
Oct 4 Experienced maintenance men
Oct 4 Maintenance Manager for Luxury Apts!
Oct 4 Maintenance Manager for Luxury Apts!
Oct 4 Maintenance Tech II for Luxury Apts!
Oct 4 Maintenance Tech II for Luxury Apts!
Oct 4 Maintenance Tech I for Luxury Apts!
Oct 4 Stucco Repair Help
Oct 4 Lyft Driver (part-time / full-time)
Oct 4 Lyft Driver (part-time / full-time)
Oct 4 Workers Needed for Local Fair (Cash Paid Daily)
Oct 4 Warehouse Team Member - Multiple Shifts Available
Oct 4 Tidal Wave Lawrenceville ǀ Car Wash Attendant
Oct 4 Tidal Wave Kennesaw ǀ Car Wash Attendant
Oct 4 Tidal Wave Gainesville ǀ Car Wash Attendant
Oct 4 Tidal Wave Dacula ǀ Car Wash Attendant
Oct 4 Tidal Wave Canton ǀ Car Wash Attendant
Oct 4 Heavy equipment operator
Oct 4 **Full Time Building Cleaner (pressure washing/brush scrubbing) Needed**
Oct 4 Amazon Picker - Up to $16.5/hr
Oct 4 Warehouse Laborer - Immediate Hire
Oct 4 Warehouse Team Member - Split Shifts (20hr/week)
Oct 4 Warehouse Team Member - Morning Shifts Available
Oct 4 Warehouse Team Member - Multiple Shifts Available
Expected output (up to oct 6):
Oct 7 General labor crew member needed
Oct 7 General Labor Demo
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 LAWN CARE & AERATION SPECIALISTS (ROSWELL, GA)
Oct 7 Craftsman Wanted
Oct 7 Home Cleaners Wanted! Make over $145+ for 1bd, $165 for 2bd cleans.
Oct 7 Sign Company In Search of Installers and Fabricators
Oct 7 ***PILE DRIVERS-DRILLERS-TRAINEES***
Oct 7 Warehouse Package Handler
Oct 7 Workers Needed for Local Fair (Cash Paid Daily)
Oct 7 Warehouse Team Member - Multiple Shifts Available
Oct 7 Construction
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 Lyft Driver (part-time / full-time)
Oct 7 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Warehouse Team Member - $3,000 Sign-On Bonus
Oct 7 Amazon Warehouse Assistant - Immediate Hire
Oct 7 Amazon Picker - Overnight Shifts Available
Oct 7 Warehouse Team Member - With Sign-On Bonus!
Oct 7 Amazon Warehouse Assistant
Oct 7 Need a Crew of 4+
Oct 6 Helper and Construction
Oct 6 Truck Driver/El Camionero
Oct 6 Workers Needed for Local Fair (Cash Paid Daily)
Oct 6 Roofing Crews Needed! ($2,000 Sign On Bonus - Start Today!)
Oct 6 Available Immediately General Labor/Driver with Truck or Van Positions
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 Automotive Technicians Needed
Oct 6 Automotive Technicians Needed
Oct 6 AUTOMOTIVE / TIRE TECHNICIAN ($800 New Hire Bonus)
Oct 6 NEED HANDYMAN FOR TASKS - at least $35/hr, Paid Daily
Oct 6 $20-25/hr. SIMPLE VALET TRASH COLLECTOR-Camp Creek/East Point 30331
Oct 6 Asphalt workers labors
Oct 6 CDL-A Company Drivers - $5K SIGN-ON BONUS!
Oct 6 EXPERIENCED HANDYMAN ***2 immediate openings
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR
Try:
import requests
from datetime import datetime
from bs4 import BeautifulSoup
link = "https://atlanta.craigslist.org/search/lab"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
}
days_to_go_back = 2
with requests.Session() as s:
s.headers.update(headers)
res = s.get(link)
soup = BeautifulSoup(res.text, "lxml")
now = datetime.now()
for item in soup.select("[id^='search-results'] li[class*='result']"):
date = " ".join(item.select_one("time.result-date").text.split())
title = item.select_one("h3.result-heading > a").get_text(strip=True)
d = datetime.strptime(item.time["datetime"][:10], "%Y-%m-%d")
if (now - d).days < days_to_go_back:
print(date, title)
Prints:
...
Oct 6 Automotive Detail /Porter/Shop Help
Oct 6 Fork Lift Operator/ Order Picker
Oct 6 Lot Operations
Oct 6 Waterproofing Foreman
Oct 6 Auto Detailer
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 Lyft Driver (part-time / full-time)
Oct 6 FULL TIME WITH PAID BENEFITS $16+ HOUR