Job Post Web Scraping
Budget: €750 – €1,500 EUR
I need a robust scraper built to harvest job-post data from a specific website and write each record directly into my database. The scraper must reliably capture three fields on every listing: the full job description, the associated company information, and the application deadline.
Please code the solution in a language and stack you’re comfortable with—Python + Scrapy, BeautifulSoup, or even Selenium are all fine so long as the run-time is headless and can be scheduled. Pagination, dynamic content, and duplicate prevention all have to be handled gracefully, and the script should pause or rotate headers/IPs if the target site begins throttling requests.
Deliverables
• A well-documented script or small module that performs the scrape and inserts the data as clean, normalized records into my database (let me know if you prefer MySQL, PostgreSQL, or a lightweight SQLite file—I can accommodate).
• A short README explaining setup, environment requirements, and how I trigger the job for future runs.
• One successful test run demonstrating that all three required data points populate in the database without manual intervention.
If you have previous experience scraping structured job boards or sites with anti-bot measures, briefly outline it when you reply so I can gauge fit. Looking forward to automating this data transfer with you.
Please code the solution in a language and stack you’re comfortable with—Python + Scrapy, BeautifulSoup, or even Selenium are all fine so long as the run-time is headless and can be scheduled. Pagination, dynamic content, and duplicate prevention all have to be handled gracefully, and the script should pause or rotate headers/IPs if the target site begins throttling requests.
Deliverables
• A well-documented script or small module that performs the scrape and inserts the data as clean, normalized records into my database (let me know if you prefer MySQL, PostgreSQL, or a lightweight SQLite file—I can accommodate).
• A short README explaining setup, environment requirements, and how I trigger the job for future runs.
• One successful test run demonstrating that all three required data points populate in the database without manual intervention.
If you have previous experience scraping structured job boards or sites with anti-bot measures, briefly outline it when you reply so I can gauge fit. Looking forward to automating this data transfer with you.