Python Web Scraping & Automation Bot
Budget: ₹12,500 – ₹37,500 INR
I'm in need of a proficient web scraping and automation expert to develop a Python-based bot. This bot should scrape the S&P 500 companies career pages (https://en.wikipedia.org/wiki/List_of_S%26P_500_companies) daily to identify new job postings that correlate with specific keywords. The key functionalities of this bot should include:
- Automated Daily Web Scraping: The bot should conduct a daily scan of the S&P 500 career pages.
- Keyword-Based Job Posting Identification: The bot is expected to filter postings based on designated keywords.
- Automation: Ideally bot should run automatically every morning, but a local application that I can run manually is also acceptable.
- Database Integration: The ideal candidate should have experience in connecting the bot to a database where the job postings can be stored. Even saving it to cloud google sheets is fine.
- Post-Scrape Management: It is essential for the bot should use this data to avoid showing duplicate listings in the future.
The primary goal of this project is to streamline and simplify my job search process, so attention to detail and accuracy are paramount.
I prefer that the bot be built using Python. While the database management system wasn't specified, please include your recommendation in your proposal.
Ideal Skills & Experience:
- Proficient in Python programming language
- Strong experience with web scraping and automation
- Previous work with building bots that scrape job postings is a plus
- Familiarity with integrating bots with databases/Excel/Airtable etc
**Note:**
Respond only if you meet the requirements and have relevant experience.
- Automated Daily Web Scraping: The bot should conduct a daily scan of the S&P 500 career pages.
- Keyword-Based Job Posting Identification: The bot is expected to filter postings based on designated keywords.
- Automation: Ideally bot should run automatically every morning, but a local application that I can run manually is also acceptable.
- Database Integration: The ideal candidate should have experience in connecting the bot to a database where the job postings can be stored. Even saving it to cloud google sheets is fine.
- Post-Scrape Management: It is essential for the bot should use this data to avoid showing duplicate listings in the future.
The primary goal of this project is to streamline and simplify my job search process, so attention to detail and accuracy are paramount.
I prefer that the bot be built using Python. While the database management system wasn't specified, please include your recommendation in your proposal.
Ideal Skills & Experience:
- Proficient in Python programming language
- Strong experience with web scraping and automation
- Previous work with building bots that scrape job postings is a plus
- Familiarity with integrating bots with databases/Excel/Airtable etc
**Note:**
Respond only if you meet the requirements and have relevant experience.