Build serverless web scrapers
Budget: $5 – $15 USD
We are looking for a Python Developer and Web Scraping specialist with knowledge of AWS cloud technologies, especially AWS Lambda. The task is to code serverless web scrapers for specific web pages.
Each web scraper will run on its own AWS Lambda Function and connect to other services for queueing, monitoring, and storage. We already have the serverless architecture running, we only need the code for the scrapers, running on Lambda Functions, customized for each webpage caracteristics. We expect to create at least one new scraper per week.
Each scraper will run at least once a month until completing a scraping task. Every scraping task might contain from 100 to above 100K documents to download, so realiability and mantainability of the scraping process are key factors for this project.
Each web scraper will run on its own AWS Lambda Function and connect to other services for queueing, monitoring, and storage. We already have the serverless architecture running, we only need the code for the scrapers, running on Lambda Functions, customized for each webpage caracteristics. We expect to create at least one new scraper per week.
Each scraper will run at least once a month until completing a scraping task. Every scraping task might contain from 100 to above 100K documents to download, so realiability and mantainability of the scraping process are key factors for this project.