Automate Python scraping script on AWS
Budget: ₹12,500 – ₹37,500 INR
We are looking for an AWS Automation Engineer to join our team on a project basis.
Time period=1 month. Budget=20,000 INR.
Role and Responsibilities:
------------------------------------------
1) Selenium Grid Setup: Set up and manage a Selenium Grid infrastructure to distribute web scraping tasks across multiple browsers efficiently.
2) AWS EC2 Deployment: Provision and manage a fleet of EC2 instances to host our web scraping scripts and Selenium nodes, ensuring high availability and scalability.
3) Docker Containerization: Containerize the scraping code and its dependencies using Docker, making it easy to scale and deploy.
4) Auto-Scaling: Implement auto-scaling mechanisms to dynamically adjust the number of instances based on workload, ensuring cost-efficiency.
5) Data Integration: Ability to seamlessly push scraped data to Redshift/Postgres databases, ensuring efficient data storage and retrieval.
Requirements:
------------------------------------------
1) Degree in Computer Science Engineering (PhD/Masters/Bachelors)
2) At least 1 year of experience in a similar role.
Time period=1 month. Budget=20,000 INR.
Role and Responsibilities:
------------------------------------------
1) Selenium Grid Setup: Set up and manage a Selenium Grid infrastructure to distribute web scraping tasks across multiple browsers efficiently.
2) AWS EC2 Deployment: Provision and manage a fleet of EC2 instances to host our web scraping scripts and Selenium nodes, ensuring high availability and scalability.
3) Docker Containerization: Containerize the scraping code and its dependencies using Docker, making it easy to scale and deploy.
4) Auto-Scaling: Implement auto-scaling mechanisms to dynamically adjust the number of instances based on workload, ensuring cost-efficiency.
5) Data Integration: Ability to seamlessly push scraped data to Redshift/Postgres databases, ensuring efficient data storage and retrieval.
Requirements:
------------------------------------------
1) Degree in Computer Science Engineering (PhD/Masters/Bachelors)
2) At least 1 year of experience in a similar role.