AWS Lambda & API Gateway for Puppeteer
Budget: $10 – $30 USD
I'm looking to automate a series of web scraping tasks using Puppeteer, and wish to build an API to manage and trigger these tasks. I plan to use AWS Lambda to execute these scripts and API Gateway to provide access to the functions.
Key requirements for the project include:
- Setting up AWS Lambda to run Puppeteer scripts for web scraping.
- Configuring those scripts to automate data gathering and downloading.
- Creating an API on API Gateway to manage the tasks, trigger the scripts, and return results.
-it should bifurcates any social media links on the website
- it should return a json with website text, links, social media texts.
The ideal candidate would be a developer with:
- Experience in AWS Lambda and API Gateway.
- Prior work with Puppeteer for web scraping.
- Strong background in creating APIs.
- Familiarity with MongoDB, as this is my preferred data storage solution.
Please include a detailed project proposal in your application. I look forward to seeing how you intend to approach this task.
Key requirements for the project include:
- Setting up AWS Lambda to run Puppeteer scripts for web scraping.
- Configuring those scripts to automate data gathering and downloading.
- Creating an API on API Gateway to manage the tasks, trigger the scripts, and return results.
-it should bifurcates any social media links on the website
- it should return a json with website text, links, social media texts.
The ideal candidate would be a developer with:
- Experience in AWS Lambda and API Gateway.
- Prior work with Puppeteer for web scraping.
- Strong background in creating APIs.
- Familiarity with MongoDB, as this is my preferred data storage solution.
Please include a detailed project proposal in your application. I look forward to seeing how you intend to approach this task.