JOB2: Develop a Node.js API with Puppeteer Cluster for Mobile Operator Dashboard (Proof of Concept)
Budget: $30 – $250 USD
IMPRTANT: WE’RE ON A TIGHT SCHEDULE AND NEED THIS COMPLETED IN 2-3 DAYS MAX!
I need an experienced developer to create a custom API with endpoints that replicate the functionality of my mobile phone operator’s web-based dashboard (control panel) using Puppeteer. The operator provides a limited official API, but the dashboard offers additional features that I want to access programmatically. This project must use Puppeteer for browser automation, with a special emphasis on implementing a Puppeteer cluster (e.g., via puppeteer-cluster) to handle parallel API calls efficiently, as we expect to invoke the API many times concurrently. We’re looking for someone who has proven experience with Puppeteer and similar automation projects. This initial task is a proof of concept to create 2-3 endpoints. If successful, we will expand the project to cover the entire website, requiring 200-300 endpoints.
Project Scope:
Dashboard Analysis: we will provide 5-6 basic functions (verify data balance, data usage, top up extra data etc.) that we want to create endpoints for.
Analyze the target dashboard (URL and specific pages will be provided privately) to understand the workflows for the functionalities we specify (e.g., button clicks, form submissions).
We will provide a detailed list of pages and functionalities that need API endpoints.
We will also provide current official API output examples for the functions that need to be added so that the new endpoint follows the exact same json layout as the current API to facilitate importing the output into our system.
Puppeteer Automation with Clustering:
Build Puppeteer scripts to:
Log into the dashboard using provided credentials (securely handled).
Navigate and execute all specified functions on the designated pages.
Handle dynamic content, errors, and timeouts robustly.
Implement a Puppeteer cluster to manage multiple browser instances, ensuring the system can handle high volumes of parallel requests without performance bottlenecks.
Extract and return relevant data (e.g., success messages, account details) from the DOM.
API Development:
Create a Node.js server (e.g., using Express) with RESTful API endpoints tailored to the functionalities we provide.
For this proof of concept, develop 2-3 endpoints (exact number and details to be provided).
Each endpoint should accept parameters (via query or body) and trigger the corresponding Puppeteer script within the cluster.
Return JSON responses with the results or error messages.
Deployment:
Deliver the solution preferably containerized and code delivery will be via a GitLab repository (we will provide access), as we will host it on our own server.
Include clear setup and deployment instructions for our team to run it locally or on our infrastructure.
Performance & Reliability:
Optimize the Puppeteer cluster for concurrent requests, ensuring scalability and minimal resource usage.
Implement session management to reduce login frequency and improve efficiency.
Deliverables:
Fully functional Node.js codebase with Puppeteer cluster scripts and 2-3 API endpoints, delivered via GitLab.
Documentation on how to use each endpoint (e.g., sample requests/responses).
Instructions for setting up and running the solution on our server.
Future Scope:
This is an initial proof-of-concept task. If the solution works as expected (reliable, scalable, and meets our needs), we will proceed with a larger project to automate the entire dashboard, creating 200-300 endpoints.
Skills Required:
Strong experience with Node.js, Puppeteer, and Express, with a proven track record of completing similar Puppeteer-based automation projects.
Expertise in implementing Puppeteer clusters for parallel task execution.
Familiarity with web scraping, DOM manipulation, and browser automation.
Knowledge of REST API design and GitLab for version control.
Bonus: Experience optimizing Puppeteer for performance in high-concurrency scenarios.
I need an experienced developer to create a custom API with endpoints that replicate the functionality of my mobile phone operator’s web-based dashboard (control panel) using Puppeteer. The operator provides a limited official API, but the dashboard offers additional features that I want to access programmatically. This project must use Puppeteer for browser automation, with a special emphasis on implementing a Puppeteer cluster (e.g., via puppeteer-cluster) to handle parallel API calls efficiently, as we expect to invoke the API many times concurrently. We’re looking for someone who has proven experience with Puppeteer and similar automation projects. This initial task is a proof of concept to create 2-3 endpoints. If successful, we will expand the project to cover the entire website, requiring 200-300 endpoints.
Project Scope:
Dashboard Analysis: we will provide 5-6 basic functions (verify data balance, data usage, top up extra data etc.) that we want to create endpoints for.
Analyze the target dashboard (URL and specific pages will be provided privately) to understand the workflows for the functionalities we specify (e.g., button clicks, form submissions).
We will provide a detailed list of pages and functionalities that need API endpoints.
We will also provide current official API output examples for the functions that need to be added so that the new endpoint follows the exact same json layout as the current API to facilitate importing the output into our system.
Puppeteer Automation with Clustering:
Build Puppeteer scripts to:
Log into the dashboard using provided credentials (securely handled).
Navigate and execute all specified functions on the designated pages.
Handle dynamic content, errors, and timeouts robustly.
Implement a Puppeteer cluster to manage multiple browser instances, ensuring the system can handle high volumes of parallel requests without performance bottlenecks.
Extract and return relevant data (e.g., success messages, account details) from the DOM.
API Development:
Create a Node.js server (e.g., using Express) with RESTful API endpoints tailored to the functionalities we provide.
For this proof of concept, develop 2-3 endpoints (exact number and details to be provided).
Each endpoint should accept parameters (via query or body) and trigger the corresponding Puppeteer script within the cluster.
Return JSON responses with the results or error messages.
Deployment:
Deliver the solution preferably containerized and code delivery will be via a GitLab repository (we will provide access), as we will host it on our own server.
Include clear setup and deployment instructions for our team to run it locally or on our infrastructure.
Performance & Reliability:
Optimize the Puppeteer cluster for concurrent requests, ensuring scalability and minimal resource usage.
Implement session management to reduce login frequency and improve efficiency.
Deliverables:
Fully functional Node.js codebase with Puppeteer cluster scripts and 2-3 API endpoints, delivered via GitLab.
Documentation on how to use each endpoint (e.g., sample requests/responses).
Instructions for setting up and running the solution on our server.
Future Scope:
This is an initial proof-of-concept task. If the solution works as expected (reliable, scalable, and meets our needs), we will proceed with a larger project to automate the entire dashboard, creating 200-300 endpoints.
Skills Required:
Strong experience with Node.js, Puppeteer, and Express, with a proven track record of completing similar Puppeteer-based automation projects.
Expertise in implementing Puppeteer clusters for parallel task execution.
Familiarity with web scraping, DOM manipulation, and browser automation.
Knowledge of REST API design and GitLab for version control.
Bonus: Experience optimizing Puppeteer for performance in high-concurrency scenarios.