Develop a Node.js API with Puppeteer Cluster for Mobile Operator Dashboard (Proof of Concept)
Budget: $8 – $15 USD
I need an experienced developer to create a custom API endpoint that replicates the functionality of my mobile phone operator’s web-based dashboard (control panel) using Puppeteer. The operator provides a limited official API, but the dashboard offers additional features that I want to access programmatically. This project must use Puppeteer for browser automation, with a special emphasis on implementing a Puppeteer cluster (e.g., via puppeteer-cluster) to handle parallel API calls efficiently, as we expect to invoke the API many times concurrently. We’re looking for someone who has proven experience with Puppeteer and similar automation projects. This initial task is a proof of concept to create 5-10 endpoints. If successful, we will expand the project to cover the entire website, requiring 200-300 endpoints.
Project Scope:
Dashboard Analysis: we will provide 5-6 basic functions (verify data balance, data usage, top up extra data etc.) that we want to create endpoints for.
Analyze the target dashboard (URL and specific pages will be provided privately) to understand the workflows for the functionalities we specify (e.g., button clicks, form submissions).
We will provide a detailed list of pages and functionalities that need API endpoints.
Puppeteer Automation with Clustering:
Build Puppeteer scripts to:
Log into the dashboard using provided credentials (securely handled).
Navigate and execute all specified functions on the designated pages.
Handle dynamic content, errors, and timeouts robustly.
Implement a Puppeteer cluster to manage multiple browser instances, ensuring the system can handle high volumes of parallel requests without performance bottlenecks.
Extract and return relevant data (e.g., success messages, account details) from the DOM.
API Development:
Create a Node.js server (e.g., using Express) with RESTful API endpoints tailored to the functionalities we provide.
For this proof of concept, develop 5-10 endpoints (exact number and details to be provided).
Each endpoint should accept parameters (via query or body) and trigger the corresponding Puppeteer script within the cluster.
Return JSON responses with the results or error messages.
Deployment:
Deliver the solution via a GitLab repository (we will provide access), as we will host it on our own server.
Include clear setup and deployment instructions for our team to run it locally or on our infrastructure.
Performance & Reliability:
Optimize the Puppeteer cluster for concurrent requests, ensuring scalability and minimal resource usage.
Implement session management to reduce login frequency and improve efficiency.
Deliverables:
Fully functional Node.js codebase with Puppeteer cluster scripts and 5-10 API endpoints, delivered via GitLab.
Documentation on how to use each endpoint (e.g., sample requests/responses).
Instructions for setting up and running the solution on our server.
Future Scope:
This is an initial proof-of-concept task. If the solution works as expected (reliable, scalable, and meets our needs), we will proceed with a larger project to automate the entire dashboard, creating 200-300 endpoints.
Skills Required:
Strong experience with Node.js, Puppeteer, and Express, with a proven track record of completing similar Puppeteer-based automation projects.
Expertise in implementing Puppeteer clusters for parallel task execution.
Familiarity with web scraping, DOM manipulation, and browser automation.
Knowledge of REST API design and GitLab for version control.
Bonus: Experience optimizing Puppeteer for performance in high-concurrency scenarios.
Budget & Timeline:
Budget: [Insert your budget, e.g., $300–$500].
Timeline: [Insert your timeline, e.g., 2–3 weeks].
Additional Notes:
We will provide the dashboard URL, specific pages, and a list of functionalities requiring endpoints upon hiring.
In your proposal, include examples of past Puppeteer projects, especially those involving automation or clustering, and your approach to handling parallel API calls.
Contact:
Please submit your proposal with a brief plan of how you’d approach this proof of concept, estimated hours, and any questions. We’ll share the dashboard details and functionality list with the selected freelancer.
Project Scope:
Dashboard Analysis: we will provide 5-6 basic functions (verify data balance, data usage, top up extra data etc.) that we want to create endpoints for.
Analyze the target dashboard (URL and specific pages will be provided privately) to understand the workflows for the functionalities we specify (e.g., button clicks, form submissions).
We will provide a detailed list of pages and functionalities that need API endpoints.
Puppeteer Automation with Clustering:
Build Puppeteer scripts to:
Log into the dashboard using provided credentials (securely handled).
Navigate and execute all specified functions on the designated pages.
Handle dynamic content, errors, and timeouts robustly.
Implement a Puppeteer cluster to manage multiple browser instances, ensuring the system can handle high volumes of parallel requests without performance bottlenecks.
Extract and return relevant data (e.g., success messages, account details) from the DOM.
API Development:
Create a Node.js server (e.g., using Express) with RESTful API endpoints tailored to the functionalities we provide.
For this proof of concept, develop 5-10 endpoints (exact number and details to be provided).
Each endpoint should accept parameters (via query or body) and trigger the corresponding Puppeteer script within the cluster.
Return JSON responses with the results or error messages.
Deployment:
Deliver the solution via a GitLab repository (we will provide access), as we will host it on our own server.
Include clear setup and deployment instructions for our team to run it locally or on our infrastructure.
Performance & Reliability:
Optimize the Puppeteer cluster for concurrent requests, ensuring scalability and minimal resource usage.
Implement session management to reduce login frequency and improve efficiency.
Deliverables:
Fully functional Node.js codebase with Puppeteer cluster scripts and 5-10 API endpoints, delivered via GitLab.
Documentation on how to use each endpoint (e.g., sample requests/responses).
Instructions for setting up and running the solution on our server.
Future Scope:
This is an initial proof-of-concept task. If the solution works as expected (reliable, scalable, and meets our needs), we will proceed with a larger project to automate the entire dashboard, creating 200-300 endpoints.
Skills Required:
Strong experience with Node.js, Puppeteer, and Express, with a proven track record of completing similar Puppeteer-based automation projects.
Expertise in implementing Puppeteer clusters for parallel task execution.
Familiarity with web scraping, DOM manipulation, and browser automation.
Knowledge of REST API design and GitLab for version control.
Bonus: Experience optimizing Puppeteer for performance in high-concurrency scenarios.
Budget & Timeline:
Budget: [Insert your budget, e.g., $300–$500].
Timeline: [Insert your timeline, e.g., 2–3 weeks].
Additional Notes:
We will provide the dashboard URL, specific pages, and a list of functionalities requiring endpoints upon hiring.
In your proposal, include examples of past Puppeteer projects, especially those involving automation or clustering, and your approach to handling parallel API calls.
Contact:
Please submit your proposal with a brief plan of how you’d approach this proof of concept, estimated hours, and any questions. We’ll share the dashboard details and functionality list with the selected freelancer.