Split a process into two systems (create a communication between them), create a workflow and graphical representation for script handling.
Budget: €250 – €750 EUR
These points are required
1. create a connection between Main Server and Scripts Server (e.g RESTful API).
2. migrate the scripts from Main Server to the scripts server.
3. improve the script handling and create a way to control the script handling in a graphical way.
The Problem
The Scheduled Tasks are child processes running within the context of the admin server (Strapi).These processes makes heavy use of Puppeteer, especially Amazon, to scrape product details and not all of the details we need are provided by the other API providers (eBay, AliExpress ...), thus we still need to scrape them from their site.
Puppeteer consumes significant amount of RAM, which causes the slow server response when a scheduled task is running.
The Prerenderer also uses Puppeteer. Everytime a scheduled task finishes, it runs the prerender. This causes a longer time for the server to be in a slow reponse state.
The Solution
The Scheduled Tasks and Prerenderer needs to be moved into a separate server, which will be named as the Scripts Server. This will allow the Main Server's resources to be alloted only for the Admin (BE) and Client (FE) sites.
A detailed description can be found in the attachment pdf
1. create a connection between Main Server and Scripts Server (e.g RESTful API).
2. migrate the scripts from Main Server to the scripts server.
3. improve the script handling and create a way to control the script handling in a graphical way.
The Problem
The Scheduled Tasks are child processes running within the context of the admin server (Strapi).These processes makes heavy use of Puppeteer, especially Amazon, to scrape product details and not all of the details we need are provided by the other API providers (eBay, AliExpress ...), thus we still need to scrape them from their site.
Puppeteer consumes significant amount of RAM, which causes the slow server response when a scheduled task is running.
The Prerenderer also uses Puppeteer. Everytime a scheduled task finishes, it runs the prerender. This causes a longer time for the server to be in a slow reponse state.
The Solution
The Scheduled Tasks and Prerenderer needs to be moved into a separate server, which will be named as the Scripts Server. This will allow the Main Server's resources to be alloted only for the Admin (BE) and Client (FE) sites.
A detailed description can be found in the attachment pdf