Assistance Needed with Optimizing Data Scraping Script for Allegro.pl
Budget: €10 – €30 EUR
Hello,
I currently have a JavaScript script that works as follows:
1. It navigates to any selected category on allegro.pl.
2. The script prompts me to input the number of pages to scrape.
3. It iterates through the specified number of pages and collects the URLs of each item listed on those pages.
4. It then visits each item’s URL and downloads the HTML file in a specific structure (I have attached an example of the required HTML structure).
Issues:
• The script is too slow because of the delays added between requests.
• If I reduce or remove the delays, Allegro’s security mechanisms detect and block the script.
• I’ve tried switching to Python, but Selenium and similar libraries face issues with Allegro’s automated access restrictions.
What I Need:
1. A solution to bypass Allegro’s security mechanisms and scrape data faster.
2. A way to efficiently download HTML files in the desired structure without being blocked.
3. Preferably, transition to a Python-based solution for better performance and flexibility.
If you are confident you can help me optimize or rewrite the script to meet these requirements, please let me know. Write me!
Looking forward to your response!
I currently have a JavaScript script that works as follows:
1. It navigates to any selected category on allegro.pl.
2. The script prompts me to input the number of pages to scrape.
3. It iterates through the specified number of pages and collects the URLs of each item listed on those pages.
4. It then visits each item’s URL and downloads the HTML file in a specific structure (I have attached an example of the required HTML structure).
Issues:
• The script is too slow because of the delays added between requests.
• If I reduce or remove the delays, Allegro’s security mechanisms detect and block the script.
• I’ve tried switching to Python, but Selenium and similar libraries face issues with Allegro’s automated access restrictions.
What I Need:
1. A solution to bypass Allegro’s security mechanisms and scrape data faster.
2. A way to efficiently download HTML files in the desired structure without being blocked.
3. Preferably, transition to a Python-based solution for better performance and flexibility.
If you are confident you can help me optimize or rewrite the script to meet these requirements, please let me know. Write me!
Looking forward to your response!