Python Web Scraping Code Optimization
Budget: $30 – $250 USD
I'm working on a web scraping project using Python and need help optimizing my code for better performance. Specifically, I'm looking for an expert who can help with:
The current version of the script scrapes data from a single web page that generates dynamic data after a click and saves the results into an Excel file. While it works, the execution time is quite slow, especially when scraping large amounts of data (multiple clicks). The code is written with basic loops, and I believe that restructuring it could significantly speed up the process.
• Optimisation of loops
• Efficient saving to Excel
Currently, I'm scraping a large data set with over 1,000,000 records. The ideal freelancer for this job should have extensive experience with Python, web scraping, and data processing. Proficiency in optimising loops and data storage methods is crucial. Please only apply if you're able to demonstrate similar past work.
The current version of the script scrapes data from a single web page that generates dynamic data after a click and saves the results into an Excel file. While it works, the execution time is quite slow, especially when scraping large amounts of data (multiple clicks). The code is written with basic loops, and I believe that restructuring it could significantly speed up the process.
• Optimisation of loops
• Efficient saving to Excel
Currently, I'm scraping a large data set with over 1,000,000 records. The ideal freelancer for this job should have extensive experience with Python, web scraping, and data processing. Proficiency in optimising loops and data storage methods is crucial. Please only apply if you're able to demonstrate similar past work.