Python Web Crawler Expert
Budget: $10 – $30 USD
I am looking for a skilled Python developer with expertise in web scraping to enhance an existing script designed to extract product data from Walmart's online store. The current script retrieves only 10 items at a time, while the website displays 40 items per page. The task involves addressing the infinite scroll or JavaScript loading mechanisms that are limiting data extraction.
Key Responsibilities:
- Modify the existing Python script to enable it to scrape all 40 items per page.
- Implement solutions to effectively handle infinite scrolling or dynamic content loading.
- Ensure the script is efficient and adheres to best practices in web scraping.
- Conduct thorough testing of the updated script to confirm accurate data retrieval.
Requirements:
- Proficiency in Python, particularly with web scraping libraries such as Selenium, Beautiful Soup, or scrapfly-sdk.
- Experience with handling JavaScript-rendered content and infinite scrolling techniques.
- Strong problem-solving skills and attention to detail.
- Ability to work independently and deliver results promptly.
Key Responsibilities:
- Modify the existing Python script to enable it to scrape all 40 items per page.
- Implement solutions to effectively handle infinite scrolling or dynamic content loading.
- Ensure the script is efficient and adheres to best practices in web scraping.
- Conduct thorough testing of the updated script to confirm accurate data retrieval.
Requirements:
- Proficiency in Python, particularly with web scraping libraries such as Selenium, Beautiful Soup, or scrapfly-sdk.
- Experience with handling JavaScript-rendered content and infinite scrolling techniques.
- Strong problem-solving skills and attention to detail.
- Ability to work independently and deliver results promptly.
Related categories:
Business, Accounting, Human Resources & Legal
JavaScript
Python
Web Scraping
Software Architecture