Python Script for Geo-Based Web page Screenshots
Budget: $3,000 – $5,000 USD
We are looking for an experienced Python developer to create a script that automates webpage screenshots from various geographic locations using proxy servers. The goal is to visually verify how a webpage renders from different regions. The script will read URLs from a Google Sheet, utilize headless Chrome with Selenium WebDriver, and save screenshots to a specified server bucket.
We already have a beta version of the script (without proxy settings), which can be used as a starting point.
--
Project Requirements:
* Google Sheets Integration: Connect to a Google Sheet containing approximately 10,000 URLs.
* Proxy Server Integration: Rotate between 30 different proxy servers to simulate access from distinct geographic locations.
* Web Scraping with Headless Chrome: Use Selenium WebDriver with headless Chrome to navigate to the URLs. Capture and save screenshots of each webpage.
Text Recognition API Integration: Utilize a Text Recognition API (e.g., Tesseract, Google Vision, or AWS Textract) to extract visible text directly from the webpages. Store the extracted text in a structured format, associating it with the corresponding URL and geographic location for seamless analysis and organization.
* Scheduling: Automate periodic checks on all URLs.
* Data Storage: Save screenshots and extracted text to a server bucket with an organized folder structure (e.g., by geographic region, timestamp, and URL). Also save the result of the text analysis in database/Google sheet.
Deliverables:
* Proof of Concept: A functional script using 3 proxy servers and 3 URLs.
Complete Script: * Fully functional script handling all 30 proxy servers and 10,000 URLs.
Ideal Candidate:
* Proficient in Python and Selenium WebDriver.
* Experience with proxy server integration and headless browsing.
* Experience in text recognition API
* Familiarity with Google Sheets API
We already have a beta version of the script (without proxy settings), which can be used as a starting point.
--
Project Requirements:
* Google Sheets Integration: Connect to a Google Sheet containing approximately 10,000 URLs.
* Proxy Server Integration: Rotate between 30 different proxy servers to simulate access from distinct geographic locations.
* Web Scraping with Headless Chrome: Use Selenium WebDriver with headless Chrome to navigate to the URLs. Capture and save screenshots of each webpage.
Text Recognition API Integration: Utilize a Text Recognition API (e.g., Tesseract, Google Vision, or AWS Textract) to extract visible text directly from the webpages. Store the extracted text in a structured format, associating it with the corresponding URL and geographic location for seamless analysis and organization.
* Scheduling: Automate periodic checks on all URLs.
* Data Storage: Save screenshots and extracted text to a server bucket with an organized folder structure (e.g., by geographic region, timestamp, and URL). Also save the result of the text analysis in database/Google sheet.
Deliverables:
* Proof of Concept: A functional script using 3 proxy servers and 3 URLs.
Complete Script: * Fully functional script handling all 30 proxy servers and 10,000 URLs.
Ideal Candidate:
* Proficient in Python and Selenium WebDriver.
* Experience with proxy server integration and headless browsing.
* Experience in text recognition API
* Familiarity with Google Sheets API