Advanced Data Collection Specialist for E-commerce Price Monitoring System
Budget: $750 – $1,500 USD
We are looking for an experienced data extraction and automation expert to develop a price monitoring and notification system for tracking product pricing trends on a major e-commerce platform. The goal is to gather publicly available pricing data, detect real-time changes, and notify end-users through email or messaging systems.
This project is ideal for professionals skilled in large-scale data processing, automation infrastructure, and system architecture with an understanding of ethical data collection practices.
- Data Extraction:
Build a system to collect publicly available product information (e.g., title, price, reviews) across diverse product categories. Must handle dynamic pages and scalable data collection routines.
- Change Detection:
Create a reliable engine to detect price fluctuations—including minor adjustments—on a frequent and scheduled basis.
- User Notification System:
Implement a notification framework via email or push alerts (via integrations like SendGrid or Twilio) based on user preferences.
- Data Storage & Retrieval:
Design a lightweight database system that supports historical price comparisons, product tracking, and optimized retrieval performance.
- Scalability:
Ensure the solution is modular and capable of handling a high number of SKUs efficiently across different product categories.
- Accuracy & Resilience:
Implement error-handling strategies and fallback logic for content loading failures, network issues, and anti-automation roadblocks.
- (Optional) Dashboard Interface:
Build a simple UI for users to manage their tracked products and configure alerts.
Note: The solution must respect all terms of use of any third-party websites and follow best practices to avoid breaching any platform restrictions.
Required Skills:
- Experience with automation tools like Selenium, Scrapy, Puppeteer, Playwright, etc.
- Proficiency in Python or JavaScript for scalable script development.
- Expertise in working with real-time pipelines or event-driven architectures.
- Familiarity with cloud environments (AWS, GCP, Azure) for deployment.
- Strong background in relational and NoSQL databases (e.g., PostgreSQL, MongoDB).
- Familiarity with proxies, rate-limiting, and best practices for maintaining system uptime.
- Ability to write clean, modular code with version control (Git).
- Prior experience with email/push notification APIs is highly preferred.
Deliverables:
- A functional system for tracking price changes with user notifications.
- Deployment and configuration documentation.
- Clean, well-commented source code.
- Brief report outlining technical decisions, workflows, and system architecture.
To Apply, Please Share:
- Relevant experience and past project samples (GitHub + LinkedIn + Resume).
- Your typical approach to designing scalable scraping or data automation systems.
- Availability and estimated timeline for delivery.
- Your preferred hourly or project-based rate.
- Start your proposal with:
"My Prior Experience in this..." so we know you’ve read the full post.
We’re excited to connect with freelancers who are driven, detail-oriented, and passionate about building smart systems.
This project is ideal for professionals skilled in large-scale data processing, automation infrastructure, and system architecture with an understanding of ethical data collection practices.
- Data Extraction:
Build a system to collect publicly available product information (e.g., title, price, reviews) across diverse product categories. Must handle dynamic pages and scalable data collection routines.
- Change Detection:
Create a reliable engine to detect price fluctuations—including minor adjustments—on a frequent and scheduled basis.
- User Notification System:
Implement a notification framework via email or push alerts (via integrations like SendGrid or Twilio) based on user preferences.
- Data Storage & Retrieval:
Design a lightweight database system that supports historical price comparisons, product tracking, and optimized retrieval performance.
- Scalability:
Ensure the solution is modular and capable of handling a high number of SKUs efficiently across different product categories.
- Accuracy & Resilience:
Implement error-handling strategies and fallback logic for content loading failures, network issues, and anti-automation roadblocks.
- (Optional) Dashboard Interface:
Build a simple UI for users to manage their tracked products and configure alerts.
Note: The solution must respect all terms of use of any third-party websites and follow best practices to avoid breaching any platform restrictions.
Required Skills:
- Experience with automation tools like Selenium, Scrapy, Puppeteer, Playwright, etc.
- Proficiency in Python or JavaScript for scalable script development.
- Expertise in working with real-time pipelines or event-driven architectures.
- Familiarity with cloud environments (AWS, GCP, Azure) for deployment.
- Strong background in relational and NoSQL databases (e.g., PostgreSQL, MongoDB).
- Familiarity with proxies, rate-limiting, and best practices for maintaining system uptime.
- Ability to write clean, modular code with version control (Git).
- Prior experience with email/push notification APIs is highly preferred.
Deliverables:
- A functional system for tracking price changes with user notifications.
- Deployment and configuration documentation.
- Clean, well-commented source code.
- Brief report outlining technical decisions, workflows, and system architecture.
To Apply, Please Share:
- Relevant experience and past project samples (GitHub + LinkedIn + Resume).
- Your typical approach to designing scalable scraping or data automation systems.
- Availability and estimated timeline for delivery.
- Your preferred hourly or project-based rate.
- Start your proposal with:
"My Prior Experience in this..." so we know you’ve read the full post.
We’re excited to connect with freelancers who are driven, detail-oriented, and passionate about building smart systems.