Automated Place Data Scraping and Processing

Job ID: 38952326

Budget: £20 – £250 GBP

I need a skilled data engineer to build an automated scraping pipeline that gathers structured data about places from various online sources. The data will need to be processed for consistency and quality before being stored in a PostgreSQL database.

Key Responsibilities:
- Scrape data from travel websites, local business directories, and social media platforms.
- Extract key data points such as name, address, user reviews and ratings, and operating hours.
- Ensure data aligns with a pre-defined Place Entity and fits within provided PlaceTypes and PlaceSubTypes enums.

Ideal Skills:
- Proficiency in web scraping tools and techniques.
- Experience with data processing and cleaning.
- Familiarity with PostgreSQL and structuring data for databases.

Data Processing Requirements:
- Data deduplication: Remove any duplicate entries to maintain data integrity.
- Data format standardization: Ensure all data is in a consistent and usable format.
- Error and anomaly detection: Identify and handle any errors or anomalies within the data.

The goal is to have a reliable and automated system that continually provides high-quality, structured data about various places.