Automated Database Scraping & Cleaning
Budget: ₹750 – ₹1,250 INR
I need an automated workflow that connects to several structured databases, extracts the raw records, and then cleans the resulting mixed data (text and numerical) so it is ready for immediate analysis.
Here’s what I expect:
• A repeatable script or small utility—Python with SQL connectors, pandas, or a comparable stack is fine—that logs in, queries the data sets, and exports them to CSV or Parquet.
• Built-in cleansing routines that standardise text fields, handle encoding issues, trim white-space, and validate numerical ranges or missing values in one pass.
• Clear inline comments plus a short README so I can adjust connection details and field mappings later on.
I will provide test credentials and a data dictionary. The job is complete when the script runs end-to-end on my sample instance and the cleaned output matches the schema I supply without errors or null-row mismatches.
Here’s what I expect:
• A repeatable script or small utility—Python with SQL connectors, pandas, or a comparable stack is fine—that logs in, queries the data sets, and exports them to CSV or Parquet.
• Built-in cleansing routines that standardise text fields, handle encoding issues, trim white-space, and validate numerical ranges or missing values in one pass.
• Clear inline comments plus a short README so I can adjust connection details and field mappings later on.
I will provide test credentials and a data dictionary. The job is complete when the script runs end-to-end on my sample instance and the cleaned output matches the schema I supply without errors or null-row mismatches.