Daily CSV Validation Automation
Budget: $250 – $750 AUD
I deal with daily CSV drops that hold logistics data from several sources, and I need them cleaned, validated, and reshaped into a polished Excel report without manual touch-ups. The flow I have in mind is straightforward:
• A Python 3.x script (pandas, openpyxl or similar) watches a folder, ingests every new file each day, and applies rule-based data validation and cleaning—date formats, missing values, duplicate tracking numbers, column consistency, you name it.
• Any row that fails a rule lands in an “Errors” sheet with a clear reason so nothing slips through.
• Once verified, the script writes the sanitised data into a client-ready Excel template, saving it to an output folder that I can immediately email or archive.
• Running it from Windows Task Scheduler is fine; a simple config (JSON/YAML) for tweaking rules or column names will future-proof the job.
Deliverables
1. Fully commented Python code and requirements.txt
2. One reusable Excel template or code-generated workbook matching my branding
3. Setup guide + short screen-capture walkthrough
4. Proof-of-concept run on my sample daily files
Acceptance criteria
– All daily logistics CSVs processed with zero critical errors
– Validation rules editable without touching the core code
– Clean Excel report generated in under two minutes for a 50 000-row file
Sample files are ready to share as soon as you’re on board.
• A Python 3.x script (pandas, openpyxl or similar) watches a folder, ingests every new file each day, and applies rule-based data validation and cleaning—date formats, missing values, duplicate tracking numbers, column consistency, you name it.
• Any row that fails a rule lands in an “Errors” sheet with a clear reason so nothing slips through.
• Once verified, the script writes the sanitised data into a client-ready Excel template, saving it to an output folder that I can immediately email or archive.
• Running it from Windows Task Scheduler is fine; a simple config (JSON/YAML) for tweaking rules or column names will future-proof the job.
Deliverables
1. Fully commented Python code and requirements.txt
2. One reusable Excel template or code-generated workbook matching my branding
3. Setup guide + short screen-capture walkthrough
4. Proof-of-concept run on my sample daily files
Acceptance criteria
– All daily logistics CSVs processed with zero critical errors
– Validation rules editable without touching the core code
– Clean Excel report generated in under two minutes for a 50 000-row file
Sample files are ready to share as soon as you’re on board.
Related categories:
Python
Data Processing
Excel
Software Architecture
JSON
Data Analysis
Automation
Pandas