Python-Based File Processing System
Budget: $1,500 – $3,000 USD
Python-Based File Processing System
Overview:
We are looking for an experienced Python developer to build a robust system for managing and modifying large datasets of files. The system will handle a database of over 3-4 million files,
Customers upload files through a web interface and request specific modifications. Your task is to create a solution that efficiently matches, modifies, and delivers these files with precision and speed.
Scope of Work:
File Matching:
Develop a script to identify 100% matches between uploaded files and our database for requested versions.
Closest Match Analysis:
Implement logic to find the closest matching file when no exact match exists (e.g., 85% similarity).
Compare key areas of the files and apply verified modifications from our database to the customer’s uploaded file.
Advanced Search for Relocated Data:
Add functionality to locate and modify matching data in different areas of the customer’s file if discrepancies arise due to structural differences.
File Delivery:
Save the modified files and upload them to a designated FTP for customer download.
also update MySQL Database
Key Requirements:
Proficiency in Python and advanced file handling techniques.
Experience with fuzzy matching algorithms (e.g., ssdeep, TLSH) and pattern recognition.
ssdeep, difflib,
Ability to optimize performance for handling large-scale datasets.
Integration experience with APIs for file exchange and system communication.
Overview:
We are looking for an experienced Python developer to build a robust system for managing and modifying large datasets of files. The system will handle a database of over 3-4 million files,
Customers upload files through a web interface and request specific modifications. Your task is to create a solution that efficiently matches, modifies, and delivers these files with precision and speed.
Scope of Work:
File Matching:
Develop a script to identify 100% matches between uploaded files and our database for requested versions.
Closest Match Analysis:
Implement logic to find the closest matching file when no exact match exists (e.g., 85% similarity).
Compare key areas of the files and apply verified modifications from our database to the customer’s uploaded file.
Advanced Search for Relocated Data:
Add functionality to locate and modify matching data in different areas of the customer’s file if discrepancies arise due to structural differences.
File Delivery:
Save the modified files and upload them to a designated FTP for customer download.
also update MySQL Database
Key Requirements:
Proficiency in Python and advanced file handling techniques.
Experience with fuzzy matching algorithms (e.g., ssdeep, TLSH) and pattern recognition.
ssdeep, difflib,
Ability to optimize performance for handling large-scale datasets.
Integration experience with APIs for file exchange and system communication.