Python Developer for PostgreSQL Upsert Automation
Budget: $30 – $250 USD
We are looking for an experienced Python developer to help us automate the process of upserting data from multiple CSV files into our PostgreSQL database using SQLAlchemy. The developer will be responsible for creating a Python script that can read CSV files from a directory, extract data, and insert or update that data into our database based on a specific primary key.
The project requires the following deliverables:
1. Python script that can read CSV files from a directory and upsert data into our PostgreSQL database
2. The script should use SQLAlchemy to interact with the database
3. The script should handle errors such as missing files, incorrect data types, and duplicate primary keys
4. The script should be scalable and able to handle large CSV files with millions of rows
5. The script should be well-documented and easy to maintain
Required Skills:
1. Strong experience in Python programming language
2. Experience in working with PostgreSQL databases and SQLAlchemy
3. Familiarity with CSV file handling and data manipulation
4. Experience in working with large datasets and optimizing performance
5. Strong problem-solving skills and attention to detail
6. Excellent communication skills
Preferred Skills:
1. Experience in data engineering and ETL processes
2. Familiarity with AWS or other cloud platforms
3. Knowledge of best practices in database design and optimization
Project Timeline:
The project should be completed within 4-6 days, starting from the date of the contract. The timeline can be adjusted based on the project scope and complexity.
Communication:
The developer will be required to provide regular updates on the project progress and communicate any issues or challenges faced during the development process. Communication can be through email, chat, or video conferencing.
Budget:
Please provide your proposed budget for this project
The project requires the following deliverables:
1. Python script that can read CSV files from a directory and upsert data into our PostgreSQL database
2. The script should use SQLAlchemy to interact with the database
3. The script should handle errors such as missing files, incorrect data types, and duplicate primary keys
4. The script should be scalable and able to handle large CSV files with millions of rows
5. The script should be well-documented and easy to maintain
Required Skills:
1. Strong experience in Python programming language
2. Experience in working with PostgreSQL databases and SQLAlchemy
3. Familiarity with CSV file handling and data manipulation
4. Experience in working with large datasets and optimizing performance
5. Strong problem-solving skills and attention to detail
6. Excellent communication skills
Preferred Skills:
1. Experience in data engineering and ETL processes
2. Familiarity with AWS or other cloud platforms
3. Knowledge of best practices in database design and optimization
Project Timeline:
The project should be completed within 4-6 days, starting from the date of the contract. The timeline can be adjusted based on the project scope and complexity.
Communication:
The developer will be required to provide regular updates on the project progress and communicate any issues or challenges faced during the development process. Communication can be through email, chat, or video conferencing.
Budget:
Please provide your proposed budget for this project