Python S3 Connection adjustment & Polars Data Utility
Budget: $10 – $30 USD
Task Description
I am looking for a Python developer to handle two distinct sub-tasks within an existing project. This is a targeted engagement estimated to take 3 hours.
As this work integrates into an active codebase, the developer must ensure that all new implementations align with the current project structure.
Deliverables
1. Template Refactor (IAM Integration)
* Task: Take an existing Python template (currently configured for anonymous S3 access) and refactor it to support IAM Authentication.
* Requirement: The refactored version must utilize Access Key ID and Secret Access Key via boto3 to establish a secure connection.
* Goal: A functional, authenticated version of the original template logic that fits into the existing workflow.
2. Polars Data Utility (New Development)
Implement a separate, standalone module (distinct from the template but within the project environment) using Polars to handle CSV, Parquet, and JSONL files:
* Schema Inference: Programmatically detect and return column names and Polars data types.
* Sample Preview: Retrieve a small sample (e.g., top 10 rows) from the S3 object.
* Optimization: The logic must be optimized (e.g., using scan_csv, scan_parquet, or scan_ndjson) to ensure the schema is inferred and the preview is generated efficiently without downloading the entire dataset.
Testing & Support
* Environment Setup: The freelancer is responsible for properly setting up their development environment to interface with the project’s existing infrastructure.
* UI Testing: The freelancer will be required to test the implementation through the UI to ensure full compatibility with the existing project's front-end requirements.
* Project Owner Support: S3 access and credentials will be provided. The Project Owner will be available to provide environment support, clarify the existing codebase, and help troubleshoot or verify the output in real-time.
Technical Stack
* Language: Python 3.12
Budget: $25
Estimation: 3hours
I am looking for a Python developer to handle two distinct sub-tasks within an existing project. This is a targeted engagement estimated to take 3 hours.
As this work integrates into an active codebase, the developer must ensure that all new implementations align with the current project structure.
Deliverables
1. Template Refactor (IAM Integration)
* Task: Take an existing Python template (currently configured for anonymous S3 access) and refactor it to support IAM Authentication.
* Requirement: The refactored version must utilize Access Key ID and Secret Access Key via boto3 to establish a secure connection.
* Goal: A functional, authenticated version of the original template logic that fits into the existing workflow.
2. Polars Data Utility (New Development)
Implement a separate, standalone module (distinct from the template but within the project environment) using Polars to handle CSV, Parquet, and JSONL files:
* Schema Inference: Programmatically detect and return column names and Polars data types.
* Sample Preview: Retrieve a small sample (e.g., top 10 rows) from the S3 object.
* Optimization: The logic must be optimized (e.g., using scan_csv, scan_parquet, or scan_ndjson) to ensure the schema is inferred and the preview is generated efficiently without downloading the entire dataset.
Testing & Support
* Environment Setup: The freelancer is responsible for properly setting up their development environment to interface with the project’s existing infrastructure.
* UI Testing: The freelancer will be required to test the implementation through the UI to ensure full compatibility with the existing project's front-end requirements.
* Project Owner Support: S3 access and credentials will be provided. The Project Owner will be available to provide environment support, clarify the existing codebase, and help troubleshoot or verify the output in real-time.
Technical Stack
* Language: Python 3.12
Budget: $25
Estimation: 3hours
Related categories:
JavaScript
Python
Software Development
Coding
Flask
Open Source
Database Management