Big Data Conversion to Parquet
Budget: ₹1,500 – ₹12,500 INR
I need an experienced big data specialist to convert 5 TB of flat JSONL data stored in an S3 bucket on AWS into Parquet format with modulo hashing for optimized query performance.
Requirements:
- Use AWS EMR for the conversion
- Handle flat JSON (key-value pairs) structure
- Implement specific field hashing as per my requirements
Ideal Skills and Experience:
- Proficient in AWS EMR
- Strong background in data format conversion
- Experience with JSONL and Parquet
- Knowledge of modulo hashing and query optimization
Please provide a detailed approach and timeline.
Requirements:
- Use AWS EMR for the conversion
- Handle flat JSON (key-value pairs) structure
- Implement specific field hashing as per my requirements
Ideal Skills and Experience:
- Proficient in AWS EMR
- Strong background in data format conversion
- Experience with JSONL and Parquet
- Knowledge of modulo hashing and query optimization
Please provide a detailed approach and timeline.
Related categories:
Python
Data Processing
MySQL
Amazon Web Services
Elasticsearch
JSON
Data Management
Big Data