Azure Data Processing Pipeline Development
Budget: $15 – $25 USD
As a client, I'm seeking an experienced Azure Cloud Engineer with proven expertise in developing data processing pipelines. The primary goal of this project is to set up and configure a robust, efficient data processing framework.
Key Responsibilities:
- Design and implement a data processing pipeline on Azure
- Ensure the pipeline is capable of handling structured data, particularly from SQL databases
- Optimize the pipeline for reliability, scalability, and performance
Ideal Skills and Experience:
- Extensive experience with Azure cloud services, particularly in a data engineering context
- Proficiency in data processing tools such as Scala-Spark, Pyspark
- Strong understanding of Unix/Linux systems and SQL
- Prior experience working with Data Warehousing, Data Lake, and Hive systems
- Proven track record in developing complex data processing pipelines
- Excellent problem-solving skills and ability to find innovative solutions to data processing challenges
This role is suited to a freelancer who is not only a technical expert in cloud engineering but also has a deep understanding of data handling and processing. If you fit these requirements, I'd be excited to discuss this project with you.
Key Responsibilities:
- Design and implement a data processing pipeline on Azure
- Ensure the pipeline is capable of handling structured data, particularly from SQL databases
- Optimize the pipeline for reliability, scalability, and performance
Ideal Skills and Experience:
- Extensive experience with Azure cloud services, particularly in a data engineering context
- Proficiency in data processing tools such as Scala-Spark, Pyspark
- Strong understanding of Unix/Linux systems and SQL
- Prior experience working with Data Warehousing, Data Lake, and Hive systems
- Proven track record in developing complex data processing pipelines
- Excellent problem-solving skills and ability to find innovative solutions to data processing challenges
This role is suited to a freelancer who is not only a technical expert in cloud engineering but also has a deep understanding of data handling and processing. If you fit these requirements, I'd be excited to discuss this project with you.