Data Engineering Azure Daily Process
Budget: $15 – $25 USD
I'm looking for a skilled Data Engineer & Developer to discuss best practices and tips on how to handle different decisions.
Project requires to move data from one Azure Storage, apply some transformation and sink into another Azure Storage.
Tools to be used are:
- Databricks (Delta tables, Pyspark
- Data Factory (logging, parameters handling, alerts)
- Auxiliary services like DataOps and Keyvault
- Pipeline orquestating: good practices at handling errors and alerts notifications
I expect it to be a few hours of meeting, understanding the iniciative, improvement suggestion, and possible some hours of code review.
Project requires to move data from one Azure Storage, apply some transformation and sink into another Azure Storage.
Tools to be used are:
- Databricks (Delta tables, Pyspark
- Data Factory (logging, parameters handling, alerts)
- Auxiliary services like DataOps and Keyvault
- Pipeline orquestating: good practices at handling errors and alerts notifications
I expect it to be a few hours of meeting, understanding the iniciative, improvement suggestion, and possible some hours of code review.