Azure SQL Database & Data Lake Expert
Budget: ₹37,500 – ₹75,000 INR
For this project, I am in need of an expert in Azure SQL Database and Azure Data Lake Storage. I have a data solution project that involves less than 1TB of data, hence would classify this as a small scale project.
Key tasks include, but are not limited to:
Data Solutions: Set up and manage an Azure SQL Database and Azure Data Lake Storage for my data solution.
Main task :
The legacy Azure SQL DWH has multiple resources including Business Central, Dynamics 365, HubSpot API, NetSuite API, ServiceNow API and Kimble one. (Alteryx and Workato sources can be ignored). Couple of other company sources need to be migrated as well.
The new Azure solution need to bring all these data into one lake using any effective azure services into a bronze layer. Further these data need to be transformed using either ADF or Data bricks into gold layer as per the sheet attached into new SQL DWH for the PBI reporting. On top of this we need to cover the data governance and data retention policy as per GDPR.
Challenges:
1. NetSuite (oracle) system need to be connected to Azure; this is contract management system.
2. One more company recently which was acquired has financial data on-prem on different tenant we need to bring in this data to the Azure cloud.
Note : Business central is for the financial data and GL entry ,Dynamics 365 is for the sales and invoicing ,Hubspot for the sales lead, Service Now is the ticketing system , Workato is the ETL tool which connects all the system,Kimble one is the project management platform. There is one billing system acquire going on we might need to bring in the data from this source also.
Data Handling: Perform necessary data normalization or transformation before storing the data in Azure SQL Database.
The ideal candidate for this role will have past experience with Azure SQL Database and Azure Data Lake Storage, good understanding of data normalization or transformation process, and the ability to handle small scale data solutions. Your expertise and professional input are crucial to the success of this project.
Key tasks include, but are not limited to:
Data Solutions: Set up and manage an Azure SQL Database and Azure Data Lake Storage for my data solution.
Main task :
The legacy Azure SQL DWH has multiple resources including Business Central, Dynamics 365, HubSpot API, NetSuite API, ServiceNow API and Kimble one. (Alteryx and Workato sources can be ignored). Couple of other company sources need to be migrated as well.
The new Azure solution need to bring all these data into one lake using any effective azure services into a bronze layer. Further these data need to be transformed using either ADF or Data bricks into gold layer as per the sheet attached into new SQL DWH for the PBI reporting. On top of this we need to cover the data governance and data retention policy as per GDPR.
Challenges:
1. NetSuite (oracle) system need to be connected to Azure; this is contract management system.
2. One more company recently which was acquired has financial data on-prem on different tenant we need to bring in this data to the Azure cloud.
Note : Business central is for the financial data and GL entry ,Dynamics 365 is for the sales and invoicing ,Hubspot for the sales lead, Service Now is the ticketing system , Workato is the ETL tool which connects all the system,Kimble one is the project management platform. There is one billing system acquire going on we might need to bring in the data from this source also.
Data Handling: Perform necessary data normalization or transformation before storing the data in Azure SQL Database.
The ideal candidate for this role will have past experience with Azure SQL Database and Azure Data Lake Storage, good understanding of data normalization or transformation process, and the ability to handle small scale data solutions. Your expertise and professional input are crucial to the success of this project.
Related categories:
Data Processing
Cloud Computing
Azure
Database Administration
Microsoft SQL Server