Azure Mixed-Source Warehouse Build
Budget: $750 – $1,500 USD
I’m ready to spin up a full-fledged data warehouse in Microsoft Azure that can comfortably host both transactional and analytical datasets. The incoming information will arrive in daily or hourly batches from three places: several SQL databases, a handful of third-party APIs, and a file-server drop of flat files.
The core of the job is to design and implement the warehouse architecture, automate the ingestion pipelines, and model the data so it is analytics-ready the moment it lands. Batch processing is perfectly adequate for our needs; no streaming component is required right now.
Preferred Azure services include Synapse Analytics (dedicated SQL pool), Data Factory for orchestration, and Data Lake Storage Gen2 as the raw landing zone, but I’m open to alternative Azure-native components if they achieve the same reliability and cost profile.
Deliverables
• High-level architecture diagram (PDF or draw.io)
• Infrastructure-as-code template (ARM, Bicep, or Terraform) that provisions all resources
• Data Factory (or equivalent) pipelines that pull from the SQL sources, call the APIs, and ingest the flat files, then load curated tables in Synapse
• Documentation covering object naming conventions, scheduling, and how to onboard a new data source
• A short video or live walk-through showing the solution running end-to-end
Acceptance criteria
1. A clean deployment from scratch using the IaC template in a fresh Azure subscription
2. All three source types land in the raw zone, are transformed, and populate dimensional or fact tables without manual intervention
3. Daily batch run completes within the agreed maintenance window and surfaces any failures through Azure Monitor alerts
If you’ve built Azure data warehouses that juggle multiple source types and prefer working in code-driven environments, let’s make this happen.
The core of the job is to design and implement the warehouse architecture, automate the ingestion pipelines, and model the data so it is analytics-ready the moment it lands. Batch processing is perfectly adequate for our needs; no streaming component is required right now.
Preferred Azure services include Synapse Analytics (dedicated SQL pool), Data Factory for orchestration, and Data Lake Storage Gen2 as the raw landing zone, but I’m open to alternative Azure-native components if they achieve the same reliability and cost profile.
Deliverables
• High-level architecture diagram (PDF or draw.io)
• Infrastructure-as-code template (ARM, Bicep, or Terraform) that provisions all resources
• Data Factory (or equivalent) pipelines that pull from the SQL sources, call the APIs, and ingest the flat files, then load curated tables in Synapse
• Documentation covering object naming conventions, scheduling, and how to onboard a new data source
• A short video or live walk-through showing the solution running end-to-end
Acceptance criteria
1. A clean deployment from scratch using the IaC template in a fresh Azure subscription
2. All three source types land in the raw zone, are transformed, and populate dimensional or fact tables without manual intervention
3. Daily batch run completes within the agreed maintenance window and surfaces any failures through Azure Monitor alerts
If you’ve built Azure data warehouses that juggle multiple source types and prefer working in code-driven environments, let’s make this happen.