Data Engineer

I need end-to-end data engineering support on Azure. The work begins with designing and standing-up robust ingestion pipelines in Azure Synapse and orchestrating them through Azure Data Factory, landing raw assets in Azure Data Lake storage. From there, you will craft efficient transformation logic—complex SQL queries and stored procedures—to cleanse, aggregate, and model the data for downstream use.

Once the curated layer is ready, the insights must be surfaced. Tableau dashboards will visualise key business metrics and KPIs, so the semantic layer and extracts you create need to feed those reports with low latency and high reliability.

Quality and governance sit at the core of this engagement. I will rely on you to embed automated data-quality checks, lineage tracking, and performance monitoring across every stage. Repetitive steps should be scripted—Python or Spark preferred—to ensure the platform remains scalable and largely hands-free once deployed. All development must adhere to our internal security standards and relevant industry privacy guidelines.

Deliverables
• Production-ready ingestion and transformation pipelines built with Azure Synapse, Azure Data Factory, and Azure Data Lake
• Documented SQL logic (views, stored procedures) for each transformation step
• Automated quality-check scripts and performance monitors
• A suite of Tableau dashboards with published data sources and clear KPI definitions
• Runbook and deployment documentation for ongoing maintenance

Acceptance criteria will be a fully operational pipeline running on schedule, dashboards refreshing without error, and audit logs confirming data integrity across at least two consecutive cycles.