Dynamic Apache Airflow DAGs
Budget: ₹750 – ₹1,250 INR
We are looking for an experienced data engineer with strong expertise in Apache Airflow, especially in dynamic DAG generation and scalable workflow design.
Key Requirements:
Strong hands-on experience with Apache Airflow
Proven experience in dynamic DAG generation
Solid experience working with AWS, especially MWAA (Managed Workflows for Apache Airflow)
Good understanding of modern data/median architecture (data pipelines, orchestration, ETL/ELT workflows)
Ability to design scalable, maintainable, and efficient workflows
Experience with debugging and optimizing Airflow performance
Nice to Have:
Experience with data lakes / warehouses (e.g., S3, Redshift, Snowflake, etc.)
Infrastructure as Code (Terraform/CloudFormation)
CI/CD for data pipelines
Project Scope:
Design and implement dynamic DAGs
Optimize existing Airflow workflows
Set up / improve MWAA environment
Provide best practices for scalable data pipeline architecture
Key Requirements:
Strong hands-on experience with Apache Airflow
Proven experience in dynamic DAG generation
Solid experience working with AWS, especially MWAA (Managed Workflows for Apache Airflow)
Good understanding of modern data/median architecture (data pipelines, orchestration, ETL/ELT workflows)
Ability to design scalable, maintainable, and efficient workflows
Experience with debugging and optimizing Airflow performance
Nice to Have:
Experience with data lakes / warehouses (e.g., S3, Redshift, Snowflake, etc.)
Infrastructure as Code (Terraform/CloudFormation)
CI/CD for data pipelines
Project Scope:
Design and implement dynamic DAGs
Optimize existing Airflow workflows
Set up / improve MWAA environment
Provide best practices for scalable data pipeline architecture
Related categories:
Linux
Data Processing
Amazon Web Services
Debugging
Redshift
Data Integration
Data Governance
Terraform
CI/CD
Snowflake