Data Engineering Expert Needed
Budget: $15 – $25 USD
Job Description:
As a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will be responsible for creating data pipelines, ensuring data quality, and optimizing data storage solutions. This role requires an expert in Python programming, with a focus on using Airflow for pipeline orchestration, AWS RDS for PostgreSQL database modeling, and Snowflake for OLAP data warehousing using dbt.
Key Responsibilities:
Design, develop, and maintain data pipelines using Apache Airflow.
Model and optimize PostgreSQL databases on AWS RDS.
Develop and manage OLAP data warehouses in Snowflake using dbt.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
Ensure data quality, consistency, and security across all systems.
Monitor and troubleshoot data pipeline issues, ensuring minimal downtime and optimal performance.
Stay up-to-date with industry best practices and emerging technologies in data engineering.
Requirements:
Bachelor's or Master’s degree in Computer Science, Engineering, or a related field.
Proven experience as a Data Engineer, with a focus on creating data pipelines and data warehousing.
Expert-level proficiency in Python programming.
Extensive experience with Apache Airflow for orchestrating data workflows.
Strong knowledge of AWS RDS and PostgreSQL database modeling.
Hands-on experience with Snowflake and dbt for OLAP data warehousing.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills.
Preferred Qualifications:
Experience with other AWS services, such as S3, Lambda, and Glue.
Familiarity with big data technologies, such as EMR and Athena.
Knowledge of ETL processes and tools.
Certifications in AWS or other relevant technologies.
Mumbai based candidates only to apply.
Only those who are ready to come to our office in Mumbai, may apply.
As a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will be responsible for creating data pipelines, ensuring data quality, and optimizing data storage solutions. This role requires an expert in Python programming, with a focus on using Airflow for pipeline orchestration, AWS RDS for PostgreSQL database modeling, and Snowflake for OLAP data warehousing using dbt.
Key Responsibilities:
Design, develop, and maintain data pipelines using Apache Airflow.
Model and optimize PostgreSQL databases on AWS RDS.
Develop and manage OLAP data warehouses in Snowflake using dbt.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
Ensure data quality, consistency, and security across all systems.
Monitor and troubleshoot data pipeline issues, ensuring minimal downtime and optimal performance.
Stay up-to-date with industry best practices and emerging technologies in data engineering.
Requirements:
Bachelor's or Master’s degree in Computer Science, Engineering, or a related field.
Proven experience as a Data Engineer, with a focus on creating data pipelines and data warehousing.
Expert-level proficiency in Python programming.
Extensive experience with Apache Airflow for orchestrating data workflows.
Strong knowledge of AWS RDS and PostgreSQL database modeling.
Hands-on experience with Snowflake and dbt for OLAP data warehousing.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills.
Preferred Qualifications:
Experience with other AWS services, such as S3, Lambda, and Glue.
Familiarity with big data technologies, such as EMR and Athena.
Knowledge of ETL processes and tools.
Certifications in AWS or other relevant technologies.
Mumbai based candidates only to apply.
Only those who are ready to come to our office in Mumbai, may apply.