AWS data engineer (EMR, AWS Glue along with Python)
Budget: ₹600 – ₹1,500 INR
Skill sets:
Python programming language, Spark, Kafka / Kinesis, EMR / Glue, S3, Redshift, Airflow,
Jenkins)
Activities developers need to perform
Load data in S3
Perform ETL functions in glue
Data tiering on S3
Filter, join, aggregation
Move data to Redshift
EMR
Apache Airflow for overall orchestrator
They will use pipeline on Glue
Python skill
Move code from Dev to Production
Python programming language, Spark, Kafka / Kinesis, EMR / Glue, S3, Redshift, Airflow,
Jenkins)
Activities developers need to perform
Load data in S3
Perform ETL functions in glue
Data tiering on S3
Filter, join, aggregation
Move data to Redshift
EMR
Apache Airflow for overall orchestrator
They will use pipeline on Glue
Python skill
Move code from Dev to Production