Azure Data Engineer realtime Job support
Budget: ₹400 – ₹750 INR
Big Data Engineer with expertise in the Hadoop Ecosystem, including on GCP, AWS and Snowflake, GCP - Big Query
Extensive experience deploying cloud-based applications using Amazon Web Services such as Amazon EC2, S3, RDS, IAM, Auto Scaling, CloudWatch, SNS, Athena, Glue, Kinesis, Lambda, EMR, Redshift, and DynamoDB.
Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena.
Proven expertise in deploying major software solutions for various high-end clients meeting the business requirements such as big data Processing, Ingestion, Analytics and Cloud Migration from On-prem to AWS Cloud using AWS EMR, S3, DynamoDB
Hands of experience in GCP, Big Query, cloud dataflow, GCS bucket, G - cloud function, Pub/sub cloud shell, GSUTIL, BQ command line utilities, Data Proc, Stack driver
Spark applications using Python to perform various data cleansing, validation, transformation, and summarization activities according to the requirement
Created data pipelines with Airflow, populating Data Lakes in GCP - Big Query
Extensive experience deploying cloud-based applications using Amazon Web Services such as Amazon EC2, S3, RDS, IAM, Auto Scaling, CloudWatch, SNS, Athena, Glue, Kinesis, Lambda, EMR, Redshift, and DynamoDB.
Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena.
Proven expertise in deploying major software solutions for various high-end clients meeting the business requirements such as big data Processing, Ingestion, Analytics and Cloud Migration from On-prem to AWS Cloud using AWS EMR, S3, DynamoDB
Hands of experience in GCP, Big Query, cloud dataflow, GCS bucket, G - cloud function, Pub/sub cloud shell, GSUTIL, BQ command line utilities, Data Proc, Stack driver
Spark applications using Python to perform various data cleansing, validation, transformation, and summarization activities according to the requirement
Created data pipelines with Airflow, populating Data Lakes in GCP - Big Query
Related categories:
Python
SQL
Azure
Hadoop
Data Analytics
ETL
PySpark
Big Data
Data Engineer
Snowflake