Data Engineer
Budget: ₹1,250 – ₹2,500 INR
I am looking for a data engineer with expertise in Python, SQL, and data visualization. The ideal candidate will be able to work with structured data and have experience in data analysis. The end goal of this project is to perform data analysis using the provided structured data.
Recommended Skills:
5+ years building large-scale streaming data platforms on the cloud.
Expert in technologies like Kafka, Flink, and Spark Streaming.
Proficiency in SQL, Python(or Java). Experience with Scala is a plus.
Expertise leveraging monitoring tools like New Relic, SolarWinds, Prometheus, Grafana,
Strong hands-on experience managing AWS big data services.
Responsibilities:
- Design and implement robust data pipelines to efficiently collect, transform, and integrate logs and metrics from different sources within our content ingestion workflow.
- Work in synergy with our talented ML Engineers to empower the development of advanced anomaly detection models and root cause analysis algorithms.
- Collaborate with Cloud Architects to optimize AWS infrastructure for seamless data extraction, storage, and processing, ensuring high performance and reliability.
- Be the guardian of data quality, applying data cleansing and validation techniques to guarantee the accuracy and integrity of the data used in the AI system.
- Embrace a dynamic and agile environment, where your skills and expertise will be harnessed to continuously innovate and improve the AI solution.
- Showcase your creativity and analytical acumen as you contribute to the development of interactive data visualizations and reports, enabling stakeholders to gain deep insights into our Sling's performance.
- Participate in knowledge-sharing sessions and workshops, fostering a culture of learning and growth within the team and across the organization
Recommended Skills:
5+ years building large-scale streaming data platforms on the cloud.
Expert in technologies like Kafka, Flink, and Spark Streaming.
Proficiency in SQL, Python(or Java). Experience with Scala is a plus.
Expertise leveraging monitoring tools like New Relic, SolarWinds, Prometheus, Grafana,
Strong hands-on experience managing AWS big data services.
Responsibilities:
- Design and implement robust data pipelines to efficiently collect, transform, and integrate logs and metrics from different sources within our content ingestion workflow.
- Work in synergy with our talented ML Engineers to empower the development of advanced anomaly detection models and root cause analysis algorithms.
- Collaborate with Cloud Architects to optimize AWS infrastructure for seamless data extraction, storage, and processing, ensuring high performance and reliability.
- Be the guardian of data quality, applying data cleansing and validation techniques to guarantee the accuracy and integrity of the data used in the AI system.
- Embrace a dynamic and agile environment, where your skills and expertise will be harnessed to continuously innovate and improve the AI solution.
- Showcase your creativity and analytical acumen as you contribute to the development of interactive data visualizations and reports, enabling stakeholders to gain deep insights into our Sling's performance.
- Participate in knowledge-sharing sessions and workshops, fostering a culture of learning and growth within the team and across the organization