Senior Data Engineer in Singapore, Malaysia or Indonesia
Budget: $50 – $0 USD
Our client, a digital consulting is looking for a Senior Data Engineer for a freelancing project of 3-6 months with hybrid working in Singapore, Indonesia or Malaysia.
As part of our client's Data CoE, the Data Engineer (DE) will serve to develop databases and data processing pipelines to enable predictive analytics, data science, and business intelligence for our clients.
The DE will also have strong analytical skills and the ability to combine data from different sources.
• To succeed in this role, the DE should know how to work with key stakeholders, BA / Data Analyst, and development team to analyse and define technical solutions to meet business and non-functional requirements
• Design, development and testing of:
ETL and data processing pipelines
Data Warehouse / DataMart’s / dimensional models
Logical and physical database schemas, indexes, views, cubes etc.
SQL queries and stored procedures
Azure infrastructure and services, and other platforms such as AWS, GCP, Ali Cloud, etc.
Production implementations of analytic algorithms
• Assist in creation and support of analytic, BI, and reporting processes
Responsibilities:
• Responsible for analysing, building and optimising the data infrastructure and data pipeline along with the clients
• Support the client to, potentially, migrate data from legacy systems to new solutions to align the system architecture with business requirements along with enabling the delivery of their data solutions
• Support the design of conceptual and logical data models and flowcharts
• Identify ways to improve data reliability, efficiency and quality
• Connect all client’s data sources into their data solution (third-party vendor)
• Analyse the client’s complete system landscape and specific system requirements and implement migration methods for existing data and/or consolidation of data into data warehouse cubes with the support of direct interfaces or API’s (where applicable)
• Support the client to improve system performance by conducting tests, troubleshooting and integrating new elements. Optimise new and current database systems. Define security and backup procedures
• Develop database solutions to ensure client’s information is stored effectively and securely
• Provide the foundations and support the client to set in place the tools for continuously monitoring and testing of the system interfaces to ensure optimized performance and security
• Analyse structural requirements for new solutions and applications that might be required for their clients
• Quantify benefit potential of Data Analytics solutions using ROI frameworks to aid in prioritization of development and definition with the client
Requirements
Qualification:
• Bachelor's degree in Computer Science, Computer Engineering or relevant field
• At least 3-5 years of hands-on experience in a similar role
• Strong knowledge of database structure systems and data mining
• Exposure and experience in modern data technologies and cloud
• Excellent organisational and analytical abilities
Technical Skills:
• Experience in SQL and NoSQL databases including Postgres
• Experience working in ETL / ELT tools: Talend, Informatica, SSIS
• Familiar with big data tools: Hadoop, Spark, Kafka, etc
• Familiar with data pipeline and workflow management tools
• Familiar with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc
• Familiar with Microsoft Azure, AWS, GCP and other cloud tools and services
• Good working knowledge of productivity tools such as Git, Jira, Confluence
• Experience with various messaging and stream processing
• Hands-on development with key technologies including and other relevant distributed computing languages, frameworks, and libraries is a plus
• Familiar with analytics software tools such as Alteryx, Power BI, Tableau
• Experience with DevOps and DataOps tools and environment
As part of our client's Data CoE, the Data Engineer (DE) will serve to develop databases and data processing pipelines to enable predictive analytics, data science, and business intelligence for our clients.
The DE will also have strong analytical skills and the ability to combine data from different sources.
• To succeed in this role, the DE should know how to work with key stakeholders, BA / Data Analyst, and development team to analyse and define technical solutions to meet business and non-functional requirements
• Design, development and testing of:
ETL and data processing pipelines
Data Warehouse / DataMart’s / dimensional models
Logical and physical database schemas, indexes, views, cubes etc.
SQL queries and stored procedures
Azure infrastructure and services, and other platforms such as AWS, GCP, Ali Cloud, etc.
Production implementations of analytic algorithms
• Assist in creation and support of analytic, BI, and reporting processes
Responsibilities:
• Responsible for analysing, building and optimising the data infrastructure and data pipeline along with the clients
• Support the client to, potentially, migrate data from legacy systems to new solutions to align the system architecture with business requirements along with enabling the delivery of their data solutions
• Support the design of conceptual and logical data models and flowcharts
• Identify ways to improve data reliability, efficiency and quality
• Connect all client’s data sources into their data solution (third-party vendor)
• Analyse the client’s complete system landscape and specific system requirements and implement migration methods for existing data and/or consolidation of data into data warehouse cubes with the support of direct interfaces or API’s (where applicable)
• Support the client to improve system performance by conducting tests, troubleshooting and integrating new elements. Optimise new and current database systems. Define security and backup procedures
• Develop database solutions to ensure client’s information is stored effectively and securely
• Provide the foundations and support the client to set in place the tools for continuously monitoring and testing of the system interfaces to ensure optimized performance and security
• Analyse structural requirements for new solutions and applications that might be required for their clients
• Quantify benefit potential of Data Analytics solutions using ROI frameworks to aid in prioritization of development and definition with the client
Requirements
Qualification:
• Bachelor's degree in Computer Science, Computer Engineering or relevant field
• At least 3-5 years of hands-on experience in a similar role
• Strong knowledge of database structure systems and data mining
• Exposure and experience in modern data technologies and cloud
• Excellent organisational and analytical abilities
Technical Skills:
• Experience in SQL and NoSQL databases including Postgres
• Experience working in ETL / ELT tools: Talend, Informatica, SSIS
• Familiar with big data tools: Hadoop, Spark, Kafka, etc
• Familiar with data pipeline and workflow management tools
• Familiar with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc
• Familiar with Microsoft Azure, AWS, GCP and other cloud tools and services
• Good working knowledge of productivity tools such as Git, Jira, Confluence
• Experience with various messaging and stream processing
• Hands-on development with key technologies including and other relevant distributed computing languages, frameworks, and libraries is a plus
• Familiar with analytics software tools such as Alteryx, Power BI, Tableau
• Experience with DevOps and DataOps tools and environment