DevOps with big data experience needed
Budget: £10 – £15 GBP
Hi,
We have a need for a DevOps engineer for a Big Data project of ours
- Build and maintain CI/CD pipelines for the product's different layers. Propose and utilize cost effective solutions for this purpose. Github actions experience will be needed
- Build and maintain test, staging and production environments on AWS mainly using Elasticbeanstalk, S3, EMR, Aurora, EC2s taking advantage of the CI/CD pipelines when possible
- Help with troubleshooting and configuration fine-tuning on several platforms (apache, haddoop, hbase etc)
- Build and maintain a local testing environment replica for developers.
- Help plan for "non hyper cloud" deployments. OpenStack, ProxMox, Kubernetes. All are on the table but the most "appropriate" one must be selected considering the architecture and CI/CD capabilities.
- Build and maintain "on prem" alternatives of the AWS structure. This will include hardware planing (server) but also deployment of several VMs (or containers at some point) with techs including php+nginx, hadoop with hbase (and phoenix), sql database (probably mysql) and CEPH object storage.
- Be the technical champion that solves "problems" for the developers to take their code to live production.
Some background
- We have a part-time DevOps on board who has done a good job so far but he is unable to coop with the workload. For this role, it is essential to be able to wrok with him, understand the current structure, fix the urgent problems and move on the planning and improving the existing approach. I expect a high load initially as we have some work piled up (ie around part time at worst.. even a full time for a month or two is also acceptable) but after that I expect the need be around 10 hours per week. If you are not ok with this kind of initial high load, please don't waste my or your time, I don't have the luxary to wait for a week to get started.
- Hadoop / HBase / Phoenix part has been the most problematic part of our structure and someone who has experience with those technologies that can help us fine tune the performance will definetaly have a longer term impact on the project.
- We are moving away from AWS to a smaller VMWare cloud.
- 3 months trial but a very long term afterwards.
We have a need for a DevOps engineer for a Big Data project of ours
- Build and maintain CI/CD pipelines for the product's different layers. Propose and utilize cost effective solutions for this purpose. Github actions experience will be needed
- Build and maintain test, staging and production environments on AWS mainly using Elasticbeanstalk, S3, EMR, Aurora, EC2s taking advantage of the CI/CD pipelines when possible
- Help with troubleshooting and configuration fine-tuning on several platforms (apache, haddoop, hbase etc)
- Build and maintain a local testing environment replica for developers.
- Help plan for "non hyper cloud" deployments. OpenStack, ProxMox, Kubernetes. All are on the table but the most "appropriate" one must be selected considering the architecture and CI/CD capabilities.
- Build and maintain "on prem" alternatives of the AWS structure. This will include hardware planing (server) but also deployment of several VMs (or containers at some point) with techs including php+nginx, hadoop with hbase (and phoenix), sql database (probably mysql) and CEPH object storage.
- Be the technical champion that solves "problems" for the developers to take their code to live production.
Some background
- We have a part-time DevOps on board who has done a good job so far but he is unable to coop with the workload. For this role, it is essential to be able to wrok with him, understand the current structure, fix the urgent problems and move on the planning and improving the existing approach. I expect a high load initially as we have some work piled up (ie around part time at worst.. even a full time for a month or two is also acceptable) but after that I expect the need be around 10 hours per week. If you are not ok with this kind of initial high load, please don't waste my or your time, I don't have the luxary to wait for a week to get started.
- Hadoop / HBase / Phoenix part has been the most problematic part of our structure and someone who has experience with those technologies that can help us fine tune the performance will definetaly have a longer term impact on the project.
- We are moving away from AWS to a smaller VMWare cloud.
- 3 months trial but a very long term afterwards.