Setup Kubernetes cluster with Spark configuration

Job ID: 37416171

Budget: €36 – €0 EUR

We are looking for someone with experience in setting up a Kubernetes cluster and configuring Spark. The Kubernetes cluster has to run 3 services: Spark, Airflow and a PostgreSQL (DWH). The spark configuration also entails setting up a dedicated Hive Metastore so that our jobs can run flawlessly. Airflow should be able to submit jobs to the spark master and we should have a good way of getting operational logs out of the Kubernetes cluster. The freelancer should help us setting up Kubernetes with Spark. We need assistance the coming 3 to 4 weeks (Nov 13 - Dec 8). I am looking to work with a reliable and experienced freelancer who can provide a high-quality job.
Related categories: Spark Kubernetes