Hadoop Cluster Simulation on 4 Ubuntu VMs
Budget: $30 – $250 USD
I'm in need of a professional who can set up and simulate a Hadoop cluster consisting of 4 Ubuntu virtual machines. This cluster needs to be equipped with Hive, Zeppelin, Spark and Kafka.
Key requirements for this project include:
- Configuring the Hive setup with redundancy, specifically 2 name nodes and 2 data nodes.
- Establishing a Kafka Consumer cluster that can listen to a producer located on an external server.
- Creating a Spark Cluster.
- Configuring Spark to consistently monitor data coming from Kafka, perform minor processing on each new message (specifically column drops and minor modifications) and save the outcomes to a Hadoop Hive table.
Time is of the essence as the project must be delivered within a one-day timeframe.
Ideal candidates for this project would have:
- Extensive experience with Hadoop and its associated components (Hive, Spark, Kafka).
- Proficiency in setting up and simulating virtual machine clusters, specifically on Ubuntu.
- Ability to deliver high-quality work under tight deadlines.
Key requirements for this project include:
- Configuring the Hive setup with redundancy, specifically 2 name nodes and 2 data nodes.
- Establishing a Kafka Consumer cluster that can listen to a producer located on an external server.
- Creating a Spark Cluster.
- Configuring Spark to consistently monitor data coming from Kafka, perform minor processing on each new message (specifically column drops and minor modifications) and save the outcomes to a Hadoop Hive table.
Time is of the essence as the project must be delivered within a one-day timeframe.
Ideal candidates for this project would have:
- Extensive experience with Hadoop and its associated components (Hive, Spark, Kafka).
- Proficiency in setting up and simulating virtual machine clusters, specifically on Ubuntu.
- Ability to deliver high-quality work under tight deadlines.