Map reduce algorithm using hadoop
Budget: $30 – $250 USD
1. A PDF document containing the results of the execution of the 2 programs. Along with each result, write a short paragraph explaining how you used your programs to generate the results. Include a description of any post-processing you performed outside of Hadoop to generate these results. Re: the “plot” required of the second program: one way to do this is to generate the necessary info (cloud-side) and then cut-and-paste it into Excel running on your laptop (and then make the visual rep of this).
a. Two receive the full credit, the document should contain screenshot of Hadoop executions!
2. A zip file containing all of your Map/Reduce programs
Now, using Python, write the two separate Map/Reduce programs (identified earlier) using Hadoop 2.10.1 on
GCP to compute the following using the sample HVAC data
You are required to run your program(s) via Hadoop 2.10.1 pseudo-distributed mode on an GCP instance. You cannot use any “add-on” to
Hadoop (such as Hive). The great majority of the data processing must be performed within Hadoop; relatively minor “post-processing” (e.g., sort) is allowed outside of Hadoop.
a. Two receive the full credit, the document should contain screenshot of Hadoop executions!
2. A zip file containing all of your Map/Reduce programs
Now, using Python, write the two separate Map/Reduce programs (identified earlier) using Hadoop 2.10.1 on
GCP to compute the following using the sample HVAC data
You are required to run your program(s) via Hadoop 2.10.1 pseudo-distributed mode on an GCP instance. You cannot use any “add-on” to
Hadoop (such as Hive). The great majority of the data processing must be performed within Hadoop; relatively minor “post-processing” (e.g., sort) is allowed outside of Hadoop.