Big Data Processing using Azure HD Insight, Azure Data Bricks, Azure ML studio
Budget: $30 – $250 AUD
1) Develop an aggregate of these reviews using your knowledge of Hadoop and MapReduce in Microsoft HDInsight.
a) Follow the same approach as the Big Data Analytics Workshop (using the wordcount method in HDInsight) to determine the contributory words for each level of rating.
b) Present the workflow of using HDInsight (you may use screen captures) along with a summary of findings and any insights for each level of rating. MapReduce documentation for HDInsight is available here
2) Azure data bricks for some insights
Provide the following:
a) A screen capture of the completed model diagram and any decision you made in training the
model. For example, rationale for some of the components used, how many records have been
used for training and how many for testing.
b) A set of metrics which presents how effective your model is.
c) Which features were most influential in driving your model?
d) Using your model predict the total trip duration for trips given below.
A report on the four activities.
• The report should be compiled in either PDF or Microsoft Word only, font size 11.
• The report should not exceed 15 pages. Diagrams, tables and any other visualisations/ screen
captures should be in the main body of the report.
• Make realistic assumptions on missing information and state these in the report.
a) Follow the same approach as the Big Data Analytics Workshop (using the wordcount method in HDInsight) to determine the contributory words for each level of rating.
b) Present the workflow of using HDInsight (you may use screen captures) along with a summary of findings and any insights for each level of rating. MapReduce documentation for HDInsight is available here
2) Azure data bricks for some insights
Provide the following:
a) A screen capture of the completed model diagram and any decision you made in training the
model. For example, rationale for some of the components used, how many records have been
used for training and how many for testing.
b) A set of metrics which presents how effective your model is.
c) Which features were most influential in driving your model?
d) Using your model predict the total trip duration for trips given below.
A report on the four activities.
• The report should be compiled in either PDF or Microsoft Word only, font size 11.
• The report should not exceed 15 pages. Diagrams, tables and any other visualisations/ screen
captures should be in the main body of the report.
• Make realistic assumptions on missing information and state these in the report.