Hadoop MapReduce with Java for the given JSON dataset Parsing, Cleaning and put the output to HDFS

Job ID: 33475229

Budget: $10 – $30 USD

Parsing, Cleaning, and Profiling of the attached file auspol2019.json by removing hashtags, emoticons, or any redundant data which is not useful for analysis. And MapReduce output will be on HDFS like the image attached named "Output" but should be clean.

Tasks:

Dataset: auspol2019.json
Programming: MapReduce with Java

Data profiling: Write MapReduce java code to characterize (profile) the data in each column.
Data cleaning: Cleaning and Profiling the tweets by removing hashtags, emoticons, or any redundant data which is not useful for analysis. Write MapReduce java code to ETL (extract, transform, load) data source. Drop some unimportant columns, Normalize data in a column, and Detect badly formatted rows.