Hadoop MapReduce with Java for the given JSON dataset Parsing, Cleaning and put the output to HDFS
Budget: $10 – $30 USD
Parsing, Cleaning, and Profiling of the attached file auspol2019.json by removing hashtags, emoticons, or any redundant data which is not useful for analysis. And MapReduce output will be on HDFS like the image attached named "Output" but should be clean.
Tasks:
Dataset: auspol2019.json
Programming: MapReduce with Java
Data profiling: Write MapReduce java code to characterize (profile) the data in each column.
Data cleaning: Cleaning and Profiling the tweets by removing hashtags, emoticons, or any redundant data which is not useful for analysis. Write MapReduce java code to ETL (extract, transform, load) data source. Drop some unimportant columns, Normalize data in a column, and Detect badly formatted rows.
Tasks:
Dataset: auspol2019.json
Programming: MapReduce with Java
Data profiling: Write MapReduce java code to characterize (profile) the data in each column.
Data cleaning: Cleaning and Profiling the tweets by removing hashtags, emoticons, or any redundant data which is not useful for analysis. Write MapReduce java code to ETL (extract, transform, load) data source. Drop some unimportant columns, Normalize data in a column, and Detect badly formatted rows.
Related categories:
Business, Accounting, Human Resources & Legal
Software Architecture
Hadoop
Map Reduce
JSON