Big Data Task

Using Spark , Hadoop and Bash to manage data and solve different tasks \
1. data-collection.ipynb (Tasks from Section 2)
2. conceptual-design.pdf (Tasks from Section 3.1)
3. data-management.ipynb (Tasks from Section 3.2)
4. data-processing-hadoop.ipynb (tasks from Section 4.1) 5. data-processing-spark.ipynb (tasks from Section 4.2)
• Data Collection (Bash)
• Data Managenent (MySQL/MongoDB)
• Data Processing using Hadoop
• Data Processing using Spark