data mining using R
Budget: $30 – $250 USD
Data Mining
This project is in two parts, A and B.
Part A Association Rule Mining
In Part A, the purpose is to mine the Zoo dataset to obtain association rules that contain a mix of positive and negative terms. For this purpose, we will build on Tutorial 3 that mined positive rules. Tutorial 3 also laid the foundation for negative rules by using the factor concept to identify items which do not occur in the dataset such as venomous=false.
association rule mining and Tutorial 3, you are required to accomplish the following tasks:
Task 1
Describe, without the use of code how a rule base can be generated which will not exceed a given size specified by a maxrules parameter. A suggested value of maxrules is 20.
Your description can be stated in pseudo code form that is iterative in nature. The process starts off with a high minsup value of 0.9 for frequent itemset generation. At each iteration the minsup threshold is reduced (say in decrements of 0.1) and the rules generated are examined. The rules that survive the quality check process (what measures would you use for quality control? – see class notes) are then compared with the maxrules threshold and the process is repeated as long as the number of quality rules do not exceed the threshold.
Task 2
Implement the process described in Task 1 and produce the R code needed. Note the rules generated should contain a mix of positive and negative terms. In your R code, clearly indicate the parameter values for each threshold that you used.
Task 3
Visualize the rules produced in Task 2 by using appropriate R code.
Part B Clustering
In this part you will use two clustering algorithms, Kmeans and DBscan on the Wholesale customers dataset UCI Machine Learning Repository: Wholesale customers Data Set. Apply the two algorithms, visualize the results, and evaluate the results using the silhouette cluster quality measure.
Task 4
Hand in the R code for each of the two algorithms. Please ensure that all parameter values are clearly indicated in your R code and documentation. For the DBSCAN algorithm use the kth nearest neighbor method discussed in class by implementing the R kNNdistplot() function.
Task 5
Using the visualization and cluster quality measures you used in Task 4 which of the two algorithms would you consider to be better? Explain your answer.
End of project specification
Note:
All code must meet good programming practices such as naming variables, modularity (using functions for repetitive tasks), and adequate comments at key points in the code.
Your code in a Google Colab sheet
A copy of your code in pdf form. This document must contain the name(s) of the persons who have undertaken the assignment. In addition, it must have a brief description of how the workload was distributed amongst the project partners if work was done in group mode.
A separate pdf document that provides written answers (not code) to the questions asked in this project specification.
This project is in two parts, A and B.
Part A Association Rule Mining
In Part A, the purpose is to mine the Zoo dataset to obtain association rules that contain a mix of positive and negative terms. For this purpose, we will build on Tutorial 3 that mined positive rules. Tutorial 3 also laid the foundation for negative rules by using the factor concept to identify items which do not occur in the dataset such as venomous=false.
association rule mining and Tutorial 3, you are required to accomplish the following tasks:
Task 1
Describe, without the use of code how a rule base can be generated which will not exceed a given size specified by a maxrules parameter. A suggested value of maxrules is 20.
Your description can be stated in pseudo code form that is iterative in nature. The process starts off with a high minsup value of 0.9 for frequent itemset generation. At each iteration the minsup threshold is reduced (say in decrements of 0.1) and the rules generated are examined. The rules that survive the quality check process (what measures would you use for quality control? – see class notes) are then compared with the maxrules threshold and the process is repeated as long as the number of quality rules do not exceed the threshold.
Task 2
Implement the process described in Task 1 and produce the R code needed. Note the rules generated should contain a mix of positive and negative terms. In your R code, clearly indicate the parameter values for each threshold that you used.
Task 3
Visualize the rules produced in Task 2 by using appropriate R code.
Part B Clustering
In this part you will use two clustering algorithms, Kmeans and DBscan on the Wholesale customers dataset UCI Machine Learning Repository: Wholesale customers Data Set. Apply the two algorithms, visualize the results, and evaluate the results using the silhouette cluster quality measure.
Task 4
Hand in the R code for each of the two algorithms. Please ensure that all parameter values are clearly indicated in your R code and documentation. For the DBSCAN algorithm use the kth nearest neighbor method discussed in class by implementing the R kNNdistplot() function.
Task 5
Using the visualization and cluster quality measures you used in Task 4 which of the two algorithms would you consider to be better? Explain your answer.
End of project specification
Note:
All code must meet good programming practices such as naming variables, modularity (using functions for repetitive tasks), and adequate comments at key points in the code.
Your code in a Google Colab sheet
A copy of your code in pdf form. This document must contain the name(s) of the persons who have undertaken the assignment. In addition, it must have a brief description of how the workload was distributed amongst the project partners if work was done in group mode.
A separate pdf document that provides written answers (not code) to the questions asked in this project specification.