Matrix Recovery / Optimization in R Programming
Budget: $10 – $30 USD
Matrix Recovery / Optimization in R Programming
Details:
The dataset “ratings.csv” contains ratings, from 1 to 5, for 300 different restaurants. In this question, you will be performing experiments on this data set to determine how missingness affects the performance of matrix completion.
Part 1 – Missing Completely at Random
Randomly select values in the ratings matrix to label as missing. Do this by generating random variates from a binomial distribution where the probability of “success” is set to 5%. At the selected locations set the entries to a value of zero. Then solve the following optimization problem: image.png
Make sure to calculate the relative reconstruction error. Repeat the optimization problem by increasing the percentage of missing data by 5% up to 50%. Plot the reconstruction error vs the percentage of missing data. Additionally, for the case where 50% of the data is missing, report the frequency of each value in the matrix with missing values, the recovered matrix, and the original full dataset.
Part 2 – Missing Not at Random
To simulate a case where customers avoid going to poorly rated restaurants, generate missing entries where ratings of value 1 or 2 are three times more likely to be missing than ratings of 3 or 4. Ratings of 5 will have a missing rate of 0%. To do so, use random variates from a binomial distribution, where the probability of “success” is adjusted for each rating value such that the overall missing rate is 5%.
Solve the same optimization problem as in Part 1 and repeat the experiment by increasing the percentage of missing data by 5% up to 50%. For the last case, report the frequency of each value in the matrix with missing values and the recovered matrix.
Finally, generate a plot of reconstruction error vs percentage of missing data. Compare with the results in Part 1.
Details:
The dataset “ratings.csv” contains ratings, from 1 to 5, for 300 different restaurants. In this question, you will be performing experiments on this data set to determine how missingness affects the performance of matrix completion.
Part 1 – Missing Completely at Random
Randomly select values in the ratings matrix to label as missing. Do this by generating random variates from a binomial distribution where the probability of “success” is set to 5%. At the selected locations set the entries to a value of zero. Then solve the following optimization problem: image.png
Make sure to calculate the relative reconstruction error. Repeat the optimization problem by increasing the percentage of missing data by 5% up to 50%. Plot the reconstruction error vs the percentage of missing data. Additionally, for the case where 50% of the data is missing, report the frequency of each value in the matrix with missing values, the recovered matrix, and the original full dataset.
Part 2 – Missing Not at Random
To simulate a case where customers avoid going to poorly rated restaurants, generate missing entries where ratings of value 1 or 2 are three times more likely to be missing than ratings of 3 or 4. Ratings of 5 will have a missing rate of 0%. To do so, use random variates from a binomial distribution, where the probability of “success” is adjusted for each rating value such that the overall missing rate is 5%.
Solve the same optimization problem as in Part 1 and repeat the experiment by increasing the percentage of missing data by 5% up to 50%. For the last case, report the frequency of each value in the matrix with missing values and the recovered matrix.
Finally, generate a plot of reconstruction error vs percentage of missing data. Compare with the results in Part 1.