Regularized Linear Models
Budget: $10 – $30 USD
Once you load data and visualize it you will see below graph (1.jpeg)
Use below functions to fit given data:
1, x, x^2, x^3, x^4, abs(x), sign(x), sin(x), cos(x), exp(x)
So dimension of your mapped data is 10 dimensional. But indeed only 2 of these functions are relevant. You can see graph 2. (2.jpeg)
Use an algorithmic approach and a mathematical approach to find these 2 features (from above 10 features). Compare the features you found using these 2 different approaches (using MSE, and R^2). Also plot the fitting as above in 1D and as in 3D, you can see graph 3 (3.jpeg)
x1 = 1 (simply x1 = x^0 = 1)
x2 = x (simply x1 = x^1 = x)
x3 = x^2
x4 = x^3
x5 = x^4
x6 = abs(x)
x7 = sign(x)
x8 = sin(x)
x9 = cos(x)
x10 = exp(x) (simply e^x)
Here are some hints for you:
Algorithmic approaches for feature selection are given in the first hour of the lecture, namely best subset selection, forward selection, and backward selection. Here, you know 2 features will be used and possible functions are also given. Normally you do not know these information in real-life but this is a homework and thus it is a simplified problem. So, show that you selected the most suitable (performance-wise and training efficiency-wise) algorithmic approach for feature selection given that mapped space will be 10 dimensional and only 2 of them will be used. You should both implement it, show your experiment and results, and also write comments to demonstrate your knowledge and understanding on the algorithmic feature selection approaches.
For mathematical approaches you have Ridge, LASSO, and Elastic-Net. Again you should choose only one of them for feature selection (either by the knowledge you gained in the lecture or trying 3 approaches and choosing the best one). Again we only need to find 2 features among 10 features. You should both implement it using Scikit-learn, show your experiment and results, and also write comments to demonstrate your knowledge and understanding on the mathematical feature selection approaches via shrinkage methods.
Compare algorithmic feature selection and mathematical feature selection approaches based on your implementation and experiments. Write your comments. Do not forget to show your fittings in 1D as nonlinear data (using selected 2 features) and also on plane where it becomes linear on the plane. Show error and performance metrics such as MSE, SSE, and R^2 (suitable ones for the regression since this is a regression problem).
Use below functions to fit given data:
1, x, x^2, x^3, x^4, abs(x), sign(x), sin(x), cos(x), exp(x)
So dimension of your mapped data is 10 dimensional. But indeed only 2 of these functions are relevant. You can see graph 2. (2.jpeg)
Use an algorithmic approach and a mathematical approach to find these 2 features (from above 10 features). Compare the features you found using these 2 different approaches (using MSE, and R^2). Also plot the fitting as above in 1D and as in 3D, you can see graph 3 (3.jpeg)
x1 = 1 (simply x1 = x^0 = 1)
x2 = x (simply x1 = x^1 = x)
x3 = x^2
x4 = x^3
x5 = x^4
x6 = abs(x)
x7 = sign(x)
x8 = sin(x)
x9 = cos(x)
x10 = exp(x) (simply e^x)
Here are some hints for you:
Algorithmic approaches for feature selection are given in the first hour of the lecture, namely best subset selection, forward selection, and backward selection. Here, you know 2 features will be used and possible functions are also given. Normally you do not know these information in real-life but this is a homework and thus it is a simplified problem. So, show that you selected the most suitable (performance-wise and training efficiency-wise) algorithmic approach for feature selection given that mapped space will be 10 dimensional and only 2 of them will be used. You should both implement it, show your experiment and results, and also write comments to demonstrate your knowledge and understanding on the algorithmic feature selection approaches.
For mathematical approaches you have Ridge, LASSO, and Elastic-Net. Again you should choose only one of them for feature selection (either by the knowledge you gained in the lecture or trying 3 approaches and choosing the best one). Again we only need to find 2 features among 10 features. You should both implement it using Scikit-learn, show your experiment and results, and also write comments to demonstrate your knowledge and understanding on the mathematical feature selection approaches via shrinkage methods.
Compare algorithmic feature selection and mathematical feature selection approaches based on your implementation and experiments. Write your comments. Do not forget to show your fittings in 1D as nonlinear data (using selected 2 features) and also on plane where it becomes linear on the plane. Show error and performance metrics such as MSE, SSE, and R^2 (suitable ones for the regression since this is a regression problem).