Machine Learning -
Budget: $15 – $25 USD
2 Data visualization and pre-processing
To understand and pre-process dataset features correctly before building the model you need to implement preprocess.py script that contains the following methods:
a) def visualize (path_to_data):
• Load the training set into a Dataframe using pandas library. Display the head records from the dataset. What types of input data do you have (numerical or categorical values)? Which data columns do you think are not necessary when training the model? Select the features that you will use when training your model. Apply the same modification to your test data
• Display the basic statistics for each feature. Count, mean, std, min, max etc.
• Display label distribution. Do we have class imbalance?
• Visualize the correlation between each feature and the label.
• Visualize the distribution for all continuous value features using histograms.
b) def split(data): train_X, test_X, train_y, test_Y
• Split the dataset into training and testing sets following 80/20 partition.
c) def preprocess_data(train_X,test_X,train_y,test_y):pr_train_X, pr_test_x,pr_train_y, p_test_y
• Preprocess categorical data using one-hot representation and drop original column. Use pandas library or sklearn.
• Preprocess continuous data by scaling their values to be in the range [0,1].
• Preprocess the label as one-hot representation.
3 Model
You need to build two different machine learning models using two different approaches :
1. Multi-Layer Perceptron (MLP).
2. A model of your choice.
Implement model.py to train and evaluate your first model (MLP) and the second model of your choice using the following methods:
a) def train_mlp( train_x, train_y, input_dim): best_model
Construct and train MLP with three layers (including input and output layers) using an appropriate activation function. Perform 10-fold cross validation to select the number of units in the middle hidden layer. Your code should select the best number of units that produce the highest accuracy form [16, 32 or 64] possible hidden unites sizes. The method should return the best model that will be used for prediction. Plot the loss of your model against each epochs. Hint: you can use keras or sklearn library.
b) deftrain(train_x,train_y):best_model
This method should build and train your selected model to capture data features. Perform 10-fold cross validation to select the best learning rate from the set [1e-2, 1e-3, 1e-4]. The method should return the best model that will be used for prediction. Plot the loss of your model against each epochs. Hint: you can use keras or sklearn library.
c) def predict_evalute (test_X,test_Y model):
• Predict the target for the test data.
• Evaluate the model on test data by calculating accuracy, F1 score and confusion
matrix.
I WANT EXPLNATION AND with zoom meeting before solving this
as soon as possible
To understand and pre-process dataset features correctly before building the model you need to implement preprocess.py script that contains the following methods:
a) def visualize (path_to_data):
• Load the training set into a Dataframe using pandas library. Display the head records from the dataset. What types of input data do you have (numerical or categorical values)? Which data columns do you think are not necessary when training the model? Select the features that you will use when training your model. Apply the same modification to your test data
• Display the basic statistics for each feature. Count, mean, std, min, max etc.
• Display label distribution. Do we have class imbalance?
• Visualize the correlation between each feature and the label.
• Visualize the distribution for all continuous value features using histograms.
b) def split(data): train_X, test_X, train_y, test_Y
• Split the dataset into training and testing sets following 80/20 partition.
c) def preprocess_data(train_X,test_X,train_y,test_y):pr_train_X, pr_test_x,pr_train_y, p_test_y
• Preprocess categorical data using one-hot representation and drop original column. Use pandas library or sklearn.
• Preprocess continuous data by scaling their values to be in the range [0,1].
• Preprocess the label as one-hot representation.
3 Model
You need to build two different machine learning models using two different approaches :
1. Multi-Layer Perceptron (MLP).
2. A model of your choice.
Implement model.py to train and evaluate your first model (MLP) and the second model of your choice using the following methods:
a) def train_mlp( train_x, train_y, input_dim): best_model
Construct and train MLP with three layers (including input and output layers) using an appropriate activation function. Perform 10-fold cross validation to select the number of units in the middle hidden layer. Your code should select the best number of units that produce the highest accuracy form [16, 32 or 64] possible hidden unites sizes. The method should return the best model that will be used for prediction. Plot the loss of your model against each epochs. Hint: you can use keras or sklearn library.
b) deftrain(train_x,train_y):best_model
This method should build and train your selected model to capture data features. Perform 10-fold cross validation to select the best learning rate from the set [1e-2, 1e-3, 1e-4]. The method should return the best model that will be used for prediction. Plot the loss of your model against each epochs. Hint: you can use keras or sklearn library.
c) def predict_evalute (test_X,test_Y model):
• Predict the target for the test data.
• Evaluate the model on test data by calculating accuracy, F1 score and confusion
matrix.
I WANT EXPLNATION AND with zoom meeting before solving this
as soon as possible