MAchine learning projects
Budget: $40 – $75 USD
You are a data scientist in healthcare. Your manager gives you two Coronavirus tweets datasets. The training dataset contains in the file: Corona_NLP_train.csv. The test dataset contains in the file: Corona_NLP_test.csv. Both have the following columns:
UserName
ScreenName
Location
TweetAt
OriginalTweet
Sentiment
We are given 2 datasets, training and testing and we need to answer questions related to them.
Visualize the top 10 positive scores of the tokens associated with positive tweets using scattertext.
Convert the column of the Tweet_texts to a matrix of token counts using CountVectorizer and unigrams and bigrams.
Perform the tf-idf analysis on the column of the Tweet_texts using Tfidfvectorizer.
Find the cosine similarity in Tweet_texts between the 200th and 20,000th tweets.
Find the corpus vector equal to the average of all the document vectors, where each document corresponds to a tweet or a row in this dataset.
For steps 15 to 20, we want to forecast Sentiment using Tweet_texts.
Build the first model based on the training dataset using the random forests and pipeline.
Check the first model on the test dataset. Is it a good model based on the selected evaluation metrics? Please justify your answer.
Create the second model using random forest, pipeline, grid search CV for the hyperparameters for the estimators. (Please see all the potential parameters at Scikit Learn's TfidVectorizerLinks to an external site. and random forestsLinks to an external site..)
Tune the second model and perform model diagnostics on the test dataset. Is it a good model? Please justify your answer.
Build the third model using pipeline, grid search CV, hyperparameter for the following classifiers:
Logistic Regression
Support Vector Machine
Random Forest
Tune the third model and perform model diagnostics on the test dataset. Is it a good model? Please justify your answer.
For the Q21 to Q25, please only works on the training dataset
. Generate the first topic model by setting five topics using LDA method and CountVectorizer.
Plot the top 15 words for each topic for the model above.
Generate the second topic model by setting five topics using LDA method and TfidfVectorizer.
. Plot the top 15 words for each topic for the second model above.
Visualize the second topic model using the dimension reduction method. (You may choose the method you like)
UserName
ScreenName
Location
TweetAt
OriginalTweet
Sentiment
We are given 2 datasets, training and testing and we need to answer questions related to them.
Visualize the top 10 positive scores of the tokens associated with positive tweets using scattertext.
Convert the column of the Tweet_texts to a matrix of token counts using CountVectorizer and unigrams and bigrams.
Perform the tf-idf analysis on the column of the Tweet_texts using Tfidfvectorizer.
Find the cosine similarity in Tweet_texts between the 200th and 20,000th tweets.
Find the corpus vector equal to the average of all the document vectors, where each document corresponds to a tweet or a row in this dataset.
For steps 15 to 20, we want to forecast Sentiment using Tweet_texts.
Build the first model based on the training dataset using the random forests and pipeline.
Check the first model on the test dataset. Is it a good model based on the selected evaluation metrics? Please justify your answer.
Create the second model using random forest, pipeline, grid search CV for the hyperparameters for the estimators. (Please see all the potential parameters at Scikit Learn's TfidVectorizerLinks to an external site. and random forestsLinks to an external site..)
Tune the second model and perform model diagnostics on the test dataset. Is it a good model? Please justify your answer.
Build the third model using pipeline, grid search CV, hyperparameter for the following classifiers:
Logistic Regression
Support Vector Machine
Random Forest
Tune the third model and perform model diagnostics on the test dataset. Is it a good model? Please justify your answer.
For the Q21 to Q25, please only works on the training dataset
. Generate the first topic model by setting five topics using LDA method and CountVectorizer.
Plot the top 15 words for each topic for the model above.
Generate the second topic model by setting five topics using LDA method and TfidfVectorizer.
. Plot the top 15 words for each topic for the second model above.
Visualize the second topic model using the dimension reduction method. (You may choose the method you like)