Group Project
Budget: $50 – $150 CAD
## Requirements
To successfully complete this project, you need to collect and process data, then train and evaluate at least two machine learning models and lastly deploy them to a website. Please see below for further details:
> Python (or a python package) is to be used everywhere it possibly can!
**Data Collection**
Data can be collected (legally) from anywhere. You may use data that you already have; or from sites that allow you to download the data, for example, [UCI Machine Learning Repository](https://archive.ics.uci.edu/) or [Kaggle Datasets](https://www.kaggle.com/datasets); or via web scraping; or via an API. We can restrict ourselves to data that would fit nicely into a spreadsheet. The content and amount of data are not the main consideration, as long as the data has:
- at least 10 variables
- three or more data types
- two or more problems: missing data, inconsistencies, errors, categorical data that needs to be converted to numeric, entries like text that need to be converted into proper features, etc.
- if the data does not have enough problems, you can substitute one problem for feature engineering (creating new features from the original features)
We are not concerned with acquiring *huge* datasets or creating super accurate models, but more with the process of creating a proper pipeline for machine learning and, for any model deployed, having a reliable estimate of its performance. Although, some effort should go into improving an initial model.
How many datasets do you need?
- Groups of 2 need **two** datasets
- Group of 3 needs **three** datasets
**Database**
After the data has been collected and processed, it should be stored in a SQLite database. At a minimum, each dataset should have its own table. Database and table creation and data insertion can be done either with the `sqlite3` package or with `Pandas`. Both SQLite and `sqlite3` come with Python.
**Machine Learning Models**
For each dataset you should train, evaluate, and save a machine learning model:
- one model should be for a *classification* problem
- the other model should be for a *regression* problem
- Group of 3: you should have 2 of one type
A *validation* dataset must be used to either select between models, or to choose between hyperparameter values of a single model.
A *test* dataset must be used to evaluate the performance of the final chosen model.
**Website**
The final models should be presented to an end-user through a website. (Deployment need only be to *localhost*). The website must be done using a Python "web framework", e.g., *flask*, *Django*, *streamlit*.
The website should have:
- a *Welcome* page that describes your project
- an *About* page for each dataset that provides:
- the source of the dataset
- definition of each variable in the dataset
- a view of a sample of the dataset used for training (pulled from the database)
- a page for each machine learning model that:
- identifies the model being used, with a brief description
- allows the end-user to enter their own data to get a prediction
**Readme.md**
This file should present the reader with a basic description of your project and how they can use it.
**Requirements.txt**
This file contains all packages necessary to run your code. This file should allow the user to install all necessary packages via the command: `pip install -r requirements.txt`
## Structure
- All project related code in a single Github repository
- All code in the repository is only FINAL code
- The repository structure is
- main folder
- data collection
- data processing
- database
- models
- model 1
- model 2
- model 3 (if required)
- website
- Readme.md
- requirements.txt
- Each subfolder should be logically organized
github repository
To successfully complete this project, you need to collect and process data, then train and evaluate at least two machine learning models and lastly deploy them to a website. Please see below for further details:
> Python (or a python package) is to be used everywhere it possibly can!
**Data Collection**
Data can be collected (legally) from anywhere. You may use data that you already have; or from sites that allow you to download the data, for example, [UCI Machine Learning Repository](https://archive.ics.uci.edu/) or [Kaggle Datasets](https://www.kaggle.com/datasets); or via web scraping; or via an API. We can restrict ourselves to data that would fit nicely into a spreadsheet. The content and amount of data are not the main consideration, as long as the data has:
- at least 10 variables
- three or more data types
- two or more problems: missing data, inconsistencies, errors, categorical data that needs to be converted to numeric, entries like text that need to be converted into proper features, etc.
- if the data does not have enough problems, you can substitute one problem for feature engineering (creating new features from the original features)
We are not concerned with acquiring *huge* datasets or creating super accurate models, but more with the process of creating a proper pipeline for machine learning and, for any model deployed, having a reliable estimate of its performance. Although, some effort should go into improving an initial model.
How many datasets do you need?
- Groups of 2 need **two** datasets
- Group of 3 needs **three** datasets
**Database**
After the data has been collected and processed, it should be stored in a SQLite database. At a minimum, each dataset should have its own table. Database and table creation and data insertion can be done either with the `sqlite3` package or with `Pandas`. Both SQLite and `sqlite3` come with Python.
**Machine Learning Models**
For each dataset you should train, evaluate, and save a machine learning model:
- one model should be for a *classification* problem
- the other model should be for a *regression* problem
- Group of 3: you should have 2 of one type
A *validation* dataset must be used to either select between models, or to choose between hyperparameter values of a single model.
A *test* dataset must be used to evaluate the performance of the final chosen model.
**Website**
The final models should be presented to an end-user through a website. (Deployment need only be to *localhost*). The website must be done using a Python "web framework", e.g., *flask*, *Django*, *streamlit*.
The website should have:
- a *Welcome* page that describes your project
- an *About* page for each dataset that provides:
- the source of the dataset
- definition of each variable in the dataset
- a view of a sample of the dataset used for training (pulled from the database)
- a page for each machine learning model that:
- identifies the model being used, with a brief description
- allows the end-user to enter their own data to get a prediction
**Readme.md**
This file should present the reader with a basic description of your project and how they can use it.
**Requirements.txt**
This file contains all packages necessary to run your code. This file should allow the user to install all necessary packages via the command: `pip install -r requirements.txt`
## Structure
- All project related code in a single Github repository
- All code in the repository is only FINAL code
- The repository structure is
- main folder
- data collection
- data processing
- database
- models
- model 1
- model 2
- model 3 (if required)
- website
- Readme.md
- requirements.txt
- Each subfolder should be logically organized
github repository