Construction of a classification model to predict the probability of default -- 2
Budget: $30 – $250 USD
Goal: Construction of a classification model to predict the probability of default for a company based on various financial ratios.
Data: Historical data on corporate financial ratios and default events for public companies over the 1995-2004 period.
Due date: May 1st, 2022
1. Create a folder. You’ll use 1 external data file as input – containing historical financial ratios and default events for public companies.
2. Import the input dataset, use the as.factor() function to convert the variable Default into a factor (this will tell R to understand its values as categorical, not as continuous). The variable Default indicates the default status of the company in a particular year – it equals 1 if the company defaults on its debt, and 0 otherwise. The variable ID serves as a company identifier, since most companies are observed over multiple years, and the variable Year designates the year of observation.
3. There are 5 financial ratios: WC2TA (working capital-to-total assets), RE2TA (retained earnings-to-total assets), EBIT2TA (earnings before interest and taxes-to-total assets), ME2TA (market value of equity-to-total assets), and S2TA (sales-to-total assets). Compute summary statistics for the data:
4. Compute and report the overall default rate for the sample (number of default events divided by the number of observed companies).
5. Use boxplots to visualize the distribution of each one of the 5 financial ratios by default status. Here is an example with 3 of the ratios:
6. Based on the plots, can you hypothesize the sign of the coefficient that each one of the financial ratios would have in a classification regression, where the default status is the dependent variable and the specific financial ratio is one of the explanatory variables (predictors)?
7. Demonstrate and explain what would happen if we fit a linear regression model (instead of a logistic model), where the default status is the dependent variable and the financial ratios are the explanatory variables (predictors)?
8. Use the function glm()to fit logistic regression models, where the default status indicator is the dependent variable and the financial ratios are the explanatory variables. Try all 16 combinations of explanatory variables, where you have 3 or more predictors in a model.
9. Use a cut-off of 50% probability to assign a model-predicted class (e.g. default or non-default) and create a table distributing the in-sample observations into correctly classified defaults and non-defaults, and incorrectly classified defaults and non-defaults. Hint: you can use the function table() (the ‘Logistic regression demo’ file on Blackboard shows how it is used), or you can use a conventional R counting. Table 4.6 on p.148 in the ISLR textbook provides an example.
10. Based on the table in item 9. compute what percentage of all observations are correctly classified by the model? What is the Type I error rate? What is the power (sensitivity) of the classification model? Hint: Table 4.7 on p.149 in the ISLR textbook has explanation how these metrics are computed.
11. Repeat steps 9. and 10. with cut-offs of 10% and 90% and report the table, percentage accuracy, Type I error rate, and power (sensitivity) of the model. How does the power (sensitivity) of the model change with the change of the cut-off? Why do you think it changes this way?
Data: Historical data on corporate financial ratios and default events for public companies over the 1995-2004 period.
Due date: May 1st, 2022
1. Create a folder. You’ll use 1 external data file as input – containing historical financial ratios and default events for public companies.
2. Import the input dataset, use the as.factor() function to convert the variable Default into a factor (this will tell R to understand its values as categorical, not as continuous). The variable Default indicates the default status of the company in a particular year – it equals 1 if the company defaults on its debt, and 0 otherwise. The variable ID serves as a company identifier, since most companies are observed over multiple years, and the variable Year designates the year of observation.
3. There are 5 financial ratios: WC2TA (working capital-to-total assets), RE2TA (retained earnings-to-total assets), EBIT2TA (earnings before interest and taxes-to-total assets), ME2TA (market value of equity-to-total assets), and S2TA (sales-to-total assets). Compute summary statistics for the data:
4. Compute and report the overall default rate for the sample (number of default events divided by the number of observed companies).
5. Use boxplots to visualize the distribution of each one of the 5 financial ratios by default status. Here is an example with 3 of the ratios:
6. Based on the plots, can you hypothesize the sign of the coefficient that each one of the financial ratios would have in a classification regression, where the default status is the dependent variable and the specific financial ratio is one of the explanatory variables (predictors)?
7. Demonstrate and explain what would happen if we fit a linear regression model (instead of a logistic model), where the default status is the dependent variable and the financial ratios are the explanatory variables (predictors)?
8. Use the function glm()to fit logistic regression models, where the default status indicator is the dependent variable and the financial ratios are the explanatory variables. Try all 16 combinations of explanatory variables, where you have 3 or more predictors in a model.
9. Use a cut-off of 50% probability to assign a model-predicted class (e.g. default or non-default) and create a table distributing the in-sample observations into correctly classified defaults and non-defaults, and incorrectly classified defaults and non-defaults. Hint: you can use the function table() (the ‘Logistic regression demo’ file on Blackboard shows how it is used), or you can use a conventional R counting. Table 4.6 on p.148 in the ISLR textbook provides an example.
10. Based on the table in item 9. compute what percentage of all observations are correctly classified by the model? What is the Type I error rate? What is the power (sensitivity) of the classification model? Hint: Table 4.7 on p.149 in the ISLR textbook has explanation how these metrics are computed.
11. Repeat steps 9. and 10. with cut-offs of 10% and 90% and report the table, percentage accuracy, Type I error rate, and power (sensitivity) of the model. How does the power (sensitivity) of the model change with the change of the cut-off? Why do you think it changes this way?