Executive Solution Overview
– ## COdata3.docx – credit_Dataset.arff.txt **## COdata3.docx** **DECISION TREES for Risk Assessment** One of the great advantages of decision trees is their _interpretability_. The rules learnt for classification are easy for a person to follow, unlike the opaque "black box" of many other methods,…
Instant Solution Download
$6.99Immediate access to the full model answer, formatted citations, and rubric breakdown.
Need a Fresh Custom Paper?
Have this question answered from scratch according to your specific instructions and deadline.
Order on TutorsGallery →Assignment Prompt & Instructions
– ## COdata3.docx
– credit_Dataset.arff.txt
**## COdata3.docx**
**DECISION TREES for Risk Assessment**
One of the great advantages of decision trees is their
_interpretability_. The rules learnt for classification are easy for a person to follow, unlike the opaque "black box" of many other methods, such as neural networks. We demonstrate the utility of this using a
[German credit](http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29) data set. You can read
[a description](http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29) of this dataset at the UCI site. The task is to predict whether a loan approval is good or bad credit risk based on 20 attributes. We've simplified the data set somewhat, particularly making attribute names and values more meaningful.
1. Download the credit_Dataset.arff dataset and load it to Weka.
2. (5 Points) When presented with a dataset, it is usually a good idea to visualise it first. Go to the
_Visualise_ tab. Click on any of the scatter plots to open a new window which shows the scatter plot for two selected attributes. Try visualising a scatter plot of
_age_ and
_duration_. Do you notice anything unusual? You can click on any data point to display all it's values.
3. (5 Points) In the previous point you should have found a data point, which seems to be corrupted, as some of its values are nonsensical. Even a single point like this can significantly affect the performance of a classifier. How do you think it would affect Decision trees? A good way to check this is to test the performance of each classifier before and after removing this datapoint.
4. (10 Points) To remove this instance from the dataset we will use a filter. We want to remove all instances, where the age of an applicant is lower than 0 years, as this suggests that the instance is corrupted. In the
_Preprocess_ tab click on
_Choose_ in the Filter pane. Select
_filters > unsupervised > instance > RemoveWithValues_. Click on the text of this filter to change the parameters. Set the attribute index to 13 (Age) and set the split point at 0. Click
_Ok_ to set the parameters and
_Apply_ to apply the filter to the data. Visualise the data again to verify that the invalid data point was removed.
5. (20 Points) On the
_Classify_ tab, select the
_Percentage split_ test option and change its value to 90%. This way, we will train the classifiers using 90% of the training data and evaluate their performance on the remaining 10%. First, train a decision tree classifier with default options. Select
_classifiers > trees > J48_ and click
_Start_.
_J48_ is the Weka implementation of the
_C4.5_ algorithm, which uses the normalized information gain criterion to build a decision tree for classification.
6. (20 Points) After training the classifier, the full decision tree is output for your perusal; you may need to scroll up for this. The tree may also be viewed in graphical form by right-clicking in the
_Result list_ and selecting
_Visualize tree_; unfortunately this format is very cluttered for large trees. Such a tree accentuates one of the strengths of decision tree algorithms: they produce classifiers which are understandable to humans. This can be an important asset in real life applications (people are seldom prepared to do what a computer program tells them if there is no clear explanation). Observe the output of the classifier and try to answer the following questions:
· How would you assess the performance of the classifier? Is the Percentage of
_Correctly Classified Instances_ a sufficient measure in this case? Why?
_Hint:_ check the number of good and bad cases in the test sample, using the confusion matrix. Each column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class. For example let us define an experiment from P positive instances and N negative instances. The four outcomes can be formulated in a 2 by 2 contingency table or confusion matrix. One benefit of a confusion matrix is that it is easy to see if the system is confusing two classes (i.e. commonly mislabeling one as another).
· Looking at the decision tree itself, are the rules it applies sensible? Are there any branches which appear absurd? At what depth of the tree? What does this suggest?
_Hint:_ Check the rules applied after following the paths: (a)
_CheckingAccount = <0, Foreign = yes, Duration >11, Job = skilled, OtherDebtors = none, Duration <= 30_ and (b)
_CheckingAccount = <0, Foreign = yes, Duration >11, Job = unskilled._
· How does the decision tree deal with classification in the case where there are zero instances in the training set corresponding to that particular path in the tree (e.g. those leaf nodes that have (0:0))?
7. (20 Points) Now, explore the effect of the
_confidenceFactor_ option. You can find this by clicking on the Classifer name (to the right of the
_Choose_ button on the Classify tab). On the
_Classifier options_ window, click on the
_More_ button to find out what the confidence factor controls. Try the values 0.1, 0.2, 0.3 and 0.5. What is the performance of the classifier at each case? Did you expect this given your observations in the previous questions? Why do you think this happens?
8. (20 Points) Suppose that it is worse to classify a customer as good when they are bad, than it is to classify a customer as bad when they are good. Which value would you pick for the confidence factor? Which performance measure would you base your decision on?
9. (Bonus: 20 Points)Finally we will create a
_random decision forest_ and compare the performance of this classifier to that of the decision tree and the decision stump. The random decision forest is an ensemble classifier that consists of many decision trees and outputs the class that is the mode of the class's output by individual trees. Again set the test option
_Percentage split_ to 90%. Select
_classifiers > trees > RandomForest_ and hit
_Start_. Again, observe the output. How high can you get the performance of the classifier by changing the number of trees (numTrees) parameter? How does the random decision forest compare performance wise to the decision tree and decision stump?
Deliverable:
· Your report including the screenshots of your implementation for each section and the results.
**credit_Dataset.arff.txt**
This file is too large to display. [View in new window](https://www.sweetstudy.com/files/credit-dataset-arff-txt)
**credit_Dataset.arff.txt**
This file is too large to display. [View in new window](https://www.sweetstudy.com/files/credit-dataset-arff-txt)
**## COdata3.docx**
**DECISION TREES for Risk Assessment**
One of the great advantages of decision trees is their
_interpretability_. The rules learnt for classification are easy for a person to follow, unlike the opaque "black box" of many other methods, such as neural networks. We demonstrate the utility of this using a
[German credit](http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29) data set. You can read
[a description](http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29) of this dataset at the UCI site. The task is to predict whether a loan approval is good or bad credit risk based on 20 attributes. We've simplified the data set somewhat, particularly making attribute names and values more meaningful.
1. Download the credit_Dataset.arff dataset and load it to Weka.
2. (5 Points) When presented with a dataset, it is usually a good idea to visualise it first. Go to the
_Visualise_ tab. Click on any of the scatter plots to open a new window which shows the scatter plot for two selected attributes. Try visualising a scatter plot of
_age_ and
_duration_. Do you notice anything unusual? You can click on any data point to display all it's values.
3. (5 Points) In the previous point you should have found a data point, which seems to be corrupted, as some of its values are nonsensical. Even a single point like this can significantly affect the performance of a classifier. How do you think it would affect Decision trees? A good way to check this is to test the performance of each classifier before and after removing this datapoint.
4. (10 Points) To remove this instance from the dataset we will use a filter. We want to remove all instances, where the age of an applicant is lower than 0 years, as this suggests that the instance is corrupted. In the
_Preprocess_ tab click on
_Choose_ in the Filter pane. Select
_filters > unsupervised > instance > RemoveWithValues_. Click on the text of this filter to change the parameters. Set the attribute index to 13 (Age) and set the split point at 0. Click
_Ok_ to set the parameters and
_Apply_ to apply the filter to the data. Visualise the data again to verify that the invalid data point was removed.
5. (20 Points) On the
_Classify_ tab, select the
_Percentage split_ test option and change its value to 90%. This way, we will train the classifiers using 90% of the training data and evaluate their performance on the remaining 10%. First, train a decision tree classifier with default options. Select
_classifiers > trees > J48_ and click
_Start_.
_J48_ is the Weka implementation of the
_C4.5_ algorithm, which uses the normalized information gain criterion to build a decision tree for classification.
6. (20 Points) After training the classifier, the full decision tree is output for your perusal; you may need to scroll up for this. The tree may also be viewed in graphical form by right-clicking in the
_Result list_ and selecting
_Visualize tree_; unfortunately this format is very cluttered for large trees. Such a tree accentuates one of the strengths of decision tree algorithms: they produce classifiers which are understandable to humans. This can be an important asset in real life applications (people are seldom prepared to do what a computer program tells them if there is no clear explanation). Observe the output of the classifier and try to answer the following questions:
· How would you assess the performance of the classifier? Is the Percentage of
_Correctly Classified Instances_ a sufficient measure in this case? Why?
_Hint:_ check the number of good and bad cases in the test sample, using the confusion matrix. Each column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class. For example let us define an experiment from P positive instances and N negative instances. The four outcomes can be formulated in a 2 by 2 contingency table or confusion matrix. One benefit of a confusion matrix is that it is easy to see if the system is confusing two classes (i.e. commonly mislabeling one as another).
· Looking at the decision tree itself, are the rules it applies sensible? Are there any branches which appear absurd? At what depth of the tree? What does this suggest?
_Hint:_ Check the rules applied after following the paths: (a)
_CheckingAccount = <0, Foreign = yes, Duration >11, Job = skilled, OtherDebtors = none, Duration <= 30_ and (b)
_CheckingAccount = <0, Foreign = yes, Duration >11, Job = unskilled._
· How does the decision tree deal with classification in the case where there are zero instances in the training set corresponding to that particular path in the tree (e.g. those leaf nodes that have (0:0))?
7. (20 Points) Now, explore the effect of the
_confidenceFactor_ option. You can find this by clicking on the Classifer name (to the right of the
_Choose_ button on the Classify tab). On the
_Classifier options_ window, click on the
_More_ button to find out what the confidence factor controls. Try the values 0.1, 0.2, 0.3 and 0.5. What is the performance of the classifier at each case? Did you expect this given your observations in the previous questions? Why do you think this happens?
8. (20 Points) Suppose that it is worse to classify a customer as good when they are bad, than it is to classify a customer as bad when they are good. Which value would you pick for the confidence factor? Which performance measure would you base your decision on?
9. (Bonus: 20 Points)Finally we will create a
_random decision forest_ and compare the performance of this classifier to that of the decision tree and the decision stump. The random decision forest is an ensemble classifier that consists of many decision trees and outputs the class that is the mode of the class's output by individual trees. Again set the test option
_Percentage split_ to 90%. Select
_classifiers > trees > RandomForest_ and hit
_Start_. Again, observe the output. How high can you get the performance of the classifier by changing the number of trees (numTrees) parameter? How does the random decision forest compare performance wise to the decision tree and decision stump?
Deliverable:
· Your report including the screenshots of your implementation for each section and the results.
**credit_Dataset.arff.txt**
This file is too large to display. [View in new window](https://www.sweetstudy.com/files/credit-dataset-arff-txt)
**## COdata3.docx**
**DECISION TREES for Risk Assessment**
One of the great advantages of decision trees is their
_interpretability_. The rules learnt for classification are easy for a person to follow, unlike the opaque "black box" of many other methods, such as neural networks. We demonstrate the utility of this using a
[German credit](http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29) data set. You can read
[a description](http://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29) of this dataset at the UCI site. The task is to predict whether a loan approval is good or bad credit risk based on 20 attributes. We've simplified the data set somewhat, particularly making attribute names and values more meaningful.
1. Download the credit_Dataset.arff dataset and load it to Weka.
2. (5 Points) When presented with a dataset, it is usually a good idea to visualise it first. Go to the
_Visualise_ tab. Click on any of the scatter plots to open a new window which shows the scatter plot for two selected attributes. Try visualising a scatter plot of
_age_ and
_duration_. Do you notice anything unusual? You can click on any data point to display all it's values.
3. (5 Points) In the previous point you should have found a data point, which seems to be corrupted, as some of its values are nonsensical. Even a single point like this can significantly affect the performance of a classifier. How do you think it would affect Decision trees? A good way to check this is to test the performance of each classifier before and after removing this datapoint.
4. (10 Points) To remove this instance from the dataset we will use a filter. We want to remove all instances, where the age of an applicant is lower than 0 years, as this suggests that the instance is corrupted. In the
_Preprocess_ tab click on
_Choose_ in the Filter pane. Select
_filters > unsupervised > instance > RemoveWithValues_. Click on the text of this filter to change the parameters. Set the attribute index to 13 (Age) and set the split point at 0. Click
_Ok_ to set the parameters and
_Apply_ to apply the filter to the data. Visualise the data again to verify that the invalid data point was removed.
5. (20 Points) On the
_Classify_ tab, select the
_Percentage split_ test option and change its value to 90%. This way, we will train the classifiers using 90% of the training data and evaluate their performance on the remaining 10%. First, train a decision tree classifier with default options. Select
_classifiers > trees > J48_ and click
_Start_.
_J48_ is the Weka implementation of the
_C4.5_ algorithm, which uses the normalized information gain criterion to build a decision tree for classification.
6. (20 Points) After training the classifier, the full decision tree is output for your perusal; you may need to scroll up for this. The tree may also be viewed in graphical form by right-clicking in the
_Result list_ and selecting
_Visualize tree_; unfortunately this format is very cluttered for large trees. Such a tree accentuates one of the strengths of decision tree algorithms: they produce classifiers which are understandable to humans. This can be an important asset in real life applications (people are seldom prepared to do what a computer program tells them if there is no clear explanation). Observe the output of the classifier and try to answer the following questions:
· How would you assess the performance of the classifier? Is the Percentage of
_Correctly Classified Instances_ a sufficient measure in this case? Why?
_Hint:_ check the number of good and bad cases in the test sample, using the confusion matrix. Each column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class. For example let us define an experiment from P positive instances and N negative instances. The four outcomes can be formulated in a 2 by 2 contingency table or confusion matrix. One benefit of a confusion matrix is that it is easy to see if the system is confusing two classes (i.e. commonly mislabeling one as another).
· Looking at the decision tree itself, are the rules it applies sensible? Are there any branches which appear absurd? At what depth of the tree? What does this suggest?
_Hint:_ Check the rules applied after following the paths: (a)
_CheckingAccount = <0, Foreign = yes, Duration >11, Job = skilled, OtherDebtors = none, Duration <= 30_ and (b)
_CheckingAccount = <0, Foreign = yes, Duration >11, Job = unskilled._
· How does the decision tree deal with classification in the case where there are zero instances in the training set corresponding to that particular path in the tree (e.g. those leaf nodes that have (0:0))?
7. (20 Points) Now, explore the effect of the
_confidenceFactor_ option. You can find this by clicking on the Classifer name (to the right of the
_Choose_ button on the Classify tab). On the
_Classifier options_ window, click on the
_More_ button to find out what the confidence factor controls. Try the values 0.1, 0.2, 0.3 and 0.5. What is the performance of the classifier at each case? Did you expect this given your observations in the previous questions? Why do you think this happens?
8. (20 Points) Suppose that it is worse to classify a customer as good when they are bad, than it is to classify a customer as bad when they are good. Which value would you pick for the confidence factor? Which performance measure would you base your decision on?
9. (Bonus: 20 Points)Finally we will create a
_random decision forest_ and compare the performance of this classifier to that of the decision tree and the decision stump. The random decision forest is an ensemble classifier that consists of many decision trees and outputs the class that is the mode of the class's output by individual trees. Again set the test option
_Percentage split_ to 90%. Select
_classifiers > trees > RandomForest_ and hit
_Start_. Again, observe the output. How high can you get the performance of the classifier by changing the number of trees (numTrees) parameter? How does the random decision forest compare performance wise to the decision tree and decision stump?
Deliverable:
· Your report including the screenshots of your implementation for each section and the results.
**credit_Dataset.arff.txt**
This file is too large to display. [View in new window](https://www.sweetstudy.com/files/credit-dataset-arff-txt)
– 1
– 2