Decision Trees Exercises Answers
This chapter goes over decision trees and their considerations.
Question 1
What is the approximate depth of a Decision Tree trained (without restrictions) on a training set with 1 million instances?
The depth of a well-balanced binary tree containing m leaves is equal to log2(m)3 , rounded up. A binary Decision Tree will end up more or less well balanced at the end of training, with one leaf per training instance if it is trained without restrictions. Thus, if the training set contains one million instances, the Decision Tree will have a depth of log2(106)≈20 .
Question 2
Is a node’s Gini impurity generally lower or greater than its parent’s? Is it generally lower/greater, or always lower/greater?
A node’s Gini impurity is generally lower than its parent’s. This is ensured by the CART training algorithm’s cost function, which splits each node in a way that minimizes the weighted sum of its children’s Gini impurities. However, if one child is smaller than the other, it is possible for it to have a higher Gini impurity than its parent, as long as this increase is more than compensated for by a decrease of the other child’s impurity.
Question 3
If a Decision Tree is overfitting the training set, is it a good idea to try decreasing max_depth?
Yes, it is a good idea to decrease the max_depth hyperparameter. This will constraining how many splits the tree is allowed to make and act as a form of regularization.
Question 4
If a Decision Tree is underfitting the training set, is it a good idea to try scaling the input features?
No, it's not a good idea to try scaling the input features if a decision tree is underfitting the training set because decision trees are not sensitive to data variance and do not require feature scaling.
Question 5
If it takes one hour to train a Decision Tree on a training set containing 1 million instances, roughly how much time will it take to train another Decision Tree on a training set containing 10 million instances?
The training algorithm complexity for Decision Tree is log-linear proportional to the number of training instances. It will take to hours plus log (10)log (m)⋅10hr as long. If m , the number of features, is 10, then it will take 20hrs. If m is 100, then it will take 30 hours.
O(n m log (m))=1hrO(n (10m log (10m)))=O(n (10m log(10)+10m log (m)))=O(10n m log(10)+10n m log (m)))Question 6
If your training set contains 100,000 instances, will setting presort=True speed up training?
The training algorithm for decision trees has a complexity of O(n m log (m)) . For small training sets, Scikit-Learn can speed up the training by presorting the data, but this slows down training on larger training sets. 100,000 instances is considered a large training set, so setting presort=True will slow down training.
Question 7
Train and fine-tune a Decision Tree for the moons dataset.
- Generate a moons dataset using make_moons(n_samples=10000, noise=0.4).
- Split it into a training set and a test set using train_test_split().
- Use grid search with cross-validation (with the help of the GridSearchCV class) to find good hyperparameter values for a DecisionTreeClassifier. Hint: try various values for max_leaf_nodes.
- Train it on the full training set using these hyperparameters, and measure your model’s performance on the test set. You should get roughly 85% to 87% accuracy.
from sklearn.datasets import make_moons
moons = make_moons(n_samples=10000, noise=0.4)from sklearn.model_selection import train_test_split
import numpy as np
np.random.seed(42)
random_state = np.random.randint(100)
X_train, X_test, y_train, y_test = train_test_split(moons[0],moons[1],test_size=0.2,random_state=random_state)import matplotlib.pyplot as plt
fig, ax = plt.subplots(1,1,layout="constrained")
print("X_train Shape:",X_train.shape)
unique_classes = np.unique(y_train)
print("Unique y_train Classes:",unique_classes)
colors = ["r","g"]
markers = ["v","s"]
for num in unique_classes:
class_indices = np.nonzero(y_train==num)
X_train_values = X_train[class_indices]
print(X_train_values.shape)
ax.scatter(X_train_values[:,0],X_train_values[:,1],c=colors[num],marker=markers[num],label=str(num),alpha=0.15)
ax.set_xlabel(r" $ x_1 $ ",fontsize=12)
ax.set_ylabel(r" $ x_2 $ ",fontsize=12)
ax.set_title("Make Moons Dataset")
ax.legend()
plt.show()X_train Shape: (8000, 2)
Unique y_train Classes: [0 1]
(3998, 2)
(4002, 2)
<Figure size 640x480 with 1 Axes>
from sklearn.tree import DecisionTreeClassifier
from sklearn.experimental import enable_halving_search_cv
from sklearn.model_selection import HalvingGridSearchCV, GridSearchCV
from sklearn.metrics import accuracy_score
print("The number of levels that the tree would contained unconstrained is approx {} (See question 1). \n Therefore, if we want to regularize the tree, we should constrain the max_depth to be less than that. (Assuming that the model will overfit the data). Also, from the graph above, I suspect the depth to be greater than or equal to 4.".format(np.floor(np.log2(X_train.shape[0]))))
param_grid = [
{"max_depth": [4, 6, 8, 12] }
]
tree_clf = HalvingGridSearchCV(DecisionTreeClassifier(),param_grid=param_grid,verbose=3,refit=True,cv=5)
tree_clf.fit(X_train,y_train)
y_pred = tree_clf.predict(X_test)
print("Accuracy Score:",accuracy_score(y_pred,y_test))
The number of levels that the tree would contained unconstrained is approx 12.0 (See question 1).
Therefore, if we want to regularize the tree, we should constrain the max_depth to be less than that. (Assuming that the model will overfit the data). Also, from the graph above, I suspect the depth to be greater than or equal to 4.
n_iterations: 2
n_required_iterations: 2
n_possible_iterations: 2
min_resources_: 2666
max_resources_: 8000
aggressive_elimination: False
factor: 3
----------
iter: 0
n_candidates: 4
n_resources: 2666
Fitting 5 folds for each of 4 candidates, totalling 20 fits
[CV 1/5] END ...max_depth=4;, score=(train=0.862, test=0.820) total time= 0.0s
[CV 2/5] END ...max_depth=4;, score=(train=0.854, test=0.871) total time= 0.0s
[CV 3/5] END ...max_depth=4;, score=(train=0.864, test=0.856) total time= 0.0s
[CV 4/5] END ...max_depth=4;, score=(train=0.866, test=0.844) total time= 0.0s
[CV 5/5] END ...max_depth=4;, score=(train=0.853, test=0.848) total time= 0.0s
[CV 1/5] END ...max_depth=6;, score=(train=0.883, test=0.811) total time= 0.0s
[CV 2/5] END ...max_depth=6;, score=(train=0.878, test=0.856) total time= 0.0s
[CV 3/5] END ...max_depth=6;, score=(train=0.880, test=0.846) total time= 0.0s
[CV 4/5] END ...max_depth=6;, score=(train=0.885, test=0.837) total time= 0.0s
[CV 5/5] END ...max_depth=6;, score=(train=0.875, test=0.850) total time= 0.0s
[CV 1/5] END ...max_depth=8;, score=(train=0.909, test=0.805) total time= 0.0s
[CV 2/5] END ...max_depth=8;, score=(train=0.900, test=0.854) total time= 0.0s
[CV 3/5] END ...max_depth=8;, score=(train=0.897, test=0.839) total time= 0.0s
[CV 4/5] END ...max_depth=8;, score=(train=0.909, test=0.831) total time= 0.0s
[CV 5/5] END ...max_depth=8;, score=(train=0.896, test=0.820) total time= 0.0s
[CV 1/5] END ..max_depth=12;, score=(train=0.959, test=0.764) total time= 0.0s
[CV 2/5] END ..max_depth=12;, score=(train=0.940, test=0.833) total time= 0.0s
[CV 3/5] END ..max_depth=12;, score=(train=0.961, test=0.812) total time= 0.0s
[CV 4/5] END ..max_depth=12;, score=(train=0.958, test=0.811) total time= 0.0s
[CV 5/5] END ..max_depth=12;, score=(train=0.944, test=0.803) total time= 0.0s
----------
iter: 1
n_candidates: 2
n_resources: 7998
Fitting 5 folds for each of 2 candidates, totalling 10 fits
[CV 1/5] END ...max_depth=6;, score=(train=0.872, test=0.844) total time= 0.0s
[CV 2/5] END ...max_depth=6;, score=(train=0.869, test=0.861) total time= 0.0s
[CV 3/5] END ...max_depth=6;, score=(train=0.869, test=0.865) total time= 0.0s
[CV 4/5] END ...max_depth=6;, score=(train=0.873, test=0.849) total time= 0.0s
[CV 5/5] END ...max_depth=6;, score=(train=0.872, test=0.859) total time= 0.0s
[CV 1/5] END ...max_depth=4;, score=(train=0.859, test=0.852) total time= 0.0s
[CV 2/5] END ...max_depth=4;, score=(train=0.857, test=0.860) total time= 0.0s
[CV 3/5] END ...max_depth=4;, score=(train=0.857, test=0.860) total time= 0.0s
[CV 4/5] END ...max_depth=4;, score=(train=0.860, test=0.852) total time= 0.0s
[CV 5/5] END ...max_depth=4;, score=(train=0.860, test=0.851) total time= 0.0s
Accuracy Score: 0.8545
param_grid = [
{"max_depth": [5, 6, 7], "criterion": ["gini", "entropy"] }
]
tree_clf = GridSearchCV(DecisionTreeClassifier(),param_grid=param_grid,verbose=3,refit=True,cv=5)
tree_clf.fit(X_train,y_train)
y_pred = tree_clf.predict(X_test)
print("Accuracy Score:",accuracy_score(y_pred,y_test))n_iterations: 2
n_required_iterations: 2
n_possible_iterations: 2
min_resources_: 2666
max_resources_: 8000
aggressive_elimination: False
factor: 3
----------
iter: 0
n_candidates: 6
n_resources: 2666
Fitting 5 folds for each of 6 candidates, totalling 30 fits
[CV 1/5] END criterion=gini, max_depth=5;, score=(train=0.868, test=0.844) total time= 0.0s
[CV 2/5] END criterion=gini, max_depth=5;, score=(train=0.861, test=0.839) total time= 0.0s
[CV 3/5] END criterion=gini, max_depth=5;, score=(train=0.872, test=0.846) total time= 0.0s
[CV 4/5] END criterion=gini, max_depth=5;, score=(train=0.879, test=0.867) total time= 0.0s
[CV 5/5] END criterion=gini, max_depth=5;, score=(train=0.871, test=0.857) total time= 0.0s
[CV 1/5] END criterion=gini, max_depth=6;, score=(train=0.884, test=0.829) total time= 0.0s
[CV 2/5] END criterion=gini, max_depth=6;, score=(train=0.876, test=0.818) total time= 0.0s
[CV 3/5] END criterion=gini, max_depth=6;, score=(train=0.883, test=0.844) total time= 0.0s
[CV 4/5] END criterion=gini, max_depth=6;, score=(train=0.894, test=0.848) total time= 0.0s
[CV 5/5] END criterion=gini, max_depth=6;, score=(train=0.879, test=0.863) total time= 0.0s
[CV 1/5] END criterion=gini, max_depth=7;, score=(train=0.891, test=0.835) total time= 0.0s
[CV 2/5] END criterion=gini, max_depth=7;, score=(train=0.889, test=0.807) total time= 0.0s
[CV 3/5] END criterion=gini, max_depth=7;, score=(train=0.891, test=0.844) total time= 0.0s
[CV 4/5] END criterion=gini, max_depth=7;, score=(train=0.904, test=0.852) total time= 0.0s
[CV 5/5] END criterion=gini, max_depth=7;, score=(train=0.887, test=0.856) total time= 0.0s
[CV 1/5] END criterion=entropy, max_depth=5;, score=(train=0.868, test=0.844) total time= 0.0s
[CV 2/5] END criterion=entropy, max_depth=5;, score=(train=0.860, test=0.835) total time= 0.0s
[CV 3/5] END criterion=entropy, max_depth=5;, score=(train=0.871, test=0.846) total time= 0.0s
[CV 4/5] END criterion=entropy, max_depth=5;, score=(train=0.869, test=0.861) total time= 0.0s
[CV 5/5] END criterion=entropy, max_depth=5;, score=(train=0.866, test=0.863) total time= 0.0s
[CV 1/5] END criterion=entropy, max_depth=6;, score=(train=0.878, test=0.842) total time= 0.0s
[CV 2/5] END criterion=entropy, max_depth=6;, score=(train=0.868, test=0.824) total time= 0.0s
[CV 3/5] END criterion=entropy, max_depth=6;, score=(train=0.878, test=0.848) total time= 0.0s
[CV 4/5] END criterion=entropy, max_depth=6;, score=(train=0.879, test=0.844) total time= 0.0s
[CV 5/5] END criterion=entropy, max_depth=6;, score=(train=0.876, test=0.874) total time= 0.0s
[CV 1/5] END criterion=entropy, max_depth=7;, score=(train=0.887, test=0.846) total time= 0.0s
[CV 2/5] END criterion=entropy, max_depth=7;, score=(train=0.886, test=0.809) total time= 0.0s
[CV 3/5] END criterion=entropy, max_depth=7;, score=(train=0.887, test=0.839) total time= 0.0s
[CV 4/5] END criterion=entropy, max_depth=7;, score=(train=0.894, test=0.842) total time= 0.0s
[CV 5/5] END criterion=entropy, max_depth=7;, score=(train=0.887, test=0.876) total time= 0.0s
----------
iter: 1
n_candidates: 2
n_resources: 7998
Fitting 5 folds for each of 2 candidates, totalling 10 fits
[CV 1/5] END criterion=entropy, max_depth=5;, score=(train=0.860, test=0.852) total time= 0.0s
[CV 2/5] END criterion=entropy, max_depth=5;, score=(train=0.859, test=0.859) total time= 0.0s
[CV 3/5] END criterion=entropy, max_depth=5;, score=(train=0.858, test=0.857) total time= 0.0s
[CV 4/5] END criterion=entropy, max_depth=5;, score=(train=0.861, test=0.850) total time= 0.0s
[CV 5/5] END criterion=entropy, max_depth=5;, score=(train=0.859, test=0.849) total time= 0.0s
[CV 1/5] END criterion=gini, max_depth=5;, score=(train=0.860, test=0.851) total time= 0.0s
[CV 2/5] END criterion=gini, max_depth=5;, score=(train=0.860, test=0.857) total time= 0.0s
[CV 3/5] END criterion=gini, max_depth=5;, score=(train=0.859, test=0.858) total time= 0.0s
[CV 4/5] END criterion=gini, max_depth=5;, score=(train=0.862, test=0.849) total time= 0.0s
[CV 5/5] END criterion=gini, max_depth=5;, score=(train=0.862, test=0.849) total time= 0.0s
Accuracy Score: 0.8565
print(tree_clf.best_params_){'criterion': 'entropy', 'max_depth': 5}
Question 8
Grow a forest.
- Continuing the previous exercise, generate 1,000 subsets of the training set, each containing 100 instances selected randomly. Hint: you can use ScikitLearn’s ShuffleSplit class for this
- Train one Decision Tree on each subset, using the best hyperparameter values found above. Evaluate these 1,000 Decision Trees on the test set. Since they were trained on smaller sets, these Decision Trees will likely perform worse than the first Decision Tree, achieving only about 80% accuracy.
- Now comes the magic. For each test set instance, generate the predictions of the 1,000 Decision Trees, and keep only the most frequent prediction (you can use SciPy’s mode() function for this). This gives you majority-vote predictions over the test set.
- Evaluate these predictions on the test set: you should obtain a slightly higher accuracy than your first model (about 0.5 to 1.5% higher). Congratulations, you have trained a Random Forest classifier!
from sklearn.model_selection import ShuffleSplit
print("""ShuffleSplit
---------------------------------------------------------------
Random Pertutation cross-validator. IIt yields indices to split data into training and test sets.
""")
shuffled_split = ShuffleSplit(n_splits=1000,train_size=0.014,test_size=0.013,random_state=random_state)
split = shuffled_split.split(X_train,y_train)
trees = []
test_set_scores = []
predictions = []
print(X_train.shape)
for i, (train_index, test_index) in enumerate(split):
print("{} Iteration:\n------------------------------------".format(i))
trees.append(DecisionTreeClassifier(criterion="entropy",max_depth=5))
trees[i].fit(X_train[train_index,:],y_train[train_index])
pred = trees[i].predict(X_test)
test_set_scores.append(accuracy_score(y_test,pred))
predictions.append(pred)
print("Accuracy Score:",test_set_scores[i])
ShuffleSplit
---------------------------------------------------------------
Random Pertutation cross-validator. IIt yields indices to split data into training and test sets.
(8000, 2)
0 Iteration:
------------------------------------
Accuracy Score: 0.8365
1 Iteration:
------------------------------------
Accuracy Score: 0.799
2 Iteration:
------------------------------------
Accuracy Score: 0.8275
3 Iteration:
------------------------------------
Accuracy Score: 0.8395
4 Iteration:
------------------------------------
Accuracy Score: 0.818
5 Iteration:
------------------------------------
Accuracy Score: 0.7925
6 Iteration:
------------------------------------
Accuracy Score: 0.803
7 Iteration:
------------------------------------
Accuracy Score: 0.7915
8 Iteration:
------------------------------------
Accuracy Score: 0.79
9 Iteration:
------------------------------------
Accuracy Score: 0.82
10 Iteration:
------------------------------------
Accuracy Score: 0.7645
11 Iteration:
------------------------------------
Accuracy Score: 0.844
12 Iteration:
------------------------------------
Accuracy Score: 0.7955
13 Iteration:
------------------------------------
Accuracy Score: 0.814
14 Iteration:
------------------------------------
Accuracy Score: 0.806
15 Iteration:
------------------------------------
Accuracy Score: 0.8185
16 Iteration:
------------------------------------
Accuracy Score: 0.745
17 Iteration:
------------------------------------
Accuracy Score: 0.848
18 Iteration:
------------------------------------
Accuracy Score: 0.7705
19 Iteration:
------------------------------------
Accuracy Score: 0.8195
20 Iteration:
------------------------------------
Accuracy Score: 0.7845
21 Iteration:
------------------------------------
Accuracy Score: 0.8175
22 Iteration:
------------------------------------
Accuracy Score: 0.7515
23 Iteration:
------------------------------------
Accuracy Score: 0.8475
24 Iteration:
------------------------------------
Accuracy Score: 0.8105
25 Iteration:
------------------------------------
Accuracy Score: 0.797
26 Iteration:
------------------------------------
Accuracy Score: 0.83
27 Iteration:
------------------------------------
Accuracy Score: 0.8195
28 Iteration:
------------------------------------
Accuracy Score: 0.7625
29 Iteration:
------------------------------------
Accuracy Score: 0.7755
30 Iteration:
------------------------------------
Accuracy Score: 0.7885
31 Iteration:
------------------------------------
Accuracy Score: 0.7985
32 Iteration:
------------------------------------
Accuracy Score: 0.797
33 Iteration:
------------------------------------
Accuracy Score: 0.8185
34 Iteration:
------------------------------------
Accuracy Score: 0.8425
35 Iteration:
------------------------------------
Accuracy Score: 0.844
36 Iteration:
------------------------------------
Accuracy Score: 0.8365
37 Iteration:
------------------------------------
Accuracy Score: 0.8025
38 Iteration:
------------------------------------
Accuracy Score: 0.798
39 Iteration:
------------------------------------
Accuracy Score: 0.849
40 Iteration:
------------------------------------
Accuracy Score: 0.8375
41 Iteration:
------------------------------------
Accuracy Score: 0.806
42 Iteration:
------------------------------------
Accuracy Score: 0.788
43 Iteration:
------------------------------------
Accuracy Score: 0.8235
44 Iteration:
------------------------------------
Accuracy Score: 0.751
45 Iteration:
------------------------------------
Accuracy Score: 0.828
46 Iteration:
------------------------------------
Accuracy Score: 0.7925
47 Iteration:
------------------------------------
Accuracy Score: 0.8305
48 Iteration:
------------------------------------
Accuracy Score: 0.8365
49 Iteration:
------------------------------------
Accuracy Score: 0.8005
50 Iteration:
------------------------------------
Accuracy Score: 0.827
51 Iteration:
------------------------------------
Accuracy Score: 0.819
52 Iteration:
------------------------------------
Accuracy Score: 0.7765
53 Iteration:
------------------------------------
Accuracy Score: 0.8065
54 Iteration:
------------------------------------
Accuracy Score: 0.822
55 Iteration:
------------------------------------
Accuracy Score: 0.8245
56 Iteration:
------------------------------------
Accuracy Score: 0.7935
57 Iteration:
------------------------------------
Accuracy Score: 0.824
58 Iteration:
------------------------------------
Accuracy Score: 0.8355
59 Iteration:
------------------------------------
Accuracy Score: 0.8045
60 Iteration:
------------------------------------
Accuracy Score: 0.802
61 Iteration:
------------------------------------
Accuracy Score: 0.8325
62 Iteration:
------------------------------------
Accuracy Score: 0.7105
63 Iteration:
------------------------------------
Accuracy Score: 0.8035
64 Iteration:
------------------------------------
Accuracy Score: 0.7785
65 Iteration:
------------------------------------
Accuracy Score: 0.7675
66 Iteration:
------------------------------------
Accuracy Score: 0.8115
67 Iteration:
------------------------------------
Accuracy Score: 0.83
68 Iteration:
------------------------------------
Accuracy Score: 0.832
69 Iteration:
------------------------------------
Accuracy Score: 0.843
70 Iteration:
------------------------------------
Accuracy Score: 0.812
71 Iteration:
------------------------------------
Accuracy Score: 0.754
72 Iteration:
------------------------------------
Accuracy Score: 0.799
73 Iteration:
------------------------------------
Accuracy Score: 0.846
74 Iteration:
------------------------------------
Accuracy Score: 0.812
75 Iteration:
------------------------------------
Accuracy Score: 0.802
76 Iteration:
------------------------------------
Accuracy Score: 0.7985
77 Iteration:
------------------------------------
Accuracy Score: 0.862
78 Iteration:
------------------------------------
Accuracy Score: 0.7895
79 Iteration:
------------------------------------
Accuracy Score: 0.8245
80 Iteration:
------------------------------------
Accuracy Score: 0.824
81 Iteration:
------------------------------------
Accuracy Score: 0.8315
82 Iteration:
------------------------------------
Accuracy Score: 0.7695
83 Iteration:
------------------------------------
Accuracy Score: 0.826
84 Iteration:
------------------------------------
Accuracy Score: 0.8195
85 Iteration:
------------------------------------
Accuracy Score: 0.808
86 Iteration:
------------------------------------
Accuracy Score: 0.8355
87 Iteration:
------------------------------------
Accuracy Score: 0.7695
88 Iteration:
------------------------------------
Accuracy Score: 0.831
89 Iteration:
------------------------------------
Accuracy Score: 0.8285
90 Iteration:
------------------------------------
Accuracy Score: 0.7615
91 Iteration:
------------------------------------
Accuracy Score: 0.794
92 Iteration:
------------------------------------
Accuracy Score: 0.8415
93 Iteration:
------------------------------------
Accuracy Score: 0.796
94 Iteration:
------------------------------------
Accuracy Score: 0.815
95 Iteration:
------------------------------------
Accuracy Score: 0.8205
96 Iteration:
------------------------------------
Accuracy Score: 0.7755
97 Iteration:
------------------------------------
Accuracy Score: 0.789
98 Iteration:
------------------------------------
Accuracy Score: 0.8245
99 Iteration:
------------------------------------
Accuracy Score: 0.835
100 Iteration:
------------------------------------
Accuracy Score: 0.776
101 Iteration:
------------------------------------
Accuracy Score: 0.794
102 Iteration:
------------------------------------
Accuracy Score: 0.8055
103 Iteration:
------------------------------------
Accuracy Score: 0.764
104 Iteration:
------------------------------------
Accuracy Score: 0.7535
105 Iteration:
------------------------------------
Accuracy Score: 0.776
106 Iteration:
------------------------------------
Accuracy Score: 0.791
107 Iteration:
------------------------------------
Accuracy Score: 0.8455
108 Iteration:
------------------------------------
Accuracy Score: 0.85
109 Iteration:
------------------------------------
Accuracy Score: 0.829
110 Iteration:
------------------------------------
Accuracy Score: 0.851
111 Iteration:
------------------------------------
Accuracy Score: 0.768
112 Iteration:
------------------------------------
Accuracy Score: 0.7295
113 Iteration:
------------------------------------
Accuracy Score: 0.7985
114 Iteration:
------------------------------------
Accuracy Score: 0.8105
115 Iteration:
------------------------------------
Accuracy Score: 0.8235
116 Iteration:
------------------------------------
Accuracy Score: 0.7695
117 Iteration:
------------------------------------
Accuracy Score: 0.758
118 Iteration:
------------------------------------
Accuracy Score: 0.8175
119 Iteration:
------------------------------------
Accuracy Score: 0.8495
120 Iteration:
------------------------------------
Accuracy Score: 0.8205
121 Iteration:
------------------------------------
Accuracy Score: 0.829
122 Iteration:
------------------------------------
Accuracy Score: 0.807
123 Iteration:
------------------------------------
Accuracy Score: 0.7925
124 Iteration:
------------------------------------
Accuracy Score: 0.8105
125 Iteration:
------------------------------------
Accuracy Score: 0.834
126 Iteration:
------------------------------------
Accuracy Score: 0.815
127 Iteration:
------------------------------------
Accuracy Score: 0.8045
128 Iteration:
------------------------------------
Accuracy Score: 0.829
129 Iteration:
------------------------------------
Accuracy Score: 0.812
130 Iteration:
------------------------------------
Accuracy Score: 0.8545
131 Iteration:
------------------------------------
Accuracy Score: 0.782
132 Iteration:
------------------------------------
Accuracy Score: 0.7755
133 Iteration:
------------------------------------
Accuracy Score: 0.799
134 Iteration:
------------------------------------
Accuracy Score: 0.7325
135 Iteration:
------------------------------------
Accuracy Score: 0.7925
136 Iteration:
------------------------------------
Accuracy Score: 0.818
137 Iteration:
------------------------------------
Accuracy Score: 0.7635
138 Iteration:
------------------------------------
Accuracy Score: 0.8055
139 Iteration:
------------------------------------
Accuracy Score: 0.748
140 Iteration:
------------------------------------
Accuracy Score: 0.823
141 Iteration:
------------------------------------
Accuracy Score: 0.801
142 Iteration:
------------------------------------
Accuracy Score: 0.761
143 Iteration:
------------------------------------
Accuracy Score: 0.8095
144 Iteration:
------------------------------------
Accuracy Score: 0.8205
145 Iteration:
------------------------------------
Accuracy Score: 0.7945
146 Iteration:
------------------------------------
Accuracy Score: 0.755
147 Iteration:
------------------------------------
Accuracy Score: 0.846
148 Iteration:
------------------------------------
Accuracy Score: 0.813
149 Iteration:
------------------------------------
Accuracy Score: 0.847
150 Iteration:
------------------------------------
Accuracy Score: 0.81
151 Iteration:
------------------------------------
Accuracy Score: 0.776
152 Iteration:
------------------------------------
Accuracy Score: 0.7895
153 Iteration:
------------------------------------
Accuracy Score: 0.838
154 Iteration:
------------------------------------
Accuracy Score: 0.8035
155 Iteration:
------------------------------------
Accuracy Score: 0.8055
156 Iteration:
------------------------------------
Accuracy Score: 0.7955
157 Iteration:
------------------------------------
Accuracy Score: 0.75
158 Iteration:
------------------------------------
Accuracy Score: 0.801
159 Iteration:
------------------------------------
Accuracy Score: 0.8115
160 Iteration:
------------------------------------
Accuracy Score: 0.7935
161 Iteration:
------------------------------------
Accuracy Score: 0.8185
162 Iteration:
------------------------------------
Accuracy Score: 0.7875
163 Iteration:
------------------------------------
Accuracy Score: 0.7675
164 Iteration:
------------------------------------
Accuracy Score: 0.797
165 Iteration:
------------------------------------
Accuracy Score: 0.7205
166 Iteration:
------------------------------------
Accuracy Score: 0.796
167 Iteration:
------------------------------------
Accuracy Score: 0.7985
168 Iteration:
------------------------------------
Accuracy Score: 0.8025
169 Iteration:
------------------------------------
Accuracy Score: 0.7905
170 Iteration:
------------------------------------
Accuracy Score: 0.71
171 Iteration:
------------------------------------
Accuracy Score: 0.8105
172 Iteration:
------------------------------------
Accuracy Score: 0.844
173 Iteration:
------------------------------------
Accuracy Score: 0.7915
174 Iteration:
------------------------------------
Accuracy Score: 0.8225
175 Iteration:
------------------------------------
Accuracy Score: 0.8255
176 Iteration:
------------------------------------
Accuracy Score: 0.8015
177 Iteration:
------------------------------------
Accuracy Score: 0.8045
178 Iteration:
------------------------------------
Accuracy Score: 0.8225
179 Iteration:
------------------------------------
Accuracy Score: 0.8355
180 Iteration:
------------------------------------
Accuracy Score: 0.7055
181 Iteration:
------------------------------------
Accuracy Score: 0.709
182 Iteration:
------------------------------------
Accuracy Score: 0.7805
183 Iteration:
------------------------------------
Accuracy Score: 0.845
184 Iteration:
------------------------------------
Accuracy Score: 0.7925
185 Iteration:
------------------------------------
Accuracy Score: 0.844
186 Iteration:
------------------------------------
Accuracy Score: 0.8195
187 Iteration:
------------------------------------
Accuracy Score: 0.766
188 Iteration:
------------------------------------
Accuracy Score: 0.849
189 Iteration:
------------------------------------
Accuracy Score: 0.8045
190 Iteration:
------------------------------------
Accuracy Score: 0.8155
191 Iteration:
------------------------------------
Accuracy Score: 0.806
192 Iteration:
------------------------------------
Accuracy Score: 0.848
193 Iteration:
------------------------------------
Accuracy Score: 0.837
194 Iteration:
------------------------------------
Accuracy Score: 0.8365
195 Iteration:
------------------------------------
Accuracy Score: 0.799
196 Iteration:
------------------------------------
Accuracy Score: 0.805
197 Iteration:
------------------------------------
Accuracy Score: 0.813
198 Iteration:
------------------------------------
Accuracy Score: 0.826
199 Iteration:
------------------------------------
Accuracy Score: 0.756
200 Iteration:
------------------------------------
Accuracy Score: 0.844
201 Iteration:
------------------------------------
Accuracy Score: 0.8045
202 Iteration:
------------------------------------
Accuracy Score: 0.8215
203 Iteration:
------------------------------------
Accuracy Score: 0.83
204 Iteration:
------------------------------------
Accuracy Score: 0.806
205 Iteration:
------------------------------------
Accuracy Score: 0.8025
206 Iteration:
------------------------------------
Accuracy Score: 0.8475
207 Iteration:
------------------------------------
Accuracy Score: 0.8315
208 Iteration:
------------------------------------
Accuracy Score: 0.732
209 Iteration:
------------------------------------
Accuracy Score: 0.7565
210 Iteration:
------------------------------------
Accuracy Score: 0.8095
211 Iteration:
------------------------------------
Accuracy Score: 0.8245
212 Iteration:
------------------------------------
Accuracy Score: 0.814
213 Iteration:
------------------------------------
Accuracy Score: 0.753
214 Iteration:
------------------------------------
Accuracy Score: 0.797
215 Iteration:
------------------------------------
Accuracy Score: 0.81
216 Iteration:
------------------------------------
Accuracy Score: 0.793
217 Iteration:
------------------------------------
Accuracy Score: 0.803
218 Iteration:
------------------------------------
Accuracy Score: 0.78
219 Iteration:
------------------------------------
Accuracy Score: 0.809
220 Iteration:
------------------------------------
Accuracy Score: 0.793
221 Iteration:
------------------------------------
Accuracy Score: 0.8245
222 Iteration:
------------------------------------
Accuracy Score: 0.8235
223 Iteration:
------------------------------------
Accuracy Score: 0.815
224 Iteration:
------------------------------------
Accuracy Score: 0.8
225 Iteration:
------------------------------------
Accuracy Score: 0.809
226 Iteration:
------------------------------------
Accuracy Score: 0.799
227 Iteration:
------------------------------------
Accuracy Score: 0.7855
228 Iteration:
------------------------------------
Accuracy Score: 0.814
229 Iteration:
------------------------------------
Accuracy Score: 0.775
230 Iteration:
------------------------------------
Accuracy Score: 0.742
231 Iteration:
------------------------------------
Accuracy Score: 0.7825
232 Iteration:
------------------------------------
Accuracy Score: 0.801
233 Iteration:
------------------------------------
Accuracy Score: 0.8515
234 Iteration:
------------------------------------
Accuracy Score: 0.7535
235 Iteration:
------------------------------------
Accuracy Score: 0.8045
236 Iteration:
------------------------------------
Accuracy Score: 0.8055
237 Iteration:
------------------------------------
Accuracy Score: 0.7525
238 Iteration:
------------------------------------
Accuracy Score: 0.8175
239 Iteration:
------------------------------------
Accuracy Score: 0.794
240 Iteration:
------------------------------------
Accuracy Score: 0.7285
241 Iteration:
------------------------------------
Accuracy Score: 0.8035
242 Iteration:
------------------------------------
Accuracy Score: 0.763
243 Iteration:
------------------------------------
Accuracy Score: 0.7985
244 Iteration:
------------------------------------
Accuracy Score: 0.827
245 Iteration:
------------------------------------
Accuracy Score: 0.7965
246 Iteration:
------------------------------------
Accuracy Score: 0.841
247 Iteration:
------------------------------------
Accuracy Score: 0.7745
248 Iteration:
------------------------------------
Accuracy Score: 0.7685
249 Iteration:
------------------------------------
Accuracy Score: 0.7755
250 Iteration:
------------------------------------
Accuracy Score: 0.823
251 Iteration:
------------------------------------
Accuracy Score: 0.7995
252 Iteration:
------------------------------------
Accuracy Score: 0.8005
253 Iteration:
------------------------------------
Accuracy Score: 0.797
254 Iteration:
------------------------------------
Accuracy Score: 0.8035
255 Iteration:
------------------------------------
Accuracy Score: 0.834
256 Iteration:
------------------------------------
Accuracy Score: 0.815
257 Iteration:
------------------------------------
Accuracy Score: 0.804
258 Iteration:
------------------------------------
Accuracy Score: 0.791
259 Iteration:
------------------------------------
Accuracy Score: 0.815
260 Iteration:
------------------------------------
Accuracy Score: 0.819
261 Iteration:
------------------------------------
Accuracy Score: 0.795
262 Iteration:
------------------------------------
Accuracy Score: 0.836
263 Iteration:
------------------------------------
Accuracy Score: 0.7905
264 Iteration:
------------------------------------
Accuracy Score: 0.793
265 Iteration:
------------------------------------
Accuracy Score: 0.796
266 Iteration:
------------------------------------
Accuracy Score: 0.7855
267 Iteration:
------------------------------------
Accuracy Score: 0.7855
268 Iteration:
------------------------------------
Accuracy Score: 0.76
269 Iteration:
------------------------------------
Accuracy Score: 0.85
270 Iteration:
------------------------------------
Accuracy Score: 0.8415
271 Iteration:
------------------------------------
Accuracy Score: 0.8055
272 Iteration:
------------------------------------
Accuracy Score: 0.797
273 Iteration:
------------------------------------
Accuracy Score: 0.814
274 Iteration:
------------------------------------
Accuracy Score: 0.776
275 Iteration:
------------------------------------
Accuracy Score: 0.782
276 Iteration:
------------------------------------
Accuracy Score: 0.8145
277 Iteration:
------------------------------------
Accuracy Score: 0.8315
278 Iteration:
------------------------------------
Accuracy Score: 0.797
279 Iteration:
------------------------------------
Accuracy Score: 0.769
280 Iteration:
------------------------------------
Accuracy Score: 0.7965
281 Iteration:
------------------------------------
Accuracy Score: 0.7675
282 Iteration:
------------------------------------
Accuracy Score: 0.816
283 Iteration:
------------------------------------
Accuracy Score: 0.7905
284 Iteration:
------------------------------------
Accuracy Score: 0.841
285 Iteration:
------------------------------------
Accuracy Score: 0.7955
286 Iteration:
------------------------------------
Accuracy Score: 0.7825
287 Iteration:
------------------------------------
Accuracy Score: 0.7995
288 Iteration:
------------------------------------
Accuracy Score: 0.758
289 Iteration:
------------------------------------
Accuracy Score: 0.8355
290 Iteration:
------------------------------------
Accuracy Score: 0.8105
291 Iteration:
------------------------------------
Accuracy Score: 0.8205
292 Iteration:
------------------------------------
Accuracy Score: 0.842
293 Iteration:
------------------------------------
Accuracy Score: 0.8
294 Iteration:
------------------------------------
Accuracy Score: 0.851
295 Iteration:
------------------------------------
Accuracy Score: 0.836
296 Iteration:
------------------------------------
Accuracy Score: 0.8005
297 Iteration:
------------------------------------
Accuracy Score: 0.806
298 Iteration:
------------------------------------
Accuracy Score: 0.793
299 Iteration:
------------------------------------
Accuracy Score: 0.815
300 Iteration:
------------------------------------
Accuracy Score: 0.827
301 Iteration:
------------------------------------
Accuracy Score: 0.825
302 Iteration:
------------------------------------
Accuracy Score: 0.8295
303 Iteration:
------------------------------------
Accuracy Score: 0.822
304 Iteration:
------------------------------------
Accuracy Score: 0.7985
305 Iteration:
------------------------------------
Accuracy Score: 0.7945
306 Iteration:
------------------------------------
Accuracy Score: 0.8225
307 Iteration:
------------------------------------
Accuracy Score: 0.84
308 Iteration:
------------------------------------
Accuracy Score: 0.7595
309 Iteration:
------------------------------------
Accuracy Score: 0.8115
310 Iteration:
------------------------------------
Accuracy Score: 0.8155
311 Iteration:
------------------------------------
Accuracy Score: 0.8175
312 Iteration:
------------------------------------
Accuracy Score: 0.8605
313 Iteration:
------------------------------------
Accuracy Score: 0.852
314 Iteration:
------------------------------------
Accuracy Score: 0.8435
315 Iteration:
------------------------------------
Accuracy Score: 0.8275
316 Iteration:
------------------------------------
Accuracy Score: 0.814
317 Iteration:
------------------------------------
Accuracy Score: 0.8125
318 Iteration:
------------------------------------
Accuracy Score: 0.8245
319 Iteration:
------------------------------------
Accuracy Score: 0.83
320 Iteration:
------------------------------------
Accuracy Score: 0.763
321 Iteration:
------------------------------------
Accuracy Score: 0.7785
322 Iteration:
------------------------------------
Accuracy Score: 0.7935
323 Iteration:
------------------------------------
Accuracy Score: 0.818
324 Iteration:
------------------------------------
Accuracy Score: 0.7925
325 Iteration:
------------------------------------
Accuracy Score: 0.7765
326 Iteration:
------------------------------------
Accuracy Score: 0.822
327 Iteration:
------------------------------------
Accuracy Score: 0.7405
328 Iteration:
------------------------------------
Accuracy Score: 0.8395
329 Iteration:
------------------------------------
Accuracy Score: 0.8165
330 Iteration:
------------------------------------
Accuracy Score: 0.7835
331 Iteration:
------------------------------------
Accuracy Score: 0.791
332 Iteration:
------------------------------------
Accuracy Score: 0.821
333 Iteration:
------------------------------------
Accuracy Score: 0.819
334 Iteration:
------------------------------------
Accuracy Score: 0.836
335 Iteration:
------------------------------------
Accuracy Score: 0.8215
336 Iteration:
------------------------------------
Accuracy Score: 0.8135
337 Iteration:
------------------------------------
Accuracy Score: 0.814
338 Iteration:
------------------------------------
Accuracy Score: 0.834
339 Iteration:
------------------------------------
Accuracy Score: 0.834
340 Iteration:
------------------------------------
Accuracy Score: 0.834
341 Iteration:
------------------------------------
Accuracy Score: 0.803
342 Iteration:
------------------------------------
Accuracy Score: 0.825
343 Iteration:
------------------------------------
Accuracy Score: 0.824
344 Iteration:
------------------------------------
Accuracy Score: 0.806
345 Iteration:
------------------------------------
Accuracy Score: 0.777
346 Iteration:
------------------------------------
Accuracy Score: 0.833
347 Iteration:
------------------------------------
Accuracy Score: 0.7595
348 Iteration:
------------------------------------
Accuracy Score: 0.7985
349 Iteration:
------------------------------------
Accuracy Score: 0.8035
350 Iteration:
------------------------------------
Accuracy Score: 0.772
351 Iteration:
------------------------------------
Accuracy Score: 0.817
352 Iteration:
------------------------------------
Accuracy Score: 0.8355
353 Iteration:
------------------------------------
Accuracy Score: 0.8265
354 Iteration:
------------------------------------
Accuracy Score: 0.82
355 Iteration:
------------------------------------
Accuracy Score: 0.7805
356 Iteration:
------------------------------------
Accuracy Score: 0.8325
357 Iteration:
------------------------------------
Accuracy Score: 0.7725
358 Iteration:
------------------------------------
Accuracy Score: 0.826
359 Iteration:
------------------------------------
Accuracy Score: 0.825
360 Iteration:
------------------------------------
Accuracy Score: 0.78
361 Iteration:
------------------------------------
Accuracy Score: 0.8435
362 Iteration:
------------------------------------
Accuracy Score: 0.7955
363 Iteration:
------------------------------------
Accuracy Score: 0.791
364 Iteration:
------------------------------------
Accuracy Score: 0.834
365 Iteration:
------------------------------------
Accuracy Score: 0.7795
366 Iteration:
------------------------------------
Accuracy Score: 0.7635
367 Iteration:
------------------------------------
Accuracy Score: 0.8195
368 Iteration:
------------------------------------
Accuracy Score: 0.819
369 Iteration:
------------------------------------
Accuracy Score: 0.8385
370 Iteration:
------------------------------------
Accuracy Score: 0.7895
371 Iteration:
------------------------------------
Accuracy Score: 0.8155
372 Iteration:
------------------------------------
Accuracy Score: 0.7855
373 Iteration:
------------------------------------
Accuracy Score: 0.8125
374 Iteration:
------------------------------------
Accuracy Score: 0.809
375 Iteration:
------------------------------------
Accuracy Score: 0.7335
376 Iteration:
------------------------------------
Accuracy Score: 0.819
377 Iteration:
------------------------------------
Accuracy Score: 0.8235
378 Iteration:
------------------------------------
Accuracy Score: 0.7625
379 Iteration:
------------------------------------
Accuracy Score: 0.778
380 Iteration:
------------------------------------
Accuracy Score: 0.7815
381 Iteration:
------------------------------------
Accuracy Score: 0.7765
382 Iteration:
------------------------------------
Accuracy Score: 0.7515
383 Iteration:
------------------------------------
Accuracy Score: 0.7855
384 Iteration:
------------------------------------
Accuracy Score: 0.8215
385 Iteration:
------------------------------------
Accuracy Score: 0.818
386 Iteration:
------------------------------------
Accuracy Score: 0.829
387 Iteration:
------------------------------------
Accuracy Score: 0.767
388 Iteration:
------------------------------------
Accuracy Score: 0.8285
389 Iteration:
------------------------------------
Accuracy Score: 0.8335
390 Iteration:
------------------------------------
Accuracy Score: 0.827
391 Iteration:
------------------------------------
Accuracy Score: 0.7415
392 Iteration:
------------------------------------
Accuracy Score: 0.8345
393 Iteration:
------------------------------------
Accuracy Score: 0.816
394 Iteration:
------------------------------------
Accuracy Score: 0.7525
395 Iteration:
------------------------------------
Accuracy Score: 0.803
396 Iteration:
------------------------------------
Accuracy Score: 0.806
397 Iteration:
------------------------------------
Accuracy Score: 0.803
398 Iteration:
------------------------------------
Accuracy Score: 0.7985
399 Iteration:
------------------------------------
Accuracy Score: 0.84
400 Iteration:
------------------------------------
Accuracy Score: 0.8185
401 Iteration:
------------------------------------
Accuracy Score: 0.82
402 Iteration:
------------------------------------
Accuracy Score: 0.755
403 Iteration:
------------------------------------
Accuracy Score: 0.802
404 Iteration:
------------------------------------
Accuracy Score: 0.771
405 Iteration:
------------------------------------
Accuracy Score: 0.784
406 Iteration:
------------------------------------
Accuracy Score: 0.797
407 Iteration:
------------------------------------
Accuracy Score: 0.766
408 Iteration:
------------------------------------
Accuracy Score: 0.853
409 Iteration:
------------------------------------
Accuracy Score: 0.8325
410 Iteration:
------------------------------------
Accuracy Score: 0.8395
411 Iteration:
------------------------------------
Accuracy Score: 0.777
412 Iteration:
------------------------------------
Accuracy Score: 0.802
413 Iteration:
------------------------------------
Accuracy Score: 0.7685
414 Iteration:
------------------------------------
Accuracy Score: 0.83
415 Iteration:
------------------------------------
Accuracy Score: 0.7955
416 Iteration:
------------------------------------
Accuracy Score: 0.7955
417 Iteration:
------------------------------------
Accuracy Score: 0.808
418 Iteration:
------------------------------------
Accuracy Score: 0.807
419 Iteration:
------------------------------------
Accuracy Score: 0.7815
420 Iteration:
------------------------------------
Accuracy Score: 0.798
421 Iteration:
------------------------------------
Accuracy Score: 0.8165
422 Iteration:
------------------------------------
Accuracy Score: 0.811
423 Iteration:
------------------------------------
Accuracy Score: 0.8045
424 Iteration:
------------------------------------
Accuracy Score: 0.807
425 Iteration:
------------------------------------
Accuracy Score: 0.845
426 Iteration:
------------------------------------
Accuracy Score: 0.7995
427 Iteration:
------------------------------------
Accuracy Score: 0.8065
428 Iteration:
------------------------------------
Accuracy Score: 0.83
429 Iteration:
------------------------------------
Accuracy Score: 0.778
430 Iteration:
------------------------------------
Accuracy Score: 0.7995
431 Iteration:
------------------------------------
Accuracy Score: 0.8415
432 Iteration:
------------------------------------
Accuracy Score: 0.8405
433 Iteration:
------------------------------------
Accuracy Score: 0.8015
434 Iteration:
------------------------------------
Accuracy Score: 0.8425
435 Iteration:
------------------------------------
Accuracy Score: 0.8285
436 Iteration:
------------------------------------
Accuracy Score: 0.82
437 Iteration:
------------------------------------
Accuracy Score: 0.831
438 Iteration:
------------------------------------
Accuracy Score: 0.8385
439 Iteration:
------------------------------------
Accuracy Score: 0.8045
440 Iteration:
------------------------------------
Accuracy Score: 0.8235
441 Iteration:
------------------------------------
Accuracy Score: 0.797
442 Iteration:
------------------------------------
Accuracy Score: 0.835
443 Iteration:
------------------------------------
Accuracy Score: 0.836
444 Iteration:
------------------------------------
Accuracy Score: 0.798
445 Iteration:
------------------------------------
Accuracy Score: 0.8255
446 Iteration:
------------------------------------
Accuracy Score: 0.8225
447 Iteration:
------------------------------------
Accuracy Score: 0.79
448 Iteration:
------------------------------------
Accuracy Score: 0.8145
449 Iteration:
------------------------------------
Accuracy Score: 0.792
450 Iteration:
------------------------------------
Accuracy Score: 0.811
451 Iteration:
------------------------------------
Accuracy Score: 0.8155
452 Iteration:
------------------------------------
Accuracy Score: 0.8515
453 Iteration:
------------------------------------
Accuracy Score: 0.802
454 Iteration:
------------------------------------
Accuracy Score: 0.787
455 Iteration:
------------------------------------
Accuracy Score: 0.818
456 Iteration:
------------------------------------
Accuracy Score: 0.8005
457 Iteration:
------------------------------------
Accuracy Score: 0.8065
458 Iteration:
------------------------------------
Accuracy Score: 0.801
459 Iteration:
------------------------------------
Accuracy Score: 0.7845
460 Iteration:
------------------------------------
Accuracy Score: 0.7995
461 Iteration:
------------------------------------
Accuracy Score: 0.8115
462 Iteration:
------------------------------------
Accuracy Score: 0.8245
463 Iteration:
------------------------------------
Accuracy Score: 0.839
464 Iteration:
------------------------------------
Accuracy Score: 0.8095
465 Iteration:
------------------------------------
Accuracy Score: 0.823
466 Iteration:
------------------------------------
Accuracy Score: 0.849
467 Iteration:
------------------------------------
Accuracy Score: 0.8395
468 Iteration:
------------------------------------
Accuracy Score: 0.8375
469 Iteration:
------------------------------------
Accuracy Score: 0.827
470 Iteration:
------------------------------------
Accuracy Score: 0.8335
471 Iteration:
------------------------------------
Accuracy Score: 0.8295
472 Iteration:
------------------------------------
Accuracy Score: 0.7795
473 Iteration:
------------------------------------
Accuracy Score: 0.8225
474 Iteration:
------------------------------------
Accuracy Score: 0.7755
475 Iteration:
------------------------------------
Accuracy Score: 0.8095
476 Iteration:
------------------------------------
Accuracy Score: 0.718
477 Iteration:
------------------------------------
Accuracy Score: 0.8045
478 Iteration:
------------------------------------
Accuracy Score: 0.8285
479 Iteration:
------------------------------------
Accuracy Score: 0.786
480 Iteration:
------------------------------------
Accuracy Score: 0.7545
481 Iteration:
------------------------------------
Accuracy Score: 0.811
482 Iteration:
------------------------------------
Accuracy Score: 0.8225
483 Iteration:
------------------------------------
Accuracy Score: 0.7925
484 Iteration:
------------------------------------
Accuracy Score: 0.847
485 Iteration:
------------------------------------
Accuracy Score: 0.7595
486 Iteration:
------------------------------------
Accuracy Score: 0.7845
487 Iteration:
------------------------------------
Accuracy Score: 0.82
488 Iteration:
------------------------------------
Accuracy Score: 0.8205
489 Iteration:
------------------------------------
Accuracy Score: 0.8355
490 Iteration:
------------------------------------
Accuracy Score: 0.82
491 Iteration:
------------------------------------
Accuracy Score: 0.8475
492 Iteration:
------------------------------------
Accuracy Score: 0.8355
493 Iteration:
------------------------------------
Accuracy Score: 0.7915
494 Iteration:
------------------------------------
Accuracy Score: 0.767
495 Iteration:
------------------------------------
Accuracy Score: 0.8205
496 Iteration:
------------------------------------
Accuracy Score: 0.7975
497 Iteration:
------------------------------------
Accuracy Score: 0.806
498 Iteration:
------------------------------------
Accuracy Score: 0.814
499 Iteration:
------------------------------------
Accuracy Score: 0.835
500 Iteration:
------------------------------------
Accuracy Score: 0.799
501 Iteration:
------------------------------------
Accuracy Score: 0.754
502 Iteration:
------------------------------------
Accuracy Score: 0.783
503 Iteration:
------------------------------------
Accuracy Score: 0.8415
504 Iteration:
------------------------------------
Accuracy Score: 0.823
505 Iteration:
------------------------------------
Accuracy Score: 0.763
506 Iteration:
------------------------------------
Accuracy Score: 0.8215
507 Iteration:
------------------------------------
Accuracy Score: 0.71
508 Iteration:
------------------------------------
Accuracy Score: 0.829
509 Iteration:
------------------------------------
Accuracy Score: 0.8065
510 Iteration:
------------------------------------
Accuracy Score: 0.8395
511 Iteration:
------------------------------------
Accuracy Score: 0.808
512 Iteration:
------------------------------------
Accuracy Score: 0.842
513 Iteration:
------------------------------------
Accuracy Score: 0.816
514 Iteration:
------------------------------------
Accuracy Score: 0.8165
515 Iteration:
------------------------------------
Accuracy Score: 0.829
516 Iteration:
------------------------------------
Accuracy Score: 0.8175
517 Iteration:
------------------------------------
Accuracy Score: 0.7625
518 Iteration:
------------------------------------
Accuracy Score: 0.8425
519 Iteration:
------------------------------------
Accuracy Score: 0.8095
520 Iteration:
------------------------------------
Accuracy Score: 0.811
521 Iteration:
------------------------------------
Accuracy Score: 0.8315
522 Iteration:
------------------------------------
Accuracy Score: 0.8015
523 Iteration:
------------------------------------
Accuracy Score: 0.8275
524 Iteration:
------------------------------------
Accuracy Score: 0.8245
525 Iteration:
------------------------------------
Accuracy Score: 0.7875
526 Iteration:
------------------------------------
Accuracy Score: 0.827
527 Iteration:
------------------------------------
Accuracy Score: 0.825
528 Iteration:
------------------------------------
Accuracy Score: 0.8475
529 Iteration:
------------------------------------
Accuracy Score: 0.8375
530 Iteration:
------------------------------------
Accuracy Score: 0.8005
531 Iteration:
------------------------------------
Accuracy Score: 0.802
532 Iteration:
------------------------------------
Accuracy Score: 0.779
533 Iteration:
------------------------------------
Accuracy Score: 0.7945
534 Iteration:
------------------------------------
Accuracy Score: 0.84
535 Iteration:
------------------------------------
Accuracy Score: 0.781
536 Iteration:
------------------------------------
Accuracy Score: 0.8145
537 Iteration:
------------------------------------
Accuracy Score: 0.8295
538 Iteration:
------------------------------------
Accuracy Score: 0.813
539 Iteration:
------------------------------------
Accuracy Score: 0.8355
540 Iteration:
------------------------------------
Accuracy Score: 0.825
541 Iteration:
------------------------------------
Accuracy Score: 0.8245
542 Iteration:
------------------------------------
Accuracy Score: 0.7765
543 Iteration:
------------------------------------
Accuracy Score: 0.817
544 Iteration:
------------------------------------
Accuracy Score: 0.7935
545 Iteration:
------------------------------------
Accuracy Score: 0.82
546 Iteration:
------------------------------------
Accuracy Score: 0.818
547 Iteration:
------------------------------------
Accuracy Score: 0.7145
548 Iteration:
------------------------------------
Accuracy Score: 0.778
549 Iteration:
------------------------------------
Accuracy Score: 0.8105
550 Iteration:
------------------------------------
Accuracy Score: 0.812
551 Iteration:
------------------------------------
Accuracy Score: 0.7855
552 Iteration:
------------------------------------
Accuracy Score: 0.832
553 Iteration:
------------------------------------
Accuracy Score: 0.817
554 Iteration:
------------------------------------
Accuracy Score: 0.7955
555 Iteration:
------------------------------------
Accuracy Score: 0.849
556 Iteration:
------------------------------------
Accuracy Score: 0.754
557 Iteration:
------------------------------------
Accuracy Score: 0.79
558 Iteration:
------------------------------------
Accuracy Score: 0.8385
559 Iteration:
------------------------------------
Accuracy Score: 0.771
560 Iteration:
------------------------------------
Accuracy Score: 0.823
561 Iteration:
------------------------------------
Accuracy Score: 0.7735
562 Iteration:
------------------------------------
Accuracy Score: 0.822
563 Iteration:
------------------------------------
Accuracy Score: 0.8285
564 Iteration:
------------------------------------
Accuracy Score: 0.839
565 Iteration:
------------------------------------
Accuracy Score: 0.802
566 Iteration:
------------------------------------
Accuracy Score: 0.805
567 Iteration:
------------------------------------
Accuracy Score: 0.848
568 Iteration:
------------------------------------
Accuracy Score: 0.7885
569 Iteration:
------------------------------------
Accuracy Score: 0.796
570 Iteration:
------------------------------------
Accuracy Score: 0.8295
571 Iteration:
------------------------------------
Accuracy Score: 0.8405
572 Iteration:
------------------------------------
Accuracy Score: 0.84
573 Iteration:
------------------------------------
Accuracy Score: 0.7925
574 Iteration:
------------------------------------
Accuracy Score: 0.8305
575 Iteration:
------------------------------------
Accuracy Score: 0.791
576 Iteration:
------------------------------------
Accuracy Score: 0.8015
577 Iteration:
------------------------------------
Accuracy Score: 0.744
578 Iteration:
------------------------------------
Accuracy Score: 0.7615
579 Iteration:
------------------------------------
Accuracy Score: 0.822
580 Iteration:
------------------------------------
Accuracy Score: 0.8015
581 Iteration:
------------------------------------
Accuracy Score: 0.8135
582 Iteration:
------------------------------------
Accuracy Score: 0.8195
583 Iteration:
------------------------------------
Accuracy Score: 0.7965
584 Iteration:
------------------------------------
Accuracy Score: 0.8445
585 Iteration:
------------------------------------
Accuracy Score: 0.842
586 Iteration:
------------------------------------
Accuracy Score: 0.8345
587 Iteration:
------------------------------------
Accuracy Score: 0.8085
588 Iteration:
------------------------------------
Accuracy Score: 0.7925
589 Iteration:
------------------------------------
Accuracy Score: 0.763
590 Iteration:
------------------------------------
Accuracy Score: 0.8385
591 Iteration:
------------------------------------
Accuracy Score: 0.8175
592 Iteration:
------------------------------------
Accuracy Score: 0.7875
593 Iteration:
------------------------------------
Accuracy Score: 0.838
594 Iteration:
------------------------------------
Accuracy Score: 0.8205
595 Iteration:
------------------------------------
Accuracy Score: 0.7305
596 Iteration:
------------------------------------
Accuracy Score: 0.808
597 Iteration:
------------------------------------
Accuracy Score: 0.814
598 Iteration:
------------------------------------
Accuracy Score: 0.8365
599 Iteration:
------------------------------------
Accuracy Score: 0.7795
600 Iteration:
------------------------------------
Accuracy Score: 0.738
601 Iteration:
------------------------------------
Accuracy Score: 0.7715
602 Iteration:
------------------------------------
Accuracy Score: 0.8305
603 Iteration:
------------------------------------
Accuracy Score: 0.8075
604 Iteration:
------------------------------------
Accuracy Score: 0.782
605 Iteration:
------------------------------------
Accuracy Score: 0.801
606 Iteration:
------------------------------------
Accuracy Score: 0.799
607 Iteration:
------------------------------------
Accuracy Score: 0.799
608 Iteration:
------------------------------------
Accuracy Score: 0.7175
609 Iteration:
------------------------------------
Accuracy Score: 0.8225
610 Iteration:
------------------------------------
Accuracy Score: 0.8425
611 Iteration:
------------------------------------
Accuracy Score: 0.7675
612 Iteration:
------------------------------------
Accuracy Score: 0.799
613 Iteration:
------------------------------------
Accuracy Score: 0.8035
614 Iteration:
------------------------------------
Accuracy Score: 0.7915
615 Iteration:
------------------------------------
Accuracy Score: 0.8045
616 Iteration:
------------------------------------
Accuracy Score: 0.837
617 Iteration:
------------------------------------
Accuracy Score: 0.8495
618 Iteration:
------------------------------------
Accuracy Score: 0.7835
619 Iteration:
------------------------------------
Accuracy Score: 0.819
620 Iteration:
------------------------------------
Accuracy Score: 0.832
621 Iteration:
------------------------------------
Accuracy Score: 0.7645
622 Iteration:
------------------------------------
Accuracy Score: 0.831
623 Iteration:
------------------------------------
Accuracy Score: 0.8065
624 Iteration:
------------------------------------
Accuracy Score: 0.745
625 Iteration:
------------------------------------
Accuracy Score: 0.826
626 Iteration:
------------------------------------
Accuracy Score: 0.81
627 Iteration:
------------------------------------
Accuracy Score: 0.7735
628 Iteration:
------------------------------------
Accuracy Score: 0.8285
629 Iteration:
------------------------------------
Accuracy Score: 0.7865
630 Iteration:
------------------------------------
Accuracy Score: 0.816
631 Iteration:
------------------------------------
Accuracy Score: 0.834
632 Iteration:
------------------------------------
Accuracy Score: 0.84
633 Iteration:
------------------------------------
Accuracy Score: 0.8085
634 Iteration:
------------------------------------
Accuracy Score: 0.7395
635 Iteration:
------------------------------------
Accuracy Score: 0.794
636 Iteration:
------------------------------------
Accuracy Score: 0.7745
637 Iteration:
------------------------------------
Accuracy Score: 0.799
638 Iteration:
------------------------------------
Accuracy Score: 0.8135
639 Iteration:
------------------------------------
Accuracy Score: 0.76
640 Iteration:
------------------------------------
Accuracy Score: 0.794
641 Iteration:
------------------------------------
Accuracy Score: 0.795
642 Iteration:
------------------------------------
Accuracy Score: 0.826
643 Iteration:
------------------------------------
Accuracy Score: 0.7945
644 Iteration:
------------------------------------
Accuracy Score: 0.833
645 Iteration:
------------------------------------
Accuracy Score: 0.807
646 Iteration:
------------------------------------
Accuracy Score: 0.8485
647 Iteration:
------------------------------------
Accuracy Score: 0.8265
648 Iteration:
------------------------------------
Accuracy Score: 0.8265
649 Iteration:
------------------------------------
Accuracy Score: 0.838
650 Iteration:
------------------------------------
Accuracy Score: 0.819
651 Iteration:
------------------------------------
Accuracy Score: 0.838
652 Iteration:
------------------------------------
Accuracy Score: 0.8
653 Iteration:
------------------------------------
Accuracy Score: 0.8125
654 Iteration:
------------------------------------
Accuracy Score: 0.8065
655 Iteration:
------------------------------------
Accuracy Score: 0.79
656 Iteration:
------------------------------------
Accuracy Score: 0.796
657 Iteration:
------------------------------------
Accuracy Score: 0.826
658 Iteration:
------------------------------------
Accuracy Score: 0.838
659 Iteration:
------------------------------------
Accuracy Score: 0.8135
660 Iteration:
------------------------------------
Accuracy Score: 0.823
661 Iteration:
------------------------------------
Accuracy Score: 0.7965
662 Iteration:
------------------------------------
Accuracy Score: 0.846
663 Iteration:
------------------------------------
Accuracy Score: 0.817
664 Iteration:
------------------------------------
Accuracy Score: 0.743
665 Iteration:
------------------------------------
Accuracy Score: 0.8045
666 Iteration:
------------------------------------
Accuracy Score: 0.787
667 Iteration:
------------------------------------
Accuracy Score: 0.739
668 Iteration:
------------------------------------
Accuracy Score: 0.823
669 Iteration:
------------------------------------
Accuracy Score: 0.7745
670 Iteration:
------------------------------------
Accuracy Score: 0.8125
671 Iteration:
------------------------------------
Accuracy Score: 0.809
672 Iteration:
------------------------------------
Accuracy Score: 0.8055
673 Iteration:
------------------------------------
Accuracy Score: 0.7775
674 Iteration:
------------------------------------
Accuracy Score: 0.8155
675 Iteration:
------------------------------------
Accuracy Score: 0.7665
676 Iteration:
------------------------------------
Accuracy Score: 0.818
677 Iteration:
------------------------------------
Accuracy Score: 0.816
678 Iteration:
------------------------------------
Accuracy Score: 0.828
679 Iteration:
------------------------------------
Accuracy Score: 0.8115
680 Iteration:
------------------------------------
Accuracy Score: 0.81
681 Iteration:
------------------------------------
Accuracy Score: 0.8285
682 Iteration:
------------------------------------
Accuracy Score: 0.8295
683 Iteration:
------------------------------------
Accuracy Score: 0.8345
684 Iteration:
------------------------------------
Accuracy Score: 0.803
685 Iteration:
------------------------------------
Accuracy Score: 0.761
686 Iteration:
------------------------------------
Accuracy Score: 0.823
687 Iteration:
------------------------------------
Accuracy Score: 0.8085
688 Iteration:
------------------------------------
Accuracy Score: 0.786
689 Iteration:
------------------------------------
Accuracy Score: 0.8205
690 Iteration:
------------------------------------
Accuracy Score: 0.818
691 Iteration:
------------------------------------
Accuracy Score: 0.8155
692 Iteration:
------------------------------------
Accuracy Score: 0.822
693 Iteration:
------------------------------------
Accuracy Score: 0.824
694 Iteration:
------------------------------------
Accuracy Score: 0.8435
695 Iteration:
------------------------------------
Accuracy Score: 0.796
696 Iteration:
------------------------------------
Accuracy Score: 0.774
697 Iteration:
------------------------------------
Accuracy Score: 0.809
698 Iteration:
------------------------------------
Accuracy Score: 0.8195
699 Iteration:
------------------------------------
Accuracy Score: 0.8215
700 Iteration:
------------------------------------
Accuracy Score: 0.8205
701 Iteration:
------------------------------------
Accuracy Score: 0.833
702 Iteration:
------------------------------------
Accuracy Score: 0.7915
703 Iteration:
------------------------------------
Accuracy Score: 0.8
704 Iteration:
------------------------------------
Accuracy Score: 0.809
705 Iteration:
------------------------------------
Accuracy Score: 0.77
706 Iteration:
------------------------------------
Accuracy Score: 0.8115
707 Iteration:
------------------------------------
Accuracy Score: 0.829
708 Iteration:
------------------------------------
Accuracy Score: 0.7675
709 Iteration:
------------------------------------
Accuracy Score: 0.7995
710 Iteration:
------------------------------------
Accuracy Score: 0.8205
711 Iteration:
------------------------------------
Accuracy Score: 0.812
712 Iteration:
------------------------------------
Accuracy Score: 0.83
713 Iteration:
------------------------------------
Accuracy Score: 0.8345
714 Iteration:
------------------------------------
Accuracy Score: 0.823
715 Iteration:
------------------------------------
Accuracy Score: 0.8275
716 Iteration:
------------------------------------
Accuracy Score: 0.817
717 Iteration:
------------------------------------
Accuracy Score: 0.83
718 Iteration:
------------------------------------
Accuracy Score: 0.7585
719 Iteration:
------------------------------------
Accuracy Score: 0.7885
720 Iteration:
------------------------------------
Accuracy Score: 0.794
721 Iteration:
------------------------------------
Accuracy Score: 0.8125
722 Iteration:
------------------------------------
Accuracy Score: 0.817
723 Iteration:
------------------------------------
Accuracy Score: 0.807
724 Iteration:
------------------------------------
Accuracy Score: 0.8325
725 Iteration:
------------------------------------
Accuracy Score: 0.8555
726 Iteration:
------------------------------------
Accuracy Score: 0.8
727 Iteration:
------------------------------------
Accuracy Score: 0.8465
728 Iteration:
------------------------------------
Accuracy Score: 0.83
729 Iteration:
------------------------------------
Accuracy Score: 0.7795
730 Iteration:
------------------------------------
Accuracy Score: 0.8195
731 Iteration:
------------------------------------
Accuracy Score: 0.7785
732 Iteration:
------------------------------------
Accuracy Score: 0.7945
733 Iteration:
------------------------------------
Accuracy Score: 0.8095
734 Iteration:
------------------------------------
Accuracy Score: 0.847
735 Iteration:
------------------------------------
Accuracy Score: 0.8215
736 Iteration:
------------------------------------
Accuracy Score: 0.8165
737 Iteration:
------------------------------------
Accuracy Score: 0.7775
738 Iteration:
------------------------------------
Accuracy Score: 0.8365
739 Iteration:
------------------------------------
Accuracy Score: 0.7945
740 Iteration:
------------------------------------
Accuracy Score: 0.814
741 Iteration:
------------------------------------
Accuracy Score: 0.795
742 Iteration:
------------------------------------
Accuracy Score: 0.7945
743 Iteration:
------------------------------------
Accuracy Score: 0.762
744 Iteration:
------------------------------------
Accuracy Score: 0.791
745 Iteration:
------------------------------------
Accuracy Score: 0.8095
746 Iteration:
------------------------------------
Accuracy Score: 0.84
747 Iteration:
------------------------------------
Accuracy Score: 0.822
748 Iteration:
------------------------------------
Accuracy Score: 0.797
749 Iteration:
------------------------------------
Accuracy Score: 0.8435
750 Iteration:
------------------------------------
Accuracy Score: 0.815
751 Iteration:
------------------------------------
Accuracy Score: 0.8405
752 Iteration:
------------------------------------
Accuracy Score: 0.7775
753 Iteration:
------------------------------------
Accuracy Score: 0.851
754 Iteration:
------------------------------------
Accuracy Score: 0.843
755 Iteration:
------------------------------------
Accuracy Score: 0.7615
756 Iteration:
------------------------------------
Accuracy Score: 0.8305
757 Iteration:
------------------------------------
Accuracy Score: 0.744
758 Iteration:
------------------------------------
Accuracy Score: 0.8005
759 Iteration:
------------------------------------
Accuracy Score: 0.7815
760 Iteration:
------------------------------------
Accuracy Score: 0.801
761 Iteration:
------------------------------------
Accuracy Score: 0.8385
762 Iteration:
------------------------------------
Accuracy Score: 0.791
763 Iteration:
------------------------------------
Accuracy Score: 0.8045
764 Iteration:
------------------------------------
Accuracy Score: 0.843
765 Iteration:
------------------------------------
Accuracy Score: 0.8375
766 Iteration:
------------------------------------
Accuracy Score: 0.8165
767 Iteration:
------------------------------------
Accuracy Score: 0.821
768 Iteration:
------------------------------------
Accuracy Score: 0.7795
769 Iteration:
------------------------------------
Accuracy Score: 0.8105
770 Iteration:
------------------------------------
Accuracy Score: 0.8295
771 Iteration:
------------------------------------
Accuracy Score: 0.805
772 Iteration:
------------------------------------
Accuracy Score: 0.783
773 Iteration:
------------------------------------
Accuracy Score: 0.789
774 Iteration:
------------------------------------
Accuracy Score: 0.759
775 Iteration:
------------------------------------
Accuracy Score: 0.8205
776 Iteration:
------------------------------------
Accuracy Score: 0.843
777 Iteration:
------------------------------------
Accuracy Score: 0.8
778 Iteration:
------------------------------------
Accuracy Score: 0.7935
779 Iteration:
------------------------------------
Accuracy Score: 0.848
780 Iteration:
------------------------------------
Accuracy Score: 0.8395
781 Iteration:
------------------------------------
Accuracy Score: 0.7785
782 Iteration:
------------------------------------
Accuracy Score: 0.826
783 Iteration:
------------------------------------
Accuracy Score: 0.7855
784 Iteration:
------------------------------------
Accuracy Score: 0.8155
785 Iteration:
------------------------------------
Accuracy Score: 0.829
786 Iteration:
------------------------------------
Accuracy Score: 0.825
787 Iteration:
------------------------------------
Accuracy Score: 0.746
788 Iteration:
------------------------------------
Accuracy Score: 0.8355
789 Iteration:
------------------------------------
Accuracy Score: 0.7525
790 Iteration:
------------------------------------
Accuracy Score: 0.7715
791 Iteration:
------------------------------------
Accuracy Score: 0.852
792 Iteration:
------------------------------------
Accuracy Score: 0.7895
793 Iteration:
------------------------------------
Accuracy Score: 0.8085
794 Iteration:
------------------------------------
Accuracy Score: 0.7835
795 Iteration:
------------------------------------
Accuracy Score: 0.8385
796 Iteration:
------------------------------------
Accuracy Score: 0.7905
797 Iteration:
------------------------------------
Accuracy Score: 0.7675
798 Iteration:
------------------------------------
Accuracy Score: 0.7825
799 Iteration:
------------------------------------
Accuracy Score: 0.7745
800 Iteration:
------------------------------------
Accuracy Score: 0.827
801 Iteration:
------------------------------------
Accuracy Score: 0.8345
802 Iteration:
------------------------------------
Accuracy Score: 0.7685
803 Iteration:
------------------------------------
Accuracy Score: 0.826
804 Iteration:
------------------------------------
Accuracy Score: 0.775
805 Iteration:
------------------------------------
Accuracy Score: 0.8265
806 Iteration:
------------------------------------
Accuracy Score: 0.8225
807 Iteration:
------------------------------------
Accuracy Score: 0.845
808 Iteration:
------------------------------------
Accuracy Score: 0.755
809 Iteration:
------------------------------------
Accuracy Score: 0.809
810 Iteration:
------------------------------------
Accuracy Score: 0.777
811 Iteration:
------------------------------------
Accuracy Score: 0.727
812 Iteration:
------------------------------------
Accuracy Score: 0.8035
813 Iteration:
------------------------------------
Accuracy Score: 0.794
814 Iteration:
------------------------------------
Accuracy Score: 0.82
815 Iteration:
------------------------------------
Accuracy Score: 0.7655
816 Iteration:
------------------------------------
Accuracy Score: 0.8065
817 Iteration:
------------------------------------
Accuracy Score: 0.8185
818 Iteration:
------------------------------------
Accuracy Score: 0.811
819 Iteration:
------------------------------------
Accuracy Score: 0.7685
820 Iteration:
------------------------------------
Accuracy Score: 0.8135
821 Iteration:
------------------------------------
Accuracy Score: 0.79
822 Iteration:
------------------------------------
Accuracy Score: 0.8215
823 Iteration:
------------------------------------
Accuracy Score: 0.841
824 Iteration:
------------------------------------
Accuracy Score: 0.8245
825 Iteration:
------------------------------------
Accuracy Score: 0.828
826 Iteration:
------------------------------------
Accuracy Score: 0.7845
827 Iteration:
------------------------------------
Accuracy Score: 0.8185
828 Iteration:
------------------------------------
Accuracy Score: 0.7875
829 Iteration:
------------------------------------
Accuracy Score: 0.797
830 Iteration:
------------------------------------
Accuracy Score: 0.835
831 Iteration:
------------------------------------
Accuracy Score: 0.8305
832 Iteration:
------------------------------------
Accuracy Score: 0.8325
833 Iteration:
------------------------------------
Accuracy Score: 0.8275
834 Iteration:
------------------------------------
Accuracy Score: 0.8435
835 Iteration:
------------------------------------
Accuracy Score: 0.835
836 Iteration:
------------------------------------
Accuracy Score: 0.787
837 Iteration:
------------------------------------
Accuracy Score: 0.735
838 Iteration:
------------------------------------
Accuracy Score: 0.826
839 Iteration:
------------------------------------
Accuracy Score: 0.7815
840 Iteration:
------------------------------------
Accuracy Score: 0.839
841 Iteration:
------------------------------------
Accuracy Score: 0.79
842 Iteration:
------------------------------------
Accuracy Score: 0.7425
843 Iteration:
------------------------------------
Accuracy Score: 0.7995
844 Iteration:
------------------------------------
Accuracy Score: 0.8315
845 Iteration:
------------------------------------
Accuracy Score: 0.819
846 Iteration:
------------------------------------
Accuracy Score: 0.827
847 Iteration:
------------------------------------
Accuracy Score: 0.805
848 Iteration:
------------------------------------
Accuracy Score: 0.8235
849 Iteration:
------------------------------------
Accuracy Score: 0.777
850 Iteration:
------------------------------------
Accuracy Score: 0.788
851 Iteration:
------------------------------------
Accuracy Score: 0.8195
852 Iteration:
------------------------------------
Accuracy Score: 0.8175
853 Iteration:
------------------------------------
Accuracy Score: 0.771
854 Iteration:
------------------------------------
Accuracy Score: 0.8085
855 Iteration:
------------------------------------
Accuracy Score: 0.806
856 Iteration:
------------------------------------
Accuracy Score: 0.789
857 Iteration:
------------------------------------
Accuracy Score: 0.831
858 Iteration:
------------------------------------
Accuracy Score: 0.8295
859 Iteration:
------------------------------------
Accuracy Score: 0.8215
860 Iteration:
------------------------------------
Accuracy Score: 0.82
861 Iteration:
------------------------------------
Accuracy Score: 0.839
862 Iteration:
------------------------------------
Accuracy Score: 0.8235
863 Iteration:
------------------------------------
Accuracy Score: 0.8185
864 Iteration:
------------------------------------
Accuracy Score: 0.811
865 Iteration:
------------------------------------
Accuracy Score: 0.7375
866 Iteration:
------------------------------------
Accuracy Score: 0.8315
867 Iteration:
------------------------------------
Accuracy Score: 0.7645
868 Iteration:
------------------------------------
Accuracy Score: 0.743
869 Iteration:
------------------------------------
Accuracy Score: 0.831
870 Iteration:
------------------------------------
Accuracy Score: 0.804
871 Iteration:
------------------------------------
Accuracy Score: 0.7695
872 Iteration:
------------------------------------
Accuracy Score: 0.7585
873 Iteration:
------------------------------------
Accuracy Score: 0.7655
874 Iteration:
------------------------------------
Accuracy Score: 0.7495
875 Iteration:
------------------------------------
Accuracy Score: 0.8265
876 Iteration:
------------------------------------
Accuracy Score: 0.84
877 Iteration:
------------------------------------
Accuracy Score: 0.8395
878 Iteration:
------------------------------------
Accuracy Score: 0.8655
879 Iteration:
------------------------------------
Accuracy Score: 0.801
880 Iteration:
------------------------------------
Accuracy Score: 0.848
881 Iteration:
------------------------------------
Accuracy Score: 0.8405
882 Iteration:
------------------------------------
Accuracy Score: 0.791
883 Iteration:
------------------------------------
Accuracy Score: 0.821
884 Iteration:
------------------------------------
Accuracy Score: 0.77
885 Iteration:
------------------------------------
Accuracy Score: 0.8275
886 Iteration:
------------------------------------
Accuracy Score: 0.821
887 Iteration:
------------------------------------
Accuracy Score: 0.8015
888 Iteration:
------------------------------------
Accuracy Score: 0.7335
889 Iteration:
------------------------------------
Accuracy Score: 0.819
890 Iteration:
------------------------------------
Accuracy Score: 0.822
891 Iteration:
------------------------------------
Accuracy Score: 0.8335
892 Iteration:
------------------------------------
Accuracy Score: 0.841
893 Iteration:
------------------------------------
Accuracy Score: 0.847
894 Iteration:
------------------------------------
Accuracy Score: 0.81
895 Iteration:
------------------------------------
Accuracy Score: 0.834
896 Iteration:
------------------------------------
Accuracy Score: 0.801
897 Iteration:
------------------------------------
Accuracy Score: 0.79
898 Iteration:
------------------------------------
Accuracy Score: 0.835
899 Iteration:
------------------------------------
Accuracy Score: 0.824
900 Iteration:
------------------------------------
Accuracy Score: 0.8235
901 Iteration:
------------------------------------
Accuracy Score: 0.777
902 Iteration:
------------------------------------
Accuracy Score: 0.8165
903 Iteration:
------------------------------------
Accuracy Score: 0.817
904 Iteration:
------------------------------------
Accuracy Score: 0.816
905 Iteration:
------------------------------------
Accuracy Score: 0.8325
906 Iteration:
------------------------------------
Accuracy Score: 0.804
907 Iteration:
------------------------------------
Accuracy Score: 0.778
908 Iteration:
------------------------------------
Accuracy Score: 0.8285
909 Iteration:
------------------------------------
Accuracy Score: 0.815
910 Iteration:
------------------------------------
Accuracy Score: 0.8505
911 Iteration:
------------------------------------
Accuracy Score: 0.8225
912 Iteration:
------------------------------------
Accuracy Score: 0.8205
913 Iteration:
------------------------------------
Accuracy Score: 0.803
914 Iteration:
------------------------------------
Accuracy Score: 0.7865
915 Iteration:
------------------------------------
Accuracy Score: 0.8355
916 Iteration:
------------------------------------
Accuracy Score: 0.735
917 Iteration:
------------------------------------
Accuracy Score: 0.825
918 Iteration:
------------------------------------
Accuracy Score: 0.7705
919 Iteration:
------------------------------------
Accuracy Score: 0.816
920 Iteration:
------------------------------------
Accuracy Score: 0.8305
921 Iteration:
------------------------------------
Accuracy Score: 0.7805
922 Iteration:
------------------------------------
Accuracy Score: 0.827
923 Iteration:
------------------------------------
Accuracy Score: 0.794
924 Iteration:
------------------------------------
Accuracy Score: 0.7935
925 Iteration:
------------------------------------
Accuracy Score: 0.7705
926 Iteration:
------------------------------------
Accuracy Score: 0.768
927 Iteration:
------------------------------------
Accuracy Score: 0.808
928 Iteration:
------------------------------------
Accuracy Score: 0.797
929 Iteration:
------------------------------------
Accuracy Score: 0.804
930 Iteration:
------------------------------------
Accuracy Score: 0.845
931 Iteration:
------------------------------------
Accuracy Score: 0.8255
932 Iteration:
------------------------------------
Accuracy Score: 0.8305
933 Iteration:
------------------------------------
Accuracy Score: 0.797
934 Iteration:
------------------------------------
Accuracy Score: 0.819
935 Iteration:
------------------------------------
Accuracy Score: 0.817
936 Iteration:
------------------------------------
Accuracy Score: 0.7595
937 Iteration:
------------------------------------
Accuracy Score: 0.802
938 Iteration:
------------------------------------
Accuracy Score: 0.8245
939 Iteration:
------------------------------------
Accuracy Score: 0.782
940 Iteration:
------------------------------------
Accuracy Score: 0.7785
941 Iteration:
------------------------------------
Accuracy Score: 0.8055
942 Iteration:
------------------------------------
Accuracy Score: 0.789
943 Iteration:
------------------------------------
Accuracy Score: 0.757
944 Iteration:
------------------------------------
Accuracy Score: 0.802
945 Iteration:
------------------------------------
Accuracy Score: 0.8185
946 Iteration:
------------------------------------
Accuracy Score: 0.7825
947 Iteration:
------------------------------------
Accuracy Score: 0.7965
948 Iteration:
------------------------------------
Accuracy Score: 0.7275
949 Iteration:
------------------------------------
Accuracy Score: 0.7895
950 Iteration:
------------------------------------
Accuracy Score: 0.8085
951 Iteration:
------------------------------------
Accuracy Score: 0.7985
952 Iteration:
------------------------------------
Accuracy Score: 0.813
953 Iteration:
------------------------------------
Accuracy Score: 0.845
954 Iteration:
------------------------------------
Accuracy Score: 0.838
955 Iteration:
------------------------------------
Accuracy Score: 0.775
956 Iteration:
------------------------------------
Accuracy Score: 0.842
957 Iteration:
------------------------------------
Accuracy Score: 0.804
958 Iteration:
------------------------------------
Accuracy Score: 0.842
959 Iteration:
------------------------------------
Accuracy Score: 0.813
960 Iteration:
------------------------------------
Accuracy Score: 0.8075
961 Iteration:
------------------------------------
Accuracy Score: 0.737
962 Iteration:
------------------------------------
Accuracy Score: 0.832
963 Iteration:
------------------------------------
Accuracy Score: 0.794
964 Iteration:
------------------------------------
Accuracy Score: 0.7995
965 Iteration:
------------------------------------
Accuracy Score: 0.8005
966 Iteration:
------------------------------------
Accuracy Score: 0.7005
967 Iteration:
------------------------------------
Accuracy Score: 0.7665
968 Iteration:
------------------------------------
Accuracy Score: 0.837
969 Iteration:
------------------------------------
Accuracy Score: 0.79
970 Iteration:
------------------------------------
Accuracy Score: 0.7975
971 Iteration:
------------------------------------
Accuracy Score: 0.746
972 Iteration:
------------------------------------
Accuracy Score: 0.762
973 Iteration:
------------------------------------
Accuracy Score: 0.8395
974 Iteration:
------------------------------------
Accuracy Score: 0.757
975 Iteration:
------------------------------------
Accuracy Score: 0.789
976 Iteration:
------------------------------------
Accuracy Score: 0.8165
977 Iteration:
------------------------------------
Accuracy Score: 0.8135
978 Iteration:
------------------------------------
Accuracy Score: 0.831
979 Iteration:
------------------------------------
Accuracy Score: 0.8495
980 Iteration:
------------------------------------
Accuracy Score: 0.8365
981 Iteration:
------------------------------------
Accuracy Score: 0.801
982 Iteration:
------------------------------------
Accuracy Score: 0.8305
983 Iteration:
------------------------------------
Accuracy Score: 0.8195
984 Iteration:
------------------------------------
Accuracy Score: 0.7145
985 Iteration:
------------------------------------
Accuracy Score: 0.7735
986 Iteration:
------------------------------------
Accuracy Score: 0.8355
987 Iteration:
------------------------------------
Accuracy Score: 0.812
988 Iteration:
------------------------------------
Accuracy Score: 0.8345
989 Iteration:
------------------------------------
Accuracy Score: 0.7465
990 Iteration:
------------------------------------
Accuracy Score: 0.821
991 Iteration:
------------------------------------
Accuracy Score: 0.7935
992 Iteration:
------------------------------------
Accuracy Score: 0.8065
993 Iteration:
------------------------------------
Accuracy Score: 0.8275
994 Iteration:
------------------------------------
Accuracy Score: 0.776
995 Iteration:
------------------------------------
Accuracy Score: 0.846
996 Iteration:
------------------------------------
Accuracy Score: 0.801
997 Iteration:
------------------------------------
Accuracy Score: 0.822
998 Iteration:
------------------------------------
Accuracy Score: 0.7905
999 Iteration:
------------------------------------
Accuracy Score: 0.799
import scipy
np_predictions = np.array(predictions)
ensemble_pred, count = scipy.stats.mode(np_predictions,axis=0) # Compute most frequent prediction for each row
print("Ensemble Accuracy Score:",accuracy_score(ensemble_pred,y_test))Ensemble Accuracy Score: 0.862