document.write( "<A HREF=/cgi-bin/jump-to-question.mpl?question=1178478>Question 1178478</A>: <I><SPAN STYLE=\"word-break: break-all;\"> This question requires a confusion matrix. I am unsure how to calculate the accuracy given the percentages. How do I go about this? Thanks. :) \r<BR>\n" );
document.write( "\n" );
document.write( "Both model A and model B achieves 80% accuracy on the balanced population, but changes to 64% and 96% accuracy respectively on the true unbalanced population. If the population is more skewed with 5% positive and 95% negative, what are the accuracies for each model?(The training population is 1,000 and the confusion matrix for model A is [500 200] <BR>\n" );
document.write( "                      [0   300] <BR>\n" );
document.write( "The confusion matrix for model B is [300 0]<BR>\n" );
document.write( "                                [200 500]).\r<BR>\n" );
document.write( "<BR>\n" );
document.write( "\n" );
document.write( " </SPAN>\n" );
document.write( "</I><HR><TABLE width=100%><TR><TD><B><A HREF=http://www.algebra.com/>Algebra.Com's</A> Answer #850338 by </B><A HREF=/tutors/aboutme.mpl?userid=CPhill><B>CPhill(1959)</B></A><img src=\"/images/stars/stars-12.gif\" alt=\"\" >&nbsp;<A HREF=/tutors/aboutme.mpl?userid=CPhill><IMG SRC=http://www.algebra.com/images/aboutme-small.gif BORDER=0 alt=\"About Me\" VALIGN=MIDDLE></A>&nbsp;<IMG SRC=http://www.algebra.com/cgi-bin/show-face.mpl?user=CPhill><BR>You can <A HREF=\"/cgi-bin/embed-solution.mpl?show=1&solution=850338\">put this solution on YOUR website!</A><BR><SPAN STYLE=\"word-break: break-all;\"> Absolutely! Let's break down how to calculate accuracy with percentages and then apply it to your confusion matrices and the skewed population scenario.\r<BR>\n" );
document.write( "\n" );
document.write( "**Understanding Accuracy and Confusion Matrices**\r<BR>\n" );
document.write( "\n" );
document.write( "* **Accuracy:** The overall correctness of a model. It's calculated as (True Positives + True Negatives) / Total Predictions.<BR>\n" );
document.write( "* **Confusion Matrix:** A table that visualizes the performance of a classification model. It typically looks like this:\r<BR>\n" );
document.write( "\n" );
document.write( "    ```<BR>\n" );
document.write( "    |                | Predicted Positive | Predicted Negative |<BR>\n" );
document.write( "    |----------------|--------------------|--------------------|<BR>\n" );
document.write( "    | Actual Positive | True Positive (TP) | False Negative (FN) |<BR>\n" );
document.write( "    | Actual Negative | False Positive (FP) | True Negative (TN) |<BR>\n" );
document.write( "    ```\r<BR>\n" );
document.write( "\n" );
document.write( "**Calculating Accuracy from a Confusion Matrix**\r<BR>\n" );
document.write( "\n" );
document.write( "1.  **Sum the Diagonals:** Add the True Positives (TP) and True Negatives (TN).<BR>\n" );
document.write( "2.  **Sum All Cells:** Add all the values in the matrix (TP + FP + FN + TN). This is the total number of predictions.<BR>\n" );
document.write( "3.  **Divide:** Divide the sum of the diagonals by the sum of all cells.\r<BR>\n" );
document.write( "\n" );
document.write( "**Applying It to Your Models**\r<BR>\n" );
document.write( "\n" );
document.write( "* **Model A:**<BR>\n" );
document.write( "    * Confusion Matrix:<BR>\n" );
document.write( "        ```<BR>\n" );
document.write( "        [500 200]<BR>\n" );
document.write( "        [0   300]<BR>\n" );
document.write( "        ```<BR>\n" );
document.write( "    * TP = 500, TN = 300, FP = 200, FN = 0<BR>\n" );
document.write( "    * Accuracy = (500 + 300) / (500 + 200 + 0 + 300) = 800 / 1000 = 0.8 or 80%\r<BR>\n" );
document.write( "\n" );
document.write( "* **Model B:**<BR>\n" );
document.write( "    * Confusion Matrix:<BR>\n" );
document.write( "        ```<BR>\n" );
document.write( "        [300 0]<BR>\n" );
document.write( "        [200 500]<BR>\n" );
document.write( "        ```<BR>\n" );
document.write( "    * TP = 300, TN = 500, FP = 0, FN = 200<BR>\n" );
document.write( "    * Accuracy = (300 + 500) / (300 + 0 + 200 + 500) = 800 / 1000 = 0.8 or 80%\r<BR>\n" );
document.write( "\n" );
document.write( "**The Unbalanced Population (64% and 96%)**\r<BR>\n" );
document.write( "\n" );
document.write( "You've mentioned that the accuracies change to 64% and 96% on the \"true unbalanced population.\" This means the original confusion matrices, while based on a balanced dataset, are being evaluated against a new dataset that is unbalanced.\r<BR>\n" );
document.write( "\n" );
document.write( "**The Highly Skewed Population (5% Positive, 95% Negative)**\r<BR>\n" );
document.write( "\n" );
document.write( "Now, let's calculate the accuracies for the even more skewed population.\r<BR>\n" );
document.write( "\n" );
document.write( "1.  **Assume a Total Population:** Since you have a 1,000 training population, for simplicity, let's assume we are still working with a population of 1000.<BR>\n" );
document.write( "2.  **Calculate Positive and Negative Counts:**<BR>\n" );
document.write( "    * Positive: 1000 * 0.05 = 50<BR>\n" );
document.write( "    * Negative: 1000 * 0.95 = 950<BR>\n" );
document.write( "3.  **Adjust the Confusion Matrices Based on the Proportions:**<BR>\n" );
document.write( "    * Since the original confusion matrix totals are 1000, we will keep the ratios of the True positives, false positives, false negatives and true negatives.<BR>\n" );
document.write( "    * Model A:<BR>\n" );
document.write( "        * TP: 500/700 * 50 = 35.71, round to 36.<BR>\n" );
document.write( "        * FP: 200/500 * 950 = 380.<BR>\n" );
document.write( "        * FN: 0.<BR>\n" );
document.write( "        * TN: 300/500 * 950 = 570.<BR>\n" );
document.write( "    * Model B:<BR>\n" );
document.write( "        * TP: 300/500 * 50 = 30.<BR>\n" );
document.write( "        * FP: 0<BR>\n" );
document.write( "        * FN: 200/500 * 50 = 20.<BR>\n" );
document.write( "        * TN: 500/500 * 950 = 950.<BR>\n" );
document.write( "4.  **Calculate Accuracies:**<BR>\n" );
document.write( "    * Model A:<BR>\n" );
document.write( "        * Accuracy = (36 + 570) / 1000 = 606 / 1000 = 0.606 or 60.6%<BR>\n" );
document.write( "    * Model B:<BR>\n" );
document.write( "        * Accuracy = (30 + 950) / 1000 = 980 / 1000 = 0.98 or 98%\r<BR>\n" );
document.write( "\n" );
document.write( "**Therefore:**\r<BR>\n" );
document.write( "\n" );
document.write( "* For the 5% positive, 95% negative population:<BR>\n" );
document.write( "    * Model A's accuracy is approximately 60.6%.<BR>\n" );
document.write( "    * Model B's accuracy is 98%.\r<BR>\n" );
document.write( "\n" );
document.write( "**Key Takeaway:**\r<BR>\n" );
document.write( "\n" );
document.write( "* When dealing with unbalanced datasets, accuracy can be misleading. Model B appears to perform much better in the skewed population.<BR>\n" );
document.write( "* It is critical to be aware of the underlying population distribution when evaluating the model.<BR>\n" );
document.write( " </SPAN>\n" );
document.write( "</TD></TR></TABLE>\n" );