Accuracy

1 · What it is

In machine learning, accuracy is the share of predictions a model got right: the number correct divided by the total. Get 40 of 50 test cases right and your accuracy is 80%. For a yes-or-no model, the right answers are true positives (a correct yes) and true negatives (a correct no). The wrong ones are false positives, where the model says yes to a real no, and false negatives, where it misses a real yes. So accuracy is (TP + TN) ÷ (TP + TN + FP + FN). The same four counts, laid out as a confusion matrix, also give related scores such as precision, recall and the F1 score, which combines those two.

The everyday word is looser. Machine learning ties accuracy, precision and recall to precise formulas, which are narrower than how people use those words day to day. In medicine, a new test’s diagnostic accuracy is about how closely its results line up with an accepted reference standard, and the US Food and Drug Administration (FDA) describes it with pairs of numbers such as sensitivity and specificity. Language model benchmarks report plain accuracy too. MMLU is a test of multiple-choice questions covering 57 tasks, and it measures the share a model answers correctly. On MMLU, guessing at random scores 25%.

Accuracy misleads most on imbalanced data, where one class is far more common than the other. Such data is the norm: among card payments, fraud can be fewer than 1 in 1,000. Google’s glossary imagines a city where it snows on 25 days per century. A model that predicts no snow every day scores 99.93%, yet it tells you nothing about when snow will come. In general, a model that always picks the larger class scores that class’s share of the test set. Balanced accuracy, built to avoid inflated scores on imbalanced data, averages the recall of each class, meaning the share of that class’s real cases the model got right. If a high score came only from the imbalance, balanced accuracy falls to chance. On a balanced test set it equals ordinary accuracy.