A confusion matrix calculator takes the four counts a binary classifier produces — true positives, false positives, false negatives and true negatives — and derives every standard evaluation metric from them. The arithmetic is not difficult. What is difficult is knowing which of those metrics to quote, because they can point in opposite directions on the same model, and the one people default to is the one most likely to mislead.
Arb Digital publishes a free tools library for practitioners, and this page sits in its AI section alongside the AI model comparison tool for comparing models on cost and capability and the sample size calculator for working out how large a test set needs to be before any of these numbers are stable. This page assumes you already have the four counts and want them turned into metrics you can defend.
What This Confusion Matrix Calculator Does
Enter TP, FP, FN and TN and it returns precision, recall, specificity, accuracy, balanced accuracy, F1, a weighted F-beta of your choosing, negative predictive value, false positive rate, prevalence and the Matthews correlation coefficient. Every one is computed from the same four numbers, with each denominator guarded so that a zero row or column returns zero rather than an error.
The headline figure is F1 rather than accuracy, deliberately. Under the headline sits the comparison that most evaluation write-ups omit: what accuracy a model would achieve by always predicting the majority class. That single line is often enough to tell you whether a reported accuracy figure means anything.
The bars show precision, recall, specificity, balanced accuracy and MCC on the same scale, because the shape of that group is more informative than any single number. A model with high specificity and low recall looks completely different from one with the reverse, and both can report the same accuracy.
How to Use It
- Fix which class is positive. Every metric on this page except accuracy and MCC changes if you swap the labels. The positive class is normally the rare or costly one — fraud, disease, defect, churn.
- Enter the four counts. TP and TN are the correct predictions, FP is a false alarm, FN is a miss. If your tooling prints the matrix as a grid, check its row and column order before transcribing.
- Choose an F-beta weighting. Leave it at β = 1 for the standard F1. Use β = 0.5 when false positives are the expensive error and β = 2 when false negatives are.
- Read the bars, not just the hero. Precision against recall and specificity against recall are the two comparisons that describe the model's actual behaviour.
- Check accuracy against the baseline line. If the gap is small, quote MCC or balanced accuracy instead and say why.
The Formula / How It's Calculated
Each metric is a ratio over a different slice of the matrix:
Precision = TP ÷ (TP + FP) — of everything flagged positive, how much was. Recall, or true positive rate = TP ÷ (TP + FN) — of everything genuinely positive, how much was caught. Specificity, or true negative rate = TN ÷ (TN + FP). Accuracy = (TP + TN) ÷ total. Balanced accuracy = (recall + specificity) ÷ 2. Negative predictive value = TN ÷ (TN + FN). False positive rate = FP ÷ (FP + TN). Prevalence = (TP + FN) ÷ total.
F-beta = (1 + β²) × precision × recall ÷ (β² × precision + recall), which reduces to the harmonic mean of precision and recall when β = 1. MCC = (TP × TN − FP × FN) ÷ √((TP+FP)(TP+FN)(TN+FP)(TN+FN)). These definitions match those set out in the scikit-learn documentation on metrics and scoring: quantifying the quality of predictions, which is the reference implementation most teams are checking against in practice.
Worked example, matching the values the page loads with. TP = 85, FP = 15, FN = 25, TN = 875, so the total is 1,000. Precision is 85 ÷ 100 = 0.8500. Recall is 85 ÷ 110 = 0.7727. Specificity is 875 ÷ 890 = 0.9831. Accuracy is 960 ÷ 1,000 = 0.9600. Balanced accuracy is (0.7727 + 0.9831) ÷ 2 = 0.8779. F1 is 2 × 0.85 × 0.7727 ÷ (0.85 + 0.7727) = 1.3136 ÷ 1.6227 = 0.8095. For MCC, the numerator is (85 × 875) − (15 × 25) = 74,375 − 375 = 74,000, and the denominator is √(100 × 110 × 890 × 900) = √8,811,000,000 ≈ 93,867, giving 0.7884. Prevalence is 110 ÷ 1,000 = 11%, so a model that always predicted negative would score 89.0% accuracy.
Why Accuracy Misleads on Imbalanced Classes
This is the section worth reading twice, because accuracy is the metric that gets reported and it is the one that breaks first.
Accuracy counts every correct prediction equally and divides by the total. When one class dominates, the majority class dominates that sum, and a model can score highly while being useless on the class you actually care about. Google's machine learning crash course makes the point directly in its treatment of classification accuracy, recall and precision: on a heavily imbalanced dataset where one class appears 1% of the time, a model that always predicts negative scores 99% accuracy despite being of no practical use.
The worked example above shows a milder version. Accuracy is 96.0%, which sounds excellent, until you notice the always-negative baseline scores 89.0%. The model has bought seven percentage points over doing nothing. Meanwhile recall is 0.7727 — it misses roughly a quarter of the actual positives — and that is the number that matters if the positive class is a fraudulent transaction or a failing component.
Three metrics survive imbalance better. Balanced accuracy averages recall and specificity, so each class contributes equally regardless of size, and the always-negative model scores exactly 0.5. MCC uses all four cells and only approaches 1 when the model does well on both classes; a degenerate always-negative classifier gives an MCC of 0, which is exactly the message you want. F1 ignores true negatives entirely, so it cannot be inflated by a large majority class, though it is asymmetric — it will change if you swap which class is positive, and MCC will not.
The practical rule: always report prevalence alongside accuracy, or report a metric that is not sensitive to it. An accuracy figure quoted without the class balance is not interpretable.
Precision and Recall Trade Against Each Other
Most classifiers produce a score rather than a label, and the label comes from comparing that score to a threshold. Moving the threshold moves precision and recall in opposite directions, which means a confusion matrix describes one operating point, not the model.
Raise the threshold and the model becomes more cautious: fewer positive predictions, so fewer false positives and higher precision, but more misses and lower recall. Lower it and the reverse happens. Nothing about the model changed — only where you chose to cut. This is why comparing two models by their confusion matrices alone is unsafe unless both were thresholded on a comparable basis.
Which direction you should move depends on the cost of each error, and those costs are rarely equal. A spam filter that marks a real invoice as spam has done more damage than one that lets a spam message through, so precision on the spam class matters more. A screening test that misses a genuine case has done more damage than one that flags an extra case for review, so recall matters more. The F-beta selector on this page exists to encode that asymmetry in a single number: β = 0.5 weights precision more, β = 2 weights recall more.
What the Four Cells Are Really Telling You
Reading a confusion matrix well means looking at the errors, not the successes. The two error cells describe different failures with different remedies.
A high false positive count relative to TP means the model is over-firing. It usually shows up as low precision and a high false positive rate, and it is often the cheapest problem to fix — raise the threshold, or add features that separate the near-miss negatives from real positives. Its operational cost is wasted review capacity.
A high false negative count means the model is not detecting. It shows up as low recall while precision may look fine, which is why a precision-only report can hide it completely. It is harder to fix, because the signal for those cases may simply not be present in the features you have.
Two more cells worth naming: negative predictive value answers "when the model says negative, how often is it right?", which is the question a user of the model actually asks when they receive a negative result. And prevalence is not a model metric at all — it describes the data. It belongs in the report because every predictive-value metric depends on it, and a model validated at one prevalence will behave differently when deployed at another.
Multi-Class Matrices and Cross-Validation
This calculator handles the binary case. Multi-class problems produce an n×n matrix, and the standard approach is to reduce it to a set of binary problems — one class against the rest — and compute these same metrics for each. Averaging them is where the choices reappear: macro-averaging treats every class equally regardless of size, micro-averaging pools the counts first and therefore weights by class frequency, and weighted averaging sits between the two. Macro and micro F1 can differ substantially on imbalanced data, so which one is quoted needs stating.
A separate caution applies to any single matrix: it is one sample. Metrics computed on a small test set have wide confidence intervals, and a 0.02 difference in F1 between two models may be noise. Cross-validation and reporting a spread rather than a point estimate are the standard remedies, and the sample size calculator will tell you how many examples you need for a given precision. The standard error calculator and Bayes theorem calculator are useful for the surrounding statistics, particularly the prevalence-dependence of predictive values.
Arb Digital designs and builds fast, dependency-free web interfaces around models — evaluation dashboards, calculators and tools that load instantly and rank.
See Web Design Services Talk to Arb DigitalCommon Mistakes to Avoid
- Reporting accuracy without prevalence — 96% means one thing at 50% prevalence and something else entirely at 11%, and the reader cannot tell which without the class balance.
- Transcribing the matrix in the wrong orientation — libraries differ on whether rows are actual or predicted, and swapping FP with FN inverts precision and recall.
- Comparing models across different thresholds — a confusion matrix is one operating point, so two matrices are only comparable if both were cut on a comparable basis.
- Optimising F1 when the errors have unequal costs — F1 weights precision and recall equally, which is almost never true of the underlying business or clinical costs.
- Treating a small test set's metrics as precise — a difference of a couple of hundredths on a few hundred examples is usually noise, not an improvement.
Related Free Tools From Arb Digital
Use the sample size calculator to size a test set, the standard error calculator for uncertainty around a proportion, and the Bayes theorem calculator for the prevalence-dependence of predictive values. The AI model comparison tool covers model selection on cost and capability, and the percentage calculator handles the supporting arithmetic. Everything else is in the free online tools hub.
Frequently Asked Questions
Because it counts every correct prediction equally, so the majority class dominates. On a dataset where the positive class appears 1% of the time, a model that always predicts negative scores 99% accuracy while detecting nothing. Always report prevalence alongside accuracy, or use balanced accuracy or MCC instead.
Precision asks what share of the items flagged positive really were positive, so it is about false alarms. Recall asks what share of the genuinely positive items were caught, so it is about misses. Moving a classifier's threshold raises one and lowers the other.
When you care about performance on both classes and want a figure that does not change if you swap which class is positive. MCC uses all four cells of the matrix and returns 0 for a degenerate classifier, whereas F1 ignores true negatives and is asymmetric between the classes.
That the predictions carry no information about the actual labels — the performance of random guessing or of always predicting one class. MCC runs from -1 for perfectly inverted predictions through 0 for no relationship to +1 for perfect agreement.
They weight precision and recall unequally. F0.5 gives precision more weight, which suits cases where a false positive is the expensive error, such as a spam filter. F2 gives recall more weight, which suits cases where a miss is the expensive error, such as a screening step feeding manual review.
Indirectly. Reduce the n-by-n matrix to one-against-the-rest binary matrices, run each through this calculator, and then decide how to average. Macro-averaging weights every class equally while micro-averaging weights by class frequency, and the two can differ substantially on imbalanced data.
Normally the rare or costly class — fraud, disease, defect, churn. Precision, recall, F1, specificity and negative predictive value all change when you swap the labels, so the choice must be stated alongside the metrics. Accuracy and MCC are the two that do not change.
This tool performs statistical arithmetic on counts you supply. It does not evaluate whether a model is fit for any particular purpose, and metrics from a classifier used in a medical, financial or safety context must be interpreted by someone qualified in that domain.
