A confidence score is only useful for routing work if it means what it says. Calibration measures the gap between stated confidence and observed accuracy — and closes it.
This guide defines calibration precisely, shows how to read a reliability diagram and compute expected calibration error, gives the published evidence that modern networks and chat-tuned language models are overconfident, compares Platt scaling, isotonic regression, histogram binning and temperature scaling, and shows how calibrated scores set human review thresholds.
Updated · 9 sources
Confidence calibration is the agreement between a model’s stated confidence and how often it is actually right. A model is calibrated if, of all the predictions it scores 0.8, about 80% are correct. Calibration is measured with reliability diagrams and expected calibration error (ECE), and corrected after training by fitting a map from raw scores to observed accuracy — Platt scaling, isotonic regression, histogram binning or temperature scaling.
Two models with the same accuracy can differ completely in how trustworthy their scores are. Accuracy asks “how often is it right?”; calibration asks “does its confidence tell you when?”.
Whether higher scores go to correct predictions more often than lower ones. Measured by accuracy at a cut-off, or by the area under the ROC curve.
Whether a score of p means a probability p of being right. Measured by reliability diagrams, ECE and the Brier score.
Group predictions into confidence bins, compare each bin’s average confidence with its accuracy, and summarise the gaps. Guo et al. used 15 equal-width bins.
Expected calibration error (ECE), M bins over n values:
ECE = Σ |B_m| / n × | accuracy(B_m) − confidence(B_m) |
Illustrative: 1,000 extracted values, three bins
bin values mean score accuracy gap weight term
0.5–0.7 200 0.62 0.55 0.07 × 0.20 = 0.014
0.7–0.9 300 0.81 0.74 0.07 × 0.30 = 0.021
0.9–1.0 500 0.96 0.90 0.06 × 0.50 = 0.030
ECE = 0.065
Overconfident in every bin: scores promise
about 6.5 points more accuracy than delivered.Read diagrams with the counts. Guo et al. note a reliability diagram does not show how many samples fall in each bin. A wild bar over ten values matters less than a small gap over ten thousand.
Guo et al. found calibration got worse as networks grew deeper and wider, even as accuracy rose, and that a one-parameter fix closed most of the gap. OpenAI’s GPT-4 report shows the same drift for a language model after post-training.
All four methods fit a map from raw score to probability on data the model was not trained on. They differ in how flexible that map is — and so in how much data they need.
A logistic curve fitted to the scores. Corrects S-shaped distortion; stable with little data.
Divides a network’s logits by one learned temperature. Keeps the ranking; in Guo et al.’s tests “surprisingly” often the most effective method.
Each score bin is replaced by its observed accuracy. Simple, but coarse and sensitive to bin edges.
A stepwise, never-decreasing map. Fixes any monotonic distortion; overfits when data is scarce.
How many labelled outcomes do you have for this score?
Calibration is a small amount of code and a large amount of discipline about data. Most failures come from calibrating on the wrong outcomes or never refitting.
Pair each score with whether the value turned out right — from labelled sets or reviewer decisions.
Fit the map on data the model never trained on; scikit-learn’s calibrator uses cross-validation for this.
One map per field, class or document type — miscalibration differs between them.
Compare ECE and Brier score on held-out folds before and after; keep the old map if it is not better.
Refit on a rolling window as data drifts, and re-read the reliability diagram.
On Botminds each field’s raw confidence is mapped to how often values like it were actually right, using your reviewers’ own decisions — and the map sharpens as evidence grows.
each review adds evidence for the next fit
Curves refit daily or on demand over a rolling window, and a reliability diagram is there to read for every field. Read human in the loop, and see how calibrated thresholds drive straight-through processing.
Confidence calibration is how well a model’s predicted confidence matches its observed accuracy. If a calibrated model gives 100 predictions a confidence of 0.8, about 80 of them are right. Calibration is measured with reliability diagrams and ECE and fixed after training with a calibration map.
ECE groups predictions into confidence bins and takes the average gap between each bin’s accuracy and its average confidence, weighted by how many predictions fall in the bin. Zero means perfect calibration on that binning.
ECE measures calibration alone, through bins. The Brier score is the mean squared error between predicted probability and the 0/1 outcome, so it rewards both calibration and the ability to separate right from wrong.
Platt scaling fits a two-parameter logistic curve and works well with little data. Isotonic regression fits any monotonic shape but overfits when data is scarce; Niculescu-Mizil and Caruana found Platt better below roughly 200–1,000 calibration cases.
A one-parameter version of Platt scaling for neural networks: divide the logits by a learned temperature before the softmax. It changes confidence without changing which class is predicted.
Pre-trained models can be well calibrated on multiple-choice questions, but post-training can hurt it: GPT-4’s technical report shows ECE rising from 0.007 to 0.074 on an MMLU subset. Calibrate LLM confidence against your own reviewed outcomes before relying on it.
Bin predictions by confidence, plot each bin’s average confidence against its accuracy, and draw the diagonal. Bars below the line are overconfident. scikit-learn’s calibration_curve function computes the points.
Bring one process and its documents. We’ll show you the agents, the controls and the record, running on your data.