# Calibration: reliability diagrams and the Brier score

Source: https://onco.cc/terms/calibration/  
OnCo record `calibration` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

A calibrated model's predicted 30 percent risk really happens about 30 percent of the time; a reliability diagram plots predicted against observed, and the Brier score measures the squared gap.

## Summary

In statistics, calibration in this sense refers to whether predicted probabilities match observed frequencies (Wikipedia). The Brier score is a strictly proper scoring rule equal to the mean squared error of predicted probabilities (Wikipedia), and Graf and colleagues extended it to censored survival data as the integrated Brier score over time. Discrimination (C-index, AUC) and calibration are independent: a model can rank patients well and still overstate every risk, and calibration is the property that breaks first under domain shift, which is why a model's reliability on a new platform has to be re-measured.

## Fields

- Kind: Term
- Last checked: 2026-09-24
- Also known as: model calibration; calibrated probabilities; reliability diagram; reliability diagrams; calibration curve; Brier score; integrated Brier score; IBS; miscalibration; bootstrap-calibrated reliability
- Tags: cansim-terms

## Notes

- Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/calibration.

## Sources

- Wikipedia: https://en.wikipedia.org/wiki/Calibration_(statistics)
- Wikipedia: Brier score: https://en.wikipedia.org/wiki/Brier_score
- Graf et al., Assessment and comparison of prognostic classification schemes for survival data (Statistics in Medicine 1999): https://doi.org/10.1002/(SICI)1097-0258(19990915/30)18:17/18<2529::AID-SIM274>3.0.CO;2-5
- Wikipedia: https://en.wikipedia.org/wiki/Calibration_(statistics)

## Connected records

- terms: [Analytical versus clinical validation](https://onco.cc/terms/analytical-vs-clinical-validation/), [C-index (concordance index), Harrell's and Uno's](https://onco.cc/terms/concordance-index/), [Cancer AI vocabulary (CanSim terms map)](https://onco.cc/terms/cancer-ai-vocabulary/), [Conformal prediction](https://onco.cc/terms/conformal-prediction/), [External validation](https://onco.cc/terms/external-validation/), [Risk score and risk percentile](https://onco.cc/terms/prognosis-risk-percentile/), [ROC-AUC, PR-AUC and time-dependent AUC](https://onco.cc/terms/roc-auc/), [Uncertainty quantification and confidence gates](https://onco.cc/terms/uncertainty-quantification/)

---
JSON: https://onco.cc/api/v1/entities/calibration.json