{"entity":{"id":"calibration","kind":"term","name":"Calibration: reliability diagrams and the Brier score","aka":["model calibration","calibrated probabilities","reliability diagram","reliability diagrams","calibration curve","Brier score","integrated Brier score","IBS","miscalibration","bootstrap-calibrated reliability"],"tldr":"A calibrated model's predicted 30 percent risk really happens about 30 percent of the time; a reliability diagram plots predicted against observed, and the Brier score measures the squared gap.","summary":"In statistics, calibration in this sense refers to whether predicted probabilities match observed frequencies (Wikipedia). The Brier score is a strictly proper scoring rule equal to the mean squared error of predicted probabilities (Wikipedia), and Graf and colleagues extended it to censored survival data as the integrated Brier score over time. Discrimination (C-index, AUC) and calibration are independent: a model can rank patients well and still overstate every risk, and calibration is the property that breaks first under domain shift, which is why a model's reliability on a new platform has to be re-measured.","asOf":"2026-09-24","wikipedia":"https://en.wikipedia.org/wiki/Calibration_(statistics)","links":[{"label":"Wikipedia: Brier score","url":"https://en.wikipedia.org/wiki/Brier_score"},{"label":"Graf et al., Assessment and comparison of prognostic classification schemes for survival data (Statistics in Medicine 1999)","url":"https://doi.org/10.1002/(SICI)1097-0258(19990915/30)18:17/18<2529::AID-SIM274>3.0.CO;2-5"},{"label":"Wikipedia","url":"https://en.wikipedia.org/wiki/Calibration_(statistics)"}],"tags":["cansim-terms"],"related":["cancer-ai-vocabulary"],"cancers":[],"sections":[],"technologies":[],"targets":[],"drugs":[],"companies":[],"institutions":[],"pathways":[],"terms":["concordance-index","roc-auc","external-validation","uncertainty-quantification"],"trials":[],"people":[],"bottlenecks":[],"keyPapers":[],"journals":[],"dependsOn":[],"notes":["Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/calibration."],"provenance":{"editedBy":"OnCo CanSim terms wave (Wikipedia summaries, standards and project pages, GDC and FDA pages, Europe PMC)","editedOn":"2026-09-24","note":"CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme"},"category":"Methods and models"},"route":"/terms/calibration/","neighbours":{"term":[{"id":"analytical-vs-clinical-validation","kind":"term","name":"Analytical versus clinical validation","route":"/terms/analytical-vs-clinical-validation/"},{"id":"concordance-index","kind":"term","name":"C-index (concordance index), Harrell's and Uno's","route":"/terms/concordance-index/"},{"id":"cancer-ai-vocabulary","kind":"term","name":"Cancer AI vocabulary (CanSim terms map)","route":"/terms/cancer-ai-vocabulary/"},{"id":"conformal-prediction","kind":"term","name":"Conformal prediction","route":"/terms/conformal-prediction/"},{"id":"external-validation","kind":"term","name":"External validation","route":"/terms/external-validation/"},{"id":"prognosis-risk-percentile","kind":"term","name":"Risk score and risk percentile","route":"/terms/prognosis-risk-percentile/"},{"id":"roc-auc","kind":"term","name":"ROC-AUC, PR-AUC and time-dependent AUC","route":"/terms/roc-auc/"},{"id":"uncertainty-quantification","kind":"term","name":"Uncertainty quantification and confidence gates","route":"/terms/uncertainty-quantification/"}]}}