# Overfitting

Source: https://onco.cc/terms/overfitting/  
OnCo record `overfitting` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Overfitting is when a model learns the chance patterns and quirks of the data used to build it, so it looks accurate there and does worse in new patients.

## Summary

When a study measures many candidate features in few patients, some features will separate good from poor outcomes by chance alone, and a signature chosen and scored in the same patients overstates how well it predicts. Michiels and colleagues reanalysed the seven largest published microarray studies of cancer prognosis using many random training and validation splits: the lists of predictive genes depended strongly on which patients were in the training set, the published results were over-optimistic because validation had been inadequate, and five of the seven studies did not classify patients better than chance. Venet and colleagues compared 47 published breast cancer outcome signatures with signatures made of random genes: 28 of them (60 per cent) were not significantly better predictors than random signatures of the same size, and more than 90 per cent of random signatures of more than 100 genes were significant outcome predictors, in part because much of the breast cancer transcriptome tracks proliferation. An outcome association alone does not establish that the biology named by a signature causes the outcome. Overfitting is not the same as bias in the data: a model can also be wrong because its development patients differ from the patients it is used on, and both lower real-world accuracy. Three habits hide the problem: scoring a model on the patients that built it, adjusting a threshold after seeing results, and relying on one favourable split. Repeated internal splits reduce optimism but do not test a new hospital, assay batch or population. The defence is a model locked before testing, evaluated in independent patients and compared with a simple baseline.

## Fields

- Kind: Term
- Last checked: 2026-11-09
- Also known as: overfitting; overfitted model; overfitted signature; optimistic accuracy

## Notes

- Reading guide (OnCo inference). A statistically significant outcome association does not by itself establish clinical usefulness. Choosing genes and a cutoff in the same patients measures fit; a locked score needs evaluation in separate patients and comparison with a simple baseline.

## Sources

- Michiels et al., Prediction of cancer outcome with microarrays: a multiple random validation strategy (Lancet 2005): https://doi.org/10.1016/S0140-6736(05)17866-0
- Venet et al., Most Random Gene Expression Signatures Are Significantly Associated with Breast Cancer Outcome (PLoS Computational Biology 2011): https://doi.org/10.1371/journal.pcbi.1002240

## Connected records

- terms: [Batch effects and harmonisation](https://onco.cc/terms/batch-effects/), [Cross-validation and stratified k-fold](https://onco.cc/terms/cross-validation/), [Data leakage in model evaluation](https://onco.cc/terms/data-leakage/), [External validation](https://onco.cc/terms/external-validation/), [Train, validation and test split discipline](https://onco.cc/terms/train-test-discipline/)

---
JSON: https://onco.cc/api/v1/entities/overfitting.json