# Benchmarks, leaderboards and contamination

Source: https://onco.cc/terms/leaderboard-benchmark/  
OnCo record `leaderboard-benchmark` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

A benchmark is a fixed dataset and task on which models are compared, and a leaderboard ranks them; both mislead once the test data has been seen in training or the metric no longer tracks the goal.

## Summary

A benchmark runs a standard set of tests to assess relative performance (Wikipedia). Cancer AI has few: TCGA tasks, drug-response splits from DrEval and IMPROVE, the Virtual Cell Challenge. Because TCGA slides and profiles are in the pretraining data of most foundation models, downstream TCGA scores are at risk of contamination, and leaderboard rank on a saturated task says little about clinical use. Publishing the exact split and seed is what makes a head-to-head comparison mean anything.

## Fields

- Kind: Term
- Last checked: 2026-09-24
- Also known as: benchmark; benchmarks; benchmark dataset; leaderboard; leaderboards; benchmark contamination; public benchmark; head-to-head comparison
- Tags: cansim-terms

## Notes

- Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/leaderboard.

## Sources

- Wikipedia: https://en.wikipedia.org/wiki/Benchmark_(computing)
- Wikipedia: https://en.wikipedia.org/wiki/Benchmark_(computing)

## Connected records

- terms: [Cancer AI vocabulary (CanSim terms map)](https://onco.cc/terms/cancer-ai-vocabulary/), [Data leakage in model evaluation](https://onco.cc/terms/data-leakage/), [Drug-response baselines and frameworks: mean-drug floor, LightGBM, DrEval, IMPROVE, DeepTTA](https://onco.cc/terms/drug-response-baselines/), [Reproducibility and negative results](https://onco.cc/terms/reproducibility/), [Virtual cell models and in-silico perturbation screens](https://onco.cc/terms/virtual-cell-models/)

---
JSON: https://onco.cc/api/v1/entities/leaderboard-benchmark.json