# Statistical significance (P values, alpha, multiplicity)

Source: https://onco.cc/terms/statistical-significance/  
OnCo record `statistical-significance` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

A result is 'statistically significant' when it would be unlikely (usually under 5%) to arise by chance if the treatment did nothing. It is a threshold, not a measure of benefit size: a tiny gain can pass it in a huge trial and a large one can miss it in a small trial, so ESMO and ASCO grade benefit separately.

## Summary

Trials are sized ('powered') to detect a pre-specified effect with 80-90% probability at a 5% two-sided alpha; testing several endpoints or several interim looks would inflate false positives unless alpha is split (allocated) or spent sequentially, so hierarchical testing means a secondary endpoint cannot be claimed if a higher one failed. 'Numerically better', 'trend' and 'nominal P' are phrases for results that did not meet the pre-set bar. Clinical meaningfulness is separate: ASCO and ESMO scales (ESMO-MCBS) grade the size of benefit, and a significant 6-week PFS gain may not matter to patients. Confidence intervals convey both size and uncertainty and are preferred to bare P values.

## Fields

- Kind: Term
- Last checked: 2026-09-09
- Also known as: statistically significant; statistical significance; not statistically significant; did not reach statistical significance; P value; p-value; P<0.05; P = 0.05; one-sided; two-sided; alpha spending; alpha allocation; type I error; multiplicity; multiple comparisons; hierarchical testing; gatekeeping; Bonferroni; powered; underpowered; sample size; statistical power; 80% power; 90% power; numerically; numerical improvement; trend toward; trend towards; clinically meaningful; clinical significance; sample sizes

## Sources

- Wikipedia: https://en.wikipedia.org/wiki/Statistical_significance
- Wikipedia: https://en.wikipedia.org/wiki/Statistical_significance

## Connected records

- terms: [Bayesian trial design](https://onco.cc/terms/bayesian-trial-design/), [Confidence interval](https://onco.cc/terms/confidence-interval/), [Futility analysis (stopped for futility)](https://onco.cc/terms/futility/), [Group sequential design, stopping rules and alpha spending](https://onco.cc/terms/group-sequential-design/), [Hazard ratio (HR)](https://onco.cc/terms/hazard-ratio/), [Interim analysis, readout and data cut-off](https://onco.cc/terms/interim-analysis/), [Non-inferiority trial](https://onco.cc/terms/non-inferiority/), [P-value](https://onco.cc/terms/p-value/), [Pre-specified vs post-hoc analysis](https://onco.cc/terms/prespecified-vs-post-hoc/), [Primary, secondary and co-primary endpoints](https://onco.cc/terms/primary-endpoint/), [Statistical power, sample size and re-estimation](https://onco.cc/terms/sample-size-re-estimation/), [Subgroup analysis (forest plots)](https://onco.cc/terms/subgroup-analysis/), [Why trials fail: underpowered, wrong endpoint, control arm drift, subgroup fishing, crossover](https://onco.cc/terms/trial-failure-modes/)
- fronts: [Drug Discovery Platforms](https://onco.cc/fronts/drug-discovery/)

---
JSON: https://onco.cc/api/v1/entities/statistical-significance.json