ideasIdea
Publish the disagreement between trial doctors and independent reviewers for every trial
Trials often have independent radiologists check whether tumours grew. How often they disagree with the treating doctors is a measure of how trustworthy the result is, and it is rarely published.
Blinded independent central review (BICR) is standard for progression-free survival in registrational trials, and discordance with investigator assessment is common. Regulators see the discordance data; the public usually does not. Requiring publication of the discordance rate, its direction by arm, and the sensitivity analysis in every trial report would let readers assess bias, and would create a dataset on when BICR matters and when it can be dropped to save cost.
Hypothesis
Systematic publication will show that differential discordance (favouring the experimental arm) is present in a minority of open-label trials but is concentrated in trials with small effect sizes, allowing targeted rather than universal BICR.
Rationale
Meta-analyses of BICR versus investigator assessment suggest overall agreement in hazard ratios but with informative exceptions; hiding the data prevents learning which trials are at risk.
What would test it
Adopt as a reporting requirement in journal and regulatory guidance; analyse the first 100 trials reporting discordance.
Maturity
early clinical
Who has to act
regulator
Cost to try
Small (under $1M)
Years to first evidence
1
Bottlenecks it attacks
- Biomarkers are not validated or standardised · Tests that decide who gets a drug are often not validated prospectively and are measured differently in every lab.
- Trial design, endpoints and cost · A phase 3 trial takes years and hundreds of millions of dollars, and often answers a question that has already moved on.