ideasIdea
Automatically detect when a trial changes its outcomes after the fact
Trials sometimes quietly swap the outcome they promised to measure for one that looks better. Software can compare the registered plan with the published paper and flag the switch.
The COMPare project manually compared registered and published outcomes in five top journals and found widespread undisclosed switching. Registry entries and publications are now structured enough for automated comparison, and pre-specified statistical analysis plans are increasingly public. A continuously running monitor that flags outcome discrepancies in oncology trials, notifies journals and posts findings publicly would deter switching.
Hypothesis
Public flagging reduces undisclosed primary outcome switching in oncology trials from the roughly 30% documented in audits to under 10% within three years.
Rationale
COMPare showed that journals correct when confronted with evidence. Automation makes the confrontation systematic rather than sporadic.
What would test it
Build the monitor over ClinicalTrials.gov and PubMed; audit its precision manually on 200 trials; publish quarterly reports and measure the switching rate.
Maturity
early clinical
Who has to act
data
Cost to try
Small (under $1M)
Years to first evidence
1
Bottlenecks it attacks
- Failures are hidden · Negative trials, failed drugs and abandoned programmes are rarely published, so the same mistakes are repeated.
- Trial design, endpoints and cost · A phase 3 trial takes years and hundreds of millions of dollars, and often answers a question that has already moved on.