A publisher accepts only significant studies at 5%, correctly conducted independently. Must only 5% of the published…
Quant ResearchLevel: IntermediatePublication selection
A publisher accepts only significant studies at 5%, correctly conducted independently. Must only 5% of the published conclusions be false?
Hints
- Reverse conditional probabilities carefully.
- Consider a world with no real effects at all.
Answer
No. The 5% controls rejection probability conditional on a true null, not the false fraction conditional on publication. If every tested hypothesis is null and only rejections are published, all published discoveries are false. Selection and the base rate matter.
Source and licence: Lehman, Leighton & Meyer — Mathematics for Computer Science (2018-06-06) · CC-BY-SA-3.0 · answer: Quantoros editorial