Cox graphically describes all the ways that EPA experts misuse their discretionary power. They don’t properly define key terms, such as causal relationship. They rely on research that doesn’t formulate predictive, falsifiable hypotheses. They use arbitrary methods to select which scientific literature will provide the “weight of evidence.” They don’t require that research use publicly accessible, independently reproducible data and algorithms. They don’t require that researchers account transparently for how their research hypotheses, statistical models, and parameterizations may artificially limit their research results. They aren’t transparent about modeling uncertainties, measurement error, and the difference between modeling assumptions and empirical results.
The Irreproducibility Crisis
These aren’t minor matters. Modern science as a whole faces a major irreproducibility crisis of improper research techniques, groupthink, and a scientific culture biased toward producing positive results. John Ioannidis, who in 2005 was the first to begin reckoning with the full scope of this irreproducibility crisis, argues that half of modern published scientific research is probably wrong. Since then it has become clear that irreproducibility is at the heart of a major breakdown in Western scientific practice, as recently argued in The American Mind by Glenn Ellmers and J. Eric Wise. Reproducibility reformers are fighting to reduce scientists’ discretionary power in their labs. Cox is striking a major blow against an irresponsible establishment by extending reproducibility reforms to government policymaking….
Cox outlines new requirements that scientific experts must use standardized vocabulary and standardized means of selecting only relevant scientific literature with falsifiable hypotheses. This should only be the beginning. The government needs to charter a new committee that will draft standard requirements to make scientific expertise reproducible throughout its agencies. We need to focus on expertise standards without getting the question mixed up with pollution standards.