Imprecise findings in COVID-19 drug trials could steer clinicians away from innovative treatments

As the COVID-19 pandemic progresses, researchers are reporting findings of randomized trials comparing standard care with care augmented by experimental drugs. The trials have small sample sizes, so estimates of treatment effects are statistically imprecise.

In a new working paper published by the National Bureau of Economic Research (NBER) today (June 8), Northwestern University economist Charles Manski, along with co-author Aleksey Tetenov of the University of Geneva, argue that the manner in which medical research articles present findings of trials assessing COVID-19 drugs may inappropriately give the impression that new treatments are not effective.

In “Statistical Decision Properties of Imprecise Trials Assessing COVID-19 Drugs,” the authors state that seeing imprecision, clinicians reading research articles may find it difficult to decide when to treat patients with experimental drugs. 

A conventional practice when comparing standard care and an innovation is to choose the innovation only if the estimated treatment effect is positive and statistically significant. 

“The requirement for conventional statistical significance creates a status quo bias in favor of what is called ‘standard care’ and against innovative treatments,” said Manski, Board of Trustees Professor in Economics and a faculty fellow with the University’s Institute for Policy Research. “My co-author and I think this is a serious problem that may affect treatment of patients.”  

Manski is available for comment. He can be reached at cfmanski@northwestern.edu.