DISPUTE OVER COVID-19 DEATH RATE

Over the past two decades, John Ioannidis, a professor of medicine, epidemiology, and population health at Stanford University, has gained a reputation as a widely respected gadfly of global science. His work consistently highlights flaws in research methods, pushing scientists and physicians to be more rigorous in evaluating and applying evidence. Ioannidis' most famous paper, "Why Most Published Research Findings Are False," has been downloaded millions of times and cited in thousands of other studies since its publication in 2005. A 2010 profile in The Atlantic stated that "Ioannidis may be one of the most influential scientists alive." John Ioannidis with audience members outside an appearance at a TEDx event in Athens, Greece, in 2015. Kostas Limitsios/flickr Ioannidis' latest work, though, has sparked pushback from many of his colleagues, with some suggesting that the researcher may be succumbing to the very sloppiness he has spent his career fighting. Since mid-March, Ioannidis has been arguing that the fatality rate of Covid-19 may be much lower than initially feared — and that, as a result, current public health restrictions could be overly strict. Last week, a Stanford-led study of Covid-19 infection rates in Santa Clara County, California, which lists Ioannidis as an author, seemed to offer some of the most forceful backing for that argument. The paper suggests that the fatality rate for Covid-19 may be 0.2 percent or less — lower than many other estimates, and much closer to the seasonal flu. These arguments have earned Ioannidis widespread attention in conservative media, where many commentators are skeptical of the overall risk of Covid-19, and critical of the restrictions currently in place across much of the world. In the past week, though, the Stanford study, and a related effort in Los Angeles, have come under fierce criticism from prominent statisticians and infectious disease experts. Critics have argued that statistical sloppiness, inaccuracies in testing, and a skewed survey method likely warped the results. Some critics have also expressed surprise that Ioannidis was involved in the Santa Clara study at all. "My quick take is that something really odd is going on with Ioannidis," wrote Alexander Rubinsteyn, a geneticist and computational biologist at the University of North Carolina at Chapel Hill, in an email to Undark. Rubinsteyn suggested that Ioannidis may simply be "so attached to being the iconoclast that defies conventional wisdom that he's unintentionally doing horrible science." He added: "Pretty much no one with statistical acumen believes these studies." For his part, Ioannidis did not respond to multiple attempts, made over several days, to reach him for comment by email and phone. Shortly after this story was published, however, he emailed Undark to express gratitude to his critics. "We have been working very hard to address all the comments raised by many colleagues and I am always grateful to receive constructive criticism," he said. "While we received thousands of congratulations for this work, I cherish far more the comments of the scientists who criticized our work and I thank them deeply for their comments."

Σελίδα 2 από 5 He's so attached to being the iconoclast that defies conventional wisdom that he's unintentionally doing horrible science. Alexander Rubinsteyn, University of North Carolina geneticist and computational biologist At issue is a single, elusive question: How many people infected with the SARS-CoV-2 virus actually die from the disease? Widely circulated figures early in the pandemic suggested that Covid-19 was killing as many as 3.4 percent of its victims, according to an estimate from the World Health Organization. Such numbers are derived by simply dividing the number of dead by the total number of reported cases. The problem, of course, is that many people who get Covid-19 are asymptomatic and never realize they are sick. Others may be unable to get a test, amid widespread testing shortages. As a result, one of the lingering questions of the pandemic has been how many people actually have the disease. If a larger number of people are getting infected but presenting with symptoms so mild or nonexistent that they never feel the need to be tested, then the overall infection fatality rate — that is, the percentage of infected people who die — will be considerably lower. Such downward adjustments in the death rate of a pandemic are entirely common, as researchers and clinicians get a better grasp on how a contagion behaves. And of course, if the fatality rate for Covid-19 drifts closer to that of the common flu, some people might question whether strict lockdowns are warranted. The question, though, is whether there is enough good information on hand yet to make that case. In an essay published by STAT in mid-March, Ioannidis himself bemoaned the lack of testing and data for Covid-19. But he also argued that it was possible that, if we had not discovered the new virus, the additional deaths it has caused may have gone largely unnoticed. And in the absence of clear information about the fatality rate of Covid-19, he suggested, long-term lockdowns could be too hasty. "If we decide to jump off the cliff, we need some data to inform us about the rationale of such an action," he wrote. Not all epidemiologists agreed. "Ioannidis doing his schtick about standards of evidence is not helpful. Everyone knows we're acting with little or partial information," wrote Yale epidemiologist Gregg Gonsalves in a Twitter thread. "We all want better data. But if you don't have it. Do you sit and wait for it in a pandemic?" Marc Lipsitch, director of Harvard's Center for Communicable Disease Dynamics, suggested that dwelling on the ambiguities of data collection might be missing the point at a time when, amid a surge in Covid-19 deaths, corpses were piling up in churches in Italy. The debate intensified late last week when Ioannidis and 16 colleagues released a draft of their new paper estimating the prevalence of Covid-19 cases in Santa Clara County, California. Using Facebook ads, the team recruited 3,330 people in Santa Clara County, California, and invited them to receive a serology test for Covid-19. Instead of looking for the presence of the virus itself, a serology test identifies the unique antibodies that the immune system produces when it encounters the SARS-CoV-2 virus. Those antibodies linger and, as a result, the tests can identify people who were sick and then recovered, and it picks up on invisible, asymptomatic infections, too. The research team tested people at three drive-in sites in the county, staffed by volunteers — including many Stanford graduate students — who had responded to emails offering a chance "to get involved with an unprecedented study that will help direct ongoing local and national Covid-19 policy!" Of those 3,330 people, 50 tested positive. Extrapolating from that result to the entire population of the county, the researchers estimated that 50- to 85-times as many people had been infected in the county as previously estimated. Given this, they posited, the actual fatality rate of Covid-19 in the county was probably closer to 0.12 to 0.20 percent of cases. The typical rate for the flu is around 0.10 percent, although estimates vary, and some suggest an even lower fatality rate.

Σελίδα 3 από 5 Still, if the results of the Santa Clara study were true, this would suggest that the SARS-CoV-2 virus is considerably less deadly than once feared, and that lockdowns, perhaps, are necessary only for the most vulnerable populations — particularly elderly people. The study was released, like many other Covid-19 studies, as a preprint, meaning it had not yet received peer review vetting from other scientists. And it quickly came under criticism from other researchers, who eviscerated its methods on Twitter and in an online forum. (Two senior authors on the paper, Eran Bendavid and Jay Bhattacharya, did not respond to requests for comment.) "The Stanford study was not a true random sample," said Prabhat Jha, an epidemiologist at the University of Toronto, in an email to Undark. In addition, he said, "their testing was flawed, so they might have many false positives." In particular, Jha and others have questioned whether recruiting samples from a Facebook ad, rather than something closer to random selection, may have skewed the results, perhaps by selecting for people who had already shown Covid-19 symptoms and wanted to see if they had antibodies, or who have a higher socioeconomic status. The preprint acknowledges these limitations as well. Because of the potential for false positives, there's also a chance the 50 positive tests could have been mostly statistical noise, rather than a meaningful result. If that were the case, the true number of infections in the county would be lower, and in turn the infection fatality rate would be higher — and perhaps more in line with what other influential studies, with estimates closer to 0.5 percent and above, have suggested. Still, the Stanford team's findings are not complete outliers, and they have pointed to other research from Germany and the Netherlands that found similar results. The issue, suggested Andrew Gelman, a statistician at Columbia University who criticized the research team for statistical errors in a post on his popular blog, is that the team is getting out in front of what the data are showing. "It's not like I'm saying they're wrong, and someone else is saying they're right. It kind of comes out that way, but it's like, yeah, I think their conclusions were a bit strong," Gelman said. "I think they have reasons for believing what they believe beyond what's in this paper. Like they have their own scientific understanding of the world, and so basically they're coming into it saying, 'Hey, we believe this. We think that this disease, this virus has a very low infection fatality rate,' and then they gather data and the data are consistent with that. "If you have reason to believe that story, these data support it," Gelman added. "If you don't have such a good reason to believe that story, you can see that the data are kind of ambiguous." The propriety of the Stanford analysis was called into further question on Friday when BuzzFeed News reported that Bhattacharya's wife had sent a message to a middle school's private email list to recruit families for the study. It included the claim that the testing would help determine if they could "return to work without fear," raising additional concerns of bias in the sample population. The Stanford data also do not seem to match up with observations in other, more hard-hit parts of the country. In New York City, for example, an estimated 15,400 people have died from Covid-19, according to figures from the city's health department. While death rates may vary from place to place, if the fatality rate of Covid-19 is as low as 0.12 percent, as the Stanford study authors claim, this would suggest that more than 12.5 million people in New York City have already been infected with Covid-19, even though only 8.3 million people live there. A. Marm Kilpatrick, an infectious disease ecologist at UC-Santa Cruz and a vocal critic of the study, summed up his thoughts in an email: "They completely ignore the data from N.Y."

Σελίδα 4 από 5 We all want better data. But if you don't have it. Do you sit and wait for it in a pandemic? Yale epidemiologist Gregg Gonsalves, in a Twitter post In the week after the study's release, Ioannidis has made the rounds on Fox News, appearing on Tucker Carlson's show and on "The Ingraham Angle," hosted by Laura Ingraham. (Bhattacharya, a former research fellow at the politically conservative Hoover Institution, has also appeared on Carlson's show talking about Covid-19 death rates). Last Sunday, Ioannidis was also on the Fox News show "Life, Liberty, and Levin," to talk about why he thought the Covid-19 risk had been overstated. As host Mark Levin criticized the widespread lockdowns and questioned the media's reporting on Covid-19 data, Ioannidis praised the public health response. "We had to base our decisions on whatever we had, and I think we did the right thing to act decisively and urgently," he said. But he also challenged estimates of the disease's danger. "The best data that we have now suggests that it's not one out of 30 or one out of a hundred people who get infected who will die. It's probably in the range of one in a thousand," Ioannidis told Levin, citing numbers that would put Covid-19 in the same fatality range as a typical case of influenza. "We also know that there's some types of people who are at much higher risk than others." "Most of the population has minimal risk," he added, "in the range of dying while you're driving from home to work and back." When Undark ran that statement by several public health experts, all of them expressed skepticism. That "claim might well be true," Jha said, "but their data are not sufficiently definitive to make their claim." In a telephone interview, Cecília Tomori, director of global public health and community health at the Johns Hopkins School of Nursing, echoed that sentiment. "The question of making sure we have the best evidence to make decisions is a valid one," she said. "But I think that there's something else when we question some of the empirical reality of how awful this disease has been. And I think that's really what's going on." It's not like I'm saying they're wrong, and someone else is saying they're right. It kind of comes out that way, but it's like, yeah, I think their conclusions were a bit strong. Andrew Gelman, Columbia University statistician Along with Ioannidis' piece in STAT, Bendavid and Bhattacharya had published their own op-ed in The Wall Street Journal, before conducting the study, questioning whether the Covid-19 quarantines were overreactions. This has led some critics to question the authors' motives. "The fact that Ioannidis and another author had published two articles…arguing that the threat of Covid-19 was overblown, and then published a study that was done in a way that was likely to produce a spuriously high estimate of seroprevalence (and thus a low [infection fatality rate]) was worrisome," wrote Kilpatrick. Accusations of biased sampling and sloppy methods have also disappointed some Stanford students who volunteered to collect samples for the study. Maria Filsinger Interrante, an MD/Ph.D. student at Stanford, is currently working long hours in the lab helping develop potential vaccines for the virus. But she spent 10 hours on a Saturday in early April helping to draw blood for the Stanford study. "If you want to prove to the world that the seroprevalence rate is higher, those are things that you would do in designing your study," she said. "Were they done intentionally? I have no idea, and I would hope not, because I think the people who ran the studies are probably great people and good scientists. But it seems to fit with getting them the answer that they wanted, and that just makes me sad."

Σελίδα 5 από 5 More data will soon be available. Bhattacharya told Science earlier this week that he was preparing a response to criticisms. And a rash of new, similar antibody studies promise to soon offer a more complete picture of the fatality rates of Covid-19 — and possibly even prove Ioannidis and his colleagues right. In his email to Undark, Ioannidis also suggested more information on his team's methodologies was forthcoming — and that it will prove their interpretation of the data to be sound. "We will be posting an updated version of the study preprint soon with far more data, additional analyses, more detail on methods and responses, that show that our data and inferences are robust," he said, adding: "I hope that scientists will be able to focus calmly on the science and not on a blame game or a clash of political agendas. I clearly do not have any political agenda, and the least plausible characterization would be that I am 'conservative,' given my track record," Ioannidis said. "My key focus remains to try to learn what I don't know and diminish my ignorance." In the meantime, critics of Ioannidis say the evidence remains murky, leaving them to wonder whether a venerated scientist known for his learned and very often justified skepticism of others' scientific work has become blind to his own methodological flaws. Travis Gerke, an epidemiologist at the Moffitt Cancer Center and a visiting scientist at the Harvard T.H. Chan School of Public Health, said that he read Ioannidis' work while a graduate student. Recently he has wondered if Ioannidis should consider re-reading his own most famous paper, "Why Most Published Research Findings are False." "His current study fits most of the high-risk criteria for falsehood that he outlines, such as publishing in a really hot scientific field with few corroborating studies, using a small bias sample, [and] reporting provocative findings in a politically charged arena," Gerke said. "If you just go through his own work," Gerke added, "he seems to be breaking all his own rules."