Assessment Finds More Systematic Reviews than Primary Studies on COVID-19

Wasteful repetition may be flooding literature about severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) with more systematic reviews than primary research, according to an assessment of literature on imaging findings of children with coronavirus disease 2019 (COVID-19).

The research letter, published in JAMA Network Open, noted that systematic reviews may be appropriate in some instances, but identifying and synthesizing evidence has become difficult with the explosion of reports.

Led by Giordano Pérez-Gaxiola, MD, MSc, of Sinaloa Pediatric Hospital’s Cochrane Associate Centre in Culiacan, Mexico, the cross-sectional study sought to map 1 question:What is the spectrum and frequency of imaging findings in children with COVID-19?

“Although there is a large amount of daily scientific publications about COVID-19, this research highlights that there is waste and duplication,” Giordano Pérez-Gaxiola, MD, MSc, of Sinaloa Pediatric Hospital’s Cochrane Associate Centre in Culiacan, Mexico, told Contagion®. “It also shows that most reviews were not accompanied by a previously registered protocol, that most reviews were quickly outdated, and none of the reviews had included all the available evidence.”

Investigators searched the Living Overview of Evidence platform, which includes databases, preprint servers, trial registries, and other sources, for systematic reviews describing imaging findings in children with COVID-19 and primary studies including at least 30 children for the period until Sept. 1.

“We were surprised that there were more systematic reviews than primary studies trying to answer a very specific clinical question,” Pérez-Gaxiola said.

The investigators identified 25 systematic reviews, including 17 primary studies, and fewer than a quarter of them (24%) had previously been registered. Each review identified 1 to 9 primary studies, and 11 eligible primary studies were not identified by any of the reviews.

The absence of primary studies in systematic reviews could be explained by the fast pace of the research or limitations to search strategies. One review included 9 of 28 eligible articles. Investigators determined that of those excluded, 4 were likely missed during the review process, and 15 were published after their search.

“Unfortunately, this study doesn’t provide answers about imaging findings of children with COVID-19,” Pérez-Gaxiola said. “It rather highlights wasteful duplication of evidence synthesis trying to answer that question. But clinicians should be aware and cautious about reading and applying information of systematic reviews that were not registered and that may have missed information.”

PROSPERO, an international database of systematic, reviews allows investigators to identify ongoing systematic reviews and avoid duplication. The research letter said replicating reviews to verify findings or extend or narrow a question may be appropriate, but the massive level of duplication seen in COVID-19 literature is “unjustified and may be unethical.”

“We encourage authors to register the protocols for their systematic reviews,” Pérez-Gaxiola said. “We believe journal editors should request all systematic reviews authors a Prospero registration (or registration in other similar platforms).”

Managing the flood of COVID-19 research has been a challenge since the World Health Organization declared the disease a pandemic in March. In April, the National Institutes of Health launched a public-private partnership to speed the development of therapeutics and vaccines to respond to the disease, calling on the “full power of the biomedical research enterprise” to come together.

Since then, the scientific community has been balancing the unprecedented global response to the pandemic with the need to preserve research integrity. Recent reports in Nature and The Atlantic detail the torrent of COVID-19 literature, including thousands of studies shared through preprints.