Abstract
Background
The COVID-19 pandemic has severely affected health systems and medical research worldwide but its impact on the global publication dynamics and non-COVID-19 research has not been measured. We hypothesized that the COVID-19 pandemic may have impacted the scientific production of non-COVID-19 research.
Methods
We conducted a comprehensive meta-research on studies (original articles, research letters and case reports) published between 01/01/2019 and 01/01/2021 in 10 high-impact medical and infectious disease journals (New England Journal of Medicine, Lancet, Journal of the American Medical Association, Nature Medicine, British Medical Journal, Annals of Internal Medicine, Lancet Global Health, Lancet Public Health, Lancet Infectious Disease and Clinical Infectious Disease). For each publication, we recorded publication date, publication type, number of authors, whether the publication was related to COVID-19, whether the publication was based on a case series, and the number of patients included in the study if the publication was based on a case report or a case series. We estimated the publication dynamics with a locally estimated scatterplot smoothing method. A Natural Language Processing algorithm was designed to calculate the number of authors for each publication. We simulated the number of non-COVID-19 studies that could have been published during the pandemic by extrapolating the publication dynamics of 2019 to 2020, and comparing the expected number to the observed number of studies.
Results
Among the 22,525 studies assessed, 6319 met the inclusion criteria, of which 1022 (16.2%) were related to COVID-19 research. A dramatic increase in the number of publications in general journals was observed from February to April 2020 from a weekly median number of publications of 4.0 (IQR: 2.8–5.5) to 19.5 (IQR: 15.8–24.8) (p < 0.001), followed afterwards by a pattern of stability with a weekly median number of publications of 10.0 (IQR: 6.0–14.0) until December 2020 (p = 0.045 in comparison with April). Two prototypical editorial strategies were found: 1) journals that maintained the volume of non-COVID-19 publications while integrating COVID-19 research and thus increased their overall scientific production, and 2) journals that decreased the volume of non-COVID-19 publications while integrating COVID-19 publications. We estimated using simulation models that the COVID pandemic was associated with a 18% decrease in the production of non-COVID-19 research. We also found a significant change of the publication type in COVID-19 research as compared with non-COVID-19 research illustrated by a decrease in the number of original articles, (47.9% in COVID-19 publications vs 71.3% in non-COVID-19 publications, p < 0.001). Last, COVID-19 publications showed a higher number of authors, especially for case reports with a median of 9.0 authors (IQR: 6.0–13.0) in COVID-19 publications, compared to a median of 4.0 authors (IQR: 3.0–6.0) in non-COVID-19 publications (p < 0.001).
Conclusion
In this meta-research gathering publications from high-impact medical journals, we have shown that the dramatic rise in COVID-19 publications was accompanied by a substantial decrease of non-COVID-19 research.
Meta-research registration
Similar content being viewed by others
Background
With a total of 3,541,881 deaths among 170,360,315 confirmed cases [1] as of May 31st, 2021, the coronavirus disease 2019 (COVID-19) pandemic has placed a strain on health systems worldwide. According to the Organization for Economic Co-operation and Development, an estimated 7 billion dollars, dedicated to COVID-19 research, were unlocked worldwide in the first 9 months of 2020 [2]. Linked to that, major collaborative efforts have been launched to urgently address COVID-19 related medical issues [3, 4], sometimes at the expense of non-COVID-19 research [5]. Some medical fields have experienced a decrease in funding allocation and publications [6], which had potentially affected patient care outside of COVID-19. For instance, the pandemic has seriously impacted cancer patients with treatment delays and reduced access to healthcare [7]. Similarly, it has had detrimental effects on organ allocation and transplantation worldwide [8], with a significant reduction in the number of transplanted organs per day, with disastrous consequences for patients whose lives depend on getting transplanted.
Overall, there has been a substantial redistribution of resources which has significantly impacted the non-COVID-19 medical research worldwide [9], including clinical trials [10, 11]. In addition, leading scientists have voiced concerns about science expediency [12] and the lowering of scientific standards [13, 14]. Together, these phenomena could have played a significant role on the dynamics of publication and worldwide medical research.
Moreover, recent research has reported a rising number of authors in COVID-19 publications [15], especially in case reports [16, 37]. Overall, innovations and discoveries have been brought and may help advancing medical research.
Although the worldwide population is progressively getting vaccinated, the COVID-19 pandemic still exerts a very harmful effect on many countries [38, 39]. In addition, the rise of many variants may challenge the efficacy of vaccines [40, 41] and delay the decrease in the number of cases and deaths. Accordingly, the medical research beyond COVID-19 is likely to be impacted in the long run. As such, we urge researchers to help continuing the evaluation on how health systems, medical research and resources are managed in pandemic time, as we attempted to accomplish in the present study.
Limitations
Several limitations should be acknowledged. First, due to the very high number of studies, the references were not assessed by two independent reviewers. However, after the screening completion, three reviewers randomly checked 20% of the references for each reviewer, and a second screening was performed if more than 3% inconsistencies were observed. Second, for the same reason, we had to restrict the analyses on the highly cited medical journals only. This may have induced a selection bias, as the editorial strategies might be different in lower-impact journals. However high-impact journals likely reflect and drive the trends in publications and are therefore relevant examples to analyze the impact of the pandemic on medical research. Third, we investigated the impact of the COVID-19 pandemic on non-COVID-19 research by focusing on the publication dynamics, the publication type, and the phenomenon of author multiplicity. Critically appraising all studies would have been ideal and enhanced our demonstration; but this would have been a gigantic work that cannot be accomplished in such study design.
Conclusion
To conclude, in this meta-research gathering original articles, research letters and case reports published in high-impact medical journals, we have shown the heterogeneity in the publication dynamics, and measured the impact of the COVID-19 pandemic in the production of non-COVID-19 studies. This study revealed how medical journals adapted to the pandemic, as some maintained the production of non-COVID-19 studies, and some decreased the production of non-COVID-19 studies. Last, we have identified an author multiplicity phenomenon in COVID-19 studies.
Availability of data and materials
The data of the manuscript are available upon reasonable request. To obtain the data, please contact the corresponding author.
Abbreviations
- COVID:
-
Coronavirus disease
- PRISMA:
-
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- NEJM:
-
New England Journal of Medicine
- JAMA:
-
Journal of American Medical Association
- BMJ:
-
British Medical Journal
- NLP:
-
Natural Language Processing
References
COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). 2020. (Accessed 19 May 2020, at https://coronavirus.jhu.edu/map.html.)
STIP COVID-19 WATCH. at https://stip.oecd.org/covid/). Accessed 31 May 2021.
Johns Hopkins University & Medicine - Coronavirus resource center. at https://coronavirus.jhu.edu/map.html). Accessed 31 May 2021.
Global coalition to accelerate COVID-19 clinical research in resource-limited settings. Lancet (London, England). 2020;395:1322–5.
Iacobucci G. Covid-19 makes the future of UK clinical research uncertain. BMJ (Clin Res ed). 2020;369:m1619.
Kourie HR, Eid R, Haddad F, Ghosn M, Sarkis DK. The future of cancer research after COVID-19 pandemic: recession? Fut Oncol (London, England). 2020;16:1493–5.
Nelson B. Covid-19 is shattering US cancer care. BMJ (Clin Res ed). 2020;369:m1544.
Loupy A, Aubert O, Reese PP, Bastien O, Bayer F, Jacquelinet C. Organ procurement and transplantation during the COVID-19 pandemic. Lancet (London, England). 2020;395:e95–e6.
Tuttle KR. Impact of the COVID-19 pandemic on clinical research. Nat Rev Nephrol. 2020;16(10):562–4. https://doi.org/10.1038/s41581-020-00336-9.
van Dorn A. COVID-19 and readjusting clinical trials. Lancet (London, England). 2020;396:523–4.
Wilkinson E. Dramatic drop in new cancer drug trials during the COVID-19 pandemic. Lancet Oncol. 2021;22:305.
London AJ, Kimmelman J. Against pandemic research exceptionalism. Science (New York, NY). 2020;368:476–7.
Glasziou PP, Sanders S, Hoffmann T. Waste in covid-19 research. BMJ. 2020;369:m1847.
Raynaud M, Zhang H, Louis K, et al. COVID-19-related medical research: a meta-research and critical appraisal. BMC Med Res Methodol. 2021;21:1.
Zdravkovic M, Berger-Estilita J, Zdravkovic B, Berger D. Scientific quality of COVID-19 and SARS CoV-2 publications in the highest impact medical journals during the early phase of the pandemic: a case control study. PLoS One. 2020;15:e0241826.
Clinical and virologic characteristics of the first 12 patients with coronavirus disease 2019 (COVID-19) in the United States. Nat Med. 2020, 26:861–8.
Zhang Y, **ao M, Zhang S, et al. Coagulopathy and antiphospholipid antibodies in patients with Covid-19. N Engl J Med. 2020;382:e38.
The rising trend in authorship. at https://thewinnower.com/papers/the-rising-trend-in-authorship. Accessed 31 May 2021.
Pintér A. Changing authorship patterns and publishing habits in the European journal of pediatric surgery: a 10-year analysis. Eur J Pediatr Surg. 2015;25:353–8.
Dang W, McInnes MD, Kielar AZ, Hong J. A comprehensive analysis of authorship in radiology journals. PLoS One. 2015;10:e0139005.
Ojerholm E, Swisher-McClure S. Authorship in radiation oncology: proliferation trends over 30 years. Int J Radiat Oncol Biol Phys. 2015;93:754–6.
Brunson JC, Wang X, Laubenbacher RC. Effects of research complexity and competition on the incidence and growth of coauthorship in biomedicine. PLoS One. 2017;12:e0173444.
Lee S, Bozeman B. The impact of research collaboration on scientific productivity. Soc Stud Sci. 2005;35:673–702.
COVID-19 Public Health Emergency of International Concern (PHEIC) Global research and innovation forum. at https://www.who.int/publications/m/item/covid-19-public-health-emergency-of-international-concern-(pheic)-global-research-and-innovation-forum). Accessed 31 May 2021.
Lipsitch M, Swerdlow DL, Finelli L. Defining the epidemiology of Covid-19 - studies needed. N Engl J Med. 2020;382:1194–6.
Brown A, Horton R. A planetary health perspective on COVID-19: a call for papers. Lancet (London, England). 2020;395:1099.
Coronavirus resource center. at https://jamanetwork.com/journals/jama/pages/coronavirus-alert). Accessed 31 May 2021.
Coronavirus (Covid-19). at https://www.nejm.org/coronavirus?query=main_nav_lg). Accessed 31 May 2021.
Berkwits M, Flanagin A, Bauchner H, Fontanarosa PB. The COVID-19 pandemic and the JAMA network. Jama. 2020;324:1159–60.
Ioannidis JPA. Coronavirus disease 2019: the harms of exaggerated information and non-evidence-based measures. Eur J Clin Investig. 2020;50:e13222.
Prudêncio M, Costa JC. Research funding after COVID-19. Nat Microbiol. 2020;5:986.
Dagens A, Sigfrid L, Cai E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the covid-19 pandemic: rapid review. BMJ (Clin Res ed). 2020;369:m1936.
Voysey M, Clemens SAC, Madhi SA, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet (London, England). 2021;397:99–111.
Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603–15.
Siemieniuk RA, Bartoszko JJ, Ge L, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ (Clin Res ed). 2020;370:m2980.
Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ (Clin Res ed). 2020;369:m1328.
The COVID-NMA initiative - A living map** and living systematic review of Covid-19 trials. at https://covid-nma.com) Accessed 31 May 2021.
Thiagarajan K. Why is India having a covid-19 surge? BMJ (Clin Res ed). 2021;373:n1124.
Burki T. No end in sight for the Brazilian COVID-19 crisis. Lancet. 2021;2(5):e180. https://doi.org/10.1016/S2666-5247(21)00095-1. Epub 2021 May 4.
Experts discuss COVID-19-variants and vaccine efficacy, immunosuppressed patients, and more. Jama. 2021. https://doi.org/10.1001/jama.2021.5938. Online ahead of print.
Gupta RK. Will SARS-CoV-2 variants of concern affect the promise of vaccines? Nat Rev Immunol. 2021;21(6):340–1. https://doi.org/10.1038/s41577-021-00556-5.
Acknowledgements
Not applicable.
Funding
This work was supported by INSERM–Action thématique incitative sur programme Avenir (ATIP-Avenir) and the Fondation Bettencourt Schueller. Both funding was used for the data collection, analysis and interpretation.
Author information
Authors and Affiliations
Contributions
Study design and lead: MR and AL; Screening and data extraction: MR, VG, KL, QD, ZD, AT, RB, NS, AG, CD, MD, SA, SN, EB, BR, AC, JD, SY, ECG, FT; Data analysis: MR, AL, VG, KL; Data interpretation: MR, VG, KL, QD, ZD, XJ, PR, AL; Figures design: MR, VG, KL, QD, ZD, PR, AL; Manuscript writing: MR, AL; Manuscript reviewing: MR, VG, KL, SA, QD, ZD, XJ, PR, AL. The corresponding author attests that all authors have read and approved the manuscript. The corresponding author attests that all authors meet the ICMJE authorship criteria.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1
: Methods: Search strategy. Figure 1. Weekly number of COVID-19 and non-COVID-19 publications with original data (all journals). Figure 2. Simulation of the number of unpublished non-COVID-19 studies. Figure 3 COVID-19 publications and the ratio of the number of authors to the number of patients. Figure 4 Publication type and COVID-19 (general journals). Figure 5 COVID-19 publications and the number of authors dynamics (general journals). Figure 6 COVID-19 publications and multiplicity of authors (general journals).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Raynaud, M., Goutaudier, V., Louis, K. et al. Impact of the COVID-19 pandemic on publication dynamics and non-COVID-19 research production. BMC Med Res Methodol 21, 255 (2021). https://doi.org/10.1186/s12874-021-01404-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12874-021-01404-9