Preprints 2: Concerns and Implications
Following the previous post on introduction and benefits of preprints, this blog post will briefly introduce concerns on preprints and some implications to readers and authors. Missed the previous post? Check the post here.
Concern of preprints
Quality
While preprints play an unprecedented role in promoting timely dissemination of biomedical knowledge during COVID-19 pandemic, skeptics argued that they may pose a risk to quality-based academic publishing because they are not peer-reviewed before entering into the public domain.
Preprint servers do implement some other processes to facilitate quality control, for example, author control (author registration and endorsement), and screening based on completeness, relevance, plagiarism, appropriate language and contents, and the ethical and legal compliance (Xie et al., 2021). Also, 41% preprints were eventually published in peer-reviewed journals.
[Image curtesy of いらすとや]
The quality of preprints may not be as concerning as one may think. For example, In 2020, 606 preprints with DOIs by HKU researchers were posted (Notes 1 & 2). The citations received by the 606 preprints were analysed with Scite (https://scite.ai, Note 3). As of 23 May 2022, 10721 citation statements mentioned an item without providing evidence for its validity, 571 provided supporting evidence, and only 70 provided contrasting evidence (Figure 1). The ratio of supporting citations to contrasting ones was about 8.15:1.
Figure 1: Citations received by preprints by HKU scholars in 2020
Notes:
- Data were extracted from Dimensions (https://app.dimensions.ai/) on 20 May 2022.
- Items with different version numbers in DOIs were treated as different items.
- Scite uses a deep learning model to identify whether a citation provides supporting or disputing evidence, or simply mentions it. If you are interested in the tool, access it via HKU Libraries: http://find.lib.hku.hk/record=HKU_IZ61623882910003414; And check out this LibGuide: https://libguides.lib.hku.hk/scite
An analysis of all HKU publications with Scite can be found here: https://blog-sc.hku.hk/scite-dashboard-analysis-of-hku-publication-citations/
Nonetheless, preprints are not peer-reviewed – and need to be examined with caution. An unfortunate case was that a 2017 research preprint reported a “horrifyingly large” spike of foetal deaths caused by lead exposure from the 2014-15 Flint, Michigan Water Crisis (Roy & Edwards, 2022). The Washington Post, The Economist, PBS NOVA, Science, The Atlantic, Axios, Detroit Free Press, BBC cited this as a fact. However, the reported 58% increase in foetal death rate was not statistically significant, and did not appear in the final 2019 peer-reviewed paper. The claim of a “horrifyingly large” surge in foetal deaths became part of the lived experience of Flint residents creating trauma which cannot be undone.
[Image curtesy of いらすとや]
Implications
A. Posting preprints
When authors may hope to enjoy the speed, readership and citation advantage lead by posting preprints, they should pay attention to the policies of the journals to which they may wish to submit their works. Different publishers may have different policies regarding preprints. For example, eLife only review articles which have been available as preprints, aiming to more effectively convey expert reviewers’ assessment to authors, readers and interested parties (eLife). The New England Journal of Medicine (NEJM) encourages authors of all submitted manuscripts related to COVID-19 to post their works on preprint servers (Rubin et al., 2020). Authors of articles published with ACM (Association for Computing Machinery) can post preprints to non-commercial repositories only (Association for Computing Machinery).
[Image curtesy of いらすとや]
When releasing preprints, authors should act with high integrity and professional accountability (Roy & Edwards, 2022). They should consider whether the issue is urgent, or is an ongoing or imminent endangerment to the public or exposing potentially illegal activity. They should carefully choose the words to prevent unnecessary panic.
B. Reading preprints
Readers should always bear in mind that preprints must be understood as an incomplete work in progress (da Silva, 2018). The responsible interpretation of a preprint’s content is the basis of responsible use and citation. For example, amidst the COVID-19 pandemic, when COVID-19 related preprints gained significant attention, since mid-March 2020, bioRxiv and medRxiv included an additional banner to caution readers that preprints are not certified by peer-review and should not guide clinical practice or be reported in news as established information (Fraser et al., 2021). These efforts aim to aware readers, including journalists, that the findings presented in preprints are yet to be vetted and should be examined with caution.
[Image curtesy of いらすとや]
C. Citing preprints
When authors write their own papers, they should be aware that preprints are citable with a unique DOI, and are generally considered to establish precedence. The guideline by COPE (Committee on Publication Ethics), of which many major publishers, such as Elsevier, Sage, Springer Nature, Taylor and Francis, and Wiley, are members, states that it is relevant to cite other preprints which authors have considered during research (COPE Council, 2018). This is in line with the expectation that other non-peer-reviewed sources such as datasets, blogs or books should be cited.
Authors should also be aware that different publishers may have different policies regarding citation of preprints. For example, Nature (Nature) states in its formatting guide that “only articles … that have been uploaded to a recognized preprint server (for example, arXiv, bioRxiv), should be in the reference list.” Such preprints should be clearly indicated as preprints in the reference list. NIH (National Institutes of Health) encourages researchers to use preprints in order to enhance the dissemination speed and the rigor of their work (National Institutes of Health, 2017). Also, in NIH applications, proposals and reports, authors can cite preprints with indication that those are preprints.
[Image curtesy of いらすとや]
Conclusion
Preprints can lead to advantages in speed, reach and citations. They also play an unprecedented role in scholarly communication amidst the COVID-19 pandemic. As preprints are prior to peer review, when reading and citing them, readers should always be cautious and not consider them as established information.
This post continues our previous post on benefits of preprints. If you are interested in benefits of preprints, please refer to this blog post.
References
Association for Computing Machinery. ACM Author Rights. Retrieved 23 May 2022 from https://authors.acm.org/author-services/author-rights
COPE Council. (2018). COPE Discussion document: Preprints. https://publicationethics.org/files/u7140/COPE_Preprints_Mar18.pdf
da Silva, J. A. T. (2018). The preprint debate: What are the issues? Medical Journal Armed Forces India, 74(2), 162-164. https://doi.org/https://doi.org/10.1016/j.mjafi.2017.08.002
eLife. Peer review. Retrieved 19 May 2022 from https://elifesciences.org/about/peer-review
Fraser, N., Brierley, L., Dey, G., Polka, J. K., Pálfy, M., Nanni, F., & Coates, J. A. (2021). The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLOS Biology, 19(4), e3000959. https://doi.org/10.1371/journal.pbio.3000959
National Institutes of Health. (2017). Reporting Preprints and Other Interim Research Products. Retrieved 23 May 2022 from https://grants.nih.gov/grants/guide/notice-files/NOT-OD-17-050.html
Nature. Formatting guide. Retrieved 23 May 2022 from https://www.nature.com/nature/for-authors/formatting-guide
Roy, S., & Edwards, M. A. (2022). Addressing the preprint dilemma. International Journal of Hygiene and Environmental Health, 240, 113896. https://doi.org/https://doi.org/10.1016/j.ijheh.2021.113896
Rubin, E. J., Baden, L. R., Morrissey, S., & Campion, E. W. (2020). Medical Journals and the 2019-nCoV Outbreak. New England Journal of Medicine, 382(9), 866-866. https://doi.org/10.1056/NEJMe2001329
Xie, B., Shen, Z., & Wang, K. (2021). Is preprint the future of science? A thirty year journey of online preprint services. arXiv e-prints, arXiv:2102.09066. https://ui.adsabs.harvard.edu/abs/2021arXiv210209066X
