Towards a Framework to Measure Open Science Practices – PLOS Open Science Indicators
— by Fanny Liu
Open Science is “transparent and accessible knowledge that is shared and developed through collaborative networks” (Vicente-Saez & Martinez-Fuentes, 2018, p. 434). It encompasses emerging trends such as open code, open data, open access, and more.
PLOS (Public Library of Science) is a non-profit, open access publisher in science and medicine, and perhaps, a journal publisher familiar to HKU scholars — In the past 3 years (2020-22), HKU scholars published 118 articles with PLOS journals.
Since 2020, as an advocator of open science, PLOS has established the Open Research Solutions programme, which aims to increase adoption of Open Science practices for sharing protocols, code, research data, and preprints (Hrynaszkiewicz, 2022). This led to PLOS’ initiative to publish data on “Open Science Indicators” observed in PLOS articles. These Open Science Indicators are conceptualized by PLOS according to the FAIR principles (Findable, Accessible, Interoperable, Reusable) and developed with DataSeer using an artificial intelligence-driven approach (Cadwallader et al., 2022).
Data were extracted from:
- All PLOS content from 2019 to June 2022 (over 66,000 articles)
- A set of approximately 6,500 comparator articles on similar topics published in non-PLOS journals
Numerical Indicators in the framework include:
- Rates of data sharing in data repositories
- Rates of code sharing
- Rates of preprint posting, in any preprint server before publication
Their first dataset is now available for download at: https://doi.org/10.6084/m9.figshare.21687686.
[Image curtesy of いらすとや]
So, from the dataset, what do we know more about open science?
1. Data sharing
From 2019-2022 Jun, the rates of research data available in a repository were as follows:
Figure 1: Rates of data sharing in a repository by year
From 2019 to 2022, growth from 23% to 28% in PLOS data, and 9% to 15% in the comparator dataset could be observed (Public Library of Science, 2022). The overall increase in data sharing in repositories shows that open data has become more and more common.
This also aligns with the current trends in government’s or funder’s requirements on open data, e.g., the new Data Management and Sharing (DMS) Policy of National Institutes of Health (NIH) and the White House Office of Science and Technology Policy (OSTP) Memorandum.
Extra tip:
Know more about current trends in open data: https://blog-sc.hku.hk/state-of-open-data-2022/
2. Code sharing
From 2019-2022 Jun, the rates of code sharing were as follows:
Figure 2: Rates of code sharing by year
Similar to data sharing, code sharing rates were on the rise from 2019 to 2022 ― from 10% to 14% in PLOS data, and from 6% to 16% in comparator data respectively (Public Library of Science, 2022).
But overall, code sharing is less common than data sharing. A possible reason is that while most research projects generate datasets, only some produce codes.
3. Preprint sharing
From 2019-2022 Jun, the rates of preprint sharing were as follows:
Figure 3: Rates of preprint sharing by year
Again, preprint sharing rates increased from 2019 to 2022. In PLOS data, the rate increased from 17% to 24%, while in comparator data, from 12% to 23% (Public Library of Science, 2022).
In 2022, the rates of preprint sharing in PLOS articles and comparator data were 24% and 23% respectively, which were similar to each other. Researchers can enjoy benefits brought by posting preprints. For example, preprints correlate to early visibility and more citations (Xie et al., 2021).
Extra tip:
Know more about benefits and concerns of preprints: https://blog-sc.hku.hk/tag/preprint/
This summary shows that an overall increase can be seen in all three Open Science Indicators (data, code, and preprint) published by PLOS.
The dataset can be analysed in other ways to understand Open Science practices. If you are interested, you can download dataset at: https://doi.org/10.6084/m9.figshare.21687686.
In the end, it is still unclear whether this framework would be adopted by more various publishers. Nonetheless, it is interesting to see efforts in developing a new framework with measurable indicators to evaluate current progress of open science.
Extra tip:
Learn about how HKU Libraries supports open science: https://lib.hku.hk/newsblog/?p=1978
References
Cadwallader, L., Morton, L., & Hrynaszkiewicz, I. (2022, 12 December). Explore the first Open Science Indicators dataset—and share your thoughts. The Official PLOS Blog. https://theplosblog.plos.org/2022/12/open-science-indicators-first-dataset/
Hrynaszkiewicz, I. (2022, 14 September). PLOS partners with DataSeer to develop Open Science Indicators. The Official PLOS Blog. https://theplosblog.plos.org/2022/09/plos-partners-with-dataseer-to-develop-open-science-indicators/
Public Library of Science. (2022). PLOS Open Science Indicators. In: Public Library of Science.
Vicente-Saez, R., & Martinez-Fuentes, C. (2018). Open Science now: A systematic literature review for an integrated definition. Journal of Business Research, 88, 428-436. https://doi.org/10.1016/j.jbusres.2017.12.043
Xie, B., Shen, Z., & Wang, K. (2021). Is preprint the future of science? A thirty year journey of online preprint services. arXiv e-prints, arXiv:2102.09066. https://ui.adsabs.harvard.edu/abs/2021arXiv210209066X
