Abstract
This study aims to validate an empirical model, at document level, that explains the interaction among content, usage, and citation within open access publications. The PLoS site was the information source for this study. Using an R API (Application Programming Interface) for PLoS ONE, 776,465 records were downloaded on August 24, 2018. Those records (from 2006 to 2018) were organized according to the PLoS’ thematic areas. The empirical framework was validated using mediation analysis. For computing the parameters’ significance, bootstrap** with 500 replications for the general model and each thematic area was used. When usage was included as the mediating variable within the model, the total effects of cognitive and social variables got better predictive capability, as expressed by the explained variance of citation (R2 = 0.282) and usage (R2 = 0.333). The same trend was observed for the indirect effects after carrying out the mediation analysis by categories. Promotion campaigns of scientific publication should reinforce the widespread adoption of easy-to-use social media because, besides the velocity and variety of diffusion channels, the extended use guarantees that journal’s papers will reach increasing audiences. This is one of the first studies that analyze the interaction effects of variables at the article-level within open access publications.
Similar content being viewed by others
References
Akella, A., Alhoori, H., Kondamudi, P. R., Freeman, C., & Zhou, H. (2021). Early indicators of scientific impact: Predicting citations with altmetrics. Journal of Informetrics, 15(2), 101128. https://doi.org/10.1016/j.joi.2020.101128
Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037//0022-3514.51.6.1173
Bjekić, J., Lazarević, L. B., Živanović, M., & Knežević, G. (2014). Psychometric evaluation of the Serbian dictionary for automatic text analysis—LIWCser. Psihologija, 47(1), 5–32. https://doi.org/10.2298/PSI1401005B
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3(4–5), 993–1022.
Bornmann, L., & Haunschild, R. (2018). Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data. PLoS ONE, 13(5), e0197133. https://doi.org/10.1371/journal.pone.0197133
Bornmann, L., Wolf, M., & Daniel, H.-D. (2012). Closed versus open reviewing of journal manuscripts: How far do comments differ in language use? Scientometrics, 91(3), 843–856. https://doi.org/10.1007/s11192-011-0569-5
Chen, W. M. Y., Bukhari, M., Cockshull, F., & Galloway, J. (2020). The relationship between citations, downloads and alternative metrics in rheumatology publications: A bibliometric study. Rheumatology, 59(2), 277–280. https://doi.org/10.1093/rheumatology/kez163
Cho, J. (2021). Altmetrics analysis of highly cited academic papers in the field of Library and Information Science. Scientometrics, 126(9), 7623–7635. https://doi.org/10.1007/s11192-021-04084-w
Dehdaridad, T. (2020). Could early tweets counts predict later citation counts? A gender study in Life Sciences and Biomedicine (2014–2016). PLoS ONE, 15(11), e0241723. https://doi.org/10.1371/journal.pone.0241723
Ebrahimy, S., Mehrad, J., Setareh, F., & Hosseinchari, M. (2016). Path analysis of the relationship between visibility and citation: The mediating roles of save, discussion, and recommendation metrics. Scientometrics, 109(3), 1497–1510. https://doi.org/10.1007/s11192-016-2130-z
Ekinci, E., & Omurca, I. (2020). Concept-LDA: Incorporating Babelfy into LDA for aspect extraction. Journal of Information Science, 46(3), 406–418. https://doi.org/10.1177/0165551519845854
Gontijo, M. C. A., & de Araújo, R. F. (2021). Impacto acadêmico e atenção on-line de pesquisas sobre inteligência artificial na área da saúde: Análise de dados bibliométricos e altmétricos. Encontros Bibli: Revista Eletrônica De Biblioteconomia e Ciência Da Informação, 26, 01–21. https://doi.org/10.5007/1518-2924.2021.e76249
Graybeal, A., Seagal, J. D., & Pennebaker, J. W. (2002). The role of story-making in disclosure writing: The psychometrics of narrative. Psychology and Health, 17(5), 571–581. https://doi.org/10.1080/08870440290025786
Guerrero-Bote, V. P., & Moya-Anegón, F. (2014). Relationship between downloads and citations at journal and paper levels, and the influence of language. Scientometrics, 101(2), 1043–1065. https://doi.org/10.1007/s11192-014-1243-5
Hassan, S.-U., Aljohani, N. R., Idrees, N., Sarwar, R., Nawaz, R., Martinez-Camara, R., Ventura, S., & Herrera, F. (2020). Predicting literature’s early impact with sentiment analysis in Twitter. Knowledge-Based Systems, 192(15), e105383. https://doi.org/10.1016/j.knosys.2019.105383
Haustein, S., & Larivière, V. (2014). A multidimensional analysis of Aslib proceedings—Using everything but the impact factor. Aslib Journal of Information Management, 66(4), 358–380. https://doi.org/10.1108/AJIM-11-2013-0127
Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495. https://doi.org/10.1371/journal.pone.0120495
Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis. The Guilford Press.
Hu, Z., Fang, S., & Liang, T. (2014). Empirical study of constructing a knowledge organization system of patent documents using topic modeling. Scientometrics, 100(3), 787–799. https://doi.org/10.1007/s11192-014-1328-1
Iacobucci, D. (2008). Mediation analysis. Sage.
JASP Team. (2020). JASP, version 0.14.1 (computer software). https://jasp-stats.org/download.
Jose, P. E. (2013). Doing statistical mediation and moderation. The Guilford Press.
Li, X., Wu, C., & Mai, F. (2019). The effect of online reviews on product sales: A joint sentiment-topic analysis. Information Management, 56(2), 172–184. https://doi.org/10.1016/j.im.2018.04.007
Maggio, L. A., Leroux, T. C., Meyer, H. S., & Artino, A. R., Jr. (2018). #MedEd: Exploring the relationship between altmetrics and traditional measures of dissemination in health professions education. Perspectives on Medical Education, 7(4), 239–247. https://doi.org/10.1007/s40037-018-0438-5
Maricato, J. M., & Vilan-Filho, J. L. (2018). The potential for altmetrics to measure other types of impact in scientific production: academic and social impact dynamics in social media and networks. Information Research, 23(1), paper 780. Retrieved from http://InformationR.net/ir/23-1/paper780.html.
Mehl, M. R., & Pennebaker, J. W. (2003). The sounds of social life: A psychometric analysis of students’ daily social environments and conversations. Journal of Personality and Social Psychology, 84(4), 857–870. https://doi.org/10.1037/0022-3514.84.4.857
Moed, H. F., & Halevi, G. (2016). On full text download and citation distributions in scientific-scholarly journals. Journal of the Association for Information Science and Technology, 67(2), 412–431. https://doi.org/10.1002/asi.23405
Ortega, J. L. (2015). Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC’s members. Journal of Informetrics, 9(1), 39–49. https://doi.org/10.1016/j.joi.2014.11.004
Pennebaker, J. W., Francis M. E., & Booth, R. J. (2001). Linguistic Inquiry and Word Count LIWC 2001 Manual. Retrieved from https://www.researchgate.net/publication/246699633_Linguistic_inquiry_and_word_count_LIWC.
Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto.
Roemer, R. C., & Borchardt, R. (2015). Issues, controversies, and opportunities for altmetrics. Library Technology Reports, 51(5), 20–30.
Sathianathen, N. J., Lane, R., Condon, B., Murphy, D. G., Lawrentschuk, N., Weight, C. J., & Lamb, A. D. (2020). Early online attention can predict citation counts for urological publications: The #UroSoMe_Score. European Urology Focus, 6(3), 458–462. https://doi.org/10.1016/j.euf.2019.10.015
Schlögl, C., & Gorraiz, J. (2010). Comparison of citation and usage indicators: The case of oncology journals. Scientometrics, 82(3), 567–580. https://doi.org/10.1007/s11192-010-0172-1
Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2014). Comparison of downloads, citations and readership data for two information systems journals. Scientometrics, 101(2), 1113–1128. https://doi.org/10.1007/s11192-014-1365-9
Smith, Z. L., Chiang, A. L., Bowman, D., & Wallace, M. B. (2019). Longitudinal relationship between social media activity and article citations in the journal Gastrointestinal Endoscopy. Gastrointestinal Endoscopy, 90(1), 77–83. https://doi.org/10.1016/j.gie.2019.03.028
Smith-Keiling, B. L., & Hyun, H. I. F. (2019). Applying a computer-assisted tool for semantic analysis of writing: Uses for STEM and ELL. Journal of Microbiology & Biology Education, 20(1), 1–6. https://doi.org/10.1128/jmbe.v20i1.1709
Strauβ, N., Alonso-Muñoz, L., & Zúñiga, H. G. (2020). Bursting the filter bubble: The mediating effect of discussion frequency on network heterogeneity. Online Information Review, 44(6), 1161–1181. https://doi.org/10.1108/OIR-11-2019-0345
Thelwall, M. (2017). Are Mendeley reader counts useful impact indicators in all fields? Scientometrics, 113(1), 1721–1731. https://doi.org/10.1007/s11192-017-2557-x
Thelwall, M., & Nevill, T. (2018). Could scientists use Altmetric.com scores to predict longer term citation counts? Journal of Informetrics, 12(1), 237–248. https://doi.org/10.1016/j.joi.2018.01.008
Thelwall, M., & Wilson, P. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971. https://doi.org/10.1016/j.joi.2014.09.011
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841. https://doi.org/10.1371/journal.pone.0064841
Vílchez-Román, C., Huamán-Delgado, F., & Alhuay-Quispe, J. (2020). Social dimension activates the usage and academic impact of Open Access publications in Andean countries: A structural modeling-based approach. Information Development. https://doi.org/10.1177/0266666920901849
Wang, Z., Chen, Y., & Glänzel, W. (2020a). Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics. Journal of Informetrics, 14(4), 101097. https://doi.org/10.1016/j.joi.2020.101097
Wang, Z., Glänzel, W., & Chen, Y. (2020b). The impact of preprints in Library and Information Science: An analysis of citations, usage and social attention indicators. Scientometrics, 125(2), 1403–1423. https://doi.org/10.1007/s11192-020-03612-4
Wei, M., & Noroozi Chakoli, A. (2020). Evaluating the relationship between the academic and social impact of open access books based on citation behaviors and social media attention. Scientometrics, 125(3), 2401–2420. https://doi.org/10.1007/s11192-020-03678-0
Wilson, T. D. (1999). Models of information behaviour research. Journal of Documentation, 55(3), 249–270. https://doi.org/10.1108/EUM0000000007145
Williams, A. E. (2017). Altmetrics: An overview and evaluation. Online Information Review, 41(3), 311–317. https://doi.org/10.1108/OIR-10-2016-0294
Wu, Q., Dbouk, W., Hasan, I., Kobeissi, N., & Zheng, L. (2021). Does gender affect innovation? Evidence from female chief technology officers. Research Policy, 50(9), 104327. https://doi.org/10.1016/j.respol.2021.104327
Zhang, Y., Wu, Y. J., Goh, M., & Liu, X. (2019). Supply chain management scholar’s research impact: Moderated mediation analysis. Library Hi Tech, 39(1), 118–135. https://doi.org/10.1108/LHT-07-2017-0149
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A
Appendix B
Links to the Comma-Separated-Values (CSV) files used in the study.
Filtered dataset (n = 758,419 records)
https://doi.org/10.6084/m9.figshare.16652707.
Analyzed dataset (n = 235,384 records)
Rights and permissions
About this article
Cite this article
Vílchez-Román, C., Vara-Horna, A. Usage, content and citation in open access publication: any interaction effects?. Scientometrics 126, 9457–9476 (2021). https://doi.org/10.1007/s11192-021-04178-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-021-04178-5