SpringerLink
Log in

Understanding the impact change of a highly cited article: a content-based citation analysis

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Researchers tend to cite highly cited articles, but how these highly cited articles influence the citing articles has been underexplored. This paper investigates how one highly cited essay, Hirsch’s “h-index” article (H-article) published in 2005, has been cited by other articles. Content-based citation analysis is applied to trace the dynamics of the article’s impact changes from 2006 to 2014. The findings confirm that citation context captures the changing impact of the H-article over time in several ways. In the first two years, average citation mention of H-article increased, yet continued to decline with fluctuation until 2014. In contrast with citation mention, average citation count stayed the same. The distribution of citation location over time also indicates three phases of the H-article “Discussion,” “Reputation,” and “Adoption” we propose in this study. Based on their locations in the citing articles and their roles in different periods, topics of citation context shifted gradually when an increasing number of other articles were co-mentioned with the H-article in the same sentences. These outcomes show that the impact of the H-article manifests in various ways within the content of these citing articles that continued to shift in nine years, data that is not captured by traditional means of citation analysis that do not weigh citation impacts over time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Aksnes, D. W. (2003). Characteristics of highly cited papers. Research Evaluation, 12(3), 159–170.

    Article  Google Scholar 

  • Alsaad, A., & Abbod, M. (2015). Enhanced topic identification algorithm for Arabic Corpora. In Proceedings of the 17th UKSIM-AMSS International Conference on Modelling and Simulation.

  • Angrosh, M., Cranefield, S., & Stanger, N. (2012). A citation centric annotation scheme for scientific Articles. Paper presented at the Proceedings of Australasian Language Technology Association Workshop.

  • Ball, P. (2005). Index aims for fair ranking of scientists. Nature, 436, 900.

    Article  Google Scholar 

  • Baneyx, A. (2008). “Publish or Perish” as citation metrics used to analyze scientific output in the humanities: International case studies in economics, geography, social sciences, philosophy, and history. Archivum immunologiae et therapiae experimentalis, 56(6), 363–371.

    Article  Google Scholar 

  • Bar-Ilan, Judit. (2008). Which h-index?—A comparison of WoS. Scopus and Google Scholar. Scientometrics, 74(2), 257–271.

    Google Scholar 

  • Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning Research, 3, 993–1022.

    MATH  Google Scholar 

  • Bornmann, L., & Daniel, H. D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.

    Article  Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173.

    Article  Google Scholar 

  • Brown, P. (1980). The half-life of the chemical literature. Journal of the American Society for Information Science, 31(1), 61–63.

    Article  Google Scholar 

  • Burton, R. E., & Kebler, R. (1960). The “half-life” of some scientific and technical literatures. American Documentation, 11(1), 18–22.

    Article  Google Scholar 

  • Cano, V. (1989). Citation behavior: Classification, utility, and location. Journal of the American Society for Information Science, 40(4), 284–290.

    Article  Google Scholar 

  • Case, D. O., & Higgins, G. M. (2000). How can we investigate citation behavior? A study of reasons for citing literature in communication. Journal of the American Society for Information Science, 51(7), 635–645.

    Article  Google Scholar 

  • Charles, J. P. (1988). Citation analysis of astronomical literature: Comments on citation half-lives. Publications of the Astronomical Society of the Pacific, 100(623), 106.

    Google Scholar 

  • Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2012). SciMAT: A new science map** analysis software tool. Journal of the American Society for Information Science and Technology, 63(8), 1609–1630.

    Article  MATH  Google Scholar 

  • Ding, Y., Liu, X., Guo, C., & Cronin, B. (2013). The distribution of references across texts: Some implications for citation analysis. Journal of Informetrics, 7, 583–592.

    Article  Google Scholar 

  • Ding, Y., & Stirling, K. (2016). Data-driven discovery: A new era of exploiting the literature and data. Journal of Data and Information Science, 1(4), 1–9.

    Article  Google Scholar 

  • Ding, Y., Zhang, G., Chambers, T., Song, M., Wang, X., & Zhai, C. (2014). Content-based citation analysis: The next generation of citation analysis. Journal of the Association for Information Science and Technology, 65(9), 1820–1833.

    Article  Google Scholar 

  • Dumais, S. T. (2004). Latent semantic analysis. Annual Review of Information Science and Technology, 38(1), 188–230.

    Article  Google Scholar 

  • Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.

    Article  Google Scholar 

  • Garfield, E. (1964). Science citation index: A new dimension in indexing. Science, 144(3619), 649–654.

    Article  Google Scholar 

  • Hack, T. F., Crooks, D., Plohman, J., & Kepron, E. (2014). Citation analysis of Canadian psycho-oncology and supportive care researchers. Supportive Care in Cancer, 22(2), 315–324.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  MATH  Google Scholar 

  • Hirsch, J. E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Sciences, 104(49), 19193–19198.

    Article  Google Scholar 

  • Hu, Z., Chen, C., & Liu, Z. (2013). Where are citations located in the body of scientific articles? A study of the distributions of citation locations. Journal of Informetrics, 7, 887–896.

    Article  Google Scholar 

  • Hu, B., Dong, X., Zhang, C., Bowman, T. D., Ding, Y., Milojević, S., et al. (2015). A lead-lag analysis of the topic evolution patterns for preprints and publications. Journal of the Association for Information Science and Technology, 66(12), 2643–2656.

    Article  Google Scholar 

  • Imperial, J., & Rodríguez-Navarro, A. (2007). Usefulness of Hirsch’s h-index to evaluate scientific research in Spain. Scientometrics, 71(2), 271–282.

    Article  Google Scholar 

  • Ioannidis, J. P. (2010). Is there a glass ceiling for highly cited scientists at the top of research universities? The FASEB Journal, 24(12), 4635–4638.

    Article  Google Scholar 

  • Järvelin, K., & Persson, O. (2008). The DCI index: Discounted cumulated impact-based research evaluation. Journal of the American Society for Information Science and Technology, 59(9), 1433–1440.

    Article  Google Scholar 

  • Jeong, Y. K., Song, M., & Ding, Y. (2014). Content-based author co-citation analysis. Journal of Informetrics, 8, 197–211.

    Article  Google Scholar 

  • Kaplan, D., Tokunaga, T., & Teufel, S. (2016). Citation block determination using textual coherence. Journal of Information Processing, 24(3), 540–553.

    Article  Google Scholar 

  • Kim, H. J., Jeong, Y. K., & Song, M. (2016). Content- and proximity-based author co-citation analysis using citation sentences. Journal of Informetrics, 10(4), 954–966.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI newsletter, 2(3), 4–6.

    Google Scholar 

  • Lazaridis, T. (2010). Ranking university departments using the mean h-index. Scientometrics, 82(2), 211–216.

    Article  Google Scholar 

  • Lee, Y. -S., Lo, R., Chen, C. -Y., Lin, P. -C., & Wang, J. -C. (2015). News topics categorization using latent Dirichlet allocation and sparse representation classifier. In Proceedings of the IEEE international conference on consumer electronics.

  • Line, M. B., & Sandison, A. (1974). Progress in documentation: “Obsolescence” and changes in the use of literature with time. Journal of documentation, 30(3), 283–350.

    Article  Google Scholar 

  • Lipetz, B. A. (1965). Improvement of the selectivity of citation indexes to science literature through inclusion of citation relationship indicators. American Documentation, 16(2), 81–90.

    Article  Google Scholar 

  • Liu, X., Zhang, J., & Guo, C. (2013). Fulltext citation analysis: A new method to enhance scholarly networks. Journal of the American Society for Information Science and Technology, 64(9), 1852–1863.

    Article  Google Scholar 

  • MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5), 342.

    Article  Google Scholar 

  • McKeown, K., Daume, H., Chaturvedi, S., Paparrizos, J., Thadani, K., Barrio, P., et al. (2016). Predicting the impact of scientific concepts using full-text features. Journal of the Association for Information Science and Technology, 67(1), 2684–2696.

    Article  Google Scholar 

  • Mikki, S. (2010). Comparing google scholar and ISI web of science for earth sciences. Scientometrics, 82(2), 321–331.

    Article  Google Scholar 

  • Minasny, B., Hartemink, A. E., & McBratney, A. (2007). Soil science and the h index. Scientometrics, 73(3), 257–264.

    Article  Google Scholar 

  • Moravcsik, M. J., & Murugesan, P. (1975). Some results on the function and quality of citations. Social Studies of Science, 5(1), 86–92.

    Article  Google Scholar 

  • Oppenheim, C. (2007). Using the h-index to rank influential British researchers in information science and librarianship. Journal of the American Society for Information Science and Technology, 58(2), 297–301.

    Article  Google Scholar 

  • Pathak, M., & Bharati, K. A. (2014). Botanical survey of India (1971–2010): A scientometric analysis. Current Science, 106(7), 964.

    Google Scholar 

  • Pulina, G., & Ana Helena Dias, F. (2007). Some bibliometric indexes for members of the Scientific Association of Animal Production (ASPA). Italian Journal of Animal Science, 6(1), 83–103.

    Article  Google Scholar 

  • Ruane, F., & Tol, R. (2008). Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics, 75(2), 395–405.

    Article  Google Scholar 

  • Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing and Management, 24(5), 513–523.

    Article  Google Scholar 

  • Schlachter, G. (1988). Obsolescence, weeding, and bibliographic love canals. RQ, 28(1), 7–9.

    Google Scholar 

  • Schreiber, M. (2008). The influence of self-citation corrections on Egghe’sg index. Scientometrics, 76(1), 187–200.

    Article  Google Scholar 

  • Schuetz, P., & Caflisch, A. (2008). Efficient modularity optimization by multistep greedy algorithm and vertex mover refinement. Physical Review E, 77(4), 046112.

    Article  Google Scholar 

  • Small, H. G. (1978). Cited documents as concept symbols. Social Studies of Science, 8(3), 327–340.

    Article  Google Scholar 

  • Small, H., Tseng, H., & Patek, M. (2017). Discovering discoveries: Identifying biomedical discoveries using citation contexts. Journal of Informetrics, 11(1), 46–62.

    Article  Google Scholar 

  • Tang, X., Wan, X., & Zhang, X. (2014). Cross-language context-aware citation recommendation in scientific articles. In Proceedings of the 37th international ACM SIGIR Conference on Research & Development in Information Retrieval.

  • Teufel, S. (2000). Argumentative zoning: Information extraction from scientific text. CiteseerX. Retrieved at http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.105.485.

  • Tsay, M.-Y. (1998). Library journal use and citation half-life in medical science. Journal of the American Society for Information Science, 49(14), 1283–1292.

    Article  Google Scholar 

  • Venable, G. T., Shepherd, B. A., Roberts, M. L., Taylor, D. R., Khan, N. R., & Klimo, P., Jr. (2014). An application of Bradford’s law: Identification of the core journals of pediatric neurosurgery and a regional comparison of citation density. Child’s Nervous System, 30(10), 1717–1727.

    Article  Google Scholar 

  • Voos, H., & Dagaev, K. S. (1976). Are all citations equal? Or, did we op. cit. your idem? Journal of Academic Librarianship, 1(6), 19–21.

    Google Scholar 

  • Wan, X., & Liu, F. (2014a). Are all literature citations equally important? Automatic citation strength estimation and its applications. Journal of the Association for Information Science and Technology, 65, 1929–1938.

    Article  Google Scholar 

  • Wan, X., & Liu, F. (2014b). WL-index: Leveraging citation mention number to quantify an individual’s scientific impact. Journal of the Association for Information Science and Technology, 65(12), 2509–2517.

    Article  Google Scholar 

  • Zhang, G., Ding, Y., & Milojević, S. (2013). Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content. Journal of the American Society for Information Science and Technology, 64(7), 1490–1503.

    Article  Google Scholar 

  • Zhao, D., & Strotmann, A. (2015). Dimensions and uncertainties of author citation rankings: Lessons learned from frequencyweighted in-text citation counting. Journal of the Association for Information Science and Technology, 67(3), 671–682.

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by Major Projects of National Social Science Fund of China (No. 16ZAD224). Thanks to Star **ng Zhao and Yi Bu for providing suggestions on the manuscript, and to our colleagues in IR and TM Group@NJUST for their technical support and suggestions. Thanks Matthew Schnaars for his helpful editing of this draft.

Author information

Authors and Affiliations

  1. Department of Information Management, Nan**g University of Science and Technology, Nan**g, 210094, China

    Chao Lu & Chengzhi Zhang

  2. School of Informatics and Computing, Indiana University Bloomington, Bloomington, IN, 47401, USA

    Chao Lu & Ying Ding

  3. School of Information Management, Wuhan University, Wuhan, 430072, China

    Ying Ding

  4. Library, Tongji University, Shanghai, 200092, China

    Ying Ding

  5. Jiangsu Key Laboratory of Data Engineering and Knowledge Service, Nan**g University, Nan**g, 210093, China

    Chengzhi Zhang

  6. Jiangsu Collaborative Innovation Center of Social Safety Science and Technology, Nan**g, 210094, China

    Chengzhi Zhang

Authors
  1. Chao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengzhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengzhi Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, C., Ding, Y. & Zhang, C. Understanding the impact change of a highly cited article: a content-based citation analysis. Scientometrics 112, 927–945 (2017). https://doi.org/10.1007/s11192-017-2398-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2398-7

Keywords

Mathematics subject classification

JEL Classification

Search

Navigation