SpringerLink
Log in

The Evidence Base of International Clinical Practice Guidelines on Prostate Cancer: A Global Framework for Clinical Research Evaluation

  • Chapter
  • First Online:
Evaluative Informetrics: The Art of Metrics-Based Research Assessment

Abstract

Clinical practice guidelines (CPGs) are being developed to ensure that medical diagnosis and treatment are based on the best available evidence. Their value depends critically on the evidence base that they cite. These references may provide a better indicator of the impact of clinical research than citation counts on papers. We searched for CPGs on prostate cancer, the most common cancer afflicting men, and found 72, from 28 countries, and seven international organisations, and identified 10,273 cited references in journals processed on the Web of Science. They involved research on the three main means of treatment—chemotherapy, radiotherapy and surgery—but relatively few on genetics and palliative care. Most countries’ CPGs over-cite research from their own country, particularly if their presence in world prostate cancer research is small. Citations on own-country CPGs show their contribution to national healthcare provision, and citations on CPGs from other countries show the esteem in which they are held internationally. There are also differences between countries in how recent are the references that they cite, and some are not basing their recommendations on the latest research or best international evidence. Citations on CPGs provide an alternative source of information for the evaluation of clinical research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Abdelsattar, Z. M., Reames, B. N., Regenbogen, S. E., Hendren, S., & Wong, S. L. (2015). Critical evaluation of the scientific content in clinical practice guidelines. Cancer, 121(5), 783–789.

    Article  Google Scholar 

  • Anonymous. (1994). American Society of Clinical Oncology. Recommendations for the use of hematopoietic colony-stimulating factors: Evidence-based, clinical practice guidelines. Journal of Clinical Oncology, 12(11), 2471–2508.

    Google Scholar 

  • Audet, A.-M., Greenfield, S., & Field, M. (1990). Medical practice guidelines: Current activities and future directions. Annals of Internal Medicine, 113(9), 709–714.

    Article  Google Scholar 

  • Bakare, V., & Lewison, G. (2017). Country over-citation ratios. Scientometrics, 113(2), 1199–1207.

    Google Scholar 

  • Begum, M., Lewison, G., Wright, J. S., Pallari, E., & Sullivan, R. (2016). European non-communicable respiratory disease research, 2002-13: Bibliometric study of outputs and funding. PLoS ONE, 11(4), e0154197.

    Article  Google Scholar 

  • Browman, G. P. (1994). Evidence-based recommendations against neoadjuvant chemotherapy for routine management of patients with squamous cell head and neck cancer. Cancer Investigation, 12(6), 662–670.

    Article  Google Scholar 

  • Browman, G. P. (2001). Development and aftercare of clinical guidelines: The balance between rigor and pragmatism. Journal of the American Medical Association, 286(12), 1509–1511.

    Article  Google Scholar 

  • Burgers, J. S., Cluzeau, F. A., Hanna, S. E., Hunt, C., & Grol, R. (2003). Characteristics of high-quality guidelines: Evaluation of 86 clinical guidelines developed in ten European countries and Canada. International Journal of Technology Assessment in Health Care, 19(1), 148–157.

    Article  Google Scholar 

  • Burgers, J., Fervers, B., Haugh, M., Brouwers, M., Browman, G., Philip, T., & Cluzeau, F. (2004). International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. Journal of Clinical Oncology, 22(10), 2000–2007.

    Google Scholar 

  • Cecamore, C., Savino, A., Salvatore, R., Cafarotti, A., Pelliccia, P., Mohn, A., & Chiarelli, F. (2011). Clinical practice guidelines: what they are, why we need them and how they should be developed through rigorous evaluation. European Journal of Pediatrics, 170(7), 831-836.

    Google Scholar 

  • Checketts, J. X., Sims, M. T., & Vassar, M. (2017). Evaluating industry payments among dermatology clinical practice guidelines authors. JAMA Dermatology, 153(12), 1229–1235.

    Article  Google Scholar 

  • Chiarodo, A. (1991). National Cancer Institute roundtable on prostate cancer: Future research directions, AACR.

    Google Scholar 

  • Cook, D. J., Greengold, N. L., Ellrodt, A. G., & Weingarten, S. R. (1997). The relation between systematic reviews and practice guidelines. Annals of Internal Medicine, 127(3), 210–216.

    Article  Google Scholar 

  • Eriksson, M., Billhult, A., Billhult, T., Pallari, E., & Lewison, G. (2020). A new database of the references on international clinical practice guidelines: A facility for the evaluation of clinical research. Scientometrics, 122(2), 1221–1235.

    Article  Google Scholar 

  • Ewalt, P. L. (1995). Clinical practice guidelines: Their impact on social work in health. Social Work, 40(3), 293.

    Google Scholar 

  • Farshad, M., Maier, C., & Gerber, C. (2011, June). Do non-scientific factors influence citation rates of orthopedic journal articles?. In: Swiss medical weekly (Vol. 141, pp. 25S–25S). Farnsburgestr 8, CH-4132 muttenz, Switzerland: EMH Swiss Medical Publishers Ltd.

    Google Scholar 

  • Fervers, B., Burgers, J. S., Haugh, M. C., Brouwers, M., Browman, G., Cluzeau, F., & Philip, T. (2005). Predictors of high quality clinical practice guidelines: Examples in oncology. International Journal for Quality in Health Care, 17(2), 123–132.

    Google Scholar 

  • Field, M. J., & Lohr, K. N. (1990). Clinical practice guidelines: Directions for a new program. National Academies Press.

    Google Scholar 

  • Gibson, P. (1993). Asthma guidelines and evidence-based medicine. The Lancet, 342(8882), 1305.

    Article  Google Scholar 

  • Glänzel, W., & Moed, H. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.

    Article  Google Scholar 

  • Grant, J. (1999). Evaluating the outcomes of biomedical research on healthcare. Research Evaluation, 8(1), 33–38.

    Article  Google Scholar 

  • Grant, J., Cottrell, R., Cluzeau, F., & Fawcett, G. (2000). Evaluating “payback” on biomedical research from papers cited in clinical guidelines: Applied bibliometric study. BMJ, 320(7242), 1107–1111.

    Article  Google Scholar 

  • Greengold, N. L., & Weingarten, S. R. (1996). Develo** evidence-based practice guidelines and pathways: The experience at the local hospital level. The Joint Commission Journal on Quality Improvement, 22(6), 391–402.

    Article  Google Scholar 

  • Grimmer, K., Dizon, J. M., Milanese, S., King, E., Beaton, K., Thorpe, O., … Kumar, S. (2014). Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Medical Research Methodology, 14(1), 63.

    Google Scholar 

  • Harris, J. S. (1997). Development, use, and evaluation of clinical practice guidelines. Journal of Occupational and Environmental Medicine, 39(1), 23–34.

    Article  Google Scholar 

  • Hayward, R., & Laupacis, A. (1993). Initiating, conducting and maintaining guidelines development programs. CMAJ: Canadian Medical Association Journal, 148(4): 507.

    Google Scholar 

  • Hayward, R. S., Wilson, M. C., Tunis, S. R., Bass, E. B., Rubin, H. R., & Haynes, R. B. (1993). More informative abstracts of articles describing clinical practice guidelines. Annals of Internal Medicine, 118(9), 731–737.

    Article  Google Scholar 

  • Horn, J., Checketts, J. X., Jawhar, O., & Vassar, M. (2018). Evaluation of industry relationships among authors of otolaryngology clinical practice guidelines. JAMA Otolaryngology-Head & Neck Surgery, 144(3), 194–201.

    Article  Google Scholar 

  • Kinchin, I. M. (2017). The importance of an engaging title or Titular colonicity: Is it a factor that influences citation rates? Journal of Biological Education, 51(1), 1–2.

    Google Scholar 

  • Kryl, D., Allen, L., Dolby, K., Sherbon, B., & Viney, I. (2012). Tracking the impact of research on policy and practice: Investigating the feasibility of using citations in clinical guidelines for research evaluation. British Medical Journal Open, 2(2), e000897.

    Google Scholar 

  • Kryworuchko, J., Stacey, D., Bai, N., & Graham, I. D. (2009). Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005). Implementation Science, 4(1), 49.

    Article  Google Scholar 

  • Legido-Quigley, H., Panteli, D., Brusamento, S., Knai, C., Saliba, V., Turk, E., … McKee, M. (2012). Clinical guidelines in the European Union: Map** the regulatory basis, development, quality control, implementation and evaluation across member states. Health policy, 107(2-3), 146–156.

    Google Scholar 

  • Lewison, G., Dawson, G., & Anderson, J. (1997). Support for UK biomedical research from tobacco industry. The Lancet, 349(9054), 778.

    Google Scholar 

  • Lewison, G., & Paraje, G. (2004). The classification of biomedical journals by research level. Scientometrics, 60(2), 145–157.

    Google Scholar 

  • Lewison, G., & Sullivan, R. (2008). The impact of cancer research: How publications influence UK cancer clinical guidelines. British Journal of Cancer, 98(12), 1944.

    Article  Google Scholar 

  • Lewison, G., & Wilcox-Jay, K. (2003). Getting biomedical research into practice—The citations from UK clinical guidelines. In Proceedings of the 9th International Conference on Scientometrics and Informetrics, Bei**g, China.

    Google Scholar 

  • Lohr, K. N. (1994). Guidelines for clinical practice: Applications for primary care. International Journal for Quality in Health Care, 6(1), 17–25.

    Article  Google Scholar 

  • Lohr, K. N. (1995). Guidelines for clinical practice: What they are and why they count. The Journal of Law, Medicine & Ethics, 23(1), 49–56.

    Article  Google Scholar 

  • Lohr, K. N., & Field, M. J. (1992). Guidelines for clinical practice: From development to use. National Academies Press.

    Google Scholar 

  • Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277.

    Article  Google Scholar 

  • Moed, H. F., Burger, W., Frankfort, J., & Van Raan, A. F. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14(3), 131–149.

    Article  Google Scholar 

  • Network, S. T. S. I. G. Who we are. Retrieved from https://www.sign.ac.uk/who-we-are.html.

  • Pallari, E., Fox, A. W., & Lewison, G. (2018a). Differential research impact in cancer practice guidelines’ evidence base: lessons from ESMO, NICE and SIGN. ESMO Open, 3(1), e000258.

    Google Scholar 

  • Pallari, E., Lewison, G., Ciani, O., Tarricone, R., Sommariva, S., Begum, M., & Sullivan, R. (2018b). The impacts of diabetes research from 31 European Countries in 2002 to 2013. Research Evaluation, 27(3), 270–282.

    Google Scholar 

  • Shekelle, P. G., Ortiz, E., Rhodes, S., Morton, S. C., Eccles, M. P., Grimshaw, J. M., & Woolf, S. H. (2001). Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: How quickly do guidelines become outdated? JAMA, 286(12), 1461–1467.

    Google Scholar 

  • Sox, H. C., & Woolf, S. H. (1993). Evidence-based practice guidelines from the US preventive services task force. Journal of the American Medical Association, 269(20), 2678.

    Article  Google Scholar 

  • Steinberg, E. P., Eknoyan, G., Levin, N. W., Eschbach, J. W., Golper, T. A., Owen, W. F., & Schwab, S. (2000). Methods used to evaluate the quality of evidence underlying the national kidney foundation-dialysis outcomes quality initiative clinical practice guidelines: Description, findings, and implications. American Journal of Kidney Diseases, 36(1), 1–11.

    Google Scholar 

  • Todd, J. S. (1990). Do practice guidelines guide practice? The New England Journal of Medicine, 322(25), 1822–1823.

    Google Scholar 

  • Van Eck, N. J., Waltman, L., van Raan, A. F., Klautz, R. J., & Peul, W. C. (2013). Citation analysis may severely underestimate the impact of clinical research as compared to basic research. PloS one, 8(4).

    Google Scholar 

  • Woolf, S. H. (1992). Practice guidelines, a new reality in medicine: II. Methods of develo** guidelines. Archives of Internal Medicine, 152(5), 946–952.

    Article  Google Scholar 

  • Woolf, S. H., Grol, R., Hutchinson, A., Eccles, M., & Grimshaw, J. (1999). Potential benefits, limitations, and harms of clinical guidelines. BMJ, 318(7182), 527–530.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Ms Shoumiya Padman, the Nuffield Research Placement for providing support for the placement of the student, and Mr  Hamish Sharp. Both students assisted the authors with the bibliography of the guidelines by running the search statements and downloading the files from the Web of Science. The authors would also like to thank Dr Philip Roe from Evaluametrics Ltd for develo** the VBA macros used for the data extraction, processing of the downloaded papers, and their analysis.

Author information

Authors and Affiliations

  1. MRC Clinical Trials and Methodology Unit, University College London, 90 High Holborn, London, WC1V 6LJ, UK

    Elena Pallari

  2. Division of Cancer Studies, School of Cancer and Pharmaceutical Sciences, King’s College London, Guy’s Hospital, London, SE1 9RT, UK

    Grant Lewison

Authors
  1. Elena Pallari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grant Lewison
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elena Pallari .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria Informatica, Automatica e Gestionale (DIAG), Sapienza University, Rome, Italy

    Cinzia Daraio

  2. Centre for R&D Monitoring (ECOOM) and Dept MSI, KU Leuven, Leuven, Belgium

    Wolfgang Glänzel

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Pallari, E., Lewison, G. (2020). The Evidence Base of International Clinical Practice Guidelines on Prostate Cancer: A Global Framework for Clinical Research Evaluation. In: Daraio, C., Glänzel, W. (eds) Evaluative Informetrics: The Art of Metrics-Based Research Assessment . Springer, Cham. https://doi.org/10.1007/978-3-030-47665-6_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-47665-6_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-47664-9

  • Online ISBN: 978-3-030-47665-6

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation