Abstract
Introduction: College students face increased risk for a variety of mental health problems but experience barriers to treatment access. Prevention programs, including those implemented by peer educators, may decrease treatment needs and increase service access. We examined the implementation of an evidence-based eating disorder prevention program, Body Project, delivered by college peer educators at 63 colleges/universities, comparing three levels of implementation support: (1) Train-the-Trainer (TTT) training; (2) TTT plus a technical assistance workshop (TTT + TA); and (3) TTT + TA with one year of quality assurance calls (TTT + TA + QA). The present study tested the degree to which indicators proposed by the Consolidated Framework for Implementation Research (CFIR) were associated with core implementation outcomes. Method: We tested whether indices of CFIR domains (i.e., perceived intervention characteristics, outer and inner setting factors, provider characteristics, and implementation process) were correlated with three implementation outcomes (program reach, fidelity, effectiveness) during a 1-year implementation period. Results: Greater program reach was associated with implementation process, specifically the completion of more implementation activities (β = 0.46). Greater program fidelity was associated with higher positive (β = 0.44) and lower negative (β = − 0.43) perceptions of the Body Project characteristics, and greater reported general support for evidence-based practices (β = 0.41). Greater effectiveness was associated with lower negative perceptions of Body Project characteristics (d = 0.49). Conclusions: Several implementation determinants proposed by the CFIR model predicted outcomes, especially intervention fidelity. Across the outcomes of interest, implementation determinants related to peer educator and supervisor perceived characteristics of the specific intervention and general attitudes towards evidence-based practices emerged as robust predictors to inform future work investigating ongoing implementation and sustainability of programs in university settings.
Similar content being viewed by others
References
Aarons, G.A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6, 61–74.
Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatric Clinics, 14(2), 255–271. https://doi.org/10.1016/j.chc.2004.04.008.
Aarons, G. A., & Sommerfeld, D. H. (2012). Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. Journal of the American Academy of Child & Adolescent Psychiatry, 51(4), 423–431. https://doi.org/10.1016/j.jaac.2012.01.018.
Ametaj, A. A., Smith, A. M., & Valentine, S. E. (2021). A stakeholder-engaged process for adapting an evidence-based intervention for posttraumatic stress disorder for peer delivery. Administration and Policy in Mental Health and Mental Health Services Research, 48(5), 793–809. https://doi.org/10.1007/s10488-021-01129-3.
Bailey, R. J., Erekson, D. M., Cattani, K., Jensen, D., Simpson, D. M., Klundt, J., Vogeler, H. A., Schmuck, D., Worthen, V. E., Caldwell, Y., Beecher, M. E., Griner, D., & Hobbs, K. (2022). Adapting stepped care: Changes to service delivery format in the context of high demand. Psychological Services, 19(3), 494–501. https://doi.org/10.1037/ser0000564. https://doi-org.ezproxy.lib.utexas.
Baker, R., Camosso-Stefinovic, J., Gillies, C., Shaw, E. J., Cheater, F., Flottorp, S., Robertson, N., Wensing, M., Fiander, M., Eccles, M. P., Cwirko, G., van Lieshout, M., J., & Jäger, C. (2015). Tailored interventions to address determinants of practice. Cochrane Database of Systematic Reviews, 4. Art. No.: CD005470. DOI: https://doi.org/10.1002/14651858.CD005470.pub3.
Bearman, S. K., Weisz, J. R., Chorpita, B. F., Hoagwood, K., Ward, A., Ugueto, A. M., & Bernstein, A. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40, 518–529. https://doi.org/10.1007/s10488-013-0485-5.
Bearman, S. K., Schneiderman, R. L., & Zoloth, E. (2017). Building an evidence base for effective supervision practices: An analogue experiment of supervision to increase EBT fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 293–307. https://doi.org/10.1007/s10488-016-0723-8.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate - a practical and powerful approach to multiple testing. Journal of Royal Statistical Society B, 57, 289–300.
Bergh, A., Arsalo, I., Malan, A. F., Patrick, M., Pattinson, R. C., & Phillips, N. (2005). Measuring implementation progress in kangaroo mother care. Acta Paediatrica, 94(8), 1102–1108. https://doi.org/10.1111/j.1651-2227.2005.tb02052.x.
Borntrager, C., Chorpita, B., Higa-McMillan, C., & Weisz, J. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60, 677–681.
Burgess, A. M., Okamura, K. H., Izmirian, S. C., Higa-McMillan, C. K., Shimabukuro, S., & Nakamura, B. J. (2017). Therapist attitudes towards evidence-based practice: A joint factor analysis. The Journal of Behavioral Health Services & Research, 44(3), 414–427.
Callahan, J. L., Almstrom, C. M., Swift, J. K., Borja, S. E., & Heath, C. J. (2009). Exploring the contribution of supervisors to intervention outcomes. Training and Education in Professional Psychology, 3(2), 72–77. https://doi.org/10.1037/a0014294. https://doi-org.ezproxy.lib.utexas.edu/.
Center for Collegiate Mental Health (2020, January). 2019 Annual Report (Publication No. STA 20–244).
Chamberlain, P., Brown, C. H., & Saldana, L. (2011). Observational measure of implementation progress in community-based settings: The stages of implementation completion (SIC). Implementation Science, 6(6), 116. https://doi.org/10.1186/1748-5908-6-116.
Collyer, H., Eisler, I., & Woolgar, M. (2020). Systematic literature review and meta-analysis of the relationship between adherence, competence and outcome in psychotherapy for children and adolescents. European Child & Adolescent Psychiatry, 29(4), 417–431. https://doi.org/10.1007/s00787-018-1265-2. https://doi-org.ezproxy.lib.utexas.edu/.
Damschroder, L. J., & Lowery, J. C. (2013). Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Science, 8(1), 1–17. https://doi.org/10.1186/1748-5908-8-51.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A Consolidated Framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50.
Dreison, K. C., White, D. A., Bauer, S. M., Salyers, M. P., & McGuire, A. B. (2018). Integrating self-determination and job demands–resources theory in predicting mental health provider burnout. Administration and Policy in Mental Health and Mental Health Services Research, 45, 121–130.
Ehrnstrom, C., Blankenheim, A., Bottiglio, M., Tomatz, M., West, A., Bryant, T., & Gerberich, S. (2022). Embedding counselors within departments across campus can meet increased demand. Student Affairs Today, 25(3), 6–7.
Erskine, H. E., Whiteford, H. A., & Pike, K. M. (2016). The global burden of eating disorders. Current Opinion in Psychiatry, 29, 346–353.
Forman, S. G., Olin, S. S., Hoagwood, K. E., Crowe, M., & Saka, N. (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1(1), 26. https://doi.org/10.1007/s12310-008-9002-5.
Fromme, K., & Corbin, W. (2004). Prevention of Heavy drinking and Associated negative consequences among mandated and Voluntary College Students. Journal of Consulting and Clinical Psychology, 72(6), 1038–1049. https://doi.org/10.1037/0022-006X.72.6.1038.
Glisson, C., Himmelman, A., Green, P., Dukes, D., Atkinson, S., & Williams, N. J. (2012). Randomized trial of the availability, responsiveness, and continuity (ARC) organizational intervention with community-based mental health programs and clinicians serving youth. Journal of the American Academy of Child and Adolescent Psychiatry, 51(8), 780–787. https://doi.org/10.1016/j.jaac.2012.05.010.
Hunt, J., & Eisenberg, D. (2010). Mental health problems and help-seeking behavior among college students. Journal of Adolescent Health, 46(1), 3–10. https://doi.org/10.1016/j.jadohealth.2009.08.008.
Izmirian, S. C., & Nakamura, B. J. (2016). Knowledge, attitudes, social desirability, and organizational characteristics in youth mental health services. The Journal of Behavioral Health Services & Research, 43(4), 630–647.
Kivimaki, M., & Elovainio, M. (1999). A short version of the Team Climate Inventory: Development and psychometric properties. Journal of Occupational and Organizational Psychology, 72(2), 241–246.
Kratz, H. E., Stahmer, A., **e, M., Marcus, S. C., Pellecchia, M., Locke, J., Beidas, R., & Mandell, D. S. (2019). The effect of implementation climate on program fidelity and student outcomes in autism support classrooms. Journal of Consulting and Clinical Psychology, 87(3), 270–281. https://doi.org/10.1037/ccp0000368. https://doi-org.ezproxy.lib.utexas.
Krause, J., Van Lieshout, J., Klomp, R., Huntink, E., Aakhus, E., Flottorp, S., & Agarwal, S. (2014). Identifying determinants of care for tailoring implementation in chronic diseases: An evaluation of different methods. Implementation Science, 9(1), 102. https://doi.org/10.1186/s13012-014-0102-3.
LeViness, P., Bershad, C., Gorman, K., Braun, L., & Murray, T. (2018). The association for university and college counseling center directors annual survey–Public version.
LeViness, P., Gorman, K., Braun, L., Koenig, L., & Bershad, C. (2019). The Association for University and College Counseling Center directors annual survey: 2019. https://www.aucccd.org/assets/documents/Survey/2019%20AUCCCD%20Survey2020-05-31-PUBLIC.pdf.
Loo, R., & Loewen, P. (2002). A confirmatory factor-analytic and psychometric examination of the team climate inventory: Full and short versions. Small Group Research, 33(2), 254–265. https://doi.org/10.1177/104649640203300205.
Lyon, A. R., & Bruns, E. J. (2019). From evidence to impact: Joining our best school mental health practices with our best implementation strategies. School Mental Health, 1–9. https://doi.org/10.1007/s12310-018-09306-w.
Malone, S., Prewitt, K., Hackett, R., Lin, J. C., McKay, V., Walsh-Bailey, C., & Luke, D. A. (2021). The clinical sustainability Assessment Tool: Measuring organizational capacity to promote sustainability in healthcare. Implementation Science Communications, 2, 1–12.
McGoey, K. E., Rispoli, K. M., Venesky, L. G., Schaffner, K. F., McGuirk, L., & Marshall, S. (2014). A preliminary investigation into teacher perceptions of the barriers to behavior intervention implementation. Journal of Applied School Psychology, 30(4), 375–390. https://doi.org/10.1080/15377903.2014.950441.
McHugh, S., Dorsey, C. N., Mettert, K., Purtle, J., Bruns, E., & Lewis, C. C. (2020). Measures of outer setting constructs for implementation research: A systematic review and analysis of psychometric quality. Implementation Research and Practice, 1, https://doi.org/10.1177/2633489520940022.
Mettert, K., Lewis, C., Dorsey, C., Halko, H., & Weiner, B. (2020). Measuring implementation outcomes: An updated systematic review of measures’ psychometric properties. Implementation Research and Practice, 1, 2633489520936644.
Ostroff, C., Kinicki, A. J., & Muhammad, R. S. (2013). Organizational culture and climate. Handbook of Psychology, Second Edition, 12.
Patel, V., Chowdhary, N., Rahman, A., & Verdeli, H. (2011). Improving access to psychological treatments: Lessons from develo** countries. Behaviour Research and Therapy, 49, 523–528.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7.
Richmond, S. A., Donaldson, A., Macpherson, A., Briedel, W., van den Berg, C., Finch, C. F., Hagel, B., & Emery, C. A. (2020). Facilitators and barriers to the implementation of iSPRINT: A sport injury prevention program in junior high schools. Clinical Journal of Sport Medicine, 30(3), 231–238. https://doi.org/10.1097/JSM.0000000000000579.
Rogers, E. M. (2003). Diffusion of innovations. New York: Free.
Schneider, B., Ehrhart, M. G., & Macey, W. H. (2011). Organizational climate research. The Handbook of Organizational Culture and Climate, 29, 12169–12012.
Shapiro, C. J., Prinz, R. J., & Sanders, M. R. (2012). Facilitators and barriers to implementation of an evidence-based parenting intervention to prevent child maltreatment: The triple P-Positive parenting program. Child Maltreatment, 17(1), 86–95. https://doi.org/10.1177/1077559511424774.
Soicher, R. N., Becker-Blease, K. A., & Bostwick, K. C. (2020). Adapting implementation science for higher education research: The systematic study of implementing evidence-based practices in college classrooms. Cognitive Research: Principles and Implications, 5(1), 1–15. https://doi.org/10.1186/s41235-020-00255-0.
Stice, E., Fisher, M., & Martinez, E. (2004). Eating disorder diagnostic scale: Additional evidence of reliability and validity. Psychological Assessment, 16(1), 60–71. https://doi.org/10.1037/1040-3590.16.1.60.
Stice, E., Butryn, M. L., Rohde, P., Shaw, H., & Marti, C. N. (2013a). An effectiveness trial of a new enhanced dissonance eating disorder prevention program among female college students. Behaviour Research and Therapy, 51(12), 862–871. 10.1016/j.brat.2013a.10.003.
Stice, E., Rohde, P., & Shaw, H. (2013b). The Body Project: A dissonance-based eating disorder prevention intervention (Programs That Work). Oxford University Press.
Stice, E., Rohde, P., Gau, J. M., Bearman, S. K., & Shaw, H. (2023). An experimental test of increasing implementation support for college peer educators delivering an evidence-based prevention program. Journal of Consulting and Clinical Psychology, 91(4), 208–220.
Twenge, J. M., Cooper, A. B., Joiner, T. E., Duffy, M. E., & Binau, S. G. (2019). Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. Journal of Abnormal Psychology, 128(3), 185–199.
van Hoeken, D., & Hoek, H. W. (2020). Review of the burden of eating disorders: Mortality, disability, costs, quality of life, and family burden. Current Opinion in Psychiatry, 33(6), 521–527.
Verschuur, R., Huskens, B., Korzilius, H., Bakker, L., Snijder, M., & Didden, R. (2020). Pivotal response treatment: A study into the relationship between therapist characteristics and fidelity of implementation. Autism, 24(2), 499–514. https://doi.org/10.1177/1362361319876213. https://doi-org.ezproxy.lib.utexas.edu/.
Vest, B. M., Wray, L. O., Brady, L. A., et al. (2020). Primary care and mental health providers’ perceptions of implementation of pharmacogenetics testing for depression prescribing. Bmc Psychiatry, 20, 518. https://doi.org/10.1186/s12888-020-02919-z.
Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211.
Weisz, J. R., Ng, M. Y., & Bearman, S. K. (2014). Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era. Clinical Psychological Science, 2(1), 58–74. https://doi.org/10.1177/2167702613501307. https://doi-org.ezproxy.lib.utexas.edu/.
White, S., Park, Y. S., Israel, T., & Cordero, E. D. (2009). Longitudinal evaluation of peer health education on a college campus: Impact on health behaviors. Journal of American College Health, 57(5), 497–505.
Wilkinson, C. B., Infantolino, Z. P., & Wacha-Montes, A. (2017). Evidence-based practice as a potential solution to burnout in university counseling center clinicians. Psychological Services, 14(4), 543–548. https://doi.org/10.1037/ser0000156.
Williams, N. J., & Beidas, R. S. (2019). Annual research review: The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry, 60(4), 430–450.
Williams, N. J., Glisson, C., Hemmelgarn, A., & Green, P. (2017). Mechanisms of change in the ARC organizational strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration and Policy in Mental Health, 44(2), 269–283. https://doi.org/10.1007/s10488-016-0742-5.
Windsor, R., Cleary, S., Ramiah, K., Clark, J., Abroms, L., & Davis, A. (2013). The Smoking Cessation and reduction in pregnancy treatment (SCRIPT) adoption scale: Evaluating the diffusion of a tobacco treatment innovation to a statewide prenatal care program and providers. Journal of Health Communication, 18(10), 1201–1220. https://doi.org/10.1080/10810730.2013.778358.
Wong, T., Pharr, J. R., Bungum, T., Coughenour, C., & Lough, N. L. (2019). Effects of peer sexual health education on college campuses: A systematic review. Health Promotion Practice, 20(5), 652–666.
Wood, C. I., Yu, Z., Sealy, D. A., Moss, I., Zigbuo-Wenzler, E., McFadden, C., Landi, D., & Brace, A. M. (2022). Mental health impacts of the COVID-19 pandemic on college students. Journal of American College Health https://doiorghttps://doi.org/10.1080/07448481.2022.2040515
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Statements and declarations
This study was supported by grant MH112743 from the National Institutes of Health. The authors adhere to the CONSORT guidelines.
Competing Interests
The authors report no conflicts of interest (financial or non-financial).
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rohde, P., Bearman, S.K., Pauling, S. et al. Setting and Provider Predictors of Implementation Success for an Eating Disorder Prevention Program Delivered by College Peer Educators. Adm Policy Ment Health 50, 912–925 (2023). https://doi.org/10.1007/s10488-023-01288-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-023-01288-5