Abstract
Since the 1970s mental health consumer-run organizations have come to offer not only mutual support, but they have also adopted agendas for broader social change. Despite an awareness of the need for system level efforts that create supportive environments for their members, there has been limited research demonstrating how their system level activities can be documented or their impacts evaluated. The purpose of this paper is to feature a method of evaluating systems change activities and impacts. The paper is based on a longitudinal study evaluating four mental health consumer-run organizations in Ontario, Canada. The study tracked system level activities and impacts using both qualitative and quantitative methodologies. The article begins by describing the development and implementation of these methods. Next it offers a critical analysis of the methods used. It concludes by reflecting on three lessons learned about capturing system level activities and impacts of mental health consumer-run organizations.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10464-007-9107-2/MediaObjects/10464_2007_9107_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10464-007-9107-2/MediaObjects/10464_2007_9107_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10464-007-9107-2/MediaObjects/10464_2007_9107_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10464-007-9107-2/MediaObjects/10464_2007_9107_Fig4_HTML.gif)
Similar content being viewed by others
References
Bassman, R. (2001). Whose reality is it anyway? Consumers/survivors/ex-patients can speak for themselves. Journal of Humanistic Psychology, 41(4), 11–35.
Behrens, T. (October, 2005). Clusters, initiatives and multi-site evaluations: The influence of funding strategies. Presentation at Crossing Borders, Crossing Boundaries (the American Evaluation Association/Canadian Evaluation Society Joint Meeting), Toronto.
Bowl, R. (2002). Psychiatric survivors and consumers in the United Kingdom – A successful social movement? Ethical and Human Sciences and Services, 4(2), 107–120.
Bronfenbrenner, U. (1979). The ecology of human development. Cambridge, MA: Harvard University Press.
Chamberlin, J. (1978). On our own: Patient-controlled alternatives to the mental health system. Toronto: McGraw-Hill.
Chamberlin, J. (1995). Rehabilitating ourselves: The psychiatric survivor movement. International Journal of Mental Health, 24, 39–46.
Chamberlin, J., Rogers, E. S., & Ellison, M. L. (1996). Self-help programs: A description of their characteristics and their members. Psychiatric Rehabilitation Journal, 19, 33–42.
Church, K. (1996). Beyond “bad manners”: The power relations of “consumer participation” in Ontario’s community mental health system. Canadian Journal of Community Mental Health, 15(2), 27–44.
CMHA et al. (2005). Consumer/survivor initiatives: Impact, outcomes and effectiveness. Retrieved March 11, 2006 from http://info.wlu.ca/∼wwwpsych/gnelson/csi%20advocacy%20paper%20july%202005.pdf.
CMHEI. (2004). Making a difference. Retrieved March 15, 2006 from http://www.ontario.cmha.ca/cmhei/images/report/Making_a_Difference.pdf.
Collins, A. (1998). In the sleep room: The story of the CIA brainwashing experiments in Canada. Toronto: Key Porter Books.
Creswell, J. W. (2002). Educational research: Planning, conducting and evaluating quantitative and qualitative research. Upper Saddle River, NJ: Merrill Prentice Hall.
CSDI. (1996). Consumer/Survivor Development Initiative project descriptions: “All 36 of the Consumer/Survivor Initiatives described in their own words.” Toronto: Consumer/Survivor Development Initiative.
Dorvil, H., Provencher, H., & Beaulieu, A. (2002). Attitudes et croyances des Montréalais et Montréalaises a l’égard des ex-patients psychiatriques vivant dans la société. Canadian Journal of Community Mental Health, 21(1), 153–178.
Everett, B. (1994). Something is happening: The contemporary consumer and psychiatric survivor movement in historical context. The Journal of Mind and Behavior, 15, 55–70.
Everett, B. (1998). Participation or exploitation? Consumers and psychiatric survivors as partners in planning mental health services. International Journal of Mental Health, 27(1), 80–97.
Everett, B. (2000). A fragile revolution: Consumers and psychiatric survivors confront the power of the mental health system. Waterloo, ON: Wilfrid Laurier University Press.
Fisher, P. A., & Ball, T. J. (2003). Tribal participatory research: Mechanisms of a collaborative model. American Journal of Community Psychology, 32, 207–216.
Foster-Fishman, P. G., & Van Egeren, L. (2002). Evaluating systems change. Paper presented at the American Evaluation Association’s Annual Conference, Washington, D.C.
Francisco, V. T., Paine, A. L., & Fawcett, S. B. (1993). A methodology for monitoring and evaluating community health coalitions. Health Education Research Theory and Practice, 8(3), 403–416.
Goering, P. N., & Streiner, D. L. (1996). Reconcilable differences: The marriage of qualitative and quantitative methods. Canadian Journal of Psychiatry, 41, 491–497.
Hardina, D., & Malott, O. W. (1996). Strategies for the empowerment of low income consumers on community-based planning boards. Journal of Progressive Human Services, 7(2), 43–61.
Hinrichs, K., & Kangas, O. (2003). When is a change big enough to be a system shift? Small system-shifting changes in German and Finnish pension policies. Social Policy and Administration, 37(6), 573–591.
Janzen, R., Nelson, G., Trainor, J., & Ochocka, J. (2006). A longitudinal study of mental health consumer/survivor initiatives: Part IV – Benefits beyond the self? A quantitative and qualitative study of system-level activities and impacts. Journal of Community Psychology, 34, 285–303.
Kemmis, S., & McTaggart, R. (2000). Participatory action research. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research 2nd ed. (pp. 567–605). Thousand Oaks: Sage.
Kyrouz, E. M., Humphreys, K., & Loomis, C. (2003). A review of the effectiveness of self-help mutual aid groups. In B. J. White & E. J. Madara (Eds.), A review of research on the effectiveness of self-help mutual aid groups 7th ed. (pp. 71–86). Cedar Knolls, NJ: American Self-Help Clearinghouse.
McLean, A. (1995). Empowerment and the psychiatric consumer/ex-patient movement in the United States: Contradictions, crisis, and change. Social Science and Medicine, 40(8), 1053–1071.
Ministry of Health and Long-term Care. (2005). Common Data Set – Mental Health (CDS-MH) Manual. Toronto: Author.
Nelson, G. (1994). The development of a mental health coalition: A case study. American Journal of Community Psychology, 22(2), 229–255.
Nelson, G., Ochocka, J., Griffin, K., & Lord, J. (1998). “Nothing about me, without me”: Participatory action research with self-help/mutual aid organizations for psychiatric consumer/survivors. American Journal of Community Psychology, 26, 881–912.
Nelson, G., Lord, J., & Ochocka, J. (2001). Shifting the paradigm in community mental health. Toronto, ON: University of Toronto Press.
Nelson, G., Ochocka, J., Janzen, R., Trainor, J., & Lauzon, S. (2004). A comprehensive evaluation framework for mental health consumer/survivor organizations: Values, conceptualization, design, and action. Canadian Journal of Program Evaluation, 19(3), 29–54.
Nelson, G., Ochocka, J., Janzen, R., & Trainor, J. (2006a). A longitudinal study of mental health consumer/survivor initiatives: Part I – Literature review and overview of the study. Journal of Community Psychology, 34, 247–260.
Nelson, G., Ochocka, J., Janzen, R., & Trainor, J. (2006b). A longitudinal study of mental health consumer/survivor initiatives: Part II – A quantitative study of impacts on new members. Journal of Community Psychology, 34, 261–272.
Nelson, G. & Prilleltensky, I. (Eds.), (2005). Community psychology: In pursuit of liberation and well-being. Basingstoke, UK: Palgrave.
Ochocka, J., Janzen, R., & Nelson, G. (2002). Sharing power and knowledge: Professional and mental health consumer/survivor researchers working together in a participatory action research project. Psychiatric Rehabilitation Journal, 25(4), 379–387.
Ochocka, J., Nelson, G., Janzen, R., & Trainor, J. (2006). A longitudinal study of mental health consumer/survivor initiatives: Part III – A qualitative study of impacts on new members. Journal of Community Psychology, 34, 273–283.
Ontario Ministry of Health. (1993). Putting people first: The reform of mental health services in Ontario. Toronto: Author.
Ontario Ministry of Health Provincial Mental Health Committee. (1988). Building community support for people: A plan for mental health in Ontario. Toronto: Author.
Reinharz, S. (1984). Alternative settings and social change. In K. Heller, R. H. Price, S. Reinharz, S. Riger & A. Wandersman (Eds.), Psychology and community change: Challenges of the future 2nd ed. (pp. 286–336). Homewood, IL: The Dorsey Press.
Rush, B., & Ogborne, A. (1991). Program logic models: Expanding their role and structure for program planning and evaluation. Canadian Journal of Program Evaluation, 6, 93–105.
Szasz T. (Ed.). (1974). The age of madness. New York: Jason Aronson.
Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining the qualitative and quantitative approaches. Thousand Oaks, CA: Sage. (Applied Social Research Methods, No 46).
Tefft, B. (1987). Advocacy coalitions as a vehicle for mental health system reform. In E. M. Bennett (Ed.), Social intervention: Theory and practice (pp. 155–185). Lewiston, New York: The Edwin Mellen Press.
Trainor, J., Shepherd, M., Boydell, K. M., Leff, A., & Crawford, E. (1997). Beyond the service paradigm: The impact and implications of consumer/survivor initiatives. Psychiatric Rehabilitation Journal, 21(2), 132–140.
Tseng, V., Chesir-Deran, D., Becker-Klein, R., Chan, M. L., Duran, V., Roberts, A., & Bardoliwalla, N. (2002). Promotion of social change: A conceptual framework. American Journal of Community Psychology, 30, 401–427.
Weitz, D. (1984). “On our own”: A self-help model. In D. P. Lumsden (Ed.), Community mental health action: Primary prevention programming in Canada (pp. 312–320). Ottawa: Canadian Public Health Association.
Yoshikawa, H., & Shinn, M. (2002). Facilitating change: Where and how should community psychology intervene? In T. A. Revenson, A. R. D’Augelli, S. E. French, D. L. Hughes, D. Livert, E. Seidman, M. Shinn & H. Yoshikawa (Eds.), A quarter century of community psychology: Readings from the American Journal of Community Psychology (pp. 33–49). New York: Kluwer Academic/Plenum Publishers.
Zimmerman, M. (2000). Empowerment theory: Psychological, organizational, and community levels of analysis. In J. Rappaport & E. Seidman (Eds.), Handbook of community psychology (pp. 43–63). New York: Kluwer Academic/Plenum.
Acknowledgement
This research was one of several projects funded under the Community Mental Health Evaluation Initiative (CMHEI) by the Ontario Ministry of Health and Long-Term Care through the Ontario Mental Health Foundation. We would like to thank staff members of the Centre for Research and Education in Human Services, who comprised the research team for this project and who assisted with data collection and analysis; members of the Ontario Peer Development Initiative and the four Consumer/Survivor Initiatives who participated on the project steering committee and guided the conceptualization, implementation, interpretation, and dissemination of the research; and the investigators for the CMHEI and research staff of the Health Systems Research and Consulting Unit, Centre for Addiction and Mental Health—Clarke Division, Toronto, for their work in coordinating all of the projects and for processing all of the data.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Janzen, R., Nelson, G., Hausfather, N. et al. Capturing system level activities and impacts of mental health consumer-run organizations. Am J Community Psychol 39, 287–299 (2007). https://doi.org/10.1007/s10464-007-9107-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-007-9107-2