Abstract
Objective
The introduction of the Milestone Project underscored the need for objective assessments of resident progress across the competencies. Therefore, the authors examined the Psychiatry Resident-In-Training Examination (PRITE) utility for measuring improvements in medical knowledge (MK).
Methods
The authors compared the mean performance for each MK subcompetency by resident year for all residents taking the PRITE from 2015 to 2017 (18,175 examination administrations). In addition, they surveyed psychiatry residency program directors regarding how well they thought they teach these subcompetencies.
Results
Increases in MK subcompetencies by resident year were significant for Psychopathology (p < 0.003), Psychotherapy (p < 0.002), and Somatic Therapies (p < 0.000). Development, Clinical Neuroscience, and Practice of Psychiatry did not show statistically significant differences between postgraduate years. Eighty psychiatry program directors responded to the survey and felt optimistic about their ability to teach the Psychopathology, Psychotherapy, Somatic Therapies, and Practice of Psychiatry subcompetencies.
Conclusions
The PRITE measured significant improvements in medical knowledge for several of the core subcompetencies. The program director’s responses would suggest that the lack of statistically significant differences found for Development and Clinical Neuroscience reflects areas in need of curricular development. The disparity between PRITE performance and program director perception of the Practice of Psychiatry subcompetency may reflect difficulties in defining the scope of this subcompetency. Overall, this suggests that structured examinations help measure improvements in certain subcompetencies and may also help identify curricular needs. However, there may be potential problems with the definition of some subcompetencies.
Similar content being viewed by others
References
Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system–rationale and benefits. N Engl J Med. 2012;366:1051–6.
Conforti LN, Yaghmour NA, Hamstra SJ, Holmboe ES, Kennedy B, Liu JJ, et al. The effect and use of milestones in the assessment of neurological surgery residents and residency programs. J Surg Educ. 2018;75:147–55.
Nabors C, Forman L, Peterson SJ, Gennarelli M, Aronow WS, DeLorenzo L, et al. Milestones: a rapid assessment method for the Clinical Competency Committee. Arch Med Sci AMS. 2017;13:201–9.
Meier AH, Gruessner A, Cooney RN. Using the ACGME milestones for resident self-evaluation and faculty engagement. J Surg Educ. 2016;73:e150–7.
Quinn SM, Worrilow CC, Jayant DA, Bailey B, Eustice E, Kohlhepp J, et al. Using milestones as evaluation metrics during an emergency medicine clerkship. J Emerg Med. 2016;51:426–31.
Colbert CY, Dannefer EF, French JC. Clinical competency committees and assessment: changing the conversation in graduate medical education. J Grad Med Educ. 2015;7:162–5.
Swing SR, Cowley DS, Bentman A. Assessing resident performance on the psychiatry milestones. Acad Psychiatry. 2014;38:294–302.
Guerrero APS, Beresin EV, Balon R, Brenner AM, Louie AK, Coverdale JH, et al. The competency movement in psychiatric education. Acad Psychiatry. 2017;41:312–4.
Beresin EV, Balon R, Coverdale J. The psychiatry milestones: new developments and challenges. Acad Psychiatry. 2014;38:249–52.
Isaak R, Stiegler M, Hobbs G, Martinelli SM, Zvara D, Arora H, et al. Comparing real-time versus delayed video assessments for evaluating ACGME sub-competency milestones in simulated patient care environments. Cureus. 2018;10:e2267.
Padilla A, Benjamin S, Lewis-Fernandez R. Assessing cultural psychiatry milestones through an objective structured clinical examination. Acad Psychiatry. 2016;40:600–3.
Dingle AD, Boland R, Travis M. The PRITE examinations: background and future directions. Acad Psychiatry. 2018;42:498–502.
Hift RJ. Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014;14:249.
Boland RJ, Lester NA, Williams E. Writing multiple-choice questions. Acad Psychiatry. 2010;34:310–6.
Thomas CR, Keepers G. The milestones for general psychiatry residency training. Acad Psychiatry. 2014;38:255–60.
The Accreditation Council for Graduate Medical Education. The Psychiatry Milestone Project [Internet]. [cited 2018 Aug 14]. Available from: https://www.acgme.org/Portals/0/PDFs/Milestones/PsychiatryMilestones.pdf?ver=2015-11-06-120520-753
Dewan M, Manring J, Satish U. The new milestones: do we need to take a step back to go a mile forward? Acad Psychiatry. 2015;39:147–50.
Fung LK, Akil M, Widge A, Roberts LW, Etkin A. Attitudes toward neuroscience education among psychiatry residents and fellows. Acad Psychiatry. 2014;38:127–34.
Benjamin S, Travis MJ, Cooper JJ, Dickey CC, Reardon CL. Neuropsychiatry and neuroscience education of psychiatry trainees: attitudes and barriers. Acad Psychiatry. 2014;38:135–40.
Fung LK, Akil M, Widge A, Roberts LW, Etkin A. Attitudes toward neuroscience education in psychiatry: a national multi-stakeholder survey. Acad Psychiatry. 2015;39:139–46.
Benjamin S, Widge A, Shaw K. Neuropsychiatry and neuroscience milestones for general psychiatry trainees. Acad Psychiatry. 2014;38:275–82.
Coverdale J, Balon R, Beresin EV, Louie AK, Tait GR, Goldsmith M, et al. Teaching clinical neuroscience to psychiatry residents: model curricula. Acad Psychiatry. 2014;38:111–5.
Ross DA, Travis MJ, Arbuckle MR. The future of psychiatry as clinical neuroscience: why not now? JAMA Psychiat. 2015;72:413–4.
National Neuroscience Curriculum Initiative. Available at www.NNCIonline.org. Accessed 31 August 2021
Acknowledgements
The authors wish to thank Laura Edgar and Stanley Hamstra of the Accreditation Council for Graduate Medical Education, Nancy Siderits of Prometric, and Kathryn Delk of the American College of Psychiatrists for their assistance and advice at various stages of this project.
Funding
The authors received support from the American College of Psychiatrists, who provided access to the required data and funding for the effort required to isolate and export the needed data.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Considerations
The protocol was reviewed by the Partners Healthcare Institutional Review Board, which determined that the project met the criteria for exemption 45 CFR 46.101(b) (1) Research conducted in established or commonly accepted educational settings involving normal educational practices.
Disclosures
The authors were all members of the PRITE Editorial Board or PRITE Commission at the time of this project. They did not receive any financial or other support as part of this role.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Boland, R.J., Dingle, A.D., Travis, M.J. et al. Using the Psychiatry Resident-In-Training Examination (PRITE) to Assess the Psychiatry Medical Knowledge Milestones in Psychiatry. Acad Psychiatry 46, 331–337 (2022). https://doi.org/10.1007/s40596-021-01537-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40596-021-01537-5