Summary
Background
Evaluation variables that correlate with future negative outcomes in surgery allow for actionable steps and early intervention. Diverse evaluation systems have been used to quantify progression of residents. This study assessed our internal evaluation system to evaluate for variables that could predict negative outcomes in surgical education and, secondarily, whether differences in resident rating existed based on faculty–resident gender dyad.
Methods
This is a retrospective single-institution review of general surgery resident evaluations (2008–2020). Demographics, simulation lab, technical skill, and percentile for overall clinical performance are given by attendings. Logistic regression models were used to determine whether any factors were associated with negative resident outcomes and to determine if the faculty–resident gender dyad influenced evaluation ratings.
Results
A total of 3117 evaluations of 116 residents were included (66% male). Twenty negative outcomes were identified. In univariate logistic regression model American Board of Surgery In-Service Training Exam (ABSITE) scores, simulation lab scores and technical evaluations ≤ 20th percentile were associated with negative outcomes (p = 0.04). In multivariate analysis, no variables were associated with negative outcomes. Male staff tended to grade residents lower than female staff, independent of resident gender.
Conclusion
Although multiple subjective and objective variables, including national standardized tests, of surgical trainees are measured and reported, none of the variables collected over 10 years in 116 residents including over 3000 evaluations were independently associated with negative outcomes. This points to a need for development of new objective variables that can predict negative outcomes in residency. Preemptive identification of “at-risk” residents remains elusive.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Morris-Wiseman LF, Nfonsam VN. Early detection and remediation of problem learners. Surg Clin North Am. 2021;101(4):611–24. https://doi.org/10.1016/j.suc.2021.05.008.
Gardner AK, AbdelFattah K. Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation: are they telling the same story? Am J Surg. 2017;214(3):547–53. https://doi.org/10.1016/j.amjsurg.2016.07.025.
Cassidy DJ, Chakraborty S, Panda N, et al. The surgical knowledge “growth curve”: predicting ABSITE scores and identifying “at-risk” residents. J Surg Educ. 2021;78(1):50–9. https://doi.org/10.1016/j.jsurg.2020.06.038.
Gardner AK, Scott DJ. Repaying in kind: examination of the reciprocity effect in faculty and resident evaluations. J Surg Educ. 2016;73(6):e91–e4. https://doi.org/10.1016/j.jsurg.2016.04.015.
Staats C. Understanding Implicit Bias. 6.
Moers F. Discretion and bias in performance evaluation: the impact of diversity and subjectivity. Account Organ Soc. 2005;30(1):67–80. https://doi.org/10.1016/j.aos.2003.11.001.
Shellito AD, de Virgilio C, Lee G, et al. Investigating association between sex and faculty teaching evaluation in general surgery residency programs: a multi-institutional study. j Am Coll Surg. 2020;231(3):309–15e. https://doi.org/10.1016/j.jamcollsurg.2020.05.026.
Dill-Macky A, Hsu CH, Neumayer LA, Nfonsam VN, Turner AP. The role of implicit bias in surgical resident evaluations. J Surg Educ. 2021; https://doi.org/10.1016/j.jsurg.2021.12.003.
Rand VE, Hudes ES, Browner WS, Wachter RM, Avins AL. Effect of evaluator and resident gender on the American board of internal medicine evaluation scores. J Gen Intern Med. 1998;13(10):670–4. https://doi.org/10.1046/j.1525-1497.1998.00202.x.
Angelo JL, Moazzez A, Neville A, Dauphine C, Lona Y, de Virgilio C. Investigating gender differences in faculty evaluations by trainees in a gender-balanced general surgery program. J Surg Educ. 2019;76(6):e132–e7. https://doi.org/10.1016/j.jsurg.2019.06.007.
Atesok K, Satava RM, Marsh JL, Hurwitz SR. Measuring surgical skills in simulation-based training. J Am Acad Orthop Surg. 2017;25(10):665–72. https://doi.org/10.5435/JAAOS-D-16-00253.
Sanfey H. Assessment of surgical training. Surgeon. 2014;12(6):350–6. https://doi.org/10.1016/j.surge.2014.07.001.
James OP, Robinson DBT, Hopkins L, et al. Strategic organisational skills predict surgical training success. Postgrad Med J. 2020; https://doi.org/10.1136/postgradmedj-2020-138763.
Deal SB, Scully RE, Wnuk G, George BC, Alseidi AA. Crowd-sourced and attending assessment of general surgery resident operative performance using global ratings scales. J Surg Educ. 2020;77(6):e214–e9. https://doi.org/10.1016/j.jsurg.2020.07.011.
Abdelsattar JM, AlJamal YN, Ruparel RK, Rowse PG, Heller SF, Farley DR. Correlation of objective assessment data with general surgery resident in-training evaluation reports and operative volumes. J Surg Educ. 2018;75(6):1430–6. https://doi.org/10.1016/j.jsurg.2018.04.016.
Ahle SL, Schuller M, Clark MJ, et al. Do end-of-rotation evaluations adequately assess readiness to operate? Acad Med. 2019;94(12):1946–52. https://doi.org/10.1097/ACM.0000000000002936.
Velez DR. Prospective factors that predict American board of surgery in-training examination performance: a systematic review. am Surg. 2021;87(12):1867–78. https://doi.org/10.1177/00031348211058626.
Gerull KM, Loe M, Seiler K, McAllister J, Salles A. Assessing gender bias in qualitative evaluations of surgical residents. Am J Surg. 2019;217(2):306–13. https://doi.org/10.1016/j.amjsurg.2018.09.029.
Meyerson SL, Odell DD, Zwischenberger JB, et al. The effect of gender on operative autonomy in general surgery residents. Surgery. 2019;166(5):738–43. https://doi.org/10.1016/j.surg.2019.06.006.
Brady JM, Bray A, Kim P, et al. Female residents give themselves lower scores than male colleagues and faculty evaluators on ACGME milestones. J Surg Educ. 2021;78(4):1305–11. https://doi.org/10.1016/j.jsurg.2020.12.003.
Skjold-Ødegaard B, Ersdal HL, Assmus J, Soreide K. Comparison of performance score for female and male residents in general surgery doing supervised real-life laparoscopic appendectomy: is there a norse shield-maiden effect? World J Surg. 2021;45(4):997–1005. https://doi.org/10.1007/s00268-020-05921-4.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
T. Szabo Yamashita, S. Lund,V. J.-H. Yeh, M. Rivera, T. J. McKenzie, J. M. Stulak, W. S. Harmsen, F. Abraha, S. F. Heller, and J. W. Jakub declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Szabo Yamashita, T., Lund, S., Yeh, V.J.H. et al. Resident evaluations: what are the predictors of future negative outcomes in surgical education?. Eur Surg 55, 107–111 (2023). https://doi.org/10.1007/s10353-023-00797-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10353-023-00797-9