Abstract
Purpose
Surgical skill assessment is essential for safe operations. In endoscopic kidney stone surgery, surgeons must perform a highly skill-dependent mental map** from the pre-operative scan to the intraoperative endoscope image. Poor mental map** can lead to incomplete exploration of the kidney and high reoperation rates. Yet there are few objective ways to evaluate competency. We propose to use unobtrusive eye-gaze measurements in the task space to evaluate skill and provide feedback.
Methods
We capture the surgeons’ eye gaze on the surgical monitor with the Microsoft Hololens 2. To enable stable and accurate gaze detection, we develop a calibration algorithm to refine the eye tracking of the Hololens. In addition, we use a QR code to locate the eye gaze on the surgical monitor. We then run a user study with three expert and three novice surgeons. Each surgeon is tasked to locate three needles representing kidney stones in three different kidney phantoms.
Results
We find that experts have more focused gaze patterns. They complete the task faster, have smaller total gaze area, and the gaze fewer times outside the area of interest. While fixation to non-fixation ratio did not show significant difference in our findings, tracking the ratio over time shows different patterns between novices and experts.
Conclusion
We show that a non-negligible difference holds between novice and expert surgeons’ gaze metrics in kidney stone identification in phantoms. Expert surgeons demonstrate more targeted gaze throughout a trial, indicating their higher level of proficiency. To improve the skill acquisition process for novice surgeons, we suggest providing sub-task specific feedback. This approach presents an objective and non-invasive method to assess surgical competence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Conti SL, Brubaker W, Chung BI, Sofer M, Hsi RS, Shinghal R, Elliott CS, Caruso T, Leppert JT (2019) Crowdsourced assessment of ureteroscopy with laser lithotripsy video feed does not correlate with trainee experience. J Endourol 33(1):42–49
Khan SR, Pearle MS, Robertson WG, Gambaro G, Canales BK, Doizi S, Traxer O, Tiselius H-G (2016) Kidney stones. Nat Rev Dis Primers 2(1):1–23
Chong E, Clark-Whitney E, Southerland A, Stubbs E, Miller C, Ajodan E, Silverman M, Lord C, Rozga A, Jones R, Rehg J (2020) Detection of eye contact with deep neural networks is as accurate as human experts. Nat Commun 11:6386
Klingner J, Kumar R, Hanrahan P (2008) Measuring the task-evoked pupillary response with a remote eye tracker. In: Proceedings of the 2008 symposium on eye tracking research & applications. ETRA ’08. Association for Computing Machinery, New York, pp 69–72
Zheng B, Jiang X, Bednarik R, Atkins M (2021) Action-related eye measures to assess surgical expertise. BJS Open 5(5):068
Richstone L, Schwartz MJ, Seideman C, Cadeddu J, Marshall S, Kavoussi LR (2010) Eye metrics as an objective assessment of surgical skill. Ann Surg 252(1):177–182
Law B, Atkins MS, Kirkpatrick AE, Lomax AJ (2004) Eye gaze patterns differentiate novice and experts in a virtual laparoscopic surgery training environment. In: Proceedings of the 2004 symposium on eye tracking research & applications. Association for Computing Machinery, San Antonio, Texas, pp 41–48
Tien G, Zheng B, Atkins MS (2011) Quantifying surgeons’ vigilance during laparoscopic operations using eyegaze tracking. In: Medicine Meets Virtual Reality 18, pp. 658–662. IOS Press, School of Computing Science, Simon Fraser University, Burnaby, BC, Canada
Khan RS, Tien G, Atkins MS, Zheng B, Panton ON, Meneghetti AT (2012) Analysis of eye gaze: do novice surgeons look at the same location as expert surgeons during a laparoscopic operation? Surg Endosc 26(12):3536–3540
Fichtel E, Lau N, Park J, Henrickson Parker S, Ponnala S, Fitzgibbons S, Safford SD (2019) Eye tracking in surgical education: gaze-based dynamic area of interest can discriminate adverse events and expertise. Surg Endosc 33(7):2249–2256
Vine SJ, Masters RS, McGrath JS, Bright E, Wilson MR (2012) Cheating experience: guiding novices to adopt the gaze strategies of experts expedites the learning of technical laparoscopic skills. Surgery 152(1):32–40
Liu S, Donaldson R, Subramaniam A, Palmer H, Champion CD, Cox ML, Appelbaum LG (2021) Develo** expert gaze pattern in laparoscopic surgery requires more than behavioral training. J Eye Mov Res 14(2)
Berges AJ, Vedula SS, Chara A, Hager GD, Ishii M, Malpani A (2022) Eye tracking and motion data predict endoscopic sinus surgery skill. Laryngoscope 133:500–505
Lu S, Sanchez Perdomo YP, Jiang X, Zheng B (2020) Integrating eye-tracking to augmented reality system for surgical training. J Med Syst 44(11):1–7
Gil AM, Birdi S, Kishibe T, Grantcharov TP (2022) Eye tracking use in surgical research: a systematic review. J Surg Res 279:774–787
Graham RL (1972) An efficient algorithm for determining the convex hull of a finite planar set. Info Pro Lett 1:132–133
Goldenberg M, Ordon M, Honey JRD, Andonian S, Lee JY (2020) Objective assessment and standard setting for basic flexible ureterorenoscopy skills among urology trainees using simulation-based methods. J Endourol 34(4):495–501
Valovska M-T, Gomez G, Fineman R, Woltmann W, Stirling L, Wollin DA (2022) Analysis of flexible ureteroscopic motion and kinematic efficiency: a simulation-based pilot study. J Endourol 36(6):855–861
Biyani CS, Kailavasan M, Rukin N, Palit V, Somani B, Jain S, Myatt A, Nabi G, Patterson J (2021) Global assessment of urological endoscopic skills (GAUEs): development and validation of a novel assessment tool to evaluate endourological skills. BJU Int 127(6):665–675
Acknowledgements
This work was supported in part by the Vanderbilt Institute for Surgery and Engineering (VISE) Physician in Residence Program.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
None.
Ethics approval
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Vanderbilt University Medical Center (IRB No. 220270).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
A Appendix
A Appendix
In Fig. 7, we show the overall procedure and workflow for the whole experiment.
To verify the accuracy of our method, we devise a new accuracy test for our setup. Five squares are placed on the virtual plane. The users are required to focus on those squares in the same way as they did in the calibration procedure. The eye gaze coordinates and squares’ coordinates are recorded. We required users to stand at four different positions in front of the monitor and do the calibration procedure. In Table 4, we show the mean errors for all five squares. We also calculate the errors after calibration. Figure 8 shows the view of calibration under four different positions.
Overall procedure and workflow for the whole experiment
We randomly choose four positions in front of the endoscope monitor and do the calibration procedure
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, Y., Reed, A., Kavoussi, N. et al. Eye gaze metrics for skill assessment and feedback in kidney stone surgery. Int J CARS 18, 1127–1134 (2023). https://doi.org/10.1007/s11548-023-02901-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-023-02901-6