Abstract
Purpose
Surgery is continuously subject to technological and medical innovations that are transforming daily surgical routines. In order to gain a better understanding and description of surgeries, the field of surgical process modelling (SPM) has recently emerged. The challenge is to support surgery through the quantitative analysis and understanding of operating room activities. Related surgical process models can then be introduced into a new generation of computer-assisted surgery systems.
Methods
In this paper, we present a review of the literature dealing with SPM. This methodological review was obtained from a search using Google Scholar on the specific keywords: “surgical process analysis”, “surgical process model” and “surgical workflow analysis”.
Results
This paper gives an overview of current approaches in the field that study the procedural aspects of surgery. We propose a classification of the domain that helps to summarise and describe the most important components of each paper we have reviewed, i.e., acquisition, modelling, analysis, application and validation/evaluation. These five aspects are presented independently along with an exhaustive list of their possible instantiations taken from the studied publications.
Conclusion
This review allows a greater understanding of the SPM field to be gained and introduces future related prospects.
Similar content being viewed by others
References
Cleary K, Chung HY, Mun SK (2005) OR 2020: the operating room of the future. Laparoendosc Adv Surg Tech 15(5):495–500
Rattner WD, Park A (2003) Advanced devices for the operating room of the future. Semin Laparosc Surg 10(2):85–88
Satava RM, Carrico CJ (1996) Advanced simulation technologies for surgical education. Bull Am College Surg 81(7):71–77
Taylor C, Draney MT, Ku JP, Parker D, Steele BN, Wang K, Zarins CK (1999) Predictive medicine: computational techniques in therapeutic decision making. Comput Aid Surg 4:231–247
MacKenzie CL, Ibbotson AJ, Cao CGL, Lomax A (2001) Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Minim Invasive Ther Allied Technol 10(3):121–128
Jannin P, Raimbault M, Morandi X, Seigneuret E, Gibaud B (2001) Design of a neurosurgical gestures model for multimodal image guided surgery. Comput Assisted Radiol Surg 1230:102–107
Münchenberg J, Brief J, Raczkowsky J, Wörn H, Hassfeld S, Mühling J (2000) Operation planning of robot supported surgical interventions. Int Conf Intell Robots Syst 1:547–552
Jannin P, Raimbault M, Morandi X, Riffaud L, Gibaud B (2003) Model of surgical procedures for multimodal image-guided neurosurgery. Computer Aided Surg 8(2):98–106
Jannin P, Morandi X (2007) Surgical models for computer-assisted neurosurgery. Neuroimage 37(3):783–791
Neumuth T, Trantakis C, Eckhardt F, Dengl M, Meixensberger J, Burgert O (2007) Supporting the analysis of inter-vention courses with surgical process models on the example of fourteen microsurgical lumbar discectomies. Int J Comput Assist Radiol Surg 2(1):436–438
Cao CGL, MacKenzie CL, Payandeh S (1996) Task and motion analysis in endoscopic surgery. In: ASME dynamic systems, 5th annual symposium on haptic interface for virtual environment and teleoperation
Ibbotson JA, MacKenzie CL, Cao CG, Lomax AJ (1999) Gaze patterns in laparoscopic surgery. Stud Health Technol Inform 7:154–160
Lo B, Darzi A, Yang G (2003) Episode classification for the analysis of tissue-instrument interaction with multiple visual cues. Med Image Comput Comput Assist Interv 1:231–237
Lin HC, Shafran I, Yuh D, Hager GD (2006) Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Computer Aided Surg 11(5):220–230
Bhatia B, Oates T, **ao Y, Hu P (2007) Real-time identification of operating room state from video. AAAI 2:1761–1766
Hu P, Ho D, MacKenzie CF, Hu H, Martz D, Jacobs J, Voit R, **ao Y (2006) Advanced Visualization platform for surgical operating room coordination. Distributed video board system. Surg Innov 13(2):129–135
Sandberg WS, Daily B, Egan MT, Stahl JE, Goldman JM, Wiklund RA, Rattner D (2005) Deliberate perioperative systems design improves operating room throughput. Anesthesiology 103:406–418
**ao Y, Hu P, Hu H, Ho D, Dexter F, Mackenzie CF, Seagull FJ (2005) An algorithm for processing vital sign monitoring data to remotely identify operating room occupancy in real-time. Anesth Analg 101(3):32–823
Ahmadi A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2006) Recovery of surgical workflow without explicit models. Med Image Comput Comput Assist Interv 9(1):420–428
James A, Vieira D, Lo BPL, Darzi A, Yang GZ (2007) Eye-gaze driven surgical workflow segmentation. Med Image Comput Comput Assist Interv 10(2):110–117
Katic D, Sudra G, Speidel S, Castrillon-Oberndorfer G, Eggers G, Dillman R (2010) Knowledge-based situation interpretation for context-aware augmented reality in dental implant surgery. Med Imaging Augment Real 6326:531–540
Klank U, Padoy N, Feussner H, Navab N (2008) Automatic feature generation in endoscopic images. Int J Comput Assisted Radiol Surg 3(3,4):331–339
Lalys F, Riffaud L, Bouget D, Jannin P (2012) A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 59(4):966–976
Nara A, Izumi K, Iseki H, Suzuki T, Nambu K, Sakurai Y (2011) Surgical workflow monitoring based on trajectory data mining. New Frontiers Artif Intell 6797:283–291
Padoy N, Horn M, Feussner H, Berger M, Navab N (2007) Recovery of surgical workflow: a model-based approach. Int J Comput Assist Radiol Surg 2(1):481–482
Padoy N, Blum T, Feuner H, Berger MO, Navab N (2008) On-line recognition of surgical activity for monitoring in the operating room. Conf Inno App Art Intell 3:1718–1724
Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2010) Statistical modeling and recognition of surgical workflow. Med Image Anal 16(3):632–641
Qi J, Jiang Z, Zhang G, Miao R, Su Q (2006) A surgical management information system driven by workflow. IEEE Conf Serv Oper Logist Inf 1014–1018
Suzuki T, Yoshimitsu K, Tamura M, Muragaki Y, Iseki H (2012) Video information management system for information guided neurosurgery. Computer Aided Surg 3(3):75–82
Thiemjarus S, James A, Yang GZ (2012) An eye-hand data fusion framework for pervasive sensing of surgical activities. Pattern Recognit 45(8):2855–2867
Blum T, Padoy N, Feussner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assisted Radiol Surg 3(5):379–386
Bouarfa L, Jonker PP, Dankelman J (2011) Discovery of high-level tasks in the operating room. J Biomed Inform 44(3): 455–462
Fischer M, Strauss G, Burgert O, Dietz A, Trantakis C, Meixensberger J, Lemke HU (2005) ENT-surgical workflow as an instrument to assess the efficiency of technological developments in medicine. Comput Assisted Radiol Surg 1281:851–855
Ko SY, Kim J, Lee WJ, Kwon DS (2007) Surgery task model for intelligent interaction between surgeon and laparoscopic assistant robot. J Robot Mechatron 8(1):38–46
Lemke HU, Trantakis C, Köchy K, Müller A, Strauss G, Meixensberger J (2004) Workflow analysis for mechatronic and imaging assistance in head surgery. Int Congress Ser 1268:830–835
Malarme P, Wikler D, Warzée N (2011) Intraoperative capture of surgical workflow. Comput Assist Radiol Surg 6:S146–S147
Ahmadi A, Padoy N, Rybachuk K, Feussner H, Heining SM, Navab N (2009) Motif discovery in OR sensor data with application to surgical workflow analysis and activity detection. M2CAI workshop, Med Image Comput Comput Assist Interv
Nomm S, Petlenkov E, Vain J, Belikov J, Miyawaki F, Yoshimitsu K (2008) Recognition of the surgeon’s motions during endoscopic operation by statistics based algorithm and neural networks based ANARX models. Proc Int Fed Automatic Control 17(1)
Burgert O, Neumuth T, Lempp F, Mudunuri R, Meixensberger J, Strauß G, Dietz A, Jannin P, Lemke HU (2006) Linking top-level ontologies and surgical workflows. Int J Comput Assist Radiol Surg 1(1):437–438
Yoshimitsu K, Masamune K, Iseki H, Fukui Y, Hashimoto D, Miyawaki F (2010) Development of scrub nurse robot (SNR) systems for endoscopic and laparoscopic surgery. Micro NanoMechatron Hum Sci 83–88
Speidel S, Sudra G, Senemaud J, Drentschew M, Müller-Stich BP, Gun C, Dillmann R (2008) Recognition of risk situations based on endoscopic instrument tracking and knowledge based situation modeling. Progress Biomed Optics Imaging 9(1):35
Sudra G, Speidel S, Fritz D, Möller-Stich BP, Gutt C, Dillmann R (2007) MEDIASSIST: MEDIcal ASSITance for intraoperative skill transfer in minimally invasive surgery using augmented reality. Progress Biomed Optics Imaging 8(2)
Meng F, D’Avolio LW, Chen AA, Taira RK, Kangarloo H (2005) Generating models of surgical procedures using UMLS concepts and multiple sequence alignment. Am Med Inform Assoc Annu Symp Proc 520–524
Neumuth T, Durstewitz N, Fischer M, Strauss G, Dietz A, Meixensberger J, Jannin P, Cleary K, Lemke HU, Burgert O (2006) Structured recording of intraoperative surgical workflows. SPIE Med Imaging PACS Surg 6145:61450A
Agarwal S, Joshi A, Finin T, Yesha Y, Ganous T (2007) A pervasive computing system for the operating room of the future. Mobile Netw Appl 12(2,3):215–228
Forestier G, Lalys F, Riffaud L, Trelhu B, Jannin P (2012) Classification of surgical processes using dynamic time war**. J Biomed Inform 45:255–264
Neumuth T, Schumann S, Strauss G, Jannin P, Meixensberger J, Dietz A, Lemke HU, Burgert O (2006) Visualization options for surgical workflows. Int J Comput Assisted Radiol Surg 1(1):438–440
Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2008) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16(1):72–82
Neumuth T, Loebe F, Jannin P (2012) Similarity metrics for surgical process models. Artif Intell Med 54(1):15–27
Neumuth T, Liebmann P, Wiedemann P, Meixensberger J (2012) Surgical workflow management schemata for cataract procedures. Process Model-based design and validation of workflow schemata. Methods Inf Med 51(4)
Riffaud L (2010) Recording of surgical processes: a study comparing senior and junior neurosurgeons during lumbar disc herniation surgery. Neurosurgery. 67(2):325–332
Kragic D, Hager G (2003) Task modelling and specification for modular sensory based human-machine cooperative systems. Int Conf Intell Robots Syst 3:3192–3197
Haro BB, Zapella L, Vidal R (2012) Surgical gesture classification from video data. Med Image Comput Comput Assist Interv 7510:34–41
Lalys F, Bouget D, Riffaud L, Jannin P (2012) Automatic knowledge-based recognition of low-level tasks in opthalmological procedures. Int J Comput Assist Radiol Surg 8(1):39–49
Miyawaki F, Masamune K, Suzuki S, Yoshimitsu K, Vain J (2005) Scrub nurse and timed-automata-based model for surgery. IEEE Ind Electron Trans 5(52):1227–1235
Neumuth T, Meissner C (2012) Online recognition of surgical instruments by information fusion. Int J Comput Assisted Radiol Surg 7(2):297–304
Weinger MB, Herndon OW, Zornow MH, Paulus MP, Gaba DM, Dallen LT (1994) An objective methodology for task analysis and workload assessment in anesthesia providers. Anesthesiology 80(1):77–92
Reiley CE, Lin HC, Yuh DD, Hager GD (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25(2):356–366
Rosen J, Hannaford B, Sinanan M, Solazzo M (2001) Objective evaluation of laparoscopic surgical skills using hidden Markov models based on haptic information and tool/tissue interactions. Stud Health Technol Inform 81:417–423
Yule S, Flin R, Paterson-Brown S, Maran N (2006) Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 193(2):140–149
Patel VL, Arocha JF et al (2001) A primer on aspects of cognition for medical informatics. J Am Med Inform Assoc 8(4):324–343
Dexter F, Epstein RH, Traub RD, **ao Y (2004) Making management decisions on the day of surgery based on operating room efficiency and patient waiting times. Anesthesiology 101(6):1444–1453
Dexter F, Epstein RH, Lee JD, Ledolter J (2009) Automatic updating of times remaining in surgical cases using Bayesian analysis of historical case duration data and instant messaging updates from anesthesia providers. Anesth Analg 108:929–940
Coles EC, Slavin G (1976) An evaluation of automatic coding of surgical pathology reports. J Clin Pathol 29(7):621–626
Jannin P, Grova C, Maurer CR (2006) Model for defining and reporting reference-based validation protocols in medical image processing. Int J Comput Assist Interv 1(2):1001–1115
Den Boer KT, de Wit LT, Davids PHP, Dankelman J, Gouma DJ (2001) Analysis of the quality and efficiency of learning laparoscopic skills. Surg Endosc 15:497–503
Sjoerdsma W, Meijer D, Jansen A, den Boer KT, Grimbergen CA (2000) Comparison of efficiencies of three techniques for colon surgery. J Laparoendosc Adv Surg Tech 10(1):47–53
Darzi A, Mackay S (2002) Skills assessment of surgeons. Surgery 131(2):121–124
Bann MS (2003) Measurement of surgical dexterity using motion analysis of simple bench skills. World J Surg 27:390–394
Dosis A, Bello F, Moorthy K, Munz Y, Gillies D, Darzi A (2004) Real-time synchronization of kinematic and video data for the comprehensive assessment of surgical skills. Stud Health Technol Inform 98:82–88
Mehta NY, Haluck RS, Frecker MI, Snyder AJ (2002) Sequence and task analysis of instrument use in common laparoscopic procedures. Surg Endosc 16(2):280–285
Malik R, White P, Macewen C (2003) Using human reliability analysis to detect surgical error in endoscopic DCR surgery. Clin Otolaryngol Allied Sci 28:456–460
Claus GP, Sjoerdsma W, Jansen A, Grimbergen CA (1995) Quantitative standardised analysis of advanced laparoscopic surgical procedures. Endosc Surg Allied Technol 3:210–213
Payandeh S, Lomax AJ, Dill J, Mackenzie CL, Cao CGL (2002) On defining metrics for assessing laparoscopic surgical skills in a virtual training environment. Stud Health Technol Inform 85:334–340
Hager G, Vagvolgyi B, Yuh D (2007) Stereoscopic video overlay with deformable registration. Medicine Meets Virtual Reality
Rosen J, Solazzo M, Hannaford B, Sinanan M (2002) Task decomposition of laparoscopic surgery for objective evaluation of surgical residents’ learning curve using hidden markov model. Comput Aided Surg 7(1):49–61
Rosen J, Brown JD, Chang L, Sinanan M, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans Biomed Eng 53(1):399–413
Voros S, Hager GD (2008) Towards “real-time” tool-tissue interaction detection in robotically assisted laparoscopy. Int Conf IEEE Biomed Robot Biomechat. 562–567
Radrich H (2008) Vision-based motion monitoring trough data fusion from a chirurgical multi-camera recording system. Diploma thesis, TUM, Munich
Radrich H, Padoy N, Ahmadi A, Feussner H, Hager G, Burschka D, Knoll A (2009) Synchronized multimodal recording system for laparoscopic minimally invasive surgeries. M2CAI workshop, Med Image Comput Comput Interv
Sielhorst T, Blum T, Navab N (2005) Synchronizing 3d movements for quantitative comparison and simultaneous visualization of actions. Int Symp Mix Augm Real 38–47
Speidel S, Benzko J, Krappe S, Sudra G, Azad P, Müller-Stich BP, Gutt C, Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. Progress Biomed Optics Imaging 10(1):37
Winter A, Brigl B, Wendt T (2003) Modeling hospital information systems (Part 1): the revised three-layer graph-based meta model 3LGM2. Method Inf Med 42(5):544–551
Wendt T, Häber A, Brigl B, Winter A (2003) Modeling hospital information systems (Part 2): using the 3LMG2 tool for modelling patient, record management. Methods Inf Med 43(3):256–267
Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH (2006) Facilitating direct entry of clinical data into electronic health record systems. J Am Med Inform Assoc 13(3):277–288
Maruster L, van der Aalst W, Weijters T, van den Bosch A, Daelemans W (2001) Automatic discovery of workflows models from hospital data. BNAIC 183–190
Sanchez D, Tentori M, Favela J (2008) Activity recognition for the smart hospital. IEEE Intell Syst 23(2):50–77
Favela J, Tentori M, Castro LA, Moran EB, Martinez-Garcia AI (2007) Activity recognition for context-aware hospital applications: issues and opportunities for the deployment of pervasive networks. Mobile Netw Appl 12(2,3):155–171
Riley R, Manias E (2005) Governing time in operating rooms. J Clin Nurs 15(5):53–546
Archer T, Macario A (2006) The drive for operating room efficiency will increase quality of patient care. Curr Opin Anaesthesiol 19:171–176
Houliston BR, Parry DT, Merry AF (2011) TADAA: towards automated detection of anaesthetic activity. Methods Inform Med 50(5):464–471
Seim AR, Meyer M, Sandberg WS (2005) Does parallel workflow impact anaesthesia quality. In: Am Med Inform Assoc Annu Symp Proc, 1053
Schleppers A, Bender H (2003) Optimised workflow and organisation—from the point of view of an anaesthesiolo-g department. Minim Invasive Ther Allied Technol 12(6):278–283
Decker K, Bauer M (2003) Ergonomics in the operating room. Minim Invasive Ther Allied Technol. 12(6):268–277
Gehbard F, Brinkmann A (2006) Management of an operating room in a university hospital. Zentralbl Chir. 131(4):341–346
Epstein RH, Dexter F (2012) Mediated interruptions of anaesthesia providers using predictions of workload from anaesthesia information management system data. Anaesth Intensive Care 40:803–812
Tiwari V, Dexter F, Rothman BS, Ehrenfeld JM, Epstein RH (2013) Explanation for the near-contant mean time remaining in surgical cases exceeding their estimated duration, necessary for appropriate display on electronic white boards. Anesth Analg 117(2):487–493
Edwards FH, Peterson RF, Bridges C, Ceithaml EL (1995) 1988: Use of a Bayesian statistical model for risk assessment in coronary artery surgery. Updated in 1995. Ann Thorac Surg 59(6):1611–1612
Biagioli B, Scolletta S, Cevenini G, Barbini E, Giomarelli P, Barbini P (2006) A multivariate Bayesian model for assessing morbidity after coronary artery surgery. Crit Care 10(3):R94
Verduijn M, Rosseel PM, Peek N, de Jonge E, de Mol BA (2007) Prognostic Bayesian networks II: an application in the domain of cardiac surgery. J Biomed Inform 40(6):619–649
Kuhan G, Marshall EC, Abidia AF, Chetter IC, McCollum PT (2002) A Bayesian hierarchical approach to comparative audit for carotid surgery. Eur J Vasc Endovasc Surg 24(6):505–515
Flin R, Youngson G, Yule S (2007) How do surgeons make intraoperative decisions. Qual Saf Health Care 16:235–239
Jalote-Parmar A, van Alfen M, Hermans JJ (2008) Workflow driven user interface for radiological system: A human factors approach. Comput Assisted Radiol Surg
Morineau T, Morandi X, Le Moëllic N, Diabira S, Haegelen C, Hénaux PL, Jannin P (2009) Decision making during preoperative surgical planning. Hum factors 51(1):66–77
Conflict of interest
None.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lalys, F., Jannin, P. Surgical process modelling: a review. Int J CARS 9, 495–511 (2014). https://doi.org/10.1007/s11548-013-0940-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-013-0940-5