Abstract
Purpose
Neuromuscular monitoring is frequently plagued by artefacts, which along with the frequent unawareness of the principles of this subtype of monitoring by many clinicians, tends to lead to a cynical attitute by clinicians towards these monitors. As such, the present study aims to derive a feature set and evaluate its discriminative performance for the purpose of Train-of-Four Ratio (TOF-R) outlier analysis during continuous intraoperative EMG-based neuromuscular monitoring.
Methods
Patient data was sourced from two devices: (1) Datex-Ohmeda Electromyography (EMG) E-NMT: a dataset derived from a prospective observational trial including 136 patients (21,891 TOF-R observations), further subdivided in two based on the type of features included; and (2) TetraGraph: a clinical case repository dataset of 388 patients (97,838 TOF-R observations). The two datasets were combined to create a synthetic set, which included shared features across the two. This process led to the training of four distinct models.
Results
The models showed an adequate bias/variance balance, suggesting no overfitting or underfitting. Models 1 and 2 consistently outperformed the others, with the former achieving an F1 score of 0.41 (0.31, 0.50) and an average precision score (95% CI) of 0.48 (0.35, 0.60). A random forest model analysis indicated that engineered TOF-R features were proportionally more influential in model performance than basic features.
Conclusions
Engineered TOF-R trend features and the resulting Cost-Sensitive Logistic Regression (CSLR) models provide useful insights and serve as a potential first step towards the automated removal of outliers for neuromuscular monitoring devices.
Trial registration
NCT04518761 (clinicaltrials.gov), registered on 19 August 2020.
Key points
• Neuromuscular monitoring is still suboptimally employed by anesthesiologists despite recent recommendations.
• Artefactual phenomena are frequently reported as a cause of distrust in neuromuscular monitoring.
• Machine-learning models based on universal neuromuscular monitoring variables can be developed to minimise artefacts and so increase end-user compliance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cammu G. Residual neuromuscular blockade and postoperative pulmonary complications: what does the recent evidence demonstrate? Curr. Anesthesiol Rep 1 (2020).
Naguib M, Brull SJ, Kopman AF, Hunter JM, Fülesdi B, Arkes HR, Elstein A, Todd MM, Johnson KB. Consensus Statement on Perioperative Use of Neuromuscular Monitoring. Anesth Analg. 2018;127:71–80. https://doi.org/10.1213/ANE.0000000000002670.
Harman A, Tung A, Fox C, Lien CA. Heuristics, overconfidence, and experience: impact on monitoring depth of neuromuscular blockade. Anesth Analg. 2019;128:1057–9. https://doi.org/10.1213/ANE.0000000000003965.
Naguib M, Brull SJ, Hunter JM, Kopman AF, Fülesdi B, Johnson KB, Arkes HR. Anesthesiologists’ overconfidence in their perceived knowledge of neuromuscular monitoring and its relevance to all aspects of medical practice: an International Survey. Anesth Analg. 2019;128:1118–26. https://doi.org/10.1213/ANE.0000000000003714.
Naguib M, Kopman AF, Lien CA, Hunter JM, Lopez A, Brull SJ. A Survey of Current Management of Neuromuscular Block in the United States and Europe: Anesth. Analg. 2010;111:110–9. https://doi.org/10.1213/ANE.0b013e3181c07428.
Todd MM, Hindman BJ, King BJ. The implementation of quantitative Electromyographic Neuromuscular monitoring in an academic Anesthesia Department. Anesth Analg. 2014;119:323–31. https://doi.org/10.1213/ANE.0000000000000261.
Kopman AF, Kopman DJ. An analysis of the TOF-watch algorithm for modifying the displayed train-of-four ratio. Acta Anaesthesiol Scand. 2006;50:1313–4. https://doi.org/10.1111/j.1399-6576.2006.01156.x.
Kopman AF, Kumar S, Klewicka MM, Neuman GG. The staircase phenomenon: implications for monitoring of neuromuscular transmission. J Am Soc Anesthesiol. 2001;95:403–7.
Carvalho H, Verdonck M, Eleveld DJ, Ramirez D, D’Haese J, Flamée P, Geerts L, Wylleman J, Cools W, Barbe K. Others: neuromuscular end-point predictive capability of published rocuronium pharmacokinetic/pharmacodynamic models: an observational trial. J Clin Anesth. 2023;90:111225.
Thilen SR, Weigel WA, Todd MM, Dutton RP, Lien CA, Grant SA, Szokol JW, Eriksson LI, Yaster M, Grant MD., others: American Society of Anesthesiologists practice guidelines for monitoring and antagonism of neuromuscular blockade: a report by the American Society of Anesthesiologists task force on neuromuscular blockade. Anesthesiology. 2023;138:13–41.
Fuchs-Buder T, Romero CS, Lewald H, Lamperti M, Afshari A, Hristovska A-M, Schmartz D, Hinkelbein J, Longrois D, Popp M. others: peri-operative management of neuromuscular blockade: a guideline from the European Society of Anaesthesiology and Intensive Care. Eur J Anaesthesiol EJA. 2023;40:82–94.
Kopman AF, Chin W, Cyriac J. Acceleromyography vs. electromyography: an ipsilateral comparison of the indirectly evoked neuromuscular response to train-of-four stimulation: Acceleromyography vs. electromyography. Acta Anaesthesiol Scand. 2005;49:316–22. https://doi.org/10.1111/j.1399-6576.2005.00643.x.
Bowdle A, Bussey L, Michaelsen K, Jelacic S, Nair B, Togashi K, Hulvershorn J. A comparison of a prototype electromyograph vs. a mechanomyograph and an acceleromyograph for assessment of neuromuscular blockade. Anaesthesia. 2020;75:187–95. https://doi.org/10.1111/anae.14872.
Nemes R, Lengyel S, Nagy G, Hampton DR, Gray M, Renew JR, Tassonyi E, Fülesdi B, Brull SJ. Ipsilateral and simultaneous comparison of responses from acceleromyography-and electromyography-based neuromuscular monitors. Anesthesiology. 2021;135:597–611.
Fuchs-Buder T, Brull SJ, Fagerlund MJ, Renew RJ, Cammu G, Murphy GS, Warlé M, Vested M, Fülesdi B, Nemes R. Others: good clinical research practice (GCRP) in pharmacodynamic studies of neuromuscular blocking agents III: the 2023 Geneva revision. Acta Anaesthesiol Scand. (2023).
Chandola V, Banerjee A, Kumar V. Anomaly detection: a survey. ACM Comput Surv. 2009;41:1–58. https://doi.org/10.1145/1541880.1541882.
Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, Shilton A, Yearwood J, Dimitrova N, Ho TB, Venkatesh S, Berk M. Guidelines for develo** and reporting machine learning predictive models in Biomedical Research: a multidisciplinary view. J Med Internet Res. 2016;18:e323. https://doi.org/10.2196/jmir.5870.
Aghazadeh-Attari J, Mobaraki K, Ahmadzadeh J, Mansorian B, Mohebbi I. Quality of observational studies in prestigious journals of occupational medicine and health based on strengthening the reporting of Observational studies in Epidemiology (STROBE) Statement: a cross-sectional study. BMC Res Notes. 2018;11:1–7.
Aggarwal CC. Outlier analysis. Data mining. Springer; 2015. pp. 237–63. https://doi.org/10.1007/978-3-319-14142-8.
Hatib F, Jian Z, Buddi S, Lee C, Settels J, Sibert K, Rinehart J, Cannesson M. Machine-learning Algorithm to Predict Hypotension based on high-fidelity arterial pressure Waveform Analysis. Anesthesiology. 2018;129:663–74. https://doi.org/10.1097/ALN.0000000000002300.
Lee CK, Hofer I, Eilon G, Baldi P, Cannesson M. Development and validation of a deep neural network model for prediction of postoperative In-hospital mortality. Anesthesiology. 1–14 (2018).
Scikit-Learn. https://scikit-learn.org/.
Haq AU, Zhang D, Peng H, Rahman SU. Combining multiple feature-ranking techniques and clustering of variables for feature selection. IEEE Access. 2019;7:151482–92.
Churchill-Davidson H, Christie T. The diagnosis of neuromuscular block in man. Br J Anaesth. 1959;31:290–301.
Christie T, Churchill-Davidson H. The St. Thomas’s hospital nerve stimulator in the diagnosis of prolonged apnoea. Lancet. 1958;271:776.
Carvalho H, Verdonck M, Cools W, Geerts L, Forget P, Poelaert J. Forty years of neuromuscular monitoring and postoperative residual curarisation: a meta-analysis and evaluation of confidence in network meta-analysis. Br J Anaesth. 2020. https://doi.org/10.1016/j.bja.2020.05.063.
Faulk DJ, Austin TM, Thomas JJ, Strupp K, Macrae AW, Yaster M. A survey of the Society for Pediatric Anesthesia on the use, monitoring, and antagonism of neuromuscular blockade. Anesth Analg. 2021;132:1518–26.
Renew JR, Tobias JD, Brull SJ. The Time to Seriously Reassess the Use and Misuse of Neuromuscular Blockade in Children Is Now, (2021).
Philips S, Stewart PA, Bilgin AB. A survey of the management of neuromuscular blockade monitoring in Australia and New Zealand. Anaesth Intensive Care. 2013;41:374–9.
Cai L, Zhu Y. Data Sci J. 2015;14:2–2. The challenges of data quality and data quality assessment in the big data era.
Maheshwari K, Shimada T, Yang D, Khanna S, Cywinski JB, Irefin SA, Ayad S, Turan A, Ruetzler K, Qiu Y. Others: hypotension prediction index for prevention of hypotension during moderate-to high-risk noncardiac surgery: a pilot randomized trial. Anesthesiology. 2020;133:1214–22.
Funding
Financial support and sponsorship: the present study has received support from the following institutions: Wetenschappelijk Fonds Willy Gepts (WFWG), Universitair Ziekenhuis Brussel, Belgium: unrestricted research grant. Agentschap Innoveren & Ondernemen (VLAIO), Flanders, Belgian Federal Government: innovation mandate. Society for Anaesthesia and Resuscitation of Belgium (BeSARPP, formerly SARB), Belgium: unrestricted research grant. Vrije Universiteit Brussel (VUB), Belgium: Industrial Research Fund.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Hugo Carvalho and Michael Verdonck. The first draft of the manuscript was written by Hugo Carvalho and Michael Verdonck and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Presentation: no previous presentations
Corresponding author
Ethics declarations
Competing interests
HC has received lecture fees from 3M (Diegem, Belgium) and Xavant Technology (Pretoria, South Africa); SJB has intellectual property assigned to Mayo Clinic (Rochester, MN); has received research support (funds to Mayo Clinic) from Merck & Co, Inc (Kenilworth, NJ) and is a consultant for Merck & Co, Inc; is a principal, shareholder, and chief medical officer in Senzime AB (publ) (Uppsala, Sweden); and is a member of the scientific/clinical advisory boards for the Doctors Company (Napa, CA), Coala Life, Inc (Irvine, CA), NMD Pharma (Aarhus, Denmark), and Takeda Pharmaceuticals (Cambridge, MA); TFB received lecture fees from MSD France. MV and JP declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Summary Statement: The present study engineered and applied a set of machine-learning models that have the potential to identify outliers in the evolutive trends of intraoperative Electromyography-based Train of Four (TOF) measurements.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Verdonck, M., Carvalho, H., Fuchs-Buder, T. et al. Machine learning based analysis and detection of trend outliers for electromyographic neuromuscular monitoring. J Clin Monit Comput (2024). https://doi.org/10.1007/s10877-024-01141-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10877-024-01141-6