1
|
Constable MD, Shum HPH, Clark S. Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review. J Cardiothorac Surg 2024; 19:94. [PMID: 38355499 PMCID: PMC10865515 DOI: 10.1186/s13019-024-02558-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2023] [Accepted: 01/30/2024] [Indexed: 02/16/2024] Open
Abstract
When technical requirements are high, and patient outcomes are critical, opportunities for monitoring and improving surgical skills via objective motion analysis feedback may be particularly beneficial. This narrative review synthesises work on technical and non-technical surgical skills, collaborative task performance, and pose estimation to illustrate new opportunities to advance cardiothoracic surgical performance with innovations from computer vision and artificial intelligence. These technological innovations are critically evaluated in terms of the benefits they could offer the cardiothoracic surgical community, and any barriers to the uptake of the technology are elaborated upon. Like some other specialities, cardiothoracic surgery has relatively few opportunities to benefit from tools with data capture technology embedded within them (as is possible with robotic-assisted laparoscopic surgery, for example). In such cases, pose estimation techniques that allow for movement tracking across a conventional operating field without using specialist equipment or markers offer considerable potential. With video data from either simulated or real surgical procedures, these tools can (1) provide insight into the development of expertise and surgical performance over a surgeon's career, (2) provide feedback to trainee surgeons regarding areas for improvement, (3) provide the opportunity to investigate what aspects of skill may be linked to patient outcomes which can (4) inform the aspects of surgical skill which should be focused on within training or mentoring programmes. Classifier or assessment algorithms that use artificial intelligence to 'learn' what expertise is from expert surgical evaluators could further assist educators in determining if trainees meet competency thresholds. With collaborative efforts between surgical teams, medical institutions, computer scientists and researchers to ensure this technology is developed with usability and ethics in mind, the developed feedback tools could improve cardiothoracic surgical practice in a data-driven way.
Collapse
Affiliation(s)
- Merryn D Constable
- Department of Psychology, Northumbria University, Newcastle-upon-Tyne, UK.
| | - Hubert P H Shum
- Department of Computer Science, Durham University, Durham, UK
| | - Stephen Clark
- Department of Applied Sciences, Northumbria University, Newcastle-upon-Tyne, UK
- Consultant Cardiothoracic and Transplant Surgeon, Freeman Hospital, Newcastle upon Tyne, UK
| |
Collapse
|
2
|
Sugiyama T, Ito M, Sugimori H, Tang M, Nakamura T, Ogasawara K, Matsuzawa H, Nakayama N, Lama S, Sutherland GR, Fujimura M. Tissue Acceleration as a Novel Metric for Surgical Performance During Carotid Endarterectomy. Oper Neurosurg (Hagerstown) 2023; 25:343-352. [PMID: 37427955 DOI: 10.1227/ons.0000000000000815] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 05/08/2023] [Indexed: 07/11/2023] Open
Abstract
BACKGROUND AND OBJECTIVES Gentle tissue handling to avoid excessive motion of affected fragile vessels during surgical dissection is essential for both surgeon proficiency and patient safety during carotid endarterectomy (CEA). However, a void remains in the quantification of these aspects during surgery. The video-based measurement of tissue acceleration is presented as a novel metric for the objective assessment of surgical performance. This study aimed to evaluate whether such metrics correlate with both surgeons' skill proficiency and adverse events during CEA. METHODS In a retrospective study including 117 patients who underwent CEA, acceleration of the carotid artery was measured during exposure through a video-based analysis. Tissue acceleration values and threshold violation error frequencies were analyzed and compared among the surgeon groups with different surgical experience (3 groups: novice , intermediate , and expert ). Multiple patient-related variables, surgeon groups, and video-based surgical performance parameters were compared between the patients with and without adverse events during CEA. RESULTS Eleven patients (9.4%) experienced adverse events after CEA, and the rate of adverse events significantly correlated with the surgeon group. The mean maximum tissue acceleration and number of errors during surgical tasks significantly decreased from novice, to intermediate, to expert surgeons, and stepwise discriminant analysis showed that the combined use of surgical performance factors could accurately discriminate between surgeon groups. The multivariate logistic regression analysis revealed that the number of errors and vulnerable carotid plaques were associated with adverse events. CONCLUSION Tissue acceleration profiles can be a novel metric for the objective assessment of surgical performance and the prediction of adverse events during surgery. Thus, this concept can be introduced into futuristic computer-aided surgeries for both surgical education and patient safety.
Collapse
Affiliation(s)
- Taku Sugiyama
- Department of Neurosurgery, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Masaki Ito
- Department of Neurosurgery, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | | | - Minghui Tang
- Department of Diagnostic Imaging, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Toshitaka Nakamura
- Department of Neurosurgery, Sapporo Azabu Neurosurgical Hospital, Sapporo, Japan
| | | | - Hitoshi Matsuzawa
- Center for Integrated Human Brain Science, Brain Research Institute, University of Niigata, Niigata, Japan
- Department of Neurosurgery, Kashiwaba Neurosurgical Hospital, Sapporo, Japan
| | - Naoki Nakayama
- Department of Neurosurgery, Kashiwaba Neurosurgical Hospital, Sapporo, Japan
| | - Sanju Lama
- Department of Clinical Neurosciences and the Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Garnette R Sutherland
- Department of Clinical Neurosciences and the Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Miki Fujimura
- Department of Neurosurgery, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| |
Collapse
|
3
|
Montgomery KB, Lindeman B. Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:765-768. [PMID: 36745875 PMCID: PMC10329982 DOI: 10.1097/acm.0000000000005165] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
In 2013, U.S. general surgery residency programs implemented a milestones assessment framework in an effort to incorporate more competency-focused evaluation methods. Developed by a group of surgical education leaders and other stakeholders working with the Accreditation Council for Graduate Medical Education and recently updated in a version 2.0, the surgery milestones framework is centered around 6 "core competencies": patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. While prior work has focused on the validity of milestones as a measure of resident performance, associations between general surgery resident milestone ratings and their post-training patient outcomes have only recently been explored in an analysis in this issue of Academic Medicine by Kendrick et al. Despite their well-designed efforts to tackle this complex problem, no relationships were identified. This accompanying commentary discusses the broader implications for the use of milestone ratings beyond their intended application, alternative assessment methods, and the challenges of developing predictive assessments in the complex setting of surgical care. Although milestone ratings have not been shown to provide the specificity needed to predict clinical outcomes in the complex settings studied by Kendrick et al, hope remains that utilization of other outcomes, assessment frameworks, and data analytic tools could augment these models and further our progress toward a predictive assessment in surgical education. Evaluation of residents in general surgery residency programs has grown both more sophisticated and complicated in the setting of increasing patient and case complexity, constraints on time, and regulation of resident supervision in the operating room. Over the last decade, surgical education research efforts related to resident assessment have focused on measuring performance through accurate and reproducible methods with evidence for their validity, as well as on attempting to refine decision making about resident preparedness for unsupervised practice.
Collapse
Affiliation(s)
- Kelsey B Montgomery
- K.B. Montgomery is a general surgery resident, Department of Surgery, University of Alabama at Birmingham, Birmingham, Alabama; ORCID: https://orcid.org/0000-0002-1284-1830
| | - Brenessa Lindeman
- B. Lindeman is associate professor, Department of Surgery, and assistant dean, Graduate Medical Education, University of Alabama at Birmingham, Birmingham, Alabama
| |
Collapse
|
4
|
Franco-González IT, Minor-Martínez A, Ordorica-Flores RM, Sossa-Azuela JH, Pérez-Escamirosa F. Objective psychomotor laparoscopic skills evaluation using a low-cost wearable device based on accelerometry: construct and concurrent validity study. Surg Endosc 2023; 37:3280-3290. [PMID: 36890413 DOI: 10.1007/s00464-023-09953-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 02/12/2023] [Indexed: 03/10/2023]
Abstract
BACKGROUND Motion analysis of surgical maneuvers provides useful quantitative information for the objective evaluation of the surgeons. However, surgical simulation laboratories for laparoscopic training do not usually integrate devices that help quantify the level of skills of the surgeons due to their limited resources and the high costs of new technologies. The purpose of this study is to present the construct and concurrent validity of a low-cost motion tracking system, based on a wireless triaxial accelerometer, employed to objectively evaluate psychomotor skills of surgeons during laparoscopic training. METHODS An accelerometry system, a wireless three-axis accelerometer with appearance of wristwatch, was placed on the dominant hand of the surgeons to register the motion during the laparoscopy practice with the EndoViS simulator, which simultaneously recorded the motion of the laparoscopic needle driver. This study included the participation of 30 surgeons (6 experts, 14 intermediates and 10 novices) who performed the task of intracorporeal knot-tying suture. Using 11 motion analysis parameters (MAPs), the performance of each participant was assessed. Subsequently, the scores of the three groups of surgeons were statistically analyzed. In addition, a validity study was conducted comparing the metrics between the accelerometry-tracking system and the EndoViS hybrid simulator. RESULTS Construct validity was achieved for 8 of the 11 metrics examined with the accelerometry system. Concurrent validity demonstrated that there is a strong correlation between the results of the accelerometry system and the EndoViS simulator in 9 of 11 parameters, showing reliability of the accelerometry system as an objective evaluation method. CONCLUSION The accelerometry system was successfully validated. This method is potentially useful to complement the objective evaluation of surgeons during laparoscopic practice in training environments such as box-trainers and simulators.
Collapse
Affiliation(s)
- Iván Tlacaélel Franco-González
- Sección de Bioelectrónica, Departamento de Ingeniería Eléctrica, Centro de Investigación Y de Estudios Avanzados del Instituto Politécnico Nacional (CINVESTAV-IPN), Av. Instituto Politécnico Nacional 2508, Col. San Pedro Zacatenco, 07360, Ciudad de México, México
| | - Arturo Minor-Martínez
- Sección de Bioelectrónica, Departamento de Ingeniería Eléctrica, Centro de Investigación Y de Estudios Avanzados del Instituto Politécnico Nacional (CINVESTAV-IPN), Av. Instituto Politécnico Nacional 2508, Col. San Pedro Zacatenco, 07360, Ciudad de México, México.
| | - Ricardo Manuel Ordorica-Flores
- Departamento de Cirugía Endoscópica, Hospital Infantil de México Federico Gómez, Calle Dr. Márquez No. 162, Cuauhtémoc, Doctores, 06720, Ciudad de México, México
| | - Juan Humberto Sossa-Azuela
- Centro de Investigación en Computación, Instituto Politécnico Nacional, Av. Juan de Dios Bátiz S/N, Esq. Miguel Othón de Mendizábal, Col. Nueva Industrial Vallejo, 07738, Ciudad de México, México
| | - Fernando Pérez-Escamirosa
- Instituto de Ciencias Aplicadas Y Tecnología (ICAT), Universidad Nacional Autónoma de México (UNAM), Circuito Exterior S/N, Ciudad Universitaria, Coyoacán, 04510, Ciudad de México, México
| |
Collapse
|
5
|
Guzmán-García C, Sánchez-González P, Margallo JAS, Snoriguzzi N, Rabazo JC, Margallo FMS, Gómez EJ, Oropesa I. Correlating Personal Resourcefulness and Psychomotor Skills: An Analysis of Stress, Visual Attention and Technical Metrics. SENSORS 2022; 22:s22030837. [PMID: 35161582 PMCID: PMC8838092 DOI: 10.3390/s22030837] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 01/12/2022] [Accepted: 01/21/2022] [Indexed: 12/10/2022]
Abstract
Modern surgical education is focused on making use of the available technologies in order to train and assess surgical skill acquisition. Innovative technologies for the automatic, objective assessment of nontechnical skills are currently under research. The main aim of this study is to determine whether personal resourcefulness can be assessed by monitoring parameters that are related to stress and visual attention and whether there is a relation between these and psychomotor skills in surgical education. For this purpose, we implemented an application in order to monitor the electrocardiogram (ECG), galvanic skin response (GSR), gaze and performance of surgeons-in-training while performing a laparoscopic box-trainer task so as to obtain technical and personal resourcefulness' metrics. Eight surgeons (6 nonexperts and 2 experts) completed the experiment. A total of 22 metrics were calculated (7 technical and 15 related to personal resourcefulness) per subject. The average values of these metrics in the presence of stressors were compared with those in their absence and depending on the participants' expertise. The results show that both the mean normalized GSR signal and average surgical instrument's acceleration change significantly when stressors are present. Additionally, the GSR and acceleration were found to be correlated, which indicates that there is a relation between psychomotor skills and personal resourcefulness.
Collapse
Affiliation(s)
- Carmen Guzmán-García
- Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicación, Centre for Biomedical Technology, Universidad Politécnica de Madrid, 28040 Madrid, Spain; (C.G.-G.); (P.S.-G.); (N.S.); (E.J.G.)
| | - Patricia Sánchez-González
- Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicación, Centre for Biomedical Technology, Universidad Politécnica de Madrid, 28040 Madrid, Spain; (C.G.-G.); (P.S.-G.); (N.S.); (E.J.G.)
- Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, 28029 Madrid, Spain
| | - Juan A. Sánchez Margallo
- Centro de Cirugía de Mínima Invasión Jesús Usón, 10071 Cáceres, Spain; (J.A.S.M.); (J.C.R.); (F.M.S.M.)
| | - Nicola Snoriguzzi
- Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicación, Centre for Biomedical Technology, Universidad Politécnica de Madrid, 28040 Madrid, Spain; (C.G.-G.); (P.S.-G.); (N.S.); (E.J.G.)
| | - José Castillo Rabazo
- Centro de Cirugía de Mínima Invasión Jesús Usón, 10071 Cáceres, Spain; (J.A.S.M.); (J.C.R.); (F.M.S.M.)
| | | | - Enrique J. Gómez
- Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicación, Centre for Biomedical Technology, Universidad Politécnica de Madrid, 28040 Madrid, Spain; (C.G.-G.); (P.S.-G.); (N.S.); (E.J.G.)
- Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, 28029 Madrid, Spain
| | - Ignacio Oropesa
- Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicación, Centre for Biomedical Technology, Universidad Politécnica de Madrid, 28040 Madrid, Spain; (C.G.-G.); (P.S.-G.); (N.S.); (E.J.G.)
- Correspondence: ; Tel.: +34-910-672-458
| |
Collapse
|
6
|
Williams TP, Klimberg V, Perez A. Tele-education assisted mentorship in surgery (TEAMS). J Surg Oncol 2021; 124:250-254. [PMID: 34245580 DOI: 10.1002/jso.26495] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Accepted: 04/03/2021] [Indexed: 12/14/2022]
Abstract
Tele-education assisted mentorship in surgery (TEAMS) is a novel methodology for surgical skills training with remote, hands-on, high-fidelity, and low-cost simulation-based education and one-to-one mentorship with longitudinal assessments. We review the background, methodology, and our experience with implementing TEAMS as an adjunct to traditional methods of surgical education and mentorship.
Collapse
Affiliation(s)
- Taylor P Williams
- Department of Surgery, University of Texas Medical Branch, Galveston, Texas, USA
| | - Vicki Klimberg
- Department of Surgery, University of Texas Medical Branch, Galveston, Texas, USA
| | - Alexander Perez
- Department of Surgery, University of Texas Medical Branch, Galveston, Texas, USA
| |
Collapse
|
7
|
Azari DP, Miller BL, Le BV, Greenberg CC, Radwin RG. Quantifying surgeon maneuevers across experience levels through marker-less hand motion kinematics of simulated surgical tasks. APPLIED ERGONOMICS 2020; 87:103136. [PMID: 32501255 DOI: 10.1016/j.apergo.2020.103136] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Revised: 04/20/2020] [Accepted: 04/21/2020] [Indexed: 06/11/2023]
Abstract
This paper compares clinician hand motion for common suturing tasks across a range of experience levels and tissue types. Medical students (32), residents (41), attending surgeons (10), and retirees (2) were recorded on digital video while suturing on one of: foam, pig feet, or porcine bowel. Depending on time in position, each medical student, resident, and attending participant was classified as junior or senior, yielding six experience categories. This work focuses on trends associated with increasing tenure observed from those medical students (10), residents (15), and attendings (10) who sutured on foam, and draws comparison across tissue types where pertinent. Utilizing custom software, the two-dimensional location of each of the participant's hands were automatically recorded in every video frame, producing a rich spatiotemporal feature set. While suturing on foam, increasing clinician experience was associated with conserved path length per cycle of the non-dominant hand, significantly reducing from junior medical students (mean = 73.63 cm, sd = 33.21 cm) to senior residents (mean = 46.16 cm, sd = 14.03 cm, p = 0.015), and again between senior residents and senior attendings (mean = 30.84 cm, sd = 14.51 cm, p = 0.045). Despite similar maneuver rates, attendings also accelerated less with their non-dominant hand (mean = 16.27 cm/s2, sd = 81.12 cm/s2, p = 0.002) than senior residents (mean = 24.84 cm/s2, sd = 68.29 cm/s2, p = 0.002). While tying, medical students moved their dominant hands slower (mean = 4.39 cm/s, sd = 1.73 cm/s, p = 0.033) than senior residents (mean = 6.53 cm/s, sd = 2.52 cm/s). These results suggest that increased psychomotor performance during early training manifest through faster dominant hand function, while later increases are characterized by conserving energy and efficiently distributing work between hands. Incorporating this scalable video-based motion analysis into regular formative assessment routines may enable greater quality and consistency of feedback throughout a surgical career.
Collapse
Affiliation(s)
- David P Azari
- Department of Industrial and Systems Engineering, 1550 Engineering Drive, University of Wisconsin-Madison, Madison, WI, 53706, USA
| | - Brady L Miller
- Department of Urology, Third Floor, 1685 Highland Avenue, University of Wisconsin-Madison, Madison, WI, 53705, USA
| | - Brian V Le
- Department of Urology, Third Floor, 1685 Highland Avenue, University of Wisconsin-Madison, Madison, WI, 53705, USA
| | - Caprice C Greenberg
- Wisconsin Surgical Outcomes Research (WiSOR) Program, Department of Surgery, Clinical Science Center, 600 Highland Avenue, University of Wisconsin-Madison, Madison, WI, 53792, USA
| | - Robert G Radwin
- Department of Industrial and Systems Engineering, 1550 Engineering Drive, University of Wisconsin-Madison, Madison, WI, 53706, USA; Department of Biomedical Engineering, 1415 Engineering Drive, University of Wisconsin-Madison, Madison, WI, 53706, USA.
| |
Collapse
|
8
|
Azari D, Greenberg C, Pugh C, Wiegmann D, Radwin R. In Search of Characterizing Surgical Skill. JOURNAL OF SURGICAL EDUCATION 2019; 76:1348-1363. [PMID: 30890315 DOI: 10.1016/j.jsurg.2019.02.010] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/22/2018] [Revised: 01/17/2019] [Accepted: 02/20/2019] [Indexed: 06/09/2023]
Abstract
OBJECTIVE This paper provides a literature review and detailed discussion of surgical skill terminology. Culminating in a novel model that proposes a set of unique definitions, this review is designed to facilitate shared understanding to study and develop metrics quantifying surgical skill. DESIGN Objective surgical skill analysis depends on consistent definitions and shared understanding of terms like performance, expertise, experience, aptitude, ability, competency, and proficiency. STRUCTURE Each term is discussed in turn, drawing from existing literature and colloquial uses. IMPLICATIONS A new model of definitions is proposed to cement a common and consistent lexicon for future skills analysis, and to quantitatively describe a surgeon's performance throughout their career.
Collapse
Affiliation(s)
- David Azari
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin
| | - Caprice Greenberg
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin; Department of Surgery, Wisconsin Surgical Outcomes Research (WiSOR) Program, University of Wisconsin-Madison, Madison, Wisconsin
| | - Carla Pugh
- Department of Surgery, Stanford University, Stanford, California
| | - Douglas Wiegmann
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin
| | - Robert Radwin
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin; Department of Biomedical Engineering, University of Wisconsin-Madison, Madison, Wisconsin.
| |
Collapse
|
9
|
Sugiyama T, Nakamura T, Ito Y, Tokairin K, Kazumata K, Nakayama N, Houkin K. A Pilot Study on Measuring Tissue Motion During Carotid Surgery Using Video-Based Analyses for the Objective Assessment of Surgical Performance. World J Surg 2019; 43:2309-2319. [DOI: 10.1007/s00268-019-05018-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
10
|
Xu J, Slagle JM, Banerjee A, Bracken B, Weinger MB. Use of a Portable Functional Near-Infrared Spectroscopy (fNIRS) System to Examine Team Experience During Crisis Event Management in Clinical Simulations. Front Hum Neurosci 2019; 13:85. [PMID: 30890926 PMCID: PMC6412154 DOI: 10.3389/fnhum.2019.00085] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2018] [Accepted: 02/18/2019] [Indexed: 11/16/2022] Open
Abstract
Objective: The aim of this study was to investigate the utilization of a portable functional near-infrared spectroscopy (fNIRS) system, the fNIRS PioneerTM, to examine team experience in high-fidelity simulation-based crisis event management (CEM) training for anesthesiologists in operating rooms. Background: Effective evaluation of team performance and experience in CEM simulations is essential for healthcare training and research. Neurophysiological measures with wearable devices can provide useful indicators of team experience to compliment traditional self-report, observer ratings, and behavioral performance measures. fNIRS measured brain blood oxygenation levels and neural synchrony can be used as indicators of workload and team engagement, which is vital for optimal team performance. Methods: Thirty-three anesthesiologists, who were attending CEM training in two-person teams, participated in this study. The participants varied in their expertise level and the simulation scenarios varied in difficulty level. The oxygenated and de-oxygenated hemoglobin (HbO and HbR) levels in the participants’ prefrontal cortex were derived from data recorded by a portable one-channel fNIRS system worn by all participants throughout CEM training. Team neural synchrony was measured by HbO/HbR wavelet transformation coherence (WTC). Observer-rated workload and self-reported workload and mood were also collected. Results: At the individual level, the pattern of HbR level corresponded to changes of workload for the individuals in different roles during different phases of a scenario; but this was not the case for HbO level. Thus, HbR level may be a better indicator for individual workload in the studied setting. However, HbR level was insensitive to differences in scenario difficulty and did not correlate with observer-rated or self-reported workload. At the team level, high levels of HbO and HbR WTC were observed during active teamwork. Furthermore, HbO WTC was sensitive to levels of scenario difficulty. Conclusion: This study showed that it was feasible to use a portable fNIRS system to study workload and team engagement in high-fidelity clinical simulations. However, more work is needed to establish the sensitivity, reliability, and validity of fNIRS measures as indicators of team experience.
Collapse
Affiliation(s)
- Jie Xu
- Faculty of Science, Center for Psychological Sciences, Zhejiang University, Hangzhou, China.,Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Jason M Slagle
- Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, TN, United States.,Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Arna Banerjee
- Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, TN, United States.,Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN, United States
| | | | - Matthew B Weinger
- Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, TN, United States.,Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN, United States.,Geriatric Research Education and Clinical Center, VA Tennessee Valley Healthcare System, Nashville, TN, United States
| |
Collapse
|
11
|
Dias RD, Gupta A, Yule SJ. Using Machine Learning to Assess Physician Competence: A Systematic Review. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2019; 94:427-439. [PMID: 30113364 DOI: 10.1097/acm.0000000000002414] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
PURPOSE To identify the different machine learning (ML) techniques that have been applied to automate physician competence assessment and evaluate how these techniques can be used to assess different competence domains in several medical specialties. METHOD In May 2017, MEDLINE, EMBASE, PsycINFO, Web of Science, ACM Digital Library, IEEE Xplore Digital Library, PROSPERO, and Cochrane Database of Systematic Reviews were searched for articles published from inception to April 30, 2017. Studies were included if they applied at least one ML technique to assess medical students', residents', fellows', or attending physicians' competence. Information on sample size, participants, study setting and design, medical specialty, ML techniques, competence domains, outcomes, and methodological quality was extracted. MERSQI was used to evaluate quality, and a qualitative narrative synthesis of the medical specialties, ML techniques, and competence domains was conducted. RESULTS Of 4,953 initial articles, 69 met inclusion criteria. General surgery (24; 34.8%) and radiology (15; 21.7%) were the most studied specialties; natural language processing (24; 34.8%), support vector machine (15; 21.7%), and hidden Markov models (14; 20.3%) were the ML techniques most often applied; and patient care (63; 91.3%) and medical knowledge (45; 65.2%) were the most assessed competence domains. CONCLUSIONS A growing number of studies have attempted to apply ML techniques to physician competence assessment. Although many studies have investigated the feasibility of certain techniques, more validation research is needed. The use of ML techniques may have the potential to integrate and analyze pragmatic information that could be used in real-time assessments and interventions.
Collapse
Affiliation(s)
- Roger D Dias
- R.D. Dias is instructor in emergency medicine, Department of Emergency Medicine and STRATUS Center for Medical Simulation, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts; ORCID: http://orcid.org/0000-0003-4959-5052. A. Gupta is research scientist, Center for Surgery and Public Health, Brigham and Women's Hospital, Boston, Massachusetts. S.J. Yule is associate professor of surgery, Harvard Medical School, and faculty, Department of Surgery and STRATUS Center for Medical Simulation, Brigham and Women's Hospital, Boston, Massachusetts
| | | | | |
Collapse
|
12
|
Georgiou K, Larentzakis A, Papavassiliou AG. Surgeons' and surgical trainees' acute stress in real operations or simulation: A systematic review. Surgeon 2017; 15:355-365. [PMID: 28716368 DOI: 10.1016/j.surge.2017.06.003] [Citation(s) in RCA: 42] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2017] [Revised: 05/29/2017] [Accepted: 06/19/2017] [Indexed: 12/16/2022]
Abstract
BACKGROUND AND PURPOSE Acute stress in surgery is ubiquitous and has an immediate impact on surgical performance and patient safety. Surgeons react with several coping strategies; however, they recognise the necessity of formal stress management training. Thus, stress assessment is a direct need. Surgical simulation is a validated standardised training milieu designed to replicate real-life situations. It replicates stress, prevents biases, and provides objective metrics. The complexity of stress mechanisms makes stress measurement difficult to quantify and interpret. This systematic review aims to identify studies that have used acute stress estimation measurements in surgeons or surgical trainees during real operations or surgical simulation, and to collectively present the rationale of these tools, with special emphasis in salivary markers. METHODS A search strategy was implemented to retrieve relevant articles from MEDLINE and SCOPUS databases. The 738 articles retrieved were reviewed for further evaluation according to the predetermined inclusion/exclusion criteria. RESULTS Thirty-three studies were included in this systematic review. The methods for acute stress assessment varied greatly among studies with the non-invasive techniques being the most commonly used. Subjective and objective tests for surgeons' acute stress assessment are being presented. CONCLUSION There is a broad spectrum of acute mental stress assessment tools in the surgical field and simulation and salivary biomarkers have recently gained popularity. There is a need to maintain a consistent methodology in future research, towards a deeper understanding of acute stress in the surgical field.
Collapse
Affiliation(s)
- Konstantinos Georgiou
- Department of Biological Chemistry, Medical School, National and Kapodistrian University of Athens, 75 Mikras Asias Street, Athens 11527, Greece.
| | - Andreas Larentzakis
- 1st Propaedeutic Surgical Clinic, Medical School, National and Kapodistrian University of Athens, 114 Vasilissis Sofias Avenue, Athens 11527, Greece.
| | - Athanasios G Papavassiliou
- Department of Biological Chemistry, Medical School, National and Kapodistrian University of Athens, 75 Mikras Asias Street, Athens 11527, Greece.
| |
Collapse
|
13
|
Pernek I, Ferscha A. A survey of context recognition in surgery. Med Biol Eng Comput 2017; 55:1719-1734. [DOI: 10.1007/s11517-017-1670-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2016] [Accepted: 06/15/2017] [Indexed: 11/30/2022]
|
14
|
Vedula SS, Ishii M, Hager GD. Objective Assessment of Surgical Technical Skill and Competency in the Operating Room. Annu Rev Biomed Eng 2017; 19:301-325. [PMID: 28375649 DOI: 10.1146/annurev-bioeng-071516-044435] [Citation(s) in RCA: 73] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
Training skillful and competent surgeons is critical to ensure high quality of care and to minimize disparities in access to effective care. Traditional models to train surgeons are being challenged by rapid advances in technology, an intensified patient-safety culture, and a need for value-driven health systems. Simultaneously, technological developments are enabling capture and analysis of large amounts of complex surgical data. These developments are motivating a "surgical data science" approach to objective computer-aided technical skill evaluation (OCASE-T) for scalable, accurate assessment; individualized feedback; and automated coaching. We define the problem space for OCASE-T and summarize 45 publications representing recent research in this domain. We find that most studies on OCASE-T are simulation based; very few are in the operating room. The algorithms and validation methodologies used for OCASE-T are highly varied; there is no uniform consensus. Future research should emphasize competency assessment in the operating room, validation against patient outcomes, and effectiveness for surgical training.
Collapse
Affiliation(s)
- S Swaroop Vedula
- Malone Center for Engineering in Healthcare, Department of Computer Science, The Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland 21218;
| | - Masaru Ishii
- Department of Otolaryngology-Head and Neck Surgery, The Johns Hopkins University School of Medicine, Baltimore, Maryland 21287
| | - Gregory D Hager
- Malone Center for Engineering in Healthcare, Department of Computer Science, The Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland 21218;
| |
Collapse
|
15
|
Levine JA. The Baetylus Theorem-the central disconnect driving consumer behavior and investment returns in Wearable Technologies. ACTA ACUST UNITED AC 2016; 7:59-65. [PMID: 27617162 PMCID: PMC5017154 DOI: 10.4236/ti.2016.73008] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
Abstract
The Wearable Technology market may increase fivefold by the end of the decade. There is almost no academic investigation as to what drives the investment hypothesis in wearable technologies. This paper seeks to examine this issue from an evidence-based perspective. There is a fundamental disconnect in how consumers view wearable sensors and how companies market them; this is called The Baetylus Theorem where people believe (falsely) that by buying a wearable sensor they will receive health benefit; data suggest that this is not the case. This idea is grounded social constructs, psychological theories and marketing approaches. A marketing proposal that fails to recognize The Baetylus Theorem and how it can be integrated into a business offering has not optimized its competitive advantage. More importantly, consumers should not falsely believe that purchasing a wearable technology, improves health.
Collapse
Affiliation(s)
- James A Levine
- Obesity Solutions, Mayo Clinic, 13400 E Shea Blvd, Scottsdale, AZ 85259 and Arizona State University, Obesity Solutions, 1000 S. Cady Mall 144, Tempe, Arizona 85287
| |
Collapse
|
16
|
Madani A, Watanabe Y, Bilgic E, Pucher PH, Vassiliou MC, Aggarwal R, Fried GM, Mitmaker EJ, Feldman LS. Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool. Surg Endosc 2016; 31:1203-1212. [PMID: 27412125 DOI: 10.1007/s00464-016-5091-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2015] [Accepted: 07/05/2016] [Indexed: 01/12/2023]
Abstract
BACKGROUND Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. METHODS An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test", VCT). Test-retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform's usability, feasibility and educational value. RESULTS Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach's α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)]. CONCLUSION This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.
Collapse
Affiliation(s)
- Amin Madani
- Department of Surgery, McGill University, Montreal, Canada.
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada.
| | - Yusuke Watanabe
- Department of Gastroenterological Surgery II, Hokkaido University, Sapporo, Japan
| | - Elif Bilgic
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada
| | - Philip H Pucher
- Department of Surgery and Cancer, Imperial College London, London, UK
| | - Melina C Vassiliou
- Department of Surgery, McGill University, Montreal, Canada
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada
| | - Rajesh Aggarwal
- Department of Surgery, McGill University, Montreal, Canada
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada
- Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Canada
| | - Gerald M Fried
- Department of Surgery, McGill University, Montreal, Canada
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada
- Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Canada
| | | | - Liane S Feldman
- Department of Surgery, McGill University, Montreal, Canada
- Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada
| |
Collapse
|