1
|
Eckhoff JA, Ban Y, Rosman G, Müller DT, Hashimoto DA, Witkowski E, Babic B, Rus D, Bruns C, Fuchs HF, Meireles O. TEsoNet: knowledge transfer in surgical phase recognition from laparoscopic sleeve gastrectomy to the laparoscopic part of Ivor-Lewis esophagectomy. Surg Endosc 2023; 37:4040-4053. [PMID: 36932188 PMCID: PMC10156818 DOI: 10.1007/s00464-023-09971-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 02/21/2023] [Indexed: 03/19/2023]
Abstract
BACKGROUND Surgical phase recognition using computer vision presents an essential requirement for artificial intelligence-assisted analysis of surgical workflow. Its performance is heavily dependent on large amounts of annotated video data, which remain a limited resource, especially concerning highly specialized procedures. Knowledge transfer from common to more complex procedures can promote data efficiency. Phase recognition models trained on large, readily available datasets may be extrapolated and transferred to smaller datasets of different procedures to improve generalizability. The conditions under which transfer learning is appropriate and feasible remain to be established. METHODS We defined ten operative phases for the laparoscopic part of Ivor-Lewis Esophagectomy through expert consensus. A dataset of 40 videos was annotated accordingly. The knowledge transfer capability of an established model architecture for phase recognition (CNN + LSTM) was adapted to generate a "Transferal Esophagectomy Network" (TEsoNet) for co-training and transfer learning from laparoscopic Sleeve Gastrectomy to the laparoscopic part of Ivor-Lewis Esophagectomy, exploring different training set compositions and training weights. RESULTS The explored model architecture is capable of accurate phase detection in complex procedures, such as Esophagectomy, even with low quantities of training data. Knowledge transfer between two upper gastrointestinal procedures is feasible and achieves reasonable accuracy with respect to operative phases with high procedural overlap. CONCLUSION Robust phase recognition models can achieve reasonable yet phase-specific accuracy through transfer learning and co-training between two related procedures, even when exposed to small amounts of training data of the target procedure. Further exploration is required to determine appropriate data amounts, key characteristics of the training procedure and temporal annotation methods required for successful transferal phase recognition. Transfer learning across different procedures addressing small datasets may increase data efficiency. Finally, to enable the surgical application of AI for intraoperative risk mitigation, coverage of rare, specialized procedures needs to be explored.
Collapse
Affiliation(s)
- J A Eckhoff
- Surgical Artificial Intelligence and Innovation Laboratory, Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC339, Boston, MA, 02114, USA. .,Department of General, Visceral, Tumor and Transplant Surgery, University Hospital Cologne, Kerpenerstrasse 62, 50937, Cologne, Germany.
| | - Y Ban
- Surgical Artificial Intelligence and Innovation Laboratory, Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC339, Boston, MA, 02114, USA.,Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA, 02139, USA
| | - G Rosman
- Surgical Artificial Intelligence and Innovation Laboratory, Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC339, Boston, MA, 02114, USA.,Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA, 02139, USA
| | - D T Müller
- Department of General, Visceral, Tumor and Transplant Surgery, University Hospital Cologne, Kerpenerstrasse 62, 50937, Cologne, Germany
| | - D A Hashimoto
- Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, OH, 44106, USA.,Department of Surgery, Case Western Reserve School of Medicine, Cleveland, OH, 44106, USA
| | - E Witkowski
- Surgical Artificial Intelligence and Innovation Laboratory, Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC339, Boston, MA, 02114, USA
| | - B Babic
- Department of General, Visceral, Tumor and Transplant Surgery, University Hospital Cologne, Kerpenerstrasse 62, 50937, Cologne, Germany
| | - D Rus
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA, 02139, USA
| | - C Bruns
- Department of General, Visceral, Tumor and Transplant Surgery, University Hospital Cologne, Kerpenerstrasse 62, 50937, Cologne, Germany
| | - H F Fuchs
- Department of General, Visceral, Tumor and Transplant Surgery, University Hospital Cologne, Kerpenerstrasse 62, 50937, Cologne, Germany
| | - O Meireles
- Surgical Artificial Intelligence and Innovation Laboratory, Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC339, Boston, MA, 02114, USA
| |
Collapse
|
2
|
Shih SP, Kantsevoy SV, Kalloo AN, Magno P, Giday SA, Ko CW, Isakovich NV, Meireles O, Hanly EJ, Marohn MR. Hybrid minimally invasive surgery--a bridge between laparoscopic and translumenal surgery. Surg Endosc 2007; 21:1450-3. [PMID: 17593460 DOI: 10.1007/s00464-007-9329-2] [Citation(s) in RCA: 47] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2006] [Revised: 10/16/2006] [Accepted: 12/04/2006] [Indexed: 12/15/2022]
Abstract
BACKGROUND The peroral transluminal approach to the peritoneal cavity appears safe, feasible, and may further reduce the invasiveness of surgery. However, flexible endoscopes have multiple limitations inside the peritoneal cavity, which can potentially be overcome by blending the use of both a laparoscope and a flexible upper endoscope--a hybrid approach. The goal of the present study was to evaluate a hybrid minimally invasive technique for cholecystectomy in a porcine model. METHODS Hybrid cholecystectomies were performed in acute experiments on 50-kg pigs under general anesthesia. Pneumoperitoneum was created with a Veress needle, and a laparoscopic 10-mm port was inserted. Under laparoscopic observation, the gastric wall incision was done with an endoscopic needle-knife and sphincterotome, and the upper endoscope was advanced into the peritoneal cavity. A laparoscopic 10-mm port was inserted into the right upper quadrant of the abdomen for gallbladder traction to facilitate exposure of the cystic duct and artery. Via the biopsy channel of the flexible endoscope, and using a knife with an isolated tip, a needle knife, and clips, both the cystic duct and artery were identified, clipped, and transected. The gallbladder itself was then dissected and retracted through the mouth, and the gastric wall incision was closed with endoscopic clips. RESULTS Five hybrid cholecystectomies were performed without complications. The laparoscopic port enabled a stable pneumoperitoneum, good traction and counter-traction, and improved spatial orientation and visualization. Necropsy did not reveal any intraperitoneal complications. CONCLUSIONS The hybrid approach increases safety of initial gastric puncture and gastric wall incision, improves orientation and navigation of the flexible endoscope inside the peritoneal cavity, simplifies peroral transgastric cholecystectomy, and could be used to decrease invasiveness of laparoscopic surgery and to facilitate development and clinical introduction of transgastric endoscopic procedures. ELECTRONIC SUPPLEMENTARY MATERIAL The online version of this article (doi:10.1007/s00464-007-9329-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- S P Shih
- Department of Surgery, Johns Hopkins University School of Medicine, 600 N. Wolfe Street, Halsted 608, Baltimore, Maryland 21287, USA
| | | | | | | | | | | | | | | | | | | |
Collapse
|
3
|
Meireles O, Kantsevoy SV, Kalloo AN, Jagannath SB, Giday SA, Magno P, Shih SP, Hanly EJ, Ko CW, Beitler DM, Marohn MR. Comparison of intraabdominal pressures using the gastroscope and laparoscope for transgastric surgery. Surg Endosc 2007; 21:998-1001. [PMID: 17404796 DOI: 10.1007/s00464-006-9167-7] [Citation(s) in RCA: 45] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2006] [Revised: 10/02/2006] [Accepted: 10/16/2006] [Indexed: 12/21/2022]
Abstract
BACKGROUND The peroral transgastric endoscopic approach for intraabdominal procedures appears to be feasible, although multiple aspects of this approach remain unclear. This study aimed to measure intraperitoneal pressure in a porcine model during the peroral transgastric endoscopic approach, comparing an endoscopic on-demand insufflator/light source with a standard autoregulated laparoscopic insufflator. METHODS All experiments were performed with 50-kg female pigs under general anesthesia. A standard upper endoscope was advanced perorally through a gastric wall incision into the peritoneal cavity. The peritoneal cavity was insufflated with operating room air from an endoscopic light source/insufflator. Intraperitoneal pressure was measured by three routes: (1) through the endoscope biopsy channel, (2) through a 5-mm transabdominal laparoscopic port, and (3) through a 16-gauge Veress needle inserted into the peritoneal cavity through the anterior abdominal wall. The source of insufflation alternated between on-demand manual insufflation through the endoscopic light source/insufflator using room air and a standard autoregulated laparoscopic insufflator using carbon dioxide (CO(2)). RESULTS Six acute experiments were performed. Intraperitoneal pressure measurements showed good correlation regardless of measurement route and were independent of the type of insufflation gas, whether room air or CO(2). On-demand insufflation with the endoscopic light source/insufflator resulted in a wide variation in pressures (range, 4-32 mmHg; mean, 16.0 +/- 11.7). Intraabdominal pressures using a standard autoregulated laparoscopic insufflator demonstrated minimal fluctuation (range, 8-15 mmHg; mean, 11.0 +/- 2.2 mmHg) around a predetermined value. CONCLUSION Use of an on-demand unregulated endoscopic light source/insufflator for translumenal surgery can cause large variation in intraperitoneal pressures and intraabdominal hypertension, leading to the risk of hemodynamic and respiratory compromise. Safety may favor well-controlled intraabdominal pressures achieved with a standard autoregulated laparoscopic insufflator.
Collapse
Affiliation(s)
- O Meireles
- Department of Surgery, Johns Hopkins University School of Medicine, Halsted 608, Baltimore, MD 21287, USA
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|