Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication

Validated simulation models in pediatric surgery: A review

INTRODUCTION

This review evaluates the validation and availability of simulation models in the field of pediatric surgery that can be used for training purposes.

METHODS

MEDLINE and EMBASE were searched for studies describing a simulation models in pediatric surgery. Articles were included if face, content and/or construct validity was described. Additionally, the costs and availability were assessed. Validation scores for each model were depicted as percentage (0-100), based on the reported data, to compare the outcomes. A score of >70% was considered adequate.

RESULTS

Forty-three studies were identified, describing the validation process of 38 simulation models. Face validity was evaluated in 33 articles, content in 36 and construct in 19. Twenty-two models received adequate validation scores (>70%). The majority (27/38, 70%) was strictly inanimate. Five models were available for purchase and eleven models were replicable based on the article.

CONCLUSION

The number of validated inanimate simulation models for pediatric surgery procedures is growing, however, few are replicable or available for widespread training purposes.

LEVEL OF EVIDENCE

Level II.

Evaluation of minimally invasive surgical skills training: comparing a neonatal esophageal atresia/tracheoesophageal fistula model with a dry box

BACKGROUND

Pediatric surgeons require highly advanced minimally invasive surgical skills to perform rare and complex surgeries in a very vulnerable population. We developed a neonatal esophageal atresia (EA) model to improve thoracoscopic surgical skills. This study aimed to evaluate the concurrent validity of the model by undertaking pre- and post-training skills assessments in two groups of students with no prior experience performing minimally invasive surgery, using the EA model and a dry box (DB).

METHODS

A pilot study was performed. The participants were randomly divided into two groups: one trained using the DB and one trained using the EA model. Both groups practiced a minimally invasive surgical suture task. The task completion time, 29-point checklist score, modified suturing error sheet score, and three-dimensional forceps movement in both groups were compared pre-and post-training by video analysis.

RESULTS

The EA model task was significantly more difficult than that of the DB. Both groups showed significant improvement in the task time, 29-point checklist score, and modified suturing error sheet score; however, the EA model training was more efficient in improving each error item. Regarding forceps movement, the EA model training significantly decreased wasted motion, whereas the DB was limited in this regard.

CONCLUSIONS

Short-term training on the EA model, which was more technically demanding than the DB, decreased technical error and wasted motion, and allowed learners to acquire surgical skills more efficiently than training with the DB model. These facts revealed the concurrent validity of the EA model.

Years of experience is more effective in defining experts in the gaze analysis of laparoscopic suturing task than task duration

In this study, the relationship between gaze patterns, task duration, and years of experience, which are commonly used to define and evaluate expert surgeons in laparoscopic surgery, was identified. Ten surgeons with 1-28 years of experience and six inexperienced students were included. Subjects used forceps to repeat the task of suturing a suture pad. Each subject wore an eye-marking recorder, and gaze points were recorded and analyzed. No significant relationship between task duration and gaze pattern was observed. However, there was a significant relationship between a surgeon's years of experience and the percentage of time spent gazing at the forceps. Subjects with more years of experience operated without looking at the forceps and fixed their gaze on the operational target. Therefore, when analyzing laparoscopic gazing patterns, it may be more appropriate to define an "expert" based on the years of experience rather than task duration.

Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review

PURPOSE

The aim of this study was to identify (1) the type of skill evaluation methods and (2) how the effect of training was evaluated in simulation-based training (SBT) in pediatric surgery.

METHODS

Databases of PubMed, Cochrane Library, and Web of Science were searched for articles published from January 2000 to January 2017. Search concepts of Medical Subject Heading terms were "surgery," "pediatrics," "simulation," and "training, evaluation."

RESULTS

Of 5858 publications identified, 43 were included. Twenty papers described simulators as assessment tools used to evaluate technical skills. Reviewers differentiated between experts and trainees using a scoring system (45%) and/or a checklist (25%). Simulators as training tools were described in 23 papers. While the training's effectiveness was measured using performance assessment scales (52%) and/or surveys (43%), no study investigated the improvement of the clinical outcomes after SBT.

CONCLUSION

Scoring, time, and motion analysis methods were used for the evaluation of basic techniques of laparoscopic skills. Only a few SBT in pediatric surgery have definite goals with clinical effect. Future research needs to demonstrate the educational effect of simulators as assessment or training tools on SBT in pediatric surgery.

A Comparison of the Effectiveness Between Three Different Endoscopic Surgical Skill Training Programs for Medical Students Using the Infant Laparoscopic Fundoplication Simulator: A Randomized Controlled Trial

Background and Aim: Training programs for developing laparoscopic suturing skills range from dry-box training to virtual simulator training. There are advantages and disadvantages to each training method; however, which training is best for medical students and young surgeons is unclear. The aim of this study was to compare the proficiency of medical students in acquiring laparoscopic suturing skills after various routes of short-term training: via a video, an expert teacher, or a virtual simulator. Materials and Methods: Seventeen medical students were registered and divided randomly into three groups: group receiving personal training while watching a training video (video group), group receiving training under the guidance of an expert (teaching group), and group receiving personal training with a virtual simulator (virtual group). The students practiced laparoscopic suturing and tying skills for 1 hour. Following their training, they performed the evaluation task of three sutures and ties using a laparoscopic fundoplication simulator. We developed a 1-year-old infant body model (body weight 10 kg) based on computed tomography data and established a pneumoperitoneum body model based on a clinical situation. Results: The path length of the assistant forceps in the virtual group tended to be longer than in the other groups. The average acceleration of the assistant forceps in the virtual group was faster than in the other groups (P = .04). There were no significant differences in the other evaluation parameters. Conclusion: A long-term and combination training study should be performed to develop the best method for training medical students and inexperienced young surgeons.

Continuous Skill Training Using the Disease-Specific Endoscopic Surgical Simulator to Promote Young Pediatric Surgeons: Learning Curve for Trainees

Purpose: This study aimed to evaluate the learning effects of continuous training with a disease-specific endoscopic surgical simulator for young pediatric surgeons. Materials and Methods: Participants trained with a laparoscopic fundoplication simulator for 1 hour every 10 days. At the start of each training session, we assessed the participant's surgical technique using the surgical skill evaluation system, which evaluates the following items: (1) task time, (2) right-left balance of suturing, (3) suture spacing between the three ligatures, (4) total path length traveled by forceps, (5) velocity of the forceps tips, (6) length of the wrap, (7) number of ligature failures, and (8) comparison of improvements according to assessment point. We examined the learning effects of this continuous training program. Results: Task time, right-left balance of suturing, and sum of the distance traveled by each forceps showed significant progressive improvements (P < .05). Suture spacing and average velocity of forceps tips did not change significantly with training (P = .5781, P = .0781, respectively). However, the ratio of traveled distance between left and right forceps significantly improved (P < .05). There was a significant trend for the wrap length to approach the target value (P < .05). According to the linear mixed-effects model, the number of training sessions required for learning was not uniform and varied depending on the skill. Conclusion: This simulator training program can help pediatric surgeons to acquire surgical skills easily, economically, and safely. In the future, we need to evaluate how surgical skills acquired during this continuous training are reflected in clinical operations.

Transitional impact of short- and long-term outcomes of a randomized controlled trial to evaluate laparoscopic versus open surgery for colorectal cancer from Japan Clinical Oncology Group Study JCOG0404

BACKGROUND

The JCOG0404 randomized controlled trial conducted to compare laparoscopic surgery (LAP) with open surgery (OP) for stage II/III colon cancer showed better short-term outcomes and equal long-term outcomes of LAP versus OP. Technical instrumentation of surgery and anticancer agents given during the registration period might have affected the outcomes.

AIM

To evaluate outcomes according to the registration periods.

METHODS

The overall registration period was divided into three periods (first: 2004-2005, second: 2006-2007 and third: 2008-2009). Short-term and long-term outcomes were compared between registration periods.

RESULTS

In total, 1057 patients were registered. Numbers of patients undergoing each approach for each of the three periods (1st/2nd/3rd) were 528 for OP (106/244/178) and 529 for LAP (106/246/177). Operation time (minutes) did not change between the periods for OP (160/156/161) or LAP (205/211/219). Blood loss (mL) gradually decreased in the latter two periods: (119/80/75) for OP and (35/28/25) for LAP. Incidence of complications (%) decreased in the latter periods for OP (27.6/20.3/21.3), whereas that for LAP remained consistently low (14.3/14.8/13.6). There was no particular trend in 5-year overall survival and recurrence-free survival depending on the period regardless of treatment. D3 dissection rates were 95% or more for all periods in both groups.

CONCLUSIONS

Operation time and survival rates did not change over time, whereas blood loss in OP improved in the latter periods. Quality control applied in this trial might have been effective in producing such safe endpoints. (ClinicalTrials.gov, number NCT00147134, UMIN Clinical Trials Registry, number C000000105.).

Feasibility of an AI-Based Measure of the Hand Motions of Expert and Novice Surgeons

This study investigated whether parameters derived from hand motions of expert and novice surgeons accurately and objectively reflect laparoscopic surgical skill levels using an artificial intelligence system consisting of a three-layer chaos neural network. Sixty-seven surgeons (23 experts and 44 novices) performed a laparoscopic skill assessment task while their hand motions were recorded using a magnetic tracking sensor. Eight parameters evaluated as measures of skill in a previous study were used as inputs to the neural network. Optimization of the neural network was achieved after seven trials with a training dataset of 38 surgeons, with a correct judgment ratio of 0.99. The neural network that prospectively worked with the remaining 29 surgeons had a correct judgment rate of 79% for distinguishing between expert and novice surgeons. In conclusion, our artificial intelligence system distinguished between expert and novice surgeons among surgeons with unknown skill levels.

Automatically rating trainee skill at a pediatric laparoscopic suturing task

BACKGROUND

Minimally invasive surgeons must acquire complex technical skills while minimizing patient risk, a challenge that is magnified in pediatric surgery. Trainees need realistic practice with frequent detailed feedback, but human grading is tedious and subjective. We aim to validate a novel motion-tracking system and algorithms that automatically evaluate trainee performance of a pediatric laparoscopic suturing task.

METHODS

Subjects (n = 32) ranging from medical students to fellows performed two trials of intracorporeal suturing in a custom pediatric laparoscopic box trainer after watching a video of ideal performance. The motions of the tools and endoscope were recorded over time using a magnetic sensing system, and both tool grip angles were recorded using handle-mounted flex sensors. An expert rated the 63 trial videos on five domains from the Objective Structured Assessment of Technical Skill (OSATS), yielding summed scores from 5 to 20. Motion data from each trial were processed to calculate 280 features. We used regularized least squares regression to identify the most predictive features from different subsets of the motion data and then built six regression tree models that predict summed OSATS score. Model accuracy was evaluated via leave-one-subject-out cross-validation.

RESULTS

The model that used all sensor data streams performed best, achieving 71% accuracy at predicting summed scores within 2 points, 89% accuracy within 4, and a correlation of 0.85 with human ratings. 59% of the rounded average OSATS score predictions were perfect, and 100% were within 1 point. This model employed 87 features, including none based on completion time, 77 from tool tip motion, 3 from tool tip visibility, and 7 from grip angle.

CONCLUSIONS

Our novel hardware and software automatically rated previously unseen trials with summed OSATS scores that closely match human expert ratings. Such a system facilitates more feedback-intensive surgical training and may yield insights into the fundamental components of surgical skill.

A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system

PURPOSE

We developed and validated a specific laparoscopic fundoplication simulator for use with the objective endoscopic surgical skills evaluation system. The aim of this study was to verify the quality of skills of surgeons.

MATERIALS AND METHODS

We developed a 1-year-old infant body model based on computed tomography data and reproduced pneumoperitoneum model based on the clinical situation. The examinees were divided into three groups: fifteen pediatric surgery experts (PSE), twenty-four pediatric surgery trainees (PSN), and ten general surgeons (GS). They each had to perform three sutures ligatures for construction of Nissen wrap. Evaluate points are time for task, the symmetry of the placement of the sutures, and the uniformity of the interval of suture ligatures in making wrap. And the total path length and velocity of forceps were measured to assess bi-hand coordination.

RESULTS

PSE were significantly superior to PSN regarding total time spent (p < 0.01) and total path length (p < 0.01). GS used both forceps faster than the other groups, and PSN used the right forceps faster than the left forceps (p < 0.05). PSE were shorter with regard to the total path length than GS (p < 0.01). PSE showed most excellent results in the symmetry of the wrap among three groups.

CONCLUSION

Our new model was used useful to validate the characteristics between GS and pediatric surgeon. Both PSE and GS have excellent bi-hand coordination and can manipulate both forceps equally and had superior skills compared to PSN. In addition, PSE performed most compact and accurate skills in the conflicted operative space.

An ex vivo liver training model continuously perfused to simulate bleeding for suture skills involved in laparoscopic liver resection: development and validity

BACKGROUND

Suture skills are essential to laparoscopic liver resection. The current suture training models are not ideal enough. The aim of this study is to develop and verify a highly simulated-bleeding continuously perfused training model (CPTM) and to evaluate its training efficacy.

METHODS

CPTM was constructed using fresh lamb liver whose portal veins were perfused with red-dyed liquid gelatin. Construct validity of CPTMs was tested in 33 participants with three levels of laparoscopic experience (experts, intermediates, and novices) who were demanded to finish one superficial stitch and one deep stitch for suture hemostasis on CPTMs. The CPTMs were also evaluated by the experts. CPTMs were compared with dry box training models (DBTMs) regarding training efficacy among the novices who were assigned to DBTM and CPTM groups to, respectively, complete a 10-day training on CPTMs or DBTMs. Before and after their assignments, their superficial stitches were assessed by completion time, suture accuracy, and suture knot performance while their deep stitches by completion time and bleeding control.

RESULTS

CPTM proved to be construct valid by both superficial and deep stitches. Significant differences were found regarding completion time (763, 271, 174 s), suture accuracy (4.4, 1.8, 0.2 mm), and suturing knot performance (12.1, 21.5, 22.0) for superficial stitches (p < 0.001), as well as regarding completion time (807, 423, 277 s) for deep stitches (p < 0.001). Positive comments were given by all experts. CPTMs helped novices to acquire laparoscopic suture skills. Their training efficacy was significantly better than that of DBTMs (p < 0.05). Learning curves of CPTM group plateaued at the sixth round for superficial stitches and at the seventh round for deep stitches.

CONCLUSION

CPTM offers trainees a highly simulated-bleeding means to acquire advanced laparoscopic suture skills. The suture skills learned on CPTMs may improve significantly at the seventh round.

The effect of forceps manipulation for expert pediatric surgeons using an endoscopic pseudo-viewpoint alternating system: the phenomenon of economical slow and fast performance in endoscopic surgery

PURPOSE

Endoscopic surgery is performed under a horizontal view in comparison to the vertical view that is associated with open surgery. We developed an endoscopic pseudo-viewpoint alternation system with out any scope action. We investigate the effect of this novel system on forceps manipulation among expert pediatric surgeons.

METHODS

Six expert pediatric surgeons performed a Nissen wrap in a fundoplication simulator either with or without this system. The constructed Nissen wrap was evaluated. The total path length and the average velocity of the forceps were also analyzed.

RESULTS

The times required either with or without this system were 587.5 ± 122.7 and 634.0 ± 212.4 s (p = 0.45), respectively. The total path lengths of right and left forceps either with or without this system were 12,309 ± 2495.5 and 15,726 ± 5649.6 mm (p = 0.07), 10,091 ± 2439.2 and 12,575 ± 5511.1 mm (p = 0.11), respectively. The average velocity of the right and left forceps with or without this system were 26.9 ± 5.29 and 31.6 ± 1.62 mm/s (p = 0.04), 21.6 ± 2.48 and 25.5 ± 6.48 mm/s (p = 0.15), respectively. There was no significance in the suture balance and suture interval.

CONCLUSION

The endoscopic pseudo-viewpoint alternation system thus made it possible for expert pediatric surgeons to carry out slow and economical forceps manipulation. These effects make it possible for surgeons to perform safe and precise surgery, thus leading to a shortening of operation time.

Procedural surgical skill assessment in laparoscopic training environments

PURPOSE

This study aimed to identify detailed differences in laparoscopic surgical processes between expert and novice surgeons in a training environment and demonstrate that surgical process modeling can be used for such detailed analysis.

METHODS

Eleven expert surgeons each of whom had performed [Formula: see text] laparoscopic procedures were compared with 10 young surgeons each of whom had performed [Formula: see text] laparoscopic procedures, and five medical students. Each examinee performed a specific skill assessment task. During tasks, instrument motion was monitored using a video capture system. From the video, the corresponding workflow was recorded by labeling the surgeons' activities according to a predefined terminology. Activities represented manual work steps performed during the task, described by a combination of a verb (representing the action), a tool, and the involved structure. The results were described as the number of occurrences (times), average duration (seconds), total duration (seconds), minimal duration (seconds), maximal duration (seconds), and occupancy percentage (%).

RESULTS

The terminology for describing the processes of this task included 10 actions, six tools, four structures, and three events for each hand. There were 63 combinations of different possible activities; significant differences in 12 activities were observed between the expert and novice groups (young surgeons and medical students). The expert group performed the task with fewer occurrences and shorter duration than did the novice group in the left hand.

CONCLUSIONS

We identified differences in surgical process between experts and novices in laparoscopic surgical simulation. Our proposed method would be useful for education and training in laparoscopic surgery.

Video-based skill assessment of endoscopic suturing in a pediatric chest model and a box trainer

PURPOSE

Pediatric endoscopic surgery requires special surgical skills because of the small working space and tissue fragility. This article presents a video-based skill assessment method for endoscopic suturing using a pediatric chest model.

MATERIALS AND METHODS

A commercial suture pad was placed in a rapid-prototyped pediatric chest model of a 1-year-old patient to simulate the thoracoscopic repair of esophageal atresia type C. Twenty-eight pediatric surgeons (9 experts, 9 intermediates, and 10 trainees) performed an endoscopic intracorporeal suturing and knot-tying task both in the pediatric chest model and in a box trainer. The tasks were video-recorded and rated by two blinded observers using the 29-point checklist method and a suturing errors score sheet method. The task completion time and the number of needle manipulations were measured.

RESULTS

The expert group showed better performance than the intermediate and trainee groups in the pediatric chest model, and the differences were larger than those in the box trainer. Significant differences between the expert and the trainee groups were observed in the items related to safety such as the skills for keeping the needle in view at all times. Significant differences between the expert and intermediate groups were observed in the items related to task quality and efficiency such as the smoothness of knot tying and the number of needle manipulations.

CONCLUSIONS

Video-based skill assessment of endoscopic suturing using the pediatric chest model and a box trainer distinguished pediatric endoscopic surgeons according to their clinical experience, and pediatric-specific skills were identified.

Objective assessment of the suture ligature method for the laparoscopic intestinal anastomosis model using a new computerized system

BACKGROUND

The purpose of this study was to develop a new objective assessment system for the suture ligature method employed in the laparoscopic intestinal anastomosis model. Suturing skills were evaluated objectively using this system.

METHODS

This study compared 17 expert surgeons, each of whom had performed >500 laparoscopic procedures, with 36 novice surgeons, each of whom had performed <15 laparoscopic procedures. Each examinee performed a specific skill assessment task using an artificial model that mimics living tissue, which is linked with the Suture Simulator Instruction Evaluation Unit. The model used internal air pressure measurements and image processing to evaluate suturing skills. Five criteria were used to evaluate the skills of participants.

RESULTS

The volumes of air pressure leak in the expert and novice groups were 21.13 ± 6.68 and 8.51 ± 8.60 kPa, respectively. The numbers of full-thickness sutures in the expert and novice groups were 2.94 ± 0.24 pairs and 2.47 ± 0.77 pairs, respectively. Suture tensions in the expert and novice groups were 60.99 ± 11.81 and 80.90 ± 16.63 %, respectively. The areas of wound-opening in the expert and novice groups were 1.76 ± 2.17 and 11.06 ± 15.37 mm(2), respectively. The performance times in the expert and novice groups were 349 ± 120 and 750 ± 269 s, respectively. Significant differences between the expert and novice groups for each criterion were observed. The acceptable range of values for each criterion except for the number of full-thickness sutures was statistically defined by the performance of the expert group.

CONCLUSIONS

Our system is useful for the quantitative assessment of suturing skill in laparoscopic surgery. We believe that this system is a useful tool for training and assessment of laparoscopic surgeons.