The role of virtual reality in surgical training in otorhinolaryngology

PURPOSE OF REVIEW

This article reviews the rationale, current status and future directions for the development and implementation of virtual reality surgical simulators as training tools.

RECENT FINDINGS

The complexity of modern surgical techniques, which utilize advanced technology, presents a dilemma for surgical training. Hands-on patient experience - the traditional apprenticeship method for teaching operations - may not apply because of the learning curve for skill acquisition and patient safety expectation. The paranasal sinuses and temporal bone have intricate anatomy with a significant amount of vital structures either within the surgical field or in close proximity. The current standard of surgical care in these areas involves the use of endoscopes, cameras and microscopes, requiring additional hand-eye coordination, an accurate command of fine motor skills, and a thorough knowledge of the anatomy under magnified vision. A surgeon's disorientation or loss of perspective can lead to complications, often catastrophic and occasionally lethal. These considerations define the ideal environment for surgical simulation; not surprisingly, significant research and validation of simulators in these areas have occurred.

SUMMARY

Virtual reality simulators are demonstrating validity as training and skills assessment tools. Future prototypes will find application for routine use in teaching, surgical planning and the development of new instruments and computer-assisted devices.

Construct Validity of the Endoscopic Sinus Surgery Simulator

OBJECTIVES

To establish discriminant validity of the endoscopic sinus surgery simulator (ES3) (Lockheed Martin, Akron, Ohio) between various health care provider experience levels and to define benchmarking criteria for skills assessment.

DESIGN

Prospective multi-institutional comparison study.

SETTING

University-based tertiary care institution.

PARTICIPANTS

Ten expert otolaryngologists, 14 otolaryngology residents, and 10 medical students.

INTERVENTIONS

Subjects completed the ES3's virtual reality curriculum (10 novice mode, 10 intermediate mode, and 3 advanced mode trials). Performance scores were recorded on each trial. Performance differences were analyzed using analysis of variance for repeated measures (experience level as between-subjects factor).

MAIN OUTCOME MEASURES

Simulator performance scores, accuracy, time to completion, and hazard disruption.

RESULTS

The novice mode accurately distinguished the 3 groups, particularly at the onset of training (mean scores: senior otolaryngologists, 66.0; residents, 42.7; students, 18.3; for the paired comparisons between groups 1 and 2 and groups 1 and 3, P = .04 and .03, respectively). Subjects were not distinguished beyond trial 5. The intermediate mode only discriminated students from other subjects (P = .008). The advanced mode did not show performance differences between groups. Scores on the novice mode predicted those on the intermediate mode, which predicted advanced mode scores (r = 0.687), but no relationship was found between novice and advanced scores. All groups performed equally well and with comparable consistency at the outset of training. Expert scores were used to define benchmark criteria of optimal performance.

CONCLUSIONS

This study completes the construct validity assessment of the ES3 by demonstrating its discriminant capabilities. It establishes expert surgeon benchmark performance criteria and shows that the ES3 can train novice subjects to attain those. The refined analysis of trial performance scores could serve educational and skills assessment purposes. Current studies are evaluating the transfer of surgical skills acquired on the ES3 to the operating room (predictive validity).

Medical students' attitudes toward the use of an endoscopic sinus surgery simulator as a training tool

BACKGROUND

Modern adult learning theory characterizes self-directed learning as most effective. While studying the effectiveness of an endoscopic sinus surgery simulator (ES3) as a training tool and acknowledging that its successful integration into a training program is dependent on its acceptance and self-driven use by trainees, we sought to determine our study subjects' attitudes toward the simulator.

METHODS

Twenty-six medical students were enrolled and trained in our ES3 study. Each student was asked to complete a 28-item questionnaire on completion of training. This questionnaire contained 10-point Likert scale instruments, yes/no questions, and one open-ended question.

RESULTS

All but four subjects responded to the questionnaire; 90.9% of the respondents rated the training benefit derived from the simulator's novice mode as 6 or greater on a 10-point scale; the mean was 7.82 (+/- 2.22). The training benefit of the intermediate mode also was scored highly, with a mean score in the 7-9 range for all but one component-heart rate response to actions performed. Trainees appreciated the simulator's ability to help them adapt to a three-dimensional space on a two-dimensional display. They also noted its strengths in elucidating intranasal anatomy.

CONCLUSION

Medical students who enrolled in our study, as a group,felt that the ES3 provided them with significant training benefits. Although subjective, these attitudes, coupled with objective data indicating that there is a measurable benefit from use of the ES3, will ensure itsfull acceptance and use in otolaryngology training programs.

The effects of previously acquired skills on sinus surgery simulator performance

OBJECTIVE

The Endoscopic Sinus Surgery Simulator is currently being studied as a tool for otolaryngology resident training. While examining performance patterns on the simulator, we sought to define any relationships that may exist between students' previously acquired skills and their performance on the simulator.

METHODS

Twenty-six medical students were enrolled in our study. After completing their trials, they were asked to complete a 28-item questionnaire. This questionnaire included yes/no questions as well as 10-point Likert scale instruments.

RESULTS

Only 4 students did not respond to the questionnaire. Significant contributions to simulator performance were elucidated for a number of previously learned skills, but most were short-lived. For example, experienced video gamers were significantly better at hazard avoidance in the simulator's novice mode (P = .03), but their advantage dissipated as they advanced to intermediate mode trials. In addition, students' handedness contributed to simulator performance for only the first 3 trials (P = .04), but this relationship, too, was no longer statistically significant in later trials.

CONCLUSIONS

Some study students had skills and characteristics that significantly aided them in their ability to perform surgery on the simulator. However, these benefits were short-lived, and these results suggest that practicing plays a more important and long-standing role than other factors in surgical simulator performance. It follows that surgical simulation should play a more prominent role in surgical education because residents who practice on simulators such as this are, in turn, practicing for the reality of the operating room.

Chronic sinusitis: a surgical perspective

Rhinosinusitis is one of the most common chronic diseases with significant symptoms and impact on the quality of life. Surgery is indicated when appropriate medical treatment fails. A review of the clinical presentations of rhinosinusitis is presented, as well as an overview of current medical and surgical treatment options. Although external approaches are still mandated in some selected instances, endoscopic sinus surgery has become the standard for surgical treatment of sinonasal pathologies, by aiming at restoring the physiological functions of the nasal and paranasal cavities. This technique presents numerous advantages with a minimally invasive approach, better visualization of structures, camera magnification of the surgical field, and allows thorough exploration of difficult-to-reach areas. The risks and complications of endoscopic sinus surgery are well identified and may be reduced significantly by the adjunct use of computer-assisted image guidance and intraoperative fluoroscopy. Surgical simulation also is a promising tool to help reduce surgical errors and improve surgeon training in the future.

Assessment of construct validity of the endoscopic sinus surgery simulator

OBJECTIVE

To study the relationship between performance on an endoscopic sinus surgery simulator (ES3) and fundamental perceptual, visuospatial, and psychomotor abilities.

DESIGN

Validation study.

SETTING

Tertiary care medical center.

PARTICIPANTS

Thirty-four medical students and 4 otolaryngology residents voluntarily enrolled.

INTERVENTIONS

Subjects performed tasks on the ES3, minimally invasive surgical trainer virtual reality (MIST-VR), pictorial surface orientation (PicSOr), and 3 visuospatial tests (cube comparison, card rotation, and map planning).

MAIN OUTCOME MEASURES

The MIST-VR was scored for time, task error, economy of hand movement, economy of diathermy, and total score. Scores were generated for the PicSOr task and visuospatial tests. Scores were correlated with time, accuracy, and total subscore on navigation, injection, and dissection tasks, as well as hazard score and total trial score on the ES3.

RESULTS

The PicSOr score was statistically significantly correlated with the hazard score on the ES3 (r = 0.50, P < .001). Cube comparison (r = 0.43, P < .01) and card rotation (r = 0.45, P < .01) scores correlated significantly with the ES3 trial score, as did the MIST-VR total score and the ES3 trial score (r = 0.57, P < .001). In a multiple regression model, the PicSOr, cube comparison, and MIST-VR total scores were statistically significant predictors of ES3 performance (r = 0.63, P < .01).

CONCLUSIONS

Scores on the ES3 correlate strongly with scores on previously validated measures of perceptual, visuospatial, and psychomotor performance. The ES3 provides a reliable assessment of factors that are important to the acquisition of minimally invasive surgical skills, demonstrating construct validity.

Effects of a twenty-four hour call period on resident performance during simulated endoscopic sinus surgery in an accreditation council for graduate medical education-compliant training program

OBJECTIVES/HYPOTHESIS

Since the early 1990s, extended resident work hours have undergone increasing scrutiny. Although previous studies have demonstrated conflicting results regarding cognitive decline secondary to fatigue, few studies have specifically examined the effects of fatigue on surgical performance. No previous studies have examined resident performance under current Accreditation Council for Graduate Medical Education (ACGME) work-hour guidelines that limit residents to an average number of work hours of 80 hours per week. The study sought to determine whether an endoscopic sinus surgery simulator (ES3) measured performance changes before and after a 24-hour on-call period in residents following mandated work-hour limitations.

STUDY DESIGN

Case control, crossover trial at a Level I trauma center.

METHODS

Eight general surgical residents were trained on the novice mode of the ES3. These residents were then tested twice both before and after on-call duties. Performance and hazard scores were compared using a paired t test.

RESULTS

No statistically significant change in the number of errors, time to task completion, or overall performance was identified in the study between the precall and postcall groups. There was a trend toward improved speed at the expense of accuracy in the postcall group. Postcall score between the two trials improved, on average, by 3.3 (P = .045).

CONCLUSION

In the study of residents following current ACGME work-hour mandates, there was no diminution in performance before and after a 24-hour on-call period. There was a trend toward improved speed at the expense of accuracy. Furthermore, repetition on the ES3 in the postcall period can result in improved ES3 proficiency.

Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training

SUMMARY BACKGROUND DATA

To inform surgeons about the practical issues to be considered for successful integration of virtual reality simulation into a surgical training program. The learning and practice of minimally invasive surgery (MIS) makes unique demands on surgical training programs. A decade ago Satava proposed virtual reality (VR) surgical simulation as a solution for this problem. Only recently have robust scientific studies supported that vision

METHODS

A review of the surgical education, human-factor, and psychology literature to identify important factors which will impinge on the successful integration of VR training into a surgical training program.

RESULTS

VR is more likely to be successful if it is systematically integrated into a well-thought-out education and training program which objectively assesses technical skills improvement proximate to the learning experience. Validated performance metrics should be relevant to the surgical task being trained but in general will require trainees to reach an objectively determined proficiency criterion, based on tightly defined metrics and perform at this level consistently. VR training is more likely to be successful if the training schedule takes place on an interval basis rather than massed into a short period of extensive practice. High-fidelity VR simulations will confer the greatest skills transfer to the in vivo surgical situation, but less expensive VR trainers will also lead to considerably improved skills generalizations.

CONCLUSIONS

VR for improved performance of MIS is now a reality. However, VR is only a training tool that must be thoughtfully introduced into a surgical training curriculum for it to successfully improve surgical technical skills.

Determinants of survival in parotid gland carcinoma: a population-based study

PURPOSE

To determine survival and factors influencing survival for parotid gland cancer.

METHODS

Cases of parotid gland malignancy were extracted from the SEER database for 1988 to 1998. Kaplan-Meier survival analysis was conducted for the most common tumor histologies. Cox proportional hazards modeling was conducted to determine the influence of age, gender, histopathology, grade, size, regional modal status, extraglandular extension, and radiation therapy on survival. Subset analysis was conducted for mucoepidermoid carcinoma according to grade.

RESULTS

Nine hundred three patients were identified with a mean age of 59.2 years. Mean follow-up was 51.8 months. Mean 5-year and 10-year actuarial survivals for the entire cohort were 87.8 months, 66.6% and 49.7%, respectively. Mean tumor size at diagnosis was 2.7 cm; 38.0% of patients had extraglandular extension of the tumor, 26.8% of patients had positive nodal disease, and 59.4% of patients received radiation therapy. Tumor histology did predict survival, with squamous cell carcinoma and acinar cell carcinoma exhibiting the poorest and best survivals, respectively. Stratified Cox proportional hazards modeling revealed that increasing age, tumor size, grade, extraglandular extension, and nodal positivity significantly negatively influenced survival (all P<or=.001); radiation therapy conferred a survival benefit (P=.090), whereas gender did not significantly affect survival. Increasing tumor grade, nodal disease, and extraglandular extension carried particularly high hazards ratios.

CONCLUSIONS

Survival in parotid gland malignancy is influenced by multiple factors. Patients with multiple poor prognostic features such as extraglandular extension, aggressive tumor histologies, and nodal disease will exhibit poorer survivals and may be candidates for more aggressive treatment protocols.

National Assessment of Business-of-Medicine Training and Its Implications for the Development of a Business-of-Medicine Curriculum

OBJECTIVES/HYPOTHESIS

The objectives were, first, to determine the current state of business training in otolaryngology residency programs in the United States and, second, to lay the groundwork for development of a business-of-medicine (BOM) curriculum.

STUDY DESIGN

Cross-sectional survey.

METHODS

A survey concerning methodology and topics for management training of residents was mailed to the chairpersons or program directors of the 102 otolaryngology residency programs. A similar survey was sent to 576 otolaryngology graduates (classes of 2000, 2001, and 2002). An interactive BOM curriculum on CD-ROM was developed based on the results.

RESULTS

The response rate among program directors was 74.5% (76 of 102), and among the otolaryngology graduates, 38.2% (220 of 575). Seventy-five percent of graduates rated their BOM training as poor or fair. Only 8% rated their BOM training as excellent. Twenty percent of the graduates responded to having a BOM course during residency. Recent graduates reported that a BOM course can best be taught through lectures and apprenticeship/mentoring, whereas program directors reported that a BOM course can best be taught through lectures and outside consultants. Graduates reported that coding compliance was the topic most neglected in residency, whereas program directors reported that coding compliance was the main topic covered in the business training. Both groups agreed that department attending physicians have the most impact on a resident's business training. Program directors reported that correct coding, planning one's entry into medical practice, risk management, and reimbursement issues are the most important topics for residents to learn, whereas recent graduates stated that the most important topics should be correct coding, office management, risk management, and reimbursement.

CONCLUSION

The present study reflects a perceived necessity for improvement of BOM training in otolaryngology residency programs. Based on this finding, the outcome measures from the survey, and the authors' own experience from business courses given in the first author's department, a BOM curriculum was developed that is general enough to target all otolaryngology residents and intended to provide business skills which result in improved use of resources and, ultimately, higher quality of care.

Identifying and reducing errors with surgical simulation

The major determinant of a patient's safety and outcome is the skill and judgment of the surgeon. While knowledge base and decision processing are evaluated during residency, technical skills-which are at the core of the profession-are not evaluated. Innovative state of the art simulation devices that train both surgical tasks and skills, without risk to patients, should allow for the detection and analysis of errors and "near misses". Studies have validated the use of a sophisticated endoscopic sinus surgery simulator (ES3) for training residents on a procedural basis. Assessments are proceeding as to whether the integration of a comprehensive ES3 training programme into the residency curriculum will have long term effects on surgical performance and patient outcomes. Using various otolaryngology residencies, subjects are exposed to mentored training on the ES3 as well as to minimally invasive trainers such as the MIST-VR. Technical errors are identified and quantified on the simulator and intraoperatively. Through a web based database, individual performance can be compared against a national standard. An upgraded version of the ES3 will be developed which will support patient specific anatomical models. This advance will allow study of the effects of simulated rehearsal of patient specific procedures (mission rehearsal) on patient outcomes and surgical errors during the actual procedure. The information gained from these studies will help usher in the next generation of surgical simulators that are anticipated to have significant impact on patient safety.

Identifying and reducing errors with surgical simulation

The major determinant of a patient's safety and outcome is the skill and judgment of the surgeon. While knowledge base and decision processing are evaluated during residency, technical skills-which are at the core of the profession-are not evaluated. Innovative state of the art simulation devices that train both surgical tasks and skills, without risk to patients, should allow for the detection and analysis of errors and "near misses". Studies have validated the use of a sophisticated endoscopic sinus surgery simulator (ES3) for training residents on a procedural basis. Assessments are proceeding as to whether the integration of a comprehensive ES3 training programme into the residency curriculum will have long term effects on surgical performance and patient outcomes. Using various otolaryngology residencies, subjects are exposed to mentored training on the ES3 as well as to minimally invasive trainers such as the MIST-VR. Technical errors are identified and quantified on the simulator and intraoperatively. Through a web based database, individual performance can be compared against a national standard. An upgraded version of the ES3 will be developed which will support patient specific anatomical models. This advance will allow study of the effects of simulated rehearsal of patient specific procedures (mission rehearsal) on patient outcomes and surgical errors during the actual procedure. The information gained from these studies will help usher in the next generation of surgical simulators that are anticipated to have significant impact on patient safety.

Learning curves, acquisition, and retention of skills trained with the endoscopic sinus surgery simulator

BACKGROUND

As an initial step in evaluating the effectiveness of training otolaryngology residents on an endoscopic sinus surgery simulator (ES3), we have assessed the ability of the ES3 to train persons inexperienced in sinus surgery (medical students) to perform certain simulated procedural tasks needed in endoscopic sinus surgery (ESS).

METHODS

A total of 26 medical students were enrolled and trained on the ES3 following a preset protocol in the three levels of difficulty and complexity (novice, intermediate, and advanced modes).

RESULTS

In the novice mode (three-dimensional abstract images are used to teach the use of endoscopic surgical equipment), medical students displayed a steep learning curve within three to five trials on the simulator and after an additional four to five trials, they reached a plateau in their learning curves to within 90% of that of experienced sinus surgeons. In the intermediate mode (ESS is performed on a simulated patient with teaching aids), medical students were able to reach a plateau in their learning curves to within 80% of that of experienced surgeons. This performance was sustained in the advanced mode (simulated sinus surgery without teaching aids). We observed that medical students, who had novice or intermediate mode training interrupted with an interval of 11-60 days, were able to resume their training without deviation from their prior learning curves.

CONCLUSION

Intensive, proctored training on the ES3 can train inexperienced persons to perform simulated ESS within a reasonable approximation of the performance of experienced sinus surgeons on the ES3 and the training that an inexperienced person receives on the simulator is not short term but is retained over a period of at least 2 months.

Navigational systems for sinus surgery: new developments

The last 15 years have seen an explosion in the understanding and treatment of paranasal sinus disease. On the heels of the development of computed tomography (CT), a new modality of paranasal intervention was propagated with the introduction of rigid telescopic technology. In the last 10 years, both computer hardware and software improvements have permitted the otolaryngologist to further this endoscopic ability with CT-guided navigational systems. This article reviews the technology, utility, and advances in the field of image-guided endoscopic sinus surgery.

The accuracy of computed tomography in the diagnosis of chronic rhinosinusitis

OBJECTIVE

To determine the sensitivity, specificity, and diagnostic accuracy of paranasal sinus computed tomography (CT) in the diagnosis of chronic rhinosinusitis (CRS).

STUDY DESIGN

Prospective dual cohort study.

METHODS

One hundred seventy-one consecutive patients undergoing endoscopic sinus surgery for CRS were evaluated with CT and staged according to the Lund system. Histopathological findings from sinus specimens were reviewed and graded. A second contemporaneous control group of 130 patients undergoing CT of the sinus regions for other reasons but without a diagnosis of CRS was also staged. Sensitivity, specificity, and the receiver-operator characteristic were determined for the sinus CT in the diagnosis of CRS. Positive and negative predictive values were also computed.

RESULTS

In the disease-positive group of patients with CRS, the mean Lund score was 9.8 (95% confidence interval, 9.0-10.6). The mean inflammatory grade on histopathological study was 2.3 (range, 0-4). For the control group (without disease), the mean Lund score was 4.3 (95% confidence interval, 3.5-5.0). The AUC for the receiver-operator characteristic was 0.802 (P <.001). Selecting a Lund score cut-off value of greater than 2 as abnormal, the sinus CT exhibited sensitivity and specificity of 94% and 41%, respectively. Increasing the cut-off value to 4 changed the sensitivity and specificity to 85% and 59%, respectively.

CONCLUSIONS

The paranasal sinus CT scan exhibits good sensitivity and above-average specificity for the diagnosis of CRS. When added to the history and physical findings, CT may add to the diagnostic accuracy of CRS.

A methodology for objective assessment of errors: an example using an endoscopic sinus surgery simulator

A well-proven methodology (the modified Delphi method) was used to generate a first-order approximation of errors that should be measured in a virtual reality surgical simulator (the ES3). The methodology and the errors derived were crafted in such a way as to be generalizable. Although some of the error measures are specific for sinus surgery, the same type of methodology can be used for other otolaryngologic, general, and [table: see text] subspecialty surgical procedures. The value of this process is that it can provide a uniform framework for investigators in surgical education and training to establish error measurements in their particular procedures or disciplines and to generate data and outcomes that are comparable, interoperable, and sharable with other investigators. Admittedly, the process is time consuming and rigorous, but it does provide a solid scientific basis to generate evidence-based data for the validation of training methods and for outcomes analysis.

Nodal metastasis in major salivary gland cancer: predictive factors and effects on survival

OBJECTIVES

To determine how regional nodal metastasis affects survival in patients with major salivary gland malignancy and to identify clinical predictors for nodal disease.

METHODS

Major salivary gland cancer cases with nodal sampling were identified from the Surveillance, Epidemiology, and End Results cancer database for 1988 through 1998. Kaplan-Meier survival analysis was conducted to compare patients with and without histopathologic evidence of nodal disease. Multivariate logistic regression analysis was used to determine the influence of clinical predictors on the presence of regional nodal disease.

RESULTS

A total of 1268 patients with major salivary gland malignancy and regional node sampling were identified. Mean age at diagnosis was 58.3 years, with a male-female ratio of 1:4. Mean tumor size was 3.0 cm. Overall mean survival time was 83 months (95% confidence interval, 80-87 months). Patients with no evidence of nodal cancer had significantly improved survival over patients with any pathologically positive nodes (mean survival time, 100 months vs 59 months, respectively; P<.001). Patient age, tumor histopathologic type, facial nerve involvement, extraglandular involvement, tumor grade, and tumor size were significant clinical predictors of nodal disease. Facial nerve involvement, tumor grade, and squamous cell carcinoma subtype exhibited the highest increased odds ratios for nodal metastasis.

CONCLUSIONS

Nodal disease significantly decreases survival in patients with major salivary gland malignancy. Tumor histopathologic type, facial nerve involvement, extraglandular tumor extension, and tumor grade are the most important predictors of nodal disease.

Comparison of endoscopic sinus surgery with and without image guidance

BACKGROUND

Image guidance based on preacquired computed tomography scans of the patient is a technique used to assist the physician during endoscopic sinus surgery (ESS). This study seeks to compare ESS with and without image guidance, analyzing a number of parameters that can impact on efficacy.

METHODS

Retrospective chart review took place at a tertiary care referral center. The study group consisted of 97 consecutive patients confirmed to have undergone ESS using an electromagnetic intraoperative image guidance system (IGS). The control group consisted of 61 consecutive patients who underwent ESS, before the IGS was available at the study hospital. The main outcomes measured were analysis of patient profile, including coexisting conditions such as asthma and polyposis, assessment of which specific sinuses underwent surgical treatment; major and minor complications; estimated blood loss (EBL); operative time; and the need for repeat surgery.

RESULTS

The IGS group had 74% of patients with polyposis; more sinuses, on average, which underwent surgical revision; one major and three minor complications; an average EBL of 134 cc, an average procedure time of 154 minutes; and one patient who needed repeat surgery in a 3-month follow-up period. The non-IGS group had 40% of patients with polyposis; seven major complications and one minor complication; an average EBL of 94 cc; and three patients who needed repeat surgery within 3 months.

CONCLUSIONS

The use of an IGS for endoscopic sinus surgery may reduce the complications associated with the procedure and allow for a more thorough operation. However, operative time and EBL may be increased.