Multi-color reflectance imaging of middle ear pathology in vivo

Label-Free Optical Technologies for Middle-Ear Diseases

Medical applications of optical technology have increased tremendously in recent decades. Label-free techniques have the unique advantage of investigating biological samples in vivo without introducing exogenous agents. This is especially beneficial for a rapid clinical translation as it reduces the need for toxicity studies and regulatory approval for exogenous labels. Emerging applications have utilized label-free optical technology for screening, diagnosis, and surgical guidance. Advancements in detection technology and rapid improvements in artificial intelligence have expedited the clinical implementation of some optical technologies. Among numerous biomedical application areas, middle-ear disease is a unique space where label-free technology has great potential. The middle ear has a unique anatomical location that can be accessed through a dark channel, the external auditory canal; it can be sampled through a tympanic membrane of approximately 100 microns in thickness. The tympanic membrane is the only membrane in the body that is surrounded by air on both sides, under normal conditions. Despite these favorable characteristics, current examination modalities for middle-ear space utilize century-old technology such as white-light otoscopy. This paper reviews existing label-free imaging technologies and their current progress in visualizing middle-ear diseases. We discuss potential opportunities, barriers, and practical considerations when transitioning label-free technology to clinical applications.

Color Dependence Analysis in a CNN-Based Computer-Aided Diagnosis System for Middle and External Ear Diseases

Artificial intelligence-assisted otologic diagnosis has been of growing interest in the scientific community, where middle and external ear disorders are the most frequent diseases in daily ENT practice. There are some efforts focused on reducing medical errors and enhancing physician capabilities using conventional artificial vision systems. However, approaches with multispectral analysis have not yet been addressed. Tissues of the tympanic membrane possess optical properties that define their characteristics in specific light spectra. This work explores color wavelengths dependence in a model that classifies four middle and external ear conditions: normal, chronic otitis media, otitis media with effusion, and earwax plug. The model is constructed under a computer-aided diagnosis system that uses a convolutional neural network architecture. We trained several models using different single-channel images by taking each color wavelength separately. The results showed that a single green channel model achieves the best overall performance in terms of accuracy (92%), sensitivity (85%), specificity (95%), precision (86%), and F1-score (85%). Our findings can be a suitable alternative for artificial intelligence diagnosis systems compared to the 50% of overall misdiagnosis of a non-specialist physician.

Identifying epithelial borders in cholesteatoma surgery using narrow band imaging

Purpose

To quantify changes in the perceived epithelial border with narrow band imaging (NBI) and white light imaging (WLI) during cholesteatoma surgery and to objectify possible benefits of NBI in otology.

Methods

Perioperative digital endoscopic images were captured during combined approach tympanoplasty at our tertiary referral center using WLI and NBI (415 nm and 540 nm wavelengths). Sixteen otologic surgeon defined the epithelial borders within 16 identical WLI and NBI photos. Pixels of these selections were calculated to analyze the quantitative difference between WLI and NBI. A questionnaire also analyzed the qualitative differences.

Results

Sixteen otologic surgeons participated in the study. Stratified per photo, only two photos yielded a significant difference: less pixels were selected with NBI than WLI. A Bland–Altman plot showed no systemic error. Stratified per otologist, four participants selected significantly more pixels with WLI than with NBI. Overall, no significant difference between selected pixels was found. Sub-analyses of surgeons with more than 5 years of experience yielded no additional findings. Despite these results, 60% believed NBI could be advantageous in defining epithelial borders, of which 83% believed NBI could reduce the risk of residual disease.

Conclusion

There was no objective difference in the identification of epithelial borders with NBI compared to WLI in cholesteatoma surgery. Therefore, we do not expect the use of NBI to evidently decrease the risk of residual cholesteatoma. However, subjective assessment does suggest a possible benefit of lighting techniques in otology.

Level of evidence

3.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00405-021-07045-4.

Shortwave infrared otoscopy for diagnosis of middle ear effusions: a machine-learning-based approach

Otitis media, a common disease marked by the presence of fluid within the middle ear space, imparts a significant global health and economic burden. Identifying an effusion through the tympanic membrane is critical to diagnostic success but remains challenging due to the inherent limitations of visible light otoscopy and user interpretation. Here we describe a powerful diagnostic approach to otitis media utilizing advancements in otoscopy and machine learning. We developed an otoscope that visualizes middle ear structures and fluid in the shortwave infrared region, holding several advantages over traditional approaches. Images were captured in vivo and then processed by a novel machine learning based algorithm. The model predicts the presence of effusions with greater accuracy than current techniques, offering specificity and sensitivity over 90%. This platform has the potential to reduce costs and resources associated with otitis media, especially as improvements are made in shortwave imaging and machine learning.

Application of artificial intelligence using a convolutional neural network for detecting cholesteatoma in endoscopic enhanced images

OBJECTIVE

We examined whether artificial intelligence (AI) used with the novel digital image enhancement system modalities (CLARA+CHROMA, SPECTRA A, and SPECTRA B) could distinguish the cholesteatoma matrix, cholesteatoma debris, and normal middle ear mucosa, and observe the middle ear cavity during middle ear cholesteatoma surgery.

METHODS

A convolutional neural network (CNN) was trained with a set of images chosen by an otologist. To evaluate the diagnostic accuracy of the constructed CNN, an independent test data set of middle ear images was collected from 14 consecutive patients with 26 cholesteatoma matrix lesions, who underwent transcanal endoscopic ear surgery at a single hospital from August 2018 to September 2019. The final test data set included 58 total images, with 1‒5 images from each modality for each case.

RESULTS

The CNN required only 10 s to analyze more than 58 test images. Using SPECTRA A and SPECTRA B, the CNN correctly diagnosed 15 and 15 of 26 cholesteatoma matrix lesions, with a sensitivity of 34.6% and 42.3%, and with a specificity of 81.3% and 87.5%, respectively.

CONCLUSION

Our preliminary study revealed that AI and novel imaging modalities are potentially useful tools for identifying and visualizing the cholesteatoma matrix during endoscopic ear surgery. The diagnostic ability of the CNN is not yet appropriate for implementation in daily clinical practice, based on our study findings. However, in the future, these techniques and AI tools could help to reduce the burden on surgeons and will facilitate telemedicine in remote and rural areas, as well as in developing countries where the number of surgeons is limited.

Short-Wave Infrared Fluorescence Chemical Sensor for Detection of Otitis Media

Otitis media (OM) or middle ear infection is one of the most common diseases in young children around the world. The diagnosis of OM is currently performed using an otoscope to detect middle ear fluid and inflammatory changes manifested in the tympanic membrane. However, conventional otoscopy cannot visualize across the tympanic membrane or sample middle ear fluid. This can lead to low diagnostic certainty and overdiagnoses of OM. To improve the diagnosis of OM, we have developed a short-wave infrared (SWIR) otoscope in combination with a protease-cleavable biosensor, 6QC-ICG, which can facilitate the detection of inflammatory proteases in the middle ear with an increase in contrast. 6QC-ICG is a fluorescently quenched probe, which is activated in the presence of cysteine cathepsin proteases that are up-regulated in inflammatory immune cells. Using a preclinical model and custom-built SWIR otomicroscope in this proof-of-concept study, we successfully demonstrated the feasibility of robustly distinguishing inflamed ears from controls (p = 0.0006). The inflamed ears showed an overall signal-to-background ratio of 2.0 with a mean fluorescence of 81 ± 17 AU, while the control ear exhibited a mean fluorescence of 41 ± 11 AU. We envision that these fluorescently quenched probes in conjunction with SWIR imaging tools have the potential to be used as an alternate/adjunct tool for objective diagnosis of OM.

Application of Multispectral Imaging in the Human Tympanic Membrane

Multispectral imaging has recently shown good performance in determining information about physiology, morphology, and composition of tissue. In the endoscopy field, many researches have shown the ability to apply multispectral or narrow-band images in surveying vascular structure based on the interaction of light wavelength with tissue composition. However, there has been no mention to assess the contrast between other components in the middle ear such as the tympanic membrane, malleus, and the surrounding area. Using CT, OCT, or ODT can clearly describe the tympanic membrane structure; nevertheless, these approaches are expensive, more complex, and time-consuming and are not suitable for most common middle ear diagnoses. Here, we show the potential of using the multispectral imaging technique to enhance the contrast of the tympanic membrane compared to the surrounding tissue. The optical absorption and scattering of biological tissues constituents are not the same at different wavelengths. In this pilot study, multiwavelength images of the tympanic membrane were captured by using the otoscope with LED light source at three distinct spectral regions: 450 nm, 530 nm, and 630 nm. Subsequently, analyses of the intensity images as well as the histogram of these images point out that the 630 nm illumination image features an evident contrast in the intensity of the tympanic membrane and malleus compared to the surrounding area. Analysis of such images could facilitate the boundary determination and segmentation of the tympanic membrane (TM) with high precision.

Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks

Convolutional neural networks (CNNs), a popular type of deep neural network, have been actively applied to image recognition, object detection, object localization, semantic segmentation, and object instance segmentation. Accordingly, the applicability of deep learning to the analysis of medical images has increased. This paper presents a novel application of state-of-the-art CNN models, such as DenseNet, to the automatic detection of the tympanic membrane (TM) and middle ear (ME) infection. We collected 2,484 oto-endoscopic images (OEIs) and classified them into one of three categories: normal, chronic otitis media (COM) with TM perforation, and otitis media with effusion (OME). Our results indicate that CNN models have significant potential for the automatic recognition of TM and ME infections, demonstrating a competitive accuracy of 95% in classifying TM and middle ear effusion (MEE) from OEIs. In addition to accuracy measurement, our approach achieves nearly perfect measures of 0.99 in terms of the average area under the receiver operating characteristics curve (AUROC). All these results indicate robust performance when recognizing TM and ME effusions in OEIs. Visualization through a class activation mapping (CAM) heatmap demonstrates that our proposed model performs prediction based on the correct region of OEIs. All these outcomes ensure the reliability of our method; hence, the study can aid otolaryngologists and primary care physicians in real-world scenarios.

Panel 1: Biotechnology, biomedical engineering and new models of otitis media

OBJECTIVE

To summarize recently published key articles on the topics of biomedical engineering, biotechnology and new models in relation to otitis media (OM).

DATA SOURCES

Electronic databases: PubMed, Ovid Medline, Cochrane Library and Clinical Evidence (BMJ Publishing).

REVIEW METHODS

Articles on biomedical engineering, biotechnology, material science, mechanical and animal models in OM published between May 2015 and May 2019 were identified and subjected to review. A total of 132 articles were ultimately included.

RESULTS

New imaging technologies for the tympanic membrane (TM) and the middle ear cavity are being developed to assess TM thickness, identify biofilms and differentiate types of middle ear effusions. Artificial intelligence (AI) has been applied to train software programs to diagnose OM with a high degree of certainty. Genetically modified mice models for OM have further investigated what predisposes some individuals to OM and consequent hearing loss. New vaccine candidates protecting against major otopathogens are being explored and developed, especially combined vaccines, targeting more than one pathogen. Transcutaneous vaccination against non-typeable Haemophilus influenzae has been successfully tried in a chinchilla model. In terms of treatment, novel technologies for trans-tympanic drug delivery are entering the clinical domain. Various growth factors and grafting materials aimed at improving healing of TM perforations show promising results in animal models.

CONCLUSION

New technologies and AI applications to improve the diagnosis of OM have shown promise in pre-clinical models and are gradually entering the clinical domain. So are novel vaccines and drug delivery approaches that may allow local treatment of OM.

IMPLICATIONS FOR PRACTICE

New diagnostic methods, potential vaccine candidates and the novel trans-tympanic drug delivery show promising results, but are not yet adapted to clinical use.

Identification of early inflammatory changes in the tympanic membrane with Raman spectroscopy

The tympanic membrane (TM) is a dynamic structure that separates the middle ear from the external auditory canal. It is also integral for the transmission of sound waves. In this study, we demonstrate the feasibility of using Raman spectroscopy to identify early chemical changes resulting from inflammation in the TM that can serve as an indicator of acute otitis media. Bacterial lipopolysaccharide (LPS) was injected trans-tympanicaly in a murine model. Presence of inflammatory response was assessed with binocular microscopy, confirmed with histopathology and immunofluorescence staining. Successful discrimination suggesting spectral differences among the control and LPS treated groups was achieved using principal component analysis. Raman imaging revealed major differences in collagen distribution and nucleic acid content. Image segmentation analysis on the trichrome stained tissue sections was performed to corroborate the Raman spectra. The spectral co-localization study suggests changes in the expression of collagen IV specific signals in LPS treated samples. The overall findings of the study support prospective application of RS in the diagnosis and therapeutic monitoring of otitis media.

Emerging Technologies for the Diagnosis of Otitis Media

OBJECTIVE

To review new experimental techniques for the diagnosis of otitis media (OM).

DATA SOURCES

Literature search in English in the following databases: MEDLINE (via PubMed), Ovid Medline, Google Scholar, and Clinical Evidence (BMJ Publishing) between January 1, 2005, and April 30, 2018. Subsequently, articles were reviewed and included only if relevant.

REVIEW METHODS

MeSH terms: ["diagnosis"] AND [all forms of OM] AND ["human"] AND ["ear"] and ["tympanic membrane"]. The retrieved innovative diagnostic techniques rely on and take advantage of the physical properties of the tympanomastoid cavity components: tympanic membrane (TM) thickness, its translucency and compliance; middle ear fluid characteristics; biofilm presence; increased tissue metabolic activity in OM states; and fluid presence in the mastoid cavity. These parameters are taken into account to establish OM diagnosis objectively. We review spectral gradient acoustic reflectometry, digital otoscopy, TM image analysis, multicolor reflectance imaging, anticonfocal middle ear assessment, optical coherence tomography, quantitative pneumatic otoscopy, transmastoid ultrasound, wideband measurements, TM thickness mapping, shortwave infrared imaging, and wideband acoustic transfer functions.

CONCLUSIONS

New experimental techniques are gradually introduced to overcome the limitations of standard otoscopy. The aforementioned techniques are still under investigation and are pending widespread clinical use. The implementation of these techniques in the market is dependent on their success in clinical trials, as well as on their future cost.

IMPLICATION FOR PRACTICE

New techniques for the diagnosis of OM can objectively evaluate the morphology of the TM, determine the presence of middle ear fluid and evaluate its content, and thus potentially replace standard otoscopy.

Differential diagnosis of otitis media with effusion using label-free Raman spectroscopy: A pilot study

Otitis media with effusion (OME) is an important and common condition affecting hearing in pediatric patients characterized by the presence of fluid in the middle ear space. The fluid is normally described as serous or mucoid based on differences in the fluid viscosity. The differential diagnosis of two OMEs, namely serous and mucoid is of significant clinical value because while the former is self-limiting, surgical procedure is commonly required for the latter. However, accurate identification of fluid types remains a challenging target unattainable with current clinical modalities due to unavailability of nonperturbative molecular tools. Here, we report an emerging spectroscopy approach featuring Raman scattering and multivariate analysis of spectral patterns to discern serous and mucoid fluids, obtained from pediatric patients undergoing myringotomy and tube placement, by providing information of differentially expressed molecules with high specificity. We demonstrate the feasibility of Raman spectroscopy-based approach to categorize middle ear effusion based on the characteristic spectral markers, notably of mucin, with classification accuracy of 91% and 93% for serous and mucoid, respectively. Our findings pave the way for further development of such a tool for fully noninvasive application that will lead to objective and accurate diagnosis thereby reducing unnecessary visits and surgical procedures.

Photonic monitoring of treatment during infection and sepsis: development of new detection strategies and potential clinical applications

Despite the strong decline in the infection-associated mortality since the development of the first antibiotics, infectious diseases are still a major cause of death in the world. With the rising number of antibiotic-resistant pathogens, the incidence of deaths caused by infections may increase strongly in the future. Survival rates in sepsis, which occurs when body response to infections becomes uncontrolled, are still very poor if an adequate therapy is not initiated immediately. Therefore, approaches to monitor the treatment efficacy are crucially needed to adapt therapeutic strategies according to the patient's response. An increasing number of photonic technologies are being considered for diagnostic purpose and monitoring of therapeutic response; however many of these strategies have not been introduced into clinical routine, yet. Here, we review photonic strategies to monitor response to treatment in patients with infectious disease, sepsis, and septic shock. We also include some selected approaches for the development of new drugs in animal models as well as new monitoring strategies which might be applicable to evaluate treatment response in humans in the future. Figure Label-free probing of blood properties using photonics.

Panel 8: Report on Recent Advances in Molecular and Cellular Biochemistry

Objectives To update the medical literature on recent cellular and molecular advances in otitis media disease models with a principal focus on developments in the past 5 years. We also aim to explain recent translational advances in cellular and molecular biology that have influenced our understanding and management of otitis media. Data Sources PubMed-indexed peer-reviewed articles. Review Methods A comprehensive review of the literature was conducted with the term otitis media and the following search terms: molecular biology, cell biology, innate immunity, oxidative stress, mucins, molecular diagnostics. Included articles were published in the English language from January 1, 2010, to July 31, 2015. Implications for Practice The molecular understanding of otitis media disease progression has rapidly advanced over the last 5 years. The roles of inflammation, mucins, and cell signaling mechanisms have been elucidated and defined. Advances in the field provide a plethora of opportunities for innovative molecular targeting in the development of novel therapeutic strategies for otitis media.

Using the shortwave infrared to image middle ear pathologies

Visualizing structures deep inside opaque biological tissues is one of the central challenges in biomedical imaging. Optical imaging with visible light provides high resolution and sensitivity; however, scattering and absorption of light by tissue limits the imaging depth to superficial features. Imaging with shortwave infrared light (SWIR, 1-2 μm) shares many advantages of visible imaging, but light scattering in tissue is reduced, providing sufficient optical penetration depth to noninvasively interrogate subsurface tissue features. However, the clinical potential of this approach has been largely unexplored because suitable detectors, until recently, have been either unavailable or cost prohibitive. Here, taking advantage of newly available detector technology, we demonstrate the potential of SWIR light to improve diagnostics through the development of a medical otoscope for determining middle ear pathologies. We show that SWIR otoscopy has the potential to provide valuable diagnostic information complementary to that provided by visible pneumotoscopy. We show that in healthy adult human ears, deeper tissue penetration of SWIR light allows better visualization of middle ear structures through the tympanic membrane, including the ossicular chain, promontory, round window niche, and chorda tympani. In addition, we investigate the potential for detection of middle ear fluid, which has significant implications for diagnosing otitis media, the overdiagnosis of which is a primary factor in increased antibiotic resistance. Middle ear fluid shows strong light absorption between 1,400 and 1,550 nm, enabling straightforward fluid detection in a model using the SWIR otoscope. Moreover, our device is easily translatable to the clinic, as the ergonomics, visual output, and operation are similar to a conventional otoscope.

Discerning the differential molecular pathology of proliferative middle ear lesions using Raman spectroscopy

Despite its widespread prevalence, middle ear pathology, especially the development of proliferative lesions, remains largely unexplored and poorly understood. Diagnostic evaluation is still predicated upon a high index of clinical suspicion on otoscopic examination of gross morphologic features. We report the first technique that has the potential to non-invasively identify two key lesions, namely cholesteatoma and myringosclerosis, by providing real-time information of differentially expressed molecules. In addition to revealing signatures consistent with the known pathobiology of these lesions, our observations provide the first evidence of the presence of carbonate- and silicate-substitutions in the calcium phosphate plaques found in myringosclerosis. Collectively, these results demonstrate the potential of Raman spectroscopy to not only provide new understanding of the etiology of these conditions by defining objective molecular markers but also aid in margin assessment to improve surgical outcome.