1
|
Baffa MDFO, Zezell DM, Bachmann L, Pereira TM, Deserno TM, Felipe JC. Deep neural networks can differentiate thyroid pathologies on infrared hyperspectral images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 247:108100. [PMID: 38442622 DOI: 10.1016/j.cmpb.2024.108100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 02/12/2024] [Accepted: 02/23/2024] [Indexed: 03/07/2024]
Abstract
BACKGROUND AND OBJECTIVE The thyroid is a gland responsible for producing important body hormones. Several pathologies can affect this gland, such as thyroiditis, hypothyroidism, and thyroid cancer. The visual histological analysis of thyroid specimens is a valuable process that enables pathologists to detect diseases with high efficiency, providing the patient with a better prognosis. Existing computer vision systems developed to aid in the analysis of histological samples have limitations in distinguishing pathologies with similar characteristics or samples containing multiple diseases. To overcome this challenge, hyperspectral images are being studied to represent biological samples based on their molecular interaction with light. METHODS In this study, we address the acquisition of infrared absorbance spectra from each voxel of histological specimens. This data is then used for the development of a multiclass fully-connected neural network model that discriminates spectral patterns, enabling the classification of voxels as healthy, cancerous, or goiter. RESULTS Through experiments using the k-fold cross-validation protocol, we obtained an average accuracy of 93.66 %, a sensitivity of 93.47 %, and a specificity of 96.93 %. Our results demonstrate the feasibility of using infrared hyperspectral imaging to characterize healthy tissue and thyroid pathologies using absorbance measurements. The proposed deep learning model has the potential to improve diagnostic efficiency and enhance patient outcomes.
Collapse
Affiliation(s)
| | | | - Luciano Bachmann
- Department of Physics, University of São Paulo, Ribeirão Preto, SP, Brazil
| | - Thiago Martini Pereira
- Department of Science and Technology, Federal University of São Paulo, São José dos Campos, SP, Brazil
| | - Thomas Martin Deserno
- Peter L. Reichertz Institute for Medical Informatics, Technische Universität Braunschweig, Braunschweig, Germany
| | - Joaquim Cezar Felipe
- Department of Computing and Mathematics, University of São Paulo, Ribeirão Preto, SP, Brazil
| |
Collapse
|
2
|
Schmidt VM, Zelger P, Wöss C, Fodor M, Hautz T, Schneeberger S, Huck CW, Arora R, Brunner A, Zelger B, Schirmer M, Pallua JD. Handheld hyperspectral imaging as a tool for the post-mortem interval estimation of human skeletal remains. Heliyon 2024; 10:e25844. [PMID: 38375262 PMCID: PMC10875450 DOI: 10.1016/j.heliyon.2024.e25844] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 01/30/2024] [Accepted: 02/02/2024] [Indexed: 02/21/2024] Open
Abstract
In forensic medicine, estimating human skeletal remains' post-mortem interval (PMI) can be challenging. Following death, bones undergo a series of chemical and physical transformations due to their interactions with the surrounding environment. Post-mortem changes have been assessed using various methods, but estimating the PMI of skeletal remains could still be improved. We propose a new methodology with handheld hyperspectral imaging (HSI) system based on the first results from 104 human skeletal remains with PMIs ranging between 1 day and 2000 years. To differentiate between forensic and archaeological bone material, the Convolutional Neural Network analyzed 65.000 distinct diagnostic spectra: the classification accuracy was 0.58, 0.62, 0.73, 0.81, and 0.98 for PMIs of 0 week-2 weeks, 2 weeks-6 months, 6 months-1 year, 1 year-10 years, and >100 years, respectively. In conclusion, HSI can be used in forensic medicine to distinguish bone materials >100 years old from those <10 years old with an accuracy of 98%. The model has adequate predictive performance, and handheld HSI could serve as a novel approach to objectively and accurately determine the PMI of human skeletal remains.
Collapse
Affiliation(s)
- Verena-Maria Schmidt
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - Philipp Zelger
- University Clinic for Hearing, Voice and Speech Disorders, Medical University of Innsbruck, Anichstrasse 35, 6020 Innsbruck, Austria
| | - Claudia Wöss
- Institute of Forensic Medicine, Medical University of Innsbruck, Muellerstraße 44, 6020 Innsbruck, Austria
| | - Margot Fodor
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Theresa Hautz
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Stefan Schneeberger
- OrganLifeTM, Department of Visceral, Transplant and Thoracic Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Christian Wolfgang Huck
- Institute of Analytical Chemistry and Radiochemistry, University of Innsbruck, 6020 Innsbruck, Austria
| | - Rohit Arora
- Department of Orthopaedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| | - Andrea Brunner
- Institute of Pathology, Neuropathology, and Molecular Pathology, Medical University of Innsbruck, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - Bettina Zelger
- Institute of Pathology, Neuropathology, and Molecular Pathology, Medical University of Innsbruck, Muellerstrasse 44, 6020 Innsbruck, Austria
| | - Michael Schirmer
- Department of Internal Medicine, Clinic II, Medical University of Innsbruck, Anichstrasse 35, 6020 Innsbruck, Austria
| | - Johannes Dominikus Pallua
- Department of Orthopaedics and Traumatology, Medical University of Innsbruck, Anichstraße 35, 6020 Innsbruck, Austria
| |
Collapse
|
3
|
Zhou X, Ma L, Mubarak HK, Palsgrove D, Sumer BD, Chen AY, Fei B. Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:016005. [PMID: 38239390 PMCID: PMC10795499 DOI: 10.1117/1.jbo.29.1.016005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 12/04/2023] [Accepted: 12/20/2023] [Indexed: 01/22/2024]
Abstract
Significance Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications. Aim The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides. Approach We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: S 0 , S 1 , S 2 , and S 3 . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors. Results By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides. Conclusions With the spatial and spectral information from the Stokes vector data cubes (S 0 , S 1 , S 2 , and S 3 ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Hasan K. Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
| | - Doreen Palsgrove
- The University of Texas Southwestern Medical Center, Department of Pathology, Dallas, Texas, United States
| | - Baran D. Sumer
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
4
|
Loperfido A, Celebrini A, Marzetti A, Bellocchi G. Current role of artificial intelligence in head and neck cancer surgery: a systematic review of literature. EXPLORATION OF TARGETED ANTI-TUMOR THERAPY 2023; 4:933-940. [PMID: 37970203 PMCID: PMC10645467 DOI: 10.37349/etat.2023.00174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 07/19/2023] [Indexed: 11/17/2023] Open
Abstract
Aim Artificial intelligence (AI) is a new field of science in which computers will provide decisions-supporting tools to help doctors make difficult clinical choices. Recent AI applications in otolaryngology include head and neck oncology, rhinology, neurotology, and laryngology. The aim of this systematic review is to describe the potential uses of AI in head and neck oncology with a special focus on the surgical field. Methods The authors performed a systematic review, in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, in the main medical databases, including PubMed, Scopus, and Cochrane Library, considering all original studies published until February 2023 about the role of AI in head and neck cancer surgery. The search strategy included a combination of the following terms: "artificial intelligence" or "machine learning" and "head and neck cancer". Results Overall, 303 papers were identified and after duplicate removal (12 papers) and excluding papers not written in English (1 paper) and off-topic (4 papers), papers were assessed for eligibility; finally, only 12 papers were included. Three main fields of clinical interest were identified: the most widely investigated included the role of AI in surgical margins assessment (7 papers); the second most frequently evaluated topic was complications assessment (4 papers); finally, only one paper dealt with the indication of salvage laryngectomy after primary radiotherapy. Conclusions The authors report the first systematic review in the literature concerning the role of AI in head and neck cancer surgery. An increasing influx of AI applications to clinical problems in otolaryngology is expected, so specialists should be increasingly prepared to manage the constant changes. It will always remain critical for clinicians to use their skills and knowledge to critically evaluate the additional information provided by AI and make the final decisions on each patient.
Collapse
Affiliation(s)
| | | | - Andrea Marzetti
- Department of Otolaryngology Head and Neck Surgery, Fabrizio Spaziani Hospital, 03100 Frosinone, Italy
| | | |
Collapse
|
5
|
Campbell JM, Habibalahi A, Handley S, Agha A, Mahbub SB, Anwer AG, Goldys EM. Emerging clinical applications in oncology for non-invasive multi- and hyperspectral imaging of cell and tissue autofluorescence. JOURNAL OF BIOPHOTONICS 2023; 16:e202300105. [PMID: 37272291 DOI: 10.1002/jbio.202300105] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 05/02/2023] [Accepted: 05/16/2023] [Indexed: 06/06/2023]
Abstract
Hyperspectral and multispectral imaging of cell and tissue autofluorescence is an emerging technology in which fluorescence imaging is applied to biological materials across multiple spectral channels. This produces a stack of images where each matched pixel contains information about the sample's spectral properties at that location. This allows precise collection of molecularly specific data from a broad range of native fluorophores. Importantly, complex information, directly reflective of biological status, is collected without staining and tissues can be characterised in situ, without biopsy. For oncology, this can spare the collection of biopsies from sensitive regions and enable accurate tumour mapping. For in vivo tumour analysis, the greatest focus has been on oral cancer, whereas for ex vivo assessment head-and-neck cancers along with colon cancer have been the most studied, followed by oral and eye cancer. This review details the scope and progress of research undertaken towards clinical translation in oncology.
Collapse
Affiliation(s)
- Jared M Campbell
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Abbas Habibalahi
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Shannon Handley
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Adnan Agha
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Saabah B Mahbub
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Ayad G Anwer
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| | - Ewa M Goldys
- Graduate School of Biomedical Engineering, University of New South Wales, Sydney, New South Wales, Australia
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, South Australia, Australia
| |
Collapse
|
6
|
Liao WC, Mukundan A, Sadiaza C, Tsao YM, Huang CW, Wang HC. Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging. BIOMEDICAL OPTICS EXPRESS 2023; 14:4383-4405. [PMID: 37799695 PMCID: PMC10549751 DOI: 10.1364/boe.492635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 10/07/2023]
Abstract
One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.
Collapse
Affiliation(s)
- Wei-Chih Liao
- Department of Internal Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan
- Graduate Institute of Epidemiology and Preventive Medicine, National Taiwan University, Taipei, Taiwan
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Cleorita Sadiaza
- Department of Mechanical Engineering, Far Eastern University, P. Paredes St., Sampaloc, Manila, 1015, Philippines
| | - Yu-Ming Tsao
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
| | - Chien-Wei Huang
- Department of Gastroenterology, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st.Rd., Lingya District, Kaohsiung City 80284, Taiwan
- Department of Nursing, Tajen University, 20, Weixin Rd., Yanpu Township, Pingtung County 90741, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi 62102, Taiwan
- Department of Medical Research, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 2, Minsheng Road, Dalin, Chiayi, 62247, Taiwan
- Director of Technology Development, Hitspectra Intelligent Technology Co., Ltd., 4F., No. 2, Fuxing 4th Rd., Qianzhen Dist., Kaohsiung City 80661, Taiwan
| |
Collapse
|
7
|
Chang M, Lee W, Jeong KY, Kim JW. Optimal Hyperspectral Band Selection for Tissue Oxygenation Mapping with Generative Adversarial Network. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38082981 DOI: 10.1109/embc40787.2023.10340032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Tissue oxygenation assessment using hyperspectral imaging is an emerging technique for the diagnosis and pre- and post-treatment monitoring of ischemic patients. However, the high spectral resolution of hyperspectral imaging leads to large data sizes and a long imaging time. In this study, we propose a method that utilizes multi-objective evolutionary algorithms to determine the optimal hyperspectral band combination when developing a deep learning model for predicting tissue oxygenation from hyperspectral images. Our results confirm that the deep learning model effectively predicts tissue oxygenation images for various oxygenation states. Moreover, we demonstrate that a high-performance prediction model can be developed using only a small number of spectral bands, indicating the potential for more efficient non-contact tissue oxygenation mapping with the proposed method.Clinical Relevance- The proposed method allows for the non-contact and efficient acquisition of two-dimensional tissue oxygenation information in various oxygenation states.
Collapse
|
8
|
Yerukala Sathipati S, Ho SY. Survival associated miRNA signature in patients with head and neck carcinomas. Heliyon 2023; 9:e17218. [PMID: 37360084 PMCID: PMC10285236 DOI: 10.1016/j.heliyon.2023.e17218] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Revised: 06/07/2023] [Accepted: 06/10/2023] [Indexed: 06/28/2023] Open
Abstract
Head and neck carcinoma (HNSC) is often diagnosed at advanced stage, incurring poor patient outcome. Despite of advances in chemoradiation and surgery approaches, limited improvements in survival rates of HNSC have been observed over the last decade. Accumulating evidences have demonstrated the importance of microRNAs (miRNAs) in carcinogenesis. In this context, we sought to identify a miRNA signature associated with the survival time in patients with HNSC. This study proposed a survival estimation method called HNSC-Sig that identified a miRNA signature consists of 25 miRNAs associated with the survival in 133 patients with HNSC. HNSC-Sig achieved 10-fold cross validation a mean correlation coefficient and a mean absolute error of 0.85 ± 0.01 and 0.46 ± 0.02 years, respectively, between actual and estimated survival times. The survival analysis revealed that five miRNAs, hsa-miR-3605-3p, hsa-miR-629-3p, hsa-miR-3127-5p, hsa-miR-497-5p, and hsa-miR-374a-5p, were significantly associated with prognosis in patients with HNSC. Comparing the relative expression difference of top 10 prioritized miRNAs, eight miRNAs, hsa-miR-629-3p, hsa-miR-3127-5p, hsa-miR-221-3p, hsa-miR-501-5p, hsa-miR-491-5p, hsa-miR-149-3p, hsa-miR-3934-5p, and hsa-miR-3170, were significantly expressed between cancer and normal groups. In addition, biological relevance, disease association, and target interactions of the miRNA signature were discussed. Our results suggest that identified miRNA signature have potential to serve as biomarker for diagnosis and clinical practice in HNSC.
Collapse
Affiliation(s)
| | - Shinn-Ying Ho
- Institute of Bioinformatics and Systems Biology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
- Department of Biological Science and Technology, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
- College of Health Sciences, Kaohsiung Medical University, Kaohsiung, Taiwan
- Center for Intelligent Drug Systems and Smart Bio-devices (IDSB), National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| |
Collapse
|
9
|
Jong LJS, Post AL, Veluponnar D, Geldof F, Sterenborg HJCM, Ruers TJM, Dashtbozorg B. Tissue Classification of Breast Cancer by Hyperspectral Unmixing. Cancers (Basel) 2023; 15:2679. [PMID: 37345015 DOI: 10.3390/cancers15102679] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 04/26/2023] [Accepted: 05/04/2023] [Indexed: 06/23/2023] Open
Abstract
(1) Background: Assessing the resection margins during breast-conserving surgery is an important clinical need to minimize the risk of recurrent breast cancer. However, currently there is no technique that can provide real-time feedback to aid surgeons in the margin assessment. Hyperspectral imaging has the potential to overcome this problem. To classify resection margins with this technique, a tissue discrimination model should be developed, which requires a dataset with accurate ground-truth labels. However, establishing such a dataset for resection specimens is difficult. (2) Methods: In this study, we therefore propose a novel approach based on hyperspectral unmixing to determine which pixels within hyperspectral images should be assigned to the ground-truth labels from histopathology. Subsequently, we use this hyperspectral-unmixing-based approach to develop a tissue discrimination model on the presence of tumor tissue within the resection margins of ex vivo breast lumpectomy specimens. (3) Results: In total, 372 measured locations were included on the lumpectomy resection surface of 189 patients. We achieved a sensitivity of 0.94, specificity of 0.85, accuracy of 0.87, Matthew's correlation coefficient of 0.71, and area under the curve of 0.92. (4) Conclusion: Using this hyperspectral-unmixing-based approach, we demonstrated that the measured locations with hyperspectral imaging on the resection surface of lumpectomy specimens could be classified with excellent performance.
Collapse
Affiliation(s)
- Lynn-Jade S Jong
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Anouk L Post
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
| | - Dinusha Veluponnar
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Freija Geldof
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Henricus J C M Sterenborg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
| | - Theo J M Ruers
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
- Department of Nanobiophysics, Faculty of Science and Technology, University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Behdad Dashtbozorg
- Department of Surgery, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, The Netherlands
| |
Collapse
|
10
|
Garcia Peraza Herrera LC, Horgan C, Ourselin S, Ebner M, Vercauteren T. Hyperspectral image segmentation: a preliminary study on the Oral and Dental Spectral Image Database (ODSI-DB). COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2023. [DOI: 10.1080/21681163.2022.2160377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Affiliation(s)
| | - Conor Horgan
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | | | - Michael Ebner
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| | - Tom Vercauteren
- King’s College London, London, UK
- Hypervision Surgical Ltd, London, UK
| |
Collapse
|
11
|
Zhou X, Mubarak HK, Ma L, Palsgrove D, Sumer BD, Fei B. Polarized hyperspectral microscopic imaging for collagen visualization on pathologic slides of head and neck squamous cell carcinoma. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2023; 12382:1238204. [PMID: 38481487 PMCID: PMC10932728 DOI: 10.1117/12.2655831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
We developed a polarized hyperspectral microscope to collect four types of Stokes vector data cubes (S0, S1, S2, and S3) of the pathologic slides with head and neck squamous cell carcinoma (HNSCC). Our system consists of an optical light microscope with a movable stage, two polarizers, two liquid crystal variable retarders (LCVRs), and a SnapScan hyperspectral camera. The polarizers and LCVRs work in tandem with the hyperspectral camera to acquire polarized hyperspectral images. Synthetic pseudo-RGB images are generated from the four Stokes vector data cubes based on a transformation function similar to the spectral response of human eye for the visualization of hyperspectral images. Collagen is the most abundant extracellular matrix (ECM) protein in the human body. A major focus of studying the ECM in tumor microenvironment is the role of collagen in both normal and abnormal function. Collagen tends to accumulate in and around tumors during cancer development and growth. In this study, we acquired images from normal regions containing normal cells and collagen fibers and from tumor regions containing cancerous squamous cells and collagen fibers on HNSCC pathologic slides. The preliminary results demonstrated that our customized polarized hyperspectral microscope is able to improve the visualization of collagen on HNSCC pathologic slides under different situations, including thick fibers of normal stroma, thin fibers of normal stroma, fibers of normal muscle cells, fibers accumulated in tumors, fibers accumulated around tumors. Our preliminary results also demonstrated that the customized polarized hyperspectral microscope is capable of extracting the spectral signatures of collagen based on Stokes vector parameters and can have various applications in pathology and oncology.
Collapse
Affiliation(s)
- Ximing Zhou
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K. Mubarak
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Doreen Palsgrove
- Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baran D. Sumer
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Baowei Fei
- Center for Imaging and Surgical Innovation, The University of Texas at Dallas, Richardson, TX
- University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
12
|
Intraoperative Assessment of Tumor Margins in Tissue Sections with Hyperspectral Imaging and Machine Learning. Cancers (Basel) 2022; 15:cancers15010213. [PMID: 36612208 PMCID: PMC9818424 DOI: 10.3390/cancers15010213] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Revised: 12/16/2022] [Accepted: 12/27/2022] [Indexed: 12/31/2022] Open
Abstract
The intraoperative assessment of tumor margins of head and neck cancer is crucial for complete tumor resection and patient outcome. The current standard is to take tumor biopsies during surgery for frozen section analysis by a pathologist after H&E staining. This evaluation is time-consuming, subjective, methodologically limited and underlies a selection bias. Optical methods such as hyperspectral imaging (HSI) are therefore of high interest to overcome these limitations. We aimed to analyze the feasibility and accuracy of an intraoperative HSI assessment on unstained tissue sections taken from seven patients with oral squamous cell carcinoma. Afterwards, the tissue sections were subjected to standard histopathological processing and evaluation. We trained different machine learning models on the HSI data, including a supervised 3D convolutional neural network to perform tumor detection. The results were congruent with the histopathological annotations. Therefore, this approach enables the delineation of tumor margins with artificial HSI-based histopathological information during surgery with high speed and accuracy on par with traditional intraoperative tumor margin assessment (Accuracy: 0.76, Specificity: 0.89, Sensitivity: 0.48). With this, we introduce HSI in combination with ML hyperspectral imaging as a potential new tool for intraoperative tumor margin assessment.
Collapse
|
13
|
Mat Lazim N, Kandhro AH, Menegaldo A, Spinato G, Verro B, Abdullah B. Autofluorescence Image-Guided Endoscopy in the Management of Upper Aerodigestive Tract Tumors. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 20:159. [PMID: 36612479 PMCID: PMC9819287 DOI: 10.3390/ijerph20010159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 12/16/2022] [Accepted: 12/20/2022] [Indexed: 06/17/2023]
Abstract
At this juncture, autofluorescence and narrow-band imaging have resurfaced in the medicine arena in parallel with current technology advancement. The emergence of newly developed optical instrumentation in addition to the discovery of new fluorescence biomolecules have contributed to a refined management of diseases and tumors, especially in the management of upper aerodigestive tract tumors. The advancement in multispectral imaging and micro-endoscopy has also escalated the trends further in the setting of the management of this tumor, in order to gain not only the best treatment outcomes but also facilitate early tumor diagnosis. This includes the usage of autofluorescence endoscopy for screening, diagnosis and treatment of this tumor. This is crucial, as microtumoral deposit at the periphery of the gross tumor can be only assessed via an enhanced endoscopy and even more precisely with autofluorescence endoscopic techniques. Overall, with this new technique, optimum management can be achieved for these patients. Hence, the treatment outcomes can be improved and patients are able to attain better prognosis and survival.
Collapse
Affiliation(s)
- Norhafiza Mat Lazim
- Department of Otorhinolaryngology-Head and Neck Surgery, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kubang Kerian 16150, Malaysia
| | - Abdul Hafeez Kandhro
- Institute of Medical Technology, Jinnah Sindh Medical University, Karachi 75510, Pakistan
| | - Anna Menegaldo
- Department of Neurosciences, Section of Otolaryngology and Regional Centre for Head and Neck Cancer, University of Padova, 31100 Treviso, Italy
| | - Giacomo Spinato
- Department of Neurosciences, Section of Otolaryngology and Regional Centre for Head and Neck Cancer, University of Padova, 31100 Treviso, Italy
- Department of Surgery, Oncology and Gastroenterology, Section of Oncology and Immunology, University of Padova, 31100 Treviso, Italy
| | - Barbara Verro
- Division of Otorhinolaryngology, Department of Biomedicine, Neuroscience and Advanced Diagnostic, University of Palermo, 90127 Palermo, Italy
| | - Baharudin Abdullah
- Department of Otorhinolaryngology-Head and Neck Surgery, School of Medical Sciences, Universiti Sains Malaysia, Health Campus, Kubang Kerian 16150, Malaysia
| |
Collapse
|
14
|
Cui R, Yu H, Xu T, Xing X, Cao X, Yan K, Chen J. Deep Learning in Medical Hyperspectral Images: A Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22249790. [PMID: 36560157 PMCID: PMC9784550 DOI: 10.3390/s22249790] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 12/11/2022] [Accepted: 12/12/2022] [Indexed: 06/13/2023]
Abstract
With the continuous progress of development, deep learning has made good progress in the analysis and recognition of images, which has also triggered some researchers to explore the area of combining deep learning with hyperspectral medical images and achieve some progress. This paper introduces the principles and techniques of hyperspectral imaging systems, summarizes the common medical hyperspectral imaging systems, and summarizes the progress of some emerging spectral imaging systems through analyzing the literature. In particular, this article introduces the more frequently used medical hyperspectral images and the pre-processing techniques of the spectra, and in other sections, it discusses the main developments of medical hyperspectral combined with deep learning for disease diagnosis. On the basis of the previous review, tne limited factors in the study on the application of deep learning to hyperspectral medical images are outlined, promising research directions are summarized, and the future research prospects are provided for subsequent scholars.
Collapse
Affiliation(s)
- Rong Cui
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - He Yu
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Tingfa Xu
- Image Engineering & Video Technology Lab, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
- Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401120, China
| | - Xiaoxue Xing
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
- Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun 130022, China
| | - Xiaorui Cao
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Kang Yan
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| | - Jiexi Chen
- College of Electronic and Information Engineering, Changchun University, Changchun 130022, China
| |
Collapse
|
15
|
Zhang L, Huang D, Chen X, Zhu L, Chen X, Xie Z, Huang G, Gao J, Shi W, Cui G. Visible near-infrared hyperspectral imaging and supervised classification for the detection of small intestinal necrosis tissue in vivo. BIOMEDICAL OPTICS EXPRESS 2022; 13:6061-6080. [PMID: 36733734 PMCID: PMC9872898 DOI: 10.1364/boe.470202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 09/23/2022] [Accepted: 09/26/2022] [Indexed: 06/18/2023]
Abstract
Complete recognition of necrotic areas during small bowel tissue resection remains challenging due to the lack of optimal intraoperative aid identification techniques. This research utilizes hyperspectral imaging techniques to automatically distinguish normal and necrotic areas of small intestinal tissue. Sample data were obtained from the animal model of small intestinal tissue of eight Japanese large-eared white rabbits developed by experienced physicians. A spectral library of normal and necrotic regions of small intestinal tissue was created and processed using six different supervised classification algorithms. The results show that hyperspectral imaging combined with supervised classification algorithms can be a suitable technique to automatically distinguish between normal and necrotic areas of small intestinal tissue. This new technique could aid physicians in objectively identify normal and necrotic areas of small intestinal tissue.
Collapse
Affiliation(s)
- LeChao Zhang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - DanFei Huang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - XiaoJing Chen
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - LiBin Zhu
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - XiaoQing Chen
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - ZhongHao Xie
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuangZao Huang
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - JunZhao Gao
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - Wen Shi
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuiHua Cui
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| |
Collapse
|
16
|
Alafeef M, Pan D. Diagnostic Approaches For COVID-19: Lessons Learned and the Path Forward. ACS NANO 2022; 16:11545-11576. [PMID: 35921264 PMCID: PMC9364978 DOI: 10.1021/acsnano.2c01697] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Accepted: 07/12/2022] [Indexed: 05/17/2023]
Abstract
Coronavirus disease 2019 (COVID-19) is a transmitted respiratory disease caused by the infection of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Although humankind has experienced several outbreaks of infectious diseases, the COVID-19 pandemic has the highest rate of infection and has had high levels of social and economic repercussions. The current COVID-19 pandemic has highlighted the limitations of existing virological tests, which have failed to be adopted at a rate to properly slow the rapid spread of SARS-CoV-2. Pandemic preparedness has developed as a focus of many governments around the world in the event of a future outbreak. Despite the largely widespread availability of vaccines, the importance of testing has not diminished to monitor the evolution of the virus and the resulting stages of the pandemic. Therefore, developing diagnostic technology that serves as a line of defense has become imperative. In particular, that test should satisfy three criteria to be widely adopted: simplicity, economic feasibility, and accessibility. At the heart of it all, it must enable early diagnosis in the course of infection to reduce spread. However, diagnostic manufacturers need guidance on the optimal characteristics of a virological test to ensure pandemic preparedness and to aid in the effective treatment of viral infections. Nanomaterials are a decisive element in developing COVID-19 diagnostic kits as well as a key contributor to enhance the performance of existing tests. Our objective is to develop a profile of the criteria that should be available in a platform as the target product. In this work, virus detection tests were evaluated from the perspective of the COVID-19 pandemic, and then we generalized the requirements to develop a target product profile for a platform for virus detection.
Collapse
Affiliation(s)
- Maha Alafeef
- Department of Chemical, Biochemical and Environmental
Engineering, University of Maryland Baltimore County, Interdisciplinary
Health Sciences Facility, 1000 Hilltop Circle, Baltimore, Maryland 21250,
United States
- Departments of Diagnostic Radiology and Nuclear
Medicine and Pediatrics, Center for Blood Oxygen Transport and Hemostasis,
University of Maryland Baltimore School of Medicine, Health Sciences
Research Facility III, 670 W Baltimore Street, Baltimore, Maryland 21201,
United States
- Department of Bioengineering, the
University of Illinois at Urbana−Champaign, Urbana, Illinois 61801,
United States
- Biomedical Engineering Department, Jordan
University of Science and Technology, Irbid 22110,
Jordan
| | - Dipanjan Pan
- Department of Chemical, Biochemical and Environmental
Engineering, University of Maryland Baltimore County, Interdisciplinary
Health Sciences Facility, 1000 Hilltop Circle, Baltimore, Maryland 21250,
United States
- Departments of Diagnostic Radiology and Nuclear
Medicine and Pediatrics, Center for Blood Oxygen Transport and Hemostasis,
University of Maryland Baltimore School of Medicine, Health Sciences
Research Facility III, 670 W Baltimore Street, Baltimore, Maryland 21201,
United States
- Department of Bioengineering, the
University of Illinois at Urbana−Champaign, Urbana, Illinois 61801,
United States
| |
Collapse
|
17
|
Hegde S, Ajila V, Zhu W, Zeng C. Review of the Use of Artificial Intelligence in Early Diagnosis and Prevention of Oral Cancer. Asia Pac J Oncol Nurs 2022; 9:100133. [PMID: 36389623 PMCID: PMC9664349 DOI: 10.1016/j.apjon.2022.100133] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 08/12/2022] [Indexed: 11/30/2022] Open
Abstract
The global occurrence of oral cancer (OC) has increased in recent years. OC that is diagnosed in its advanced stages results in morbidity and mortality. The use of technology may be beneficial for early detection and diagnosis and thus help the clinician with better patient management. The advent of artificial intelligence (AI) has the potential to improve OC screening. AI can precisely analyze an enormous dataset from various imaging modalities and provide assistance in the field of oncology. This review focused on the applications of AI in the early diagnosis and prevention of OC. A literature search was conducted in the PubMed and Scopus databases using the search terminology “oral cancer” and “artificial intelligence.” Further information regarding the topic was collected by scrutinizing the reference lists of selected articles. Based on the information obtained, this article reviews and discusses the applications and advantages of AI in OC screening, early diagnosis, disease prediction, treatment planning, and prognosis. Limitations and the future scope of AI in OC research are also highlighted.
Collapse
|
18
|
Construction of a new automatic grading system for jaw bone mineral density level based on deep learning using cone beam computed tomography. Sci Rep 2022; 12:12841. [PMID: 35896558 PMCID: PMC9329319 DOI: 10.1038/s41598-022-16074-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 07/04/2022] [Indexed: 11/17/2022] Open
Abstract
To develop and verify an automatic classification method using artificial intelligence deep learning to determine the bone mineral density level of the implant site in oral implant surgery from radiographic data obtained from cone beam computed tomography (CBCT) images. Seventy patients with mandibular dentition defects were scanned using CBCT. These Digital Imaging and Communications in Medicine data were cut into 605 training sets, and then the data were processed with data standardization, and the Hounsfiled Unit (HU) value level was determined as follows: Type 1, 1000–2000; type 2, 700–1000; type 3, 400–700; type 4, 100–400; and type 5, − 200–100. Four trained dental implant physicians manually identified and classified the area of the jaw bone density level in the image using the software LabelMe. Then, with the assistance of the HU value generated by LabelMe, a physician with 20 years of clinical experience confirmed the labeling level. Finally, the HU mean values of various categories marked by dental implant physicians were compared to the mean values detected by the artificial intelligence model to assess the accuracy of artificial intelligence classification. After the model was trained on 605 training sets, the statistical results of the HU mean values of various categories in the dataset detected by the model were almost the same as the HU grading interval on the data annotation. This new classification provides a more detailed solution to guide surgeons to adjust the drilling rate and tool selection during preoperative decision-making and intraoperative hole preparation for oral implant surgery.
Collapse
|
19
|
Lu J, Wang Q, Wang Z, Liu J, Guo Y, Pan C, Li X, Che J, Shi Z, Zhang S. Log P analyzation-based discovery of GSH activated biotin-tagged fluorescence probe for selective colorectal cancer imaging. Eur J Med Chem 2022; 239:114555. [PMID: 35763866 DOI: 10.1016/j.ejmech.2022.114555] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Revised: 06/16/2022] [Accepted: 06/17/2022] [Indexed: 11/28/2022]
Abstract
Targeted activatable fluorescent probes could provide an effective approach for colorectal cancer imaging. In this study, F1 was found as an effective targeted activatable fluorescent probe based on log P analysis. In vitro experiments demonstrated that the initial fluorescence of the developed probe F1 was initially well quenched, and the fluorescence increased after the probe interacted with glutathione. Cell imaging results showed that the probe had good cell permeability and selectivity. Remarkably, F1 displayed enhanced tumor tissue fluorescence in MC-38 tumor-bearing mice. Notably, it showed selectivity in imaging clinical specimens of human colorectal cancer tissues. Accordingly, this study shows that log P analysis can facilitate the developing efficient of biotin-tagged activatable probes, and the identified F1 has a good potential in clinical colorectal cancer diagnosis.
Collapse
Affiliation(s)
- Jialiang Lu
- The Second Affiliated Hospital of Zhejiang Chinese Medical University (Xinhua Hospital of Zhejiang Province), Hangzhou, 310005, China; Hangzhou Institute of Innovative Medicine, Institute of Drug Discovery and Design, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Qianqian Wang
- The First Clinical Medical College of Zhejiang Chinese Medical University, Hangzhou, 310000, China
| | - Zhaojun Wang
- The First Clinical Medical College of Zhejiang Chinese Medical University, Hangzhou, 310000, China
| | - Jinguo Liu
- The First Clinical Medical College of Zhejiang Chinese Medical University, Hangzhou, 310000, China
| | - Yu Guo
- Hangzhou Institute of Innovative Medicine, Institute of Drug Discovery and Design, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Chenghao Pan
- Hangzhou Institute of Innovative Medicine, Institute of Drug Discovery and Design, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Xin Li
- Hangzhou Institute of Innovative Medicine, Institute of Drug Discovery and Design, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Jinxin Che
- Hangzhou Institute of Innovative Medicine, Institute of Drug Discovery and Design, College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, 310058, China.
| | - Zheng Shi
- The Second Affiliated Hospital of Zhejiang Chinese Medical University (Xinhua Hospital of Zhejiang Province), Hangzhou, 310005, China; The Department of Pharmacy, The First Affiliated Hospital of Zhejiang Chinese Medical University (Zhejiang Provincial Hospital of Traditional Chinese Medicine), Hangzhou, China.
| | - Shuo Zhang
- The Second Affiliated Hospital of Zhejiang Chinese Medical University (Xinhua Hospital of Zhejiang Province), Hangzhou, 310005, China; The First Clinical Medical College of Zhejiang Chinese Medical University, Hangzhou, 310000, China.
| |
Collapse
|
20
|
Voskuil FJ, Vonk J, van der Vegt B, Kruijff S, Ntziachristos V, van der Zaag PJ, Witjes MJH, van Dam GM. Intraoperative imaging in pathology-assisted surgery. Nat Biomed Eng 2022; 6:503-514. [PMID: 34750537 DOI: 10.1038/s41551-021-00808-8] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2019] [Accepted: 06/17/2021] [Indexed: 12/12/2022]
Abstract
The pathological assessment of surgical specimens during surgery can reduce the incidence of positive resection margins, which otherwise can result in additional surgeries or aggressive therapeutic regimens. To improve patient outcomes, intraoperative spectroscopic, fluorescence-based, structural, optoacoustic and radiological imaging techniques are being tested on freshly excised tissue. The specific clinical setting and tumour type largely determine whether endogenous or exogenous contrast is to be detected and whether the tumour specificity of the detected biomarker, image resolution, image-acquisition times or penetration depth are to be prioritized. In this Perspective, we describe current clinical standards for intraoperative tissue analysis and discuss how intraoperative imaging is being implemented. We also discuss potential implementations of intraoperative pathology-assisted surgery for clinical decision-making.
Collapse
Affiliation(s)
- Floris J Voskuil
- Department of Oral and Maxillofacial Surgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands.,Department of Pathology and Medical Biology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Jasper Vonk
- Department of Oral and Maxillofacial Surgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Bert van der Vegt
- Department of Pathology and Medical Biology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Schelto Kruijff
- Department of Surgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands.,Department of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Vasilis Ntziachristos
- Chair for Biological Imaging, Center for Translational Cancer Research, Technical University of Munich, Klinikum rechts der Isar, Munich, Germany.,Institute of Biological and Medical Imaging, Helmholtz Zentrum München, Neuherberg, Germany
| | - Pieter J van der Zaag
- Phillips Research Laboratories, Eindhoven, The Netherlands.,Department of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands.,Molecular Biophysics, Zernike Institute, University of Groningen, Groningen, The Netherlands
| | - Max J H Witjes
- Department of Oral and Maxillofacial Surgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Gooitzen M van Dam
- Department of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands. .,AxelaRx/TRACER BV, Groningen, The Netherlands.
| |
Collapse
|
21
|
Pterygium and Ocular Surface Squamous Neoplasia: Optical Biopsy Using a Novel Autofluorescence Multispectral Imaging Technique. Cancers (Basel) 2022; 14:cancers14061591. [PMID: 35326744 PMCID: PMC8946656 DOI: 10.3390/cancers14061591] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Revised: 03/07/2022] [Accepted: 03/11/2022] [Indexed: 02/04/2023] Open
Abstract
In this study, differentiation of pterygium vs. ocular surface squamous neoplasia based on multispectral autofluorescence imaging technique was investigated. Fifty (N = 50) patients with histopathological diagnosis of pterygium (PTG) and/or ocular surface squamous neoplasia (OSSN) were recruited. Fixed unstained biopsy specimens were imaged by multispectral microscopy. Tissue autofluorescence images were obtained with a custom-built fluorescent microscope with 59 spectral channels, each with specific excitation and emission wavelength ranges, suitable for the most abundant tissue fluorophores such as elastin, flavins, porphyrin, and lipofuscin. Images were analyzed using a new classification framework called fused-classification, designed to minimize interpatient variability, as an established support vector machine learning method. Normal, PTG, and OSSN regions were automatically detected and delineated, with accuracy evaluated against expert assessment by a specialist in OSSN pathology. Signals from spectral channels yielding signals from elastin, flavins, porphyrin, and lipofuscin were significantly different between regions classified as normal, PTG, and OSSN (p < 0.01). Differential diagnosis of PTG/OSSN and normal tissue had accuracy, sensitivity, and specificity of 88 ± 6%, 84 ± 10% and 91 ± 6%, respectively. Our automated diagnostic method generated maps of the reasonably well circumscribed normal/PTG and OSSN interface. PTG and OSSN margins identified by our automated analysis were in close agreement with the margins found in the H&E sections. Such a map can be rapidly generated on a real time basis and potentially used for intraoperative assessment.
Collapse
|
22
|
Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging. Sci Rep 2022; 12:4508. [PMID: 35296685 PMCID: PMC8927097 DOI: 10.1038/s41598-022-07524-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 02/17/2022] [Indexed: 12/24/2022] Open
Abstract
Esophageal cancer is the sixth leading cause of cancer-related death worldwide. Histopathological confirmation is a key step in tumor diagnosis. Therefore, simplification in decision-making by discrimination between malignant and non-malignant cells of histological specimens can be provided by combination of new imaging technology and artificial intelligence (AI). In this work, hyperspectral imaging (HSI) data from 95 patients were used to classify three different histopathological features (squamous epithelium cells, esophageal adenocarcinoma (EAC) cells, and tumor stroma cells), based on a multi-layer perceptron with two hidden layers. We achieved an accuracy of 78% for EAC and stroma cells, and 80% for squamous epithelium. HSI combined with machine learning algorithms is a promising and innovative technique, which allows image acquisition beyond Red–Green–Blue (RGB) images. Further method validation and standardization will be necessary, before automated tumor cell identification algorithms can be used in daily clinical practice.
Collapse
|
23
|
van Vliet-Pérez SM, van de Berg NJ, Manni F, Lai M, Rijstenberg L, Hendriks BHW, Dankelman J, Ewing-Graham PC, Nieuwenhuyzen-de Boer GM, van Beekhuizen HJ. Hyperspectral Imaging for Tissue Classification after Advanced Stage Ovarian Cancer Surgery-A Pilot Study. Cancers (Basel) 2022; 14:cancers14061422. [PMID: 35326577 PMCID: PMC8946803 DOI: 10.3390/cancers14061422] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Revised: 02/24/2022] [Accepted: 03/08/2022] [Indexed: 02/05/2023] Open
Abstract
The most important prognostic factor for the survival of advanced-stage epithelial ovarian cancer (EOC) is the completeness of cytoreductive surgery (CRS). Therefore, an intraoperative technique to detect microscopic tumors would be of great value. The aim of this pilot study is to assess the feasibility of near-infrared hyperspectral imaging (HSI) for EOC detection in ex vivo tissue samples. Images were collected during CRS in 11 patients in the wavelength range of 665−975 nm, and processed by calibration, normalization, and noise filtering. A linear support vector machine (SVM) was employed to classify healthy and tumorous tissue (defined as >50% tumor cells). Classifier performance was evaluated using leave-one-out cross-validation. Images of 26 tissue samples from 10 patients were included, containing 26,446 data points that were matched to histopathology. Tumorous tissue could be classified with an area under the curve of 0.83, a sensitivity of 0.81, a specificity of 0.70, and Matthew’s correlation coefficient of 0.41. This study paves the way to in vivo and intraoperative use of HSI during CRS. Hyperspectral imaging can scan a whole tissue surface in a fast and non-contact way. Our pilot study demonstrates that HSI and SVM learning can be used to discriminate EOC from surrounding tissue.
Collapse
Affiliation(s)
- Sharline M. van Vliet-Pérez
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
- Department of Radiotherapy, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands
- Correspondence:
| | - Nick J. van de Berg
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
| | - Francesca Manni
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (F.M.); (M.L.)
| | - Marco Lai
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands; (F.M.); (M.L.)
| | - Lucia Rijstenberg
- Department of Pathology, Erasmus University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (L.R.); (P.C.E.-G.)
| | - Benno H. W. Hendriks
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
| | - Jenny Dankelman
- Department of Biomechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands; (N.J.v.d.B.); (B.H.W.H.); (J.D.)
| | - Patricia C. Ewing-Graham
- Department of Pathology, Erasmus University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (L.R.); (P.C.E.-G.)
| | - Gatske M. Nieuwenhuyzen-de Boer
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
- Department of Gynecology, Albert Schweitzer Hospital, 3318 AT Dordrecht, The Netherlands
| | - Heleen J. van Beekhuizen
- Department of Gynecological Oncology, Erasmus MC Cancer Institute, University Medical Center Rotterdam, 3015 GD Rotterdam, The Netherlands; (G.M.N.-d.B.); (H.J.v.B.)
| |
Collapse
|
24
|
Zhou X, Ma L, Mubarak HK, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12039:120390G. [PMID: 36798940 PMCID: PMC9930132 DOI: 10.1117/12.2614624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
The study is to incorporate polarized hyperspectral imaging (PHSI) with deep learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we firstly collected the Stokes vector data cubes (S0, S1, S2, and S3) of histologic slides from 17 patients with SCC by the PHSI microscope, under the wavelength range from 467 nm to 750 nm. Secondly, we generated the synthetic RGB images from the original Stokes vector data cubes. Thirdly, we cropped the synthetic RGB images into image patches at the image size of 96×96 pixels, and then set up a ResNet50-based convolutional neural network (CNN) to classify the image patches of the four Stokes vector parameters (S0, S1, S2, and S3) by application of transfer learning. To test the performances of the model, each time we trained the model based on the image patches (S0, S1, S2, and S3) of 16 patients out of 17 patients, and used the trained model to calculate the testing accuracy based on the image patches of the rest 1 patient (S0, S1, S2, and S3). We repeated the process for 6 times and obtained 24 testing accuracies (S0, S1, S2, and S3) from 6 different patients out of the 17 patients. The preliminary results showed that the average testing accuracy (84.2%) on S3 outperformed the average testing accuracy (83.5%) on S0. Furthermore, 4 of 6 testing accuracies of S3 (96.0%, 87.3%, 82.8%, and 86.7%) outperformed the testing accuracies of S0 (93.3%, 85.2%, 80.2%, and 79.0%). The study demonstrated the potential of using polarized hyperspectral imaging and deep learning for automatic detection of head and neck SCC on pathologic slides.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Hasan K Mubarak
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - James V. Little
- Emory University, Department of Pathology and Laboratory Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta, GA
| | - Larry L. Myers
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baran D. Sumer
- Univ. of Texas Southwestern Medical Center, Dept. of Otolaryngology, Dallas, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX
- Univ. of Texas Southwestern Medical Center, Dept. of Radiology, Dallas, TX
| |
Collapse
|
25
|
Leitch K, Halicek M, Shahedi M, Little JV, Chen AY, Fei B. Detecting Aggressive Papillary Thyroid Carcinoma Using Hyperspectral Imaging and Radiomic Features. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12033:1203322. [PMID: 36798628 PMCID: PMC9929637 DOI: 10.1117/12.2611842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Hyperspectral imaging (HSI) and radiomics have the potential to improve the accuracy of tumor malignancy prediction and assessment. In this work, we extracted radiomic features of fresh surgical papillary thyroid carcinoma (PTC) specimen that were imaged with HSI. A total of 107 unique radiomic features were extracted. This study includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. With the dilated hyperspectral images, the shape feature of least axis length was able to predict the tumor aggressiveness with a high accuracy. The HSI-based radiomic method may provide a useful tool to aid oncologists in determining tumors with intermediate to high risk and in clinical decision making.
Collapse
Affiliation(s)
- Ka’Toria Leitch
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta, GA
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University School of Medicine, Atlanta, GA
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Richardson, TX
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
26
|
Tran MH, Ma L, Litter JV, Chen AY, Fei B. Thyroid Carcinoma Detection on Whole Histologic Slides Using Hyperspectral Imaging and Deep Learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12039:120390H. [PMID: 36798939 PMCID: PMC9929647 DOI: 10.1117/12.2612963] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Hyperspectral imaging (HSI), a non-invasive imaging modality, has been successfully used in many different biological and medical applications. One such application is in the field of oncology, where hyperspectral imaging is being used on histologic samples. This study compares the performances of different image classifiers using different imaging modalities as training data. From a database of 33 fixed tissues from head and neck patients with follicular thyroid carcinoma, we produced three different datasets: an RGB image dataset that was acquired from a whole slide image scanner, a hyperspectral (HS) dataset that was acquired with a compact hyperspectral camera, and an HS-synthesized RGB image dataset. Three separate deep learning classifiers were trained using the three datasets. We show that the deep learning classifier trained on HSI data has an area under the receiver operator characteristic curve (AUC-ROC) of 0.966, higher than that of the classifiers trained on RGB and HSI-synthesized RGB data. This study demonstrates that hyperspectral images improve the performance of cancer classification on whole histologic slides. Hyperspectral imaging and deep learning provide an automatic tool for thyroid cancer detection on whole histologic slides.
Collapse
Affiliation(s)
- Minh Ha Tran
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX
- Univ. of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, TX
| | - Ling Ma
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX
- Univ. of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, TX
| | - James V. Litter
- Emory Univ. School of Medicine, Dept. of Pathology and Laboratory Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory Univ. School of Medicine, Dept. of Otolaryngology, Atlanta, GA
| | - Baowei Fei
- Univ. of Texas at Dallas, Dept. of Bioengineering, Richardson, TX
- Univ. of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
27
|
Vohra N, Liu H, Nelson AH, Bailey K, El-Shenawee M. Hyperspectral terahertz imaging and optical clearance for cancer classification in breast tumor surgical specimen. J Med Imaging (Bellingham) 2022; 9:014002. [PMID: 35036473 PMCID: PMC8752447 DOI: 10.1117/1.jmi.9.1.014002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Accepted: 12/21/2021] [Indexed: 01/14/2023] Open
Abstract
Purpose: We investigate the enhancement in terahertz (THz) images of freshly excised breast tumors upon treatment with an optical clearance agent. The hyperspectral imaging and spectral classifications are used to quantitatively demonstrate the image enhancement. Glycerol solution with 60% concentration is applied to excised breast tumor specimens for various time durations to investigate the effectiveness on image enhancement. Approach: THz reflection spectroscopy is utilized to obtain the absorption coefficient and the index of refraction of untreated and glycerol-treated tissues at each frequency up to 3 THz. Two classifiers, spectral angular mapping (SAM) based on several kernels and Euclidean minimum distance (EMD) are implemented to evaluate the effectiveness of the treatment. The testing raw data is obtained from five breast cancer specimens: two untreated specimens and three specimens treated with glycerol solution for 20, 40, or 60 min. All tumors used in the testing data have healthy tissues adjacent to cancerous ones consistent with the challenge faced in lumpectomy surgeries. Results: The glycerol-treated tissues showed a decrease in the absorption coefficients compared with untreated tissues, especially as the period of treatment increased. Although the sensitivity metric of the classifier presented higher values in the untreated tissues compared with the treated ones, the specificity and accuracy metrics demonstrated higher values for the treated tissues compared with the untreated ones. Conclusions: The biocompatible glycerol solution is a potential optical clearance agent in THz imaging while keeping the histopathology imaging intact. The SAM technique provided a good classification of cancerous tissues despite the small amount of cancer in the training data (only 7%). The SAM exponential kernel and EMD presented classification accuracy of ∼ 80 % to 85% compared with linear and polynomial kernels that provided accuracy ranging from 70% to 80%. Overall, glycerol treatment provides a potential improvement in cancer classification in freshly excised breast tumors.
Collapse
Affiliation(s)
- Nagma Vohra
- University of Arkansas, Department of Electrical Engineering, Fayetteville, Arkansas, United States
| | - Haoyan Liu
- University of Arkansas, Department of Computer Science and Engineering, Fayetteville, Arkansas, United States
| | - Alexander H. Nelson
- University of Arkansas, Department of Computer Science and Engineering, Fayetteville, Arkansas, United States
| | - Keith Bailey
- Charles River Laboratory, Mattawan, Michigan, United States
| | - Magda El-Shenawee
- University of Arkansas, Department of Electrical Engineering, Fayetteville, Arkansas, United States
| |
Collapse
|
28
|
Bassler MC, Stefanakis M, Sequeira I, Ostertag E, Wagner A, Bartsch JW, Roeßler M, Mandic R, Reddmann EF, Lorenz A, Rebner K, Brecht M. Comparison of Whiskbroom and Pushbroom darkfield elastic light scattering spectroscopic imaging for head and neck cancer identification in a mouse model. Anal Bioanal Chem 2021; 413:7363-7383. [PMID: 34799750 PMCID: PMC8626402 DOI: 10.1007/s00216-021-03726-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Revised: 08/30/2021] [Accepted: 10/08/2021] [Indexed: 12/20/2022]
Abstract
The early detection of head and neck cancer is a prolonged challenging task. It requires a precise and accurate identification of tissue alterations as well as a distinct discrimination of cancerous from healthy tissue areas. A novel approach for this purpose uses microspectroscopic techniques with special focus on hyperspectral imaging (HSI) methods. Our proof-of-principle study presents the implementation and application of darkfield elastic light scattering spectroscopy (DF ELSS) as a non-destructive, high-resolution, and fast imaging modality to distinguish lingual healthy from altered tissue regions in a mouse model. The main aspect of our study deals with the comparison of two varying HSI detection principles, which are a point-by-point and line scanning imaging, and whether one might be more appropriate in differentiating several tissue types. Statistical models are formed by deploying a principal component analysis (PCA) with the Bayesian discriminant analysis (DA) on the elastic light scattering (ELS) spectra. Overall accuracy, sensitivity, and precision values of 98% are achieved for both models whereas the overall specificity results in 99%. An additional classification of model-unknown ELS spectra is performed. The predictions are verified with histopathological evaluations of identical HE-stained tissue areas to prove the model’s capability of tissue distinction. In the context of our proof-of-principle study, we assess the Pushbroom PCA-DA model to be more suitable for tissue type differentiations and thus tissue classification. In addition to the HE-examination in head and neck cancer diagnosis, the usage of HSI-based statistical models might be conceivable in a daily clinical routine.
Collapse
Affiliation(s)
- Miriam C Bassler
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany
| | - Mona Stefanakis
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany
| | - Inês Sequeira
- Institute of Dentistry, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - Edwin Ostertag
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Alexandra Wagner
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany
| | - Jörg W Bartsch
- Department of Neurosurgery, Philipps University Marburg, Baldingerstraße, 35033, Marburg, Germany
| | - Marion Roeßler
- Department of Pathology, Philipps University Marburg, Baldingerstraße, 35033, Marburg, Germany
| | - Robert Mandic
- Department of Otorhinolaryngology, Philipps University Marburg, Baldingerstraße, 35033, Marburg, Germany
| | - Eike F Reddmann
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Anita Lorenz
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Karsten Rebner
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany
| | - Marc Brecht
- Process Analysis and Technology (PA&T), Reutlingen University, Alteburgstr. 150, 72762, Reutlingen, Germany.
- Institute of Physical and Theoretical Chemistry, University of Tübingen, Auf der Morgenstelle 18, 72076, Tübingen, Germany.
| |
Collapse
|
29
|
Zhang Y, Yu S, Zhu X, Ning X, Liu W, Wang C, Liu X, Zhao D, Zheng Y, Bao J. Explainable liver tumor delineation in surgical specimens using hyperspectral imaging and deep learning. BIOMEDICAL OPTICS EXPRESS 2021; 12:4510-4529. [PMID: 34457429 PMCID: PMC8367264 DOI: 10.1364/boe.432654] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2021] [Revised: 06/17/2021] [Accepted: 06/18/2021] [Indexed: 05/08/2023]
Abstract
Surgical removal is the primary treatment for liver cancer, but frequent recurrence caused by residual malignant tissue remains an important challenge, as recurrence leads to high mortality. It is unreliable to distinguish tumors from normal tissues merely under visual inspection. Hyperspectral imaging (HSI) has been proved to be a promising technology for intra-operative use by capturing the spatial and spectral information of tissue in a fast, non-contact and label-free manner. In this work, we investigated the feasibility of HSI for liver tumor delineation on surgical specimens using a multi-task U-Net framework. Measurements are performed on 19 patients and a dataset of 36 specimens was collected with corresponding pathological results serving as the ground truth. The developed framework can achieve an overall sensitivity of 94.48% and a specificity of 87.22%, outperforming the baseline SVM method by a large margin. In particular, we propose to add explanations on the well-trained model from the spatial and spectral dimensions to show the contribution of pixels and spectral channels explicitly. On that basis, a novel saliency-weighted channel selection method is further proposed to select a small subset of 5 spectral channels which provide essentially as much information as using all 224 channels. According to the dominant channels, the absorption difference of hemoglobin and bile content in the normal and malignant tissues seems to be promising markers that could be further exploited.
Collapse
Affiliation(s)
- Yating Zhang
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Si Yu
- Department of Liver Surgery, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Xueyu Zhu
- Department of Mathematics, University of Iowa, Iowa City, IA 52242, USA
| | - Xuefei Ning
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Wei Liu
- Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences, School of Basic Medicine, Peking Union Medical College, Beijing, China
| | - Chuting Wang
- Department of Liver Surgery, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Xiaohu Liu
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
- Currently with the School of Biomedical Engineering, School of Ophthalmology & Optometry, Wenzhou Medical University. Xueyuan Road 270, Wenzhou 325027, China
| | - Ding Zhao
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Yongchang Zheng
- Department of Liver Surgery, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Jie Bao
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| |
Collapse
|
30
|
Mahmood H, Shaban M, Rajpoot N, Khurram SA. Artificial Intelligence-based methods in head and neck cancer diagnosis: an overview. Br J Cancer 2021; 124:1934-1940. [PMID: 33875821 PMCID: PMC8184820 DOI: 10.1038/s41416-021-01386-x] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Revised: 03/11/2021] [Accepted: 03/31/2021] [Indexed: 02/02/2023] Open
Abstract
BACKGROUND This paper reviews recent literature employing Artificial Intelligence/Machine Learning (AI/ML) methods for diagnostic evaluation of head and neck cancers (HNC) using automated image analysis. METHODS Electronic database searches using MEDLINE via OVID, EMBASE and Google Scholar were conducted to retrieve articles using AI/ML for diagnostic evaluation of HNC (2009-2020). No restrictions were placed on the AI/ML method or imaging modality used. RESULTS In total, 32 articles were identified. HNC sites included oral cavity (n = 16), nasopharynx (n = 3), oropharynx (n = 3), larynx (n = 2), salivary glands (n = 2), sinonasal (n = 1) and in five studies multiple sites were studied. Imaging modalities included histological (n = 9), radiological (n = 8), hyperspectral (n = 6), endoscopic/clinical (n = 5), infrared thermal (n = 1) and optical (n = 1). Clinicopathologic/genomic data were used in two studies. Traditional ML methods were employed in 22 studies (69%), deep learning (DL) in eight studies (25%) and a combination of these methods in two studies (6%). CONCLUSIONS There is an increasing volume of studies exploring the role of AI/ML to aid HNC detection using a range of imaging modalities. These methods can achieve high degrees of accuracy that can exceed the abilities of human judgement in making data predictions. Large-scale multi-centric prospective studies are required to aid deployment into clinical practice.
Collapse
Affiliation(s)
- Hanya Mahmood
- Academic Unit of Oral & Maxillofacial Surgery, School of Clinical Dentistry, University of Sheffield, Sheffield, UK.
| | - Muhammad Shaban
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, Coventry, UK
| | - Syed A Khurram
- Unit of Oral & Maxillofacial Pathology, School of Clinical Dentistry, University of Sheffield, Sheffield, UK
| |
Collapse
|
31
|
Yin L, Cao Z, Wang K, Tian J, Yang X, Zhang J. A review of the application of machine learning in molecular imaging. ANNALS OF TRANSLATIONAL MEDICINE 2021; 9:825. [PMID: 34268438 PMCID: PMC8246214 DOI: 10.21037/atm-20-5877] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Accepted: 10/02/2020] [Indexed: 12/12/2022]
Abstract
Molecular imaging (MI) is a science that uses imaging methods to reflect the changes of molecular level in living state and conduct qualitative and quantitative studies on its biological behaviors in imaging. Optical molecular imaging (OMI) and nuclear medical imaging are two key research fields of MI. OMI technology refers to the optical information generated by the imaging target (such as tumors) due to drug intervention and other reasons. By collecting the optical information, researchers can track the motion trajectory of the imaging target at the molecular level. Owing to its high specificity and sensitivity, OMI has been widely used in preclinical research and clinical surgery. Nuclear medical imaging mainly detects ionizing radiation emitted by radioactive substances. It can provide molecular information for early diagnosis, effective treatment and basic research of diseases, which has become one of the frontiers and hot topics in the field of medicine in the world today. Both OMI and nuclear medical imaging technology require a lot of data processing and analysis. In recent years, artificial intelligence technology, especially neural network-based machine learning (ML) technology, has been widely used in MI because of its powerful data processing capability. It provides a feasible strategy to deal with large and complex data for the requirement of MI. In this review, we will focus on the applications of ML methods in OMI and nuclear medical imaging.
Collapse
Affiliation(s)
- Lin Yin
- Key Laboratory of Molecular Imaging of Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, Beijing, China.,School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Zhen Cao
- Peking University First Hospital, Beijing, China
| | - Kun Wang
- Key Laboratory of Molecular Imaging of Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, Beijing, China.,School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jie Tian
- Key Laboratory of Molecular Imaging of Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, Beijing, China.,School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.,Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, Beihang University, Beijing, China
| | - Xing Yang
- Peking University First Hospital, Beijing, China
| | | |
Collapse
|
32
|
Trajanovski S, Shan C, Weijtmans PJC, de Koning SGB, Ruers TJM. Tongue Tumor Detection in Hyperspectral Images Using Deep Learning Semantic Segmentation. IEEE Trans Biomed Eng 2021; 68:1330-1340. [PMID: 32976092 DOI: 10.1109/tbme.2020.3026683] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
OBJECTIVE The utilization of hyperspectral imaging (HSI) in real-time tumor segmentation during a surgery have recently received much attention, but it remains a very challenging task. METHODS In this work, we propose semantic segmentation methods, and compare them with other relevant deep learning algorithms for tongue tumor segmentation. To the best of our knowledge, this is the first work using deep learning semantic segmentation for tumor detection in HSI data using channel selection, and accounting for more spatial tissue context, and global comparison between the prediction map, and the annotation per sample. Results, and Conclusion: On a clinical data set with tongue squamous cell carcinoma, our best method obtains very strong results of average dice coefficient, and area under the ROC-curve of [Formula: see text], and [Formula: see text], respectively on the original spatial image size. The results show that a very good performance can be achieved even with a limited amount of data. We demonstrate that important information regarding tumor decision is encoded in various channels, but some channel selection, and filtering is beneficial over the full spectra. Moreover, we use both visual (VIS), and near-infrared (NIR) spectrum, rather than commonly used only VIS spectrum; although VIS spectrum is generally of higher significance, we demonstrate NIR spectrum is crucial for tumor capturing in some cases. SIGNIFICANCE The HSI technology augmented with accurate deep learning algorithms has a huge potential to be a promising alternative to digital pathology or a doctors' supportive tool in real-time surgeries.
Collapse
|
33
|
Jansen-Winkeln B, Barberio M, Chalopin C, Schierle K, Diana M, Köhler H, Gockel I, Maktabi M. Feedforward Artificial Neural Network-Based Colorectal Cancer Detection Using Hyperspectral Imaging: A Step towards Automatic Optical Biopsy. Cancers (Basel) 2021; 13:cancers13050967. [PMID: 33669082 PMCID: PMC7956537 DOI: 10.3390/cancers13050967] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 02/18/2021] [Accepted: 02/20/2021] [Indexed: 12/14/2022] Open
Abstract
Simple Summary Detection of colorectal carcinoma is performed visually by investigators and is confirmed pathologically. With hyperspectral imaging, an expanded spectral range of optical information is now available for analysis. The acquired recordings were analyzed with a neural network, and it was possible to differentiate tumor from healthy mucosa in colorectal carcinoma by automatic classification with high reliability. Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. This is a step towards optical biopsy. Abstract Currently, colorectal cancer (CRC) is mainly identified via a visual assessment during colonoscopy, increasingly used artificial intelligence algorithms, or surgery. Subsequently, CRC is confirmed through a histopathological examination by a pathologist. Hyperspectral imaging (HSI), a non-invasive optical imaging technology, has shown promising results in the medical field. In the current study, we combined HSI with several artificial intelligence algorithms to discriminate CRC. Between July 2019 and May 2020, 54 consecutive patients undergoing colorectal resections for CRC were included. The tumor was imaged from the mucosal side with a hyperspectral camera. The image annotations were classified into three groups (cancer, CA; adenomatous margin around the central tumor, AD; and healthy mucosa, HM). Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. Hyperspectral imaging combined with automatic classification can be used to differentiate between CRC and healthy mucosa. Additionally, the biological changes induced by chemotherapy to the tissue are detectable with HSI.
Collapse
Affiliation(s)
- Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (M.B.); (I.G.)
- Correspondence: ; Tel.: +49-341-9717211; Fax: +49-341-9728167
| | - Manuel Barberio
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (M.B.); (I.G.)
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France;
- Department of General Surgery, Hospital Card. G. Panico, 73039 Tricase, Italy
| | - Claire Chalopin
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (C.C.); (H.K.); (M.M.)
| | - Katrin Schierle
- Institute of Pathology, University Hospital Leipzig, 04103 Leipzig, Germany;
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France;
| | - Hannes Köhler
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (C.C.); (H.K.); (M.M.)
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, 04103 Leipzig, Germany; (M.B.); (I.G.)
| | - Marianne Maktabi
- Innovation Center Computer-Assisted Surgery (ICCAS), University of Leipzig, 04103 Leipzig, Germany; (C.C.); (H.K.); (M.M.)
| |
Collapse
|
34
|
Nedu ME, Tertis M, Cristea C, Georgescu AV. Methylene Blue and Proflavine as Intraarterial Marker for Functional Perforazome-Comparative Study. J Pers Med 2021; 11:147. [PMID: 33669636 PMCID: PMC7922966 DOI: 10.3390/jpm11020147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Revised: 02/13/2021] [Accepted: 02/17/2021] [Indexed: 11/21/2022] Open
Abstract
Methylene blue (MB) is both a dye and a medicine known and used for a long time including as lymphatic tracer in melanoma and breast cancer for revealing sentinel lymph nodes. Proflavine (PRO) is an acriflavine dye, used as bacteriostatic disinfectant against many gram-positive bacteria that was also successfully applied to evaluate morphopathological changes in tissues. This study was performed on a group of twenty-eight Wistar rats and had as its main objective the in vivo evaluation of the use of MB and PRO as perforator tracers. The two dyes proved to be effective functional perforasome tracers with medium inflammatory infiltrate in the skin of the island perforator flap which heals perfectly at 14 days with complete absence of the inflammatory reaction. At the same injected amount, PRO seems to determine a greater inflammatory reaction compared with MB, but in smaller concentration, the inflammatory response is absent in the case of PRO. In conclusion, both substances tested within this in vivo study are good functional perforasome tracers, but PRO has the advantage of the absence of inflammatory reaction when using lower concentrations, while preserving unalerted its efficiency as tracer.
Collapse
Affiliation(s)
- Maria-Eliza Nedu
- Department of Plastic Surgery, Faculty of Medicine, Iuliu Haţieganu University of Medicine and Pharmacy, 400347 Cluj-Napoca, Romania; (M.-E.N.); (A.V.G.)
| | - Mihaela Tertis
- Department of Analytical Chemistry, Faculty of Pharmacy, Iuliu Haţieganu University of Medicine and Pharmacy, 400349 Cluj-Napoca, Romania;
| | - Cecilia Cristea
- Department of Analytical Chemistry, Faculty of Pharmacy, Iuliu Haţieganu University of Medicine and Pharmacy, 400349 Cluj-Napoca, Romania;
| | - Alexandru Valentin Georgescu
- Department of Plastic Surgery, Faculty of Medicine, Iuliu Haţieganu University of Medicine and Pharmacy, 400347 Cluj-Napoca, Romania; (M.-E.N.); (A.V.G.)
| |
Collapse
|
35
|
Laimer J, Bruckmoser E, Helten T, Kofler B, Zelger B, Brunner A, Zelger B, Huck CW, Tappert M, Rogge D, Schirmer M, Pallua JD. Hyperspectral imaging as a diagnostic tool to differentiate between amalgam tattoos and other dark pigmented intraoral lesions. JOURNAL OF BIOPHOTONICS 2021; 14:e202000424. [PMID: 33210464 DOI: 10.1002/jbio.202000424] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 11/13/2020] [Accepted: 11/15/2020] [Indexed: 06/11/2023]
Abstract
The goal of this project is to identify any in-depth benefits and drawbacks in the diagnosis of amalgam tattoos and other pigmented intraoral lesions using hyperspectral imagery collected from amalgam tattoos, benign, and malignant melanocytic neoplasms. Software solutions capable of classifying pigmented lesions of the skin already exist, but conventional red, green and blue images may be reaching an upper limit in their performance. Emerging technologies, such as hyperspectral imaging (HSI) utilize more than a hundred, continuous data channels, while also collecting data in the infrared. A total of 18 paraffin-embedded human tissue specimens of dark pigmented intraoral lesions (including the lip) were analyzed using visible and near-infrared (VIS-NIR) hyperspectral imagery obtained from HE-stained histopathological slides. Transmittance data were collected between 450 and 900 nm using a snapshot camera mounted to a microscope with a halogen light source. VIS-NIR spectra collected from different specimens, such as melanocytic cells and other tissues (eg, epithelium), produced distinct and diagnostic spectra that were used to identify these materials in several regions of interest, making it possible to distinguish between intraoral amalgam tattoos (intramucosal metallic foreign bodies) and melanocytic lesions of the intraoral mucosa and the lip (each with P < .01 using the independent t test). HSI is presented as a diagnostic tool for the rapidly growing field of digital pathology. In this preliminary study, amalgam tattoos were reliably differentiated from melanocytic lesions of the oral cavity and the lip.
Collapse
Affiliation(s)
- Johannes Laimer
- University Hospital for Craniomaxillofacial and Oral Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | | | - Tom Helten
- University Hospital for Craniomaxillofacial and Oral Surgery, Medical University of Innsbruck, Innsbruck, Austria
| | - Barbara Kofler
- University Hospital of Otorhinolaryngology, Medical University of Innsbruck, Innsbruck, Austria
| | - Bettina Zelger
- Institute of Pathology, Neuropathology and Molecular Pathology, Medical University of Innsbruck, Innsbruck, Austria
| | - Andrea Brunner
- Institute of Pathology, Neuropathology and Molecular Pathology, Medical University of Innsbruck, Innsbruck, Austria
| | - Bernhard Zelger
- University Hospital for Dermatology, Venereology and Allergology, Medical University of Innsbruck, Innsbruck, Austria
| | - Christian W Huck
- Institute of Analytical Chemistry and Radiochemistry, Leopold Franzens University of Innsbruck, Innsbruck, Austria
| | - Michelle Tappert
- Hyperspectral Intelligence Inc., Gibsons, British Columbia, Canada
| | - Derek Rogge
- Hyperspectral Intelligence Inc., Gibsons, British Columbia, Canada
| | - Michael Schirmer
- Department of Internal Medicine, Clinic II, Medical University of Innsbruck, Innsbruck, Austria
| | - Johannes D Pallua
- Institute of Pathology, Neuropathology and Molecular Pathology, Medical University of Innsbruck, Innsbruck, Austria
- University Hospital for Orthopedics and Traumatology, Medical University of Innsbruck, Innsbruck, Austria
| |
Collapse
|
36
|
Ma L, Shahedi M, Shi T, Halicek M, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Pixel-level Tumor Margin Assessment of Surgical Specimen with Hyperspectral Imaging and Deep Learning Classification. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11598:1159811. [PMID: 35755403 PMCID: PMC9232191 DOI: 10.1117/12.2581046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Surgery is a major treatment method for squamous cell carcinoma (SCC). During surgery, insufficient tumor margin may lead to local recurrence of cancer. Hyperspectral imaging (HSI) is a promising optical imaging technique for in vivo cancer detection and tumor margin assessment. In this study, a fully convolutional network (FCN) was implemented for tumor classification and margin assessment on hyperspectral images of SCC. The FCN was trained and validated with hyperspectral images of 25 ex vivo SCC surgical specimens from 20 different patients. The network was evaluated per patient and achieved pixel-level tissue classification with an average area under the curve (AUC) of 0.88, as well as 0.83 accuracy, 0.84 sensitivity, and 0.70 specificity across all the 20 patients. The 95% Hausdorff distance of assessed tumor margin in 17 patients was less than 2 mm, and the classification time of each tissue specimen took less than 10 seconds. The proposed methods can potentially facilitate intraoperative tumor margin assessment and improve surgical outcomes.
Collapse
Affiliation(s)
- Ling Ma
- Department of Bioengineering, University of Texas at Dallas
- State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin University
| | - Maysam Shahedi
- Department of Bioengineering, University of Texas at Dallas
| | - Ted Shi
- Department of Bioengineering, University of Texas at Dallas
| | - Martin Halicek
- Department of Bioengineering, University of Texas at Dallas
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University
| | - Larry L. Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center
- Department of Radiology, University of Texas Southwestern Medical Center
| |
Collapse
|
37
|
Zhou X, Ma L, Brown W, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and machine learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11603:116030Q. [PMID: 34955584 PMCID: PMC8699168 DOI: 10.1117/12.2582330] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
The aim of this study is to incorporate polarized hyperspectral imaging (PHSI) with machine learning for automatic detection of head and neck squamous cell carcinoma (SCC) on hematoxylin and eosin (H&E) stained tissue slides. A polarized hyperspectral imaging microscope had been developed in our group. In this paper, we imaged 20 H&E stained tissue slides from 10 patients with SCC of the larynx by the PHSI microscope. Several machine learning algorithms, including support vector machine (SVM), random forest, Gaussian naive Bayes, and logistic regression, were applied to the collected image data for the automatic detection of SCC on the H&E stained tissue slides. The performance of these methods was compared among the collected PHSI data, the pseudo-RGB images generated from the PHSI data, and the PHSI data after applying the principal component analysis (PCA) transformation. The results suggest that SVM is a superior classifier for the classification task based on the PHSI data cubes compared to the other three classifiers. The incorporate of four Stokes vector parameters improved the classification accuracy. Finally, the PCA transformed image data did not improve the accuracy as it might lose some important information from the original PHSI data. The preliminary results show that polarized hyperspectral imaging can have many potential applications in digital pathology.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
| | - William Brown
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
| | - James V. Little
- Emory University, Department of Pathology and Laboratory
Medicine, Atlanta, GA
| | - Amy Y. Chen
- Emory University, Department of Otolaryngology, Atlanta,
GA
| | - Larry L. Myers
- Univ. of Texas Southwestern Medical Center, Dept. of
Otolaryngology, Dallas, TX
| | - Baran D. Sumer
- Univ. of Texas Southwestern Medical Center, Dept. of
Otolaryngology, Dallas, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering and Center for Imaging and Surgical Innovation, Richardson, TX
- Univ. of Texas Southwestern Medical Center, Advanced
Imaging Research Center, Dallas, TX
- Univ. of Texas Southwestern Medical Center, Dept. of
Radiology, Dallas, TX
| |
Collapse
|
38
|
Ma L, Fei B. Comprehensive review of surgical microscopes: technology development and medical applications. JOURNAL OF BIOMEDICAL OPTICS 2021; 26:JBO-200292VRR. [PMID: 33398948 PMCID: PMC7780882 DOI: 10.1117/1.jbo.26.1.010901] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Accepted: 12/04/2020] [Indexed: 05/06/2023]
Abstract
SIGNIFICANCE Surgical microscopes provide adjustable magnification, bright illumination, and clear visualization of the surgical field and have been increasingly used in operating rooms. State-of-the-art surgical microscopes are integrated with various imaging modalities, such as optical coherence tomography (OCT), fluorescence imaging, and augmented reality (AR) for image-guided surgery. AIM This comprehensive review is based on the literature of over 500 papers that cover the technology development and applications of surgical microscopy over the past century. The aim of this review is threefold: (i) providing a comprehensive technical overview of surgical microscopes, (ii) providing critical references for microscope selection and system development, and (iii) providing an overview of various medical applications. APPROACH More than 500 references were collected and reviewed. A timeline of important milestones during the evolution of surgical microscope is provided in this study. An in-depth technical overview of the optical system, mechanical system, illumination, visualization, and integration with advanced imaging modalities is provided. Various medical applications of surgical microscopes in neurosurgery and spine surgery, ophthalmic surgery, ear-nose-throat (ENT) surgery, endodontics, and plastic and reconstructive surgery are described. RESULTS Surgical microscopy has been significantly advanced in the technical aspects of high-end optics, bright and shadow-free illumination, stable and flexible mechanical design, and versatile visualization. New imaging modalities, such as hyperspectral imaging, OCT, fluorescence imaging, photoacoustic microscopy, and laser speckle contrast imaging, are being integrated with surgical microscopes. Advanced visualization and AR are being added to surgical microscopes as new features that are changing clinical practices in the operating room. CONCLUSIONS The combination of new imaging technologies and surgical microscopy will enable surgeons to perform challenging procedures and improve surgical outcomes. With advanced visualization and improved ergonomics, the surgical microscope has become a powerful tool in neurosurgery, spinal, ENT, ophthalmic, plastic and reconstructive surgeries.
Collapse
Affiliation(s)
- Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
39
|
Manni F, van der Sommen F, Fabelo H, Zinger S, Shan C, Edström E, Elmi-Terander A, Ortega S, Marrero Callicó G, de With PHN. Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach. SENSORS (BASEL, SWITZERLAND) 2020; 20:E6955. [PMID: 33291409 PMCID: PMC7730670 DOI: 10.3390/s20236955] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Revised: 12/01/2020] [Accepted: 12/03/2020] [Indexed: 12/16/2022]
Abstract
The primary treatment for malignant brain tumors is surgical resection. While gross total resection improves the prognosis, a supratotal resection may result in neurological deficits. On the other hand, accurate intraoperative identification of the tumor boundaries may be very difficult, resulting in subtotal resections. Histological examination of biopsies can be used repeatedly to help achieve gross total resection but this is not practically feasible due to the turn-around time of the tissue analysis. Therefore, intraoperative techniques to recognize tissue types are investigated to expedite the clinical workflow for tumor resection and improve outcome by aiding in the identification and removal of the malignant lesion. Hyperspectral imaging (HSI) is an optical imaging technique with the power of extracting additional information from the imaged tissue. Because HSI images cannot be visually assessed by human observers, we instead exploit artificial intelligence techniques and leverage a Convolutional Neural Network (CNN) to investigate the potential of HSI in twelve in vivo specimens. The proposed framework consists of a 3D-2D hybrid CNN-based approach to create a joint extraction of spectral and spatial information from hyperspectral images. A comparison study was conducted exploiting a 2D CNN, a 1D DNN and two conventional classification methods (SVM, and the SVM classifier combined with the 3D-2D hybrid CNN) to validate the proposed network. An overall accuracy of 80% was found when tumor, healthy tissue and blood vessels were classified, clearly outperforming the state-of-the-art approaches. These results can serve as a basis for brain tumor classification using HSI, and may open future avenues for image-guided neurosurgical applications.
Collapse
Affiliation(s)
- Francesca Manni
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (F.v.d.S.); (S.Z.); (P.H.N.d.W.)
| | - Fons van der Sommen
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (F.v.d.S.); (S.Z.); (P.H.N.d.W.)
| | - Himar Fabelo
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (H.F.); (S.O.); (G.M.C.)
| | - Svitlana Zinger
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (F.v.d.S.); (S.Z.); (P.H.N.d.W.)
| | - Caifeng Shan
- College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao 266590, China;
| | - Erik Edström
- Department of Neurosurgery, Karolinska University Hospital and Department of Clinical Neuroscience, Karolinska Institutet, SE-171 46 Stockholm, Sweden; (E.E.); (A.E.-T.)
| | - Adrian Elmi-Terander
- Department of Neurosurgery, Karolinska University Hospital and Department of Clinical Neuroscience, Karolinska Institutet, SE-171 46 Stockholm, Sweden; (E.E.); (A.E.-T.)
| | - Samuel Ortega
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (H.F.); (S.O.); (G.M.C.)
| | - Gustavo Marrero Callicó
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (H.F.); (S.O.); (G.M.C.)
| | - Peter H. N. de With
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (F.v.d.S.); (S.Z.); (P.H.N.d.W.)
| |
Collapse
|
40
|
Saiko G, Lombardi P, Au Y, Queen D, Armstrong D, Harding K. Hyperspectral imaging in wound care: A systematic review. Int Wound J 2020; 17:1840-1856. [PMID: 32830443 PMCID: PMC7949456 DOI: 10.1111/iwj.13474] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Revised: 07/07/2020] [Accepted: 07/15/2020] [Indexed: 01/18/2023] Open
Abstract
Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.
Collapse
Affiliation(s)
| | | | | | | | - David Armstrong
- Keck School of MedicineUniversity of Southern California, Los AngelesCaliforniaCaliforniaCanada
| | - Keith Harding
- School of MedicineCardiff UniversityWalesUK
- A*STARSingapore
| |
Collapse
|
41
|
Tama BA, Kim DH, Kim G, Kim SW, Lee S. Recent Advances in the Application of Artificial Intelligence in Otorhinolaryngology-Head and Neck Surgery. Clin Exp Otorhinolaryngol 2020; 13:326-339. [PMID: 32631041 PMCID: PMC7669308 DOI: 10.21053/ceo.2020.00654] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 05/24/2020] [Accepted: 06/09/2020] [Indexed: 12/12/2022] Open
Abstract
This study presents an up-to-date survey of the use of artificial intelligence (AI) in the field of otorhinolaryngology, considering opportunities, research challenges, and research directions. We searched PubMed, the Cochrane Central Register of Controlled Trials, Embase, and the Web of Science. We initially retrieved 458 articles. The exclusion of non-English publications and duplicates yielded a total of 90 remaining studies. These 90 studies were divided into those analyzing medical images, voice, medical devices, and clinical diagnoses and treatments. Most studies (42.2%, 38/90) used AI for image-based analysis, followed by clinical diagnoses and treatments (24 studies). Each of the remaining two subcategories included 14 studies. Machine learning and deep learning have been extensively applied in the field of otorhinolaryngology. However, the performance of AI models varies and research challenges remain.
Collapse
Affiliation(s)
- Bayu Adhi Tama
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, Korea
| | - Do Hyun Kim
- Department of Otolaryngology-Head and Neck Surgery, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Gyuwon Kim
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, Korea
| | - Soo Whan Kim
- Department of Otolaryngology-Head and Neck Surgery, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
| | - Seungchul Lee
- Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang, Korea
- Graduate School of Artificial Intelligence, Pohang University of Science and Technology, Pohang, Korea
| |
Collapse
|
42
|
Saiko G, Lombardi P, Au Y, Queen D, Armstrong D, Harding K. Hyperspectral imaging in wound care: A systematic review. Int Wound J 2020. [PMID: 32830443 DOI: 10.1111/iwj.13474.] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023] Open
Abstract
Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.
Collapse
Affiliation(s)
| | | | | | | | - David Armstrong
- Keck School of Medicine, University of Southern California, Los Angeles, California, California, Canada
| | - Keith Harding
- School of Medicine, Cardiff University, Wales, UK.,A*STAR, Singapore
| |
Collapse
|
43
|
Aboughaleb IH, Aref MH, El-Sharkawy YH. Hyperspectral imaging for diagnosis and detection of ex-vivo breast cancer. Photodiagnosis Photodyn Ther 2020; 31:101922. [PMID: 32726640 DOI: 10.1016/j.pdpdt.2020.101922] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2020] [Revised: 06/09/2020] [Accepted: 07/10/2020] [Indexed: 01/28/2023]
Abstract
BACKGROUND AND PURPOSE Breast cancer is one of the most widely recognized tumors. .Diagnosis made in the early stage of disease may imporve outcomes. The discovery of malignant growth utilizing noninvasive light intrusive methods in lieu of conventional excisional biopsy may assist in achieving this goal. MATERIALS AND METHODS The change of the optical properties of ex-vivo breast tissues provides different responses to light transmission, absorption, and particularly the reflection over the spectrum range. We offer the use of Hyperspectral imaging (HSI) with advanced image processing and pattern recognition in order to analyze HSI data for breast cancer detection. The spectral signatures were mined and evaluated in both malignant and normal tissue. K-mean clustering was designed for classifying hyperspectral data in order to evaluate and detection of cancer tissue. This method was used to detect ex-vivo breast cancer. Spatial spectral images were created to high spot the differences in the reflectance properties of malignant versus normal tissue. RESULTS Trials showed that the superficial spectral reflection images within 500 nm wavelength showed high variance (214.65) between cancerous and normal breast tissues. On the other hand, image within 620 nm wavelength showed low variance (0.0020).However, the superimposed of spectral region 420-620 nm was proposed as the optimum bandwidth. Finally, the proposed HS imaging system was capable to discriminate the tumor region from normal tissue of the ex-vivo breast sample with sensitivity and a specificity of 95 % and 96 %. CONCLUSIONS High sensitivity and specificity were achieved, which proposes potential for HSI as an edge evaluation method to enhance the surgical outcome compared to the presently available techniques in the clinics.
Collapse
Affiliation(s)
- Ibrahim H Aboughaleb
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| | - Mohamed Hisham Aref
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| | - Yasser H El-Sharkawy
- Military Technical College, Biomedical Engineering Department, El-Fangary Street, Cairo, Egypt.
| |
Collapse
|
44
|
Ma L, Halicek M, Zhou X, Dormer J, Fei B. Hyperspectral Microscopic Imaging for Automatic Detection of Head and Neck Squamous Cell Carcinoma Using Histologic Image and Machine Learning. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11320. [PMID: 32476708 DOI: 10.1117/12.2549369] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
The purpose of this study is to develop hyperspectral imaging (HSI) for automatic detection of head and neck cancer cells on histologic slides. A compact hyperspectral microscopic system is developed in this study. Histologic slides from 15 patients with squamous cell carcinoma (SCC) of the larynx and hypopharynx are imaged with the system. The proposed nuclei segmentation method based on principle component analysis (PCA) can extract most nuclei in the hyperspectral image without extracting other sub-cellular components. Both spectra-based support vector machine (SVM) and patch-based convolutional neural network (CNN) are used for nuclei classification. CNNs were trained with both hyperspectral images and pseudo RGB images of extracted nuclei, in order to evaluate the usefulness of extra information provided by hyperspectral imaging. The average accuracy of spectra-based SVM classification is 68%. The average AUC and average accuracy of the HSI patch-based CNN classification is 0.94 and 82.4%, respectively. The hyperspectral microscopic imaging and classification methods provide an automatic tool to aid pathologists in detecting SCC on histologic slides.
Collapse
Affiliation(s)
- Ling Ma
- Univ. of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080
| | - Martin Halicek
- Univ. of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080
| | - Ximing Zhou
- Univ. of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080
| | - James Dormer
- Univ. of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080
| | - Baowei Fei
- Univ. of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080.,Univ. of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, TX.,Univ. of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
45
|
Mondal SB, O'Brien CM, Bishop K, Fields RC, Margenthaler JA, Achilefu S. Repurposing Molecular Imaging and Sensing for Cancer Image-Guided Surgery. J Nucl Med 2020; 61:1113-1122. [PMID: 32303598 DOI: 10.2967/jnumed.118.220426] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2019] [Accepted: 03/05/2020] [Indexed: 12/25/2022] Open
Abstract
Gone are the days when medical imaging was used primarily to visualize anatomic structures. The emergence of molecular imaging (MI), championed by radiolabeled 18F-FDG PET, has expanded the information content derived from imaging to include pathophysiologic and molecular processes. Cancer imaging, in particular, has leveraged advances in MI agents and technology to improve the accuracy of tumor detection, interrogate tumor heterogeneity, monitor treatment response, focus surgical resection, and enable image-guided biopsy. Surgeons are actively latching on to the incredible opportunities provided by medical imaging for preoperative planning, intraoperative guidance, and postoperative monitoring. From label-free techniques to enabling cancer-selective imaging agents, image-guided surgery provides surgical oncologists and interventional radiologists both macroscopic and microscopic views of cancer in the operating room. This review highlights the current state of MI and sensing approaches available for surgical guidance. Salient features of nuclear, optical, and multimodal approaches will be discussed, including their strengths, limitations, and clinical applications. To address the increasing complexity and diversity of methods available today, this review provides a framework to identify a contrast mechanism, suitable modality, and device. Emerging low-cost, portable, and user-friendly imaging systems make the case for adopting some of these technologies as the global standard of care in surgical practice.
Collapse
Affiliation(s)
- Suman B Mondal
- Department of Radiology, Washington University, St. Louis, Missouri
| | | | - Kevin Bishop
- Department of Radiology, Washington University, St. Louis, Missouri
| | - Ryan C Fields
- Department of Surgery and Siteman Cancer Center, Washington University School of Medicine, St. Louis, Missouri
| | - Julie A Margenthaler
- Department of Surgery and Siteman Cancer Center, Washington University School of Medicine, St. Louis, Missouri
| | - Samuel Achilefu
- Department of Radiology, Washington University, St. Louis, Missouri .,Department of Biomedical Engineering, Washington University, St. Louis, Missouri; and.,Department of Biochemistry and Molecular Biophysics, Washington University, St. Louis, Missouri
| |
Collapse
|
46
|
|
47
|
Halicek M, Dormer JD, Little JV, Chen AY, Fei B. Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning. BIOMEDICAL OPTICS EXPRESS 2020; 11:1383-1400. [PMID: 32206417 PMCID: PMC7075628 DOI: 10.1364/boe.381257] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2019] [Revised: 01/24/2020] [Accepted: 01/25/2020] [Indexed: 05/05/2023]
Abstract
The performance of hyperspectral imaging (HSI) for tumor detection is investigated in ex-vivo specimens from the thyroid (N = 200) and salivary glands (N = 16) from 82 patients. Tissues were imaged with HSI in broadband reflectance and autofluorescence modes. For comparison, the tissues were imaged with two fluorescent dyes. Additionally, HSI was used to synthesize three-band RGB multiplex images to represent the human-eye response and Gaussian RGBs, which are referred to as HSI-synthesized RGB images. Using histological ground truths, deep learning algorithms were developed for tumor detection. For the classification of thyroid tumors, HSI-synthesized RGB images achieved the best performance with an AUC score of 0.90. In salivary glands, HSI had the best performance with 0.92 AUC score. This study demonstrates that HSI could aid surgeons and pathologists in detecting tumors of the thyroid and salivary glands.
Collapse
Affiliation(s)
- Martin Halicek
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080, USA
- Emory University and Georgia Institute of Technology, Department of Biomedical Engineering, Atlanta, GA 30332, USA
| | - James D. Dormer
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080, USA
| | - James V. Little
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, GA 30322, USA
| | - Amy Y. Chen
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, GA 30322, USA
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX 75080, USA
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX 75080, USA
| |
Collapse
|
48
|
Zhang Y, Wu X, He L, Meng C, Du S, Bao J, Zheng Y. Applications of hyperspectral imaging in the detection and diagnosis of solid tumors. Transl Cancer Res 2020; 9:1265-1277. [PMID: 35117471 PMCID: PMC8798535 DOI: 10.21037/tcr.2019.12.53] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2019] [Accepted: 11/28/2019] [Indexed: 11/09/2022]
Abstract
Hyperspectral imaging (HSI) is an emerging new technology in solid tumor diagnosis and detection. It incorporates traditional imaging and spectroscopy together to obtain both spatial and spectral information from tissues simultaneously in a non-invasive manner. This imaging modality is based on the principle that different tissues inherit different spectral reflectance responses that present as unique spectral fingerprints. HSI captures those composition-specific fingerprints to identify cancerous and normal tissues. It becomes a promising tool for performing tumor diagnosis and detection from the label-free histopathological examination to real-time intraoperative assistance. This review introduces the basic principles of HSI and summarizes its methodology and recent advances in solid tumor detection. In particular, the advantages of HSI applied to solid tumors are highlighted to show its potential for clinical use.
Collapse
Affiliation(s)
- Yating Zhang
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Xiaoqian Wu
- Department of Liver Surgery, Peking Union Medicine Collage Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Li He
- Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Chan Meng
- Department of Pathology, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Shunda Du
- Department of Liver Surgery, Peking Union Medicine Collage Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| | - Jie Bao
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Yongchang Zheng
- Department of Liver Surgery, Peking Union Medicine Collage Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing 100730, China
| |
Collapse
|
49
|
Zhou X, Ma L, Halicek M, Dormer J, Fei B. Development of a new polarized hyperspectral imaging microscope. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11213:1121308. [PMID: 32577044 PMCID: PMC7309561 DOI: 10.1117/12.2549676] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
In this study, we proposed and designed a transmission mode polarized hyperspectral imaging microscope (PHSIM). The hyperspectral imaging (HSI) component is based on the snapscan with a hyperspectral camera. The HSI wavelength range is from 467-700 nm. Polarized light imaging is realized by the integration of two polarizers and two liquid crystal variable retarders (LCVR), which is capable of full Stokes polarimetric imaging. The new imaging device was tested for the detection of squamous cell carcinoma (SCC) in H&E stained oral tissue slides of 8 patients. One normal area and one cancerous area on each slide are selected to make the comparison. The preliminary results indicated that the spectral curves of the Stokes vector parameters (S0, S1, S2, S3) of the normal area on the H&E stained oral tissue slides are different from those of SCC in certain wavelength range. Further work is required to apply the new polarized hyperspectral imaging microscope to a large number of patient samples and to test the PHSIM system in different cancer types.
Collapse
Affiliation(s)
- Ximing Zhou
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Ling Ma
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Martin Halicek
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - James Dormer
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
| | - Baowei Fei
- The University of Texas at Dallas, Department of Bioengineering, Richardson, TX
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX
| |
Collapse
|
50
|
Lu G, Fakurnejad S, Martin BA, van den Berg NS, van Keulen S, Nishio N, Zhu AJ, Chirita SU, Zhou Q, Gao RW, Kong CS, Fischbein N, Penta M, Colevas AD, Rosenthal EL. Predicting Therapeutic Antibody Delivery into Human Head and Neck Cancers. Clin Cancer Res 2020; 26:2582-2594. [PMID: 31980465 DOI: 10.1158/1078-0432.ccr-19-3717] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2019] [Revised: 12/20/2019] [Accepted: 01/21/2020] [Indexed: 11/16/2022]
Abstract
PURPOSE The efficacy of antibody-based therapeutics depends on successful drug delivery into solid tumors; therefore, there is a clinical need to measure intratumoral antibody distribution. This study aims to develop and validate an imaging and computation platform to directly quantify and predict antibody delivery into human head and neck cancers in a clinical study. EXPERIMENTAL DESIGN Twenty-four patients received systemic infusion of a near-infrared fluorescence-labeled therapeutic antibody followed by surgical tumor resection. A computational platform was developed to quantify the extent of heterogeneity of intratumoral antibody distribution. Both univariate and multivariate regression analyses were used to select the most predictive tumor biological factors for antibody delivery. Quantitative image features from the pretreatment MRI were extracted and correlated with fluorescence imaging of antibody delivery. RESULTS This study not only confirmed heterogeneous intratumoral antibody distribution in-line with many preclinical reports, but also quantified the extent of interpatient, intertumor, and intratumor heterogeneity of antibody delivery. This study demonstrated the strong predictive value of tumor size for intratumoral antibody accumulation and its significant impact on antibody distribution in both primary tumor and lymph node metastasis. Furthermore, this study established the feasibility of using contrast-enhanced MRI to predict antibody delivery. CONCLUSIONS This study provides a clinically translatable platform to measure antibody delivery into solid tumors and yields valuable insight into clinically relevant antibody tumor penetration, with implications in the selection of patients amenable to antibody therapy and the design of more effective dosing strategies.
Collapse
Affiliation(s)
- Guolan Lu
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | | | - Brock A Martin
- Department of Pathology, Stanford University School of Medicine, Stanford, California
| | - Nynke S van den Berg
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | - Stan van Keulen
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | - Naoki Nishio
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | - Ashley J Zhu
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | - Stefania U Chirita
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | - Quan Zhou
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California
| | - Rebecca W Gao
- Stanford University School of Medicine, Stanford, California
| | - Christina S Kong
- Department of Pathology, Stanford University School of Medicine, Stanford, California
| | - Nancy Fischbein
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| | - Mrudula Penta
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| | - Alexander D Colevas
- Department of Medicine, Division of Oncology, Stanford University School of Medicine, Stanford, California
| | - Eben L Rosenthal
- Department of Otolaryngology - Head and Neck Surgery, Stanford University School of Medicine, Stanford, California. .,Department of Radiology, Stanford University School of Medicine, Stanford, California
| |
Collapse
|