1
|
Norose T, Ohike N, Nakaya D, Kamiya K, Sugiura Y, Takatsuki M, Koizumi H, Okawa C, Ohya A, Sasaki M, Aoki R, Nakahara K, Kobayashi S, Tateishi K, Koike J. Investigation of the usefulness of a bile duct biopsy and bile cytology using a hyperspectral camera and machine learning. Pathol Int 2024. [PMID: 38787324 DOI: 10.1111/pin.13438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 04/15/2024] [Accepted: 05/04/2024] [Indexed: 05/25/2024]
Abstract
To improve the efficiency of pathological diagnoses, the development of automatic pathological diagnostic systems using artificial intelligence (AI) is progressing; however, problems include the low interpretability of AI technology and the need for large amounts of data. We herein report the usefulness of a general-purpose method that combines a hyperspectral camera with machine learning. As a result of analyzing bile duct biopsy and bile cytology specimens, which are especially difficult to determine as benign or malignant, using multiple machine learning models, both were able to identify benign or malignant cells with an accuracy rate of more than 80% (93.3% for bile duct biopsy specimens and 83.2% for bile cytology specimens). This method has the potential to contribute to the diagnosis and treatment of bile duct cancer and is expected to be widely applied and utilized in general pathological diagnoses.
Collapse
Affiliation(s)
- Tomoko Norose
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Nobuyuki Ohike
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | | | | | - Yoshiya Sugiura
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Misato Takatsuki
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Hirotaka Koizumi
- Division of Molecular Pathology, Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Chie Okawa
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Aya Ohya
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Miyu Sasaki
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Ruka Aoki
- Division of Pathology, St. Marianna University Hospital, Kawasaki, Japan
| | - Kazunari Nakahara
- Department of Gastroenterology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Shinjiro Kobayashi
- Department of Gastroenterological and General Surgery, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Keisuke Tateishi
- Department of Gastroenterology, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Junki Koike
- Department of Pathology, St. Marianna University School of Medicine, Kawasaki, Japan
| |
Collapse
|
2
|
Jiang S, Ma D, Tan X, Yang M, Jiao Q, Xu L. Bibliometric analysis of the current status and trends on medical hyperspectral imaging. Front Med (Lausanne) 2023; 10:1235955. [PMID: 37795419 PMCID: PMC10545955 DOI: 10.3389/fmed.2023.1235955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 08/30/2023] [Indexed: 10/06/2023] Open
Abstract
Hyperspectral imaging (HSI) is a promising technology that can provide valuable support for the advancement of the medical field. Bibliometrics can analyze a vast number of publications on both macroscopic and microscopic levels, providing scholars with essential foundations to shape future directions. The purpose of this study is to comprehensively review the existing literature on medical hyperspectral imaging (MHSI). Based on the Web of Science (WOS) database, this study systematically combs through literature using bibliometric methods and visualization software such as VOSviewer and CiteSpace to draw scientific conclusions. The analysis yielded 2,274 articles from 73 countries/regions, involving 7,401 authors, 2,037 institutions, 1,038 journals/conferences, and a total of 7,522 keywords. The field of MHSI is currently in a positive stage of development and has conducted extensive research worldwide. This research encompasses not only HSI technology but also its application to diverse medical research subjects, such as skin, cancer, tumors, etc., covering a wide range of hardware constructions and software algorithms. In addition to advancements in hardware, the future should focus on the development of algorithm standards for specific medical research targets and cultivate medical professionals of managing vast amounts of technical information.
Collapse
Affiliation(s)
| | | | | | | | | | - Liang Xu
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin,China
| |
Collapse
|
3
|
Zhao Y, Kusama S, Furutani Y, Huang WH, Luo CW, Fuji T. High-speed scanless entire bandwidth mid-infrared chemical imaging. Nat Commun 2023; 14:3929. [PMID: 37402722 DOI: 10.1038/s41467-023-39628-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 06/19/2023] [Indexed: 07/06/2023] Open
Abstract
Mid-infrared spectroscopy probes molecular vibrations to identify chemical species and functional groups. Therefore, mid-infrared hyperspectral imaging is one of the most powerful and promising candidates for chemical imaging using optical methods. Yet high-speed and entire bandwidth mid-infrared hyperspectral imaging has not been realized. Here we report a mid-infrared hyperspectral chemical imaging technique that uses chirped pulse upconversion of sub-cycle pulses at the image plane. This technique offers a lateral resolution of 15 µm, and the field of view is adjustable between 800 µm × 600 µm to 12 mm × 9 mm. The hyperspectral imaging produces a 640 × 480 pixel image in 8 s, which covers a spectral range of 640-3015 cm-1, comprising 1069 wavelength points and offering a wavenumber resolution of 2.6-3.7 cm-1. For discrete frequency mid-infrared imaging, the measurement speed reaches a frame rate of 5 kHz, the repetition rate of the laser. As a demonstration, we effectively identified and mapped different components in a microfluidic device, plant cell, and mouse embryo section. The great capacity and latent force of this technique in chemical imaging promise to be applied to many fields such as chemical analysis, biology, and medicine.
Collapse
Affiliation(s)
- Yue Zhao
- Laser Science Laboratory, Toyota Technological Institute, 2-12-1 Hisakata, Tempaku-ku, Nagoya, 468-8511, Japan.
- Graduate School of Engineering College of Design and Manufacturing Technology, Muroran Institute of Technology, 27-1 Mizumoto-cho, Muroran, Hokkaido, 050-8585, Japan.
| | - Shota Kusama
- Laser Science Laboratory, Toyota Technological Institute, 2-12-1 Hisakata, Tempaku-ku, Nagoya, 468-8511, Japan
| | - Yuji Furutani
- Department of Life Science and Applied Chemistry, Nagoya Institute of Technology, Showa-Ku, Nagoya, 466-8555, Japan
- Optobiotechnology Research Center, Nagoya Institute of Technology, Showa-Ku, Nagoya, 466-8555, Japan
| | - Wei-Hong Huang
- Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu, 30010, Taiwan
| | - Chih-Wei Luo
- Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu, 30010, Taiwan
| | - Takao Fuji
- Laser Science Laboratory, Toyota Technological Institute, 2-12-1 Hisakata, Tempaku-ku, Nagoya, 468-8511, Japan.
| |
Collapse
|
4
|
Spectral Similarity Measures for In Vivo Human Tissue Discrimination Based on Hyperspectral Imaging. Diagnostics (Basel) 2023; 13:diagnostics13020195. [PMID: 36673005 PMCID: PMC9857871 DOI: 10.3390/diagnostics13020195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 12/08/2022] [Accepted: 12/21/2022] [Indexed: 01/06/2023] Open
Abstract
PROBLEM Similarity measures are widely used as an approved method for spectral discrimination or identification with their applications in different areas of scientific research. Even though a range of works have been presented, only a few showed slightly promising results for human tissue, and these were mostly focused on pathological and non-pathological tissue classification. METHODS In this work, several spectral similarity measures on hyperspectral (HS) images of in vivo human tissue were evaluated for tissue discrimination purposes. Moreover, we introduced two new hybrid spectral measures, called SID-JM-TAN(SAM) and SID-JM-TAN(SCA). We analyzed spectral signatures obtained from 13 different human tissue types and two different materials (gauze, instruments), collected from HS images of 100 patients during surgeries. RESULTS The quantitative results showed the reliable performance of the different similarity measures and the proposed hybrid measures for tissue discrimination purposes. The latter produced higher discrimination values, up to 6.7 times more than the classical spectral similarity measures. Moreover, an application of the similarity measures was presented to support the annotations of the HS images. We showed that the automatic checking of tissue-annotated thyroid and colon tissues was successful in 73% and 60% of the total spectra, respectively. The hybrid measures showed the highest performance. Furthermore, the automatic labeling of wrongly annotated tissues was similar for all measures, with an accuracy of up to 90%. CONCLUSION In future work, the proposed spectral similarity measures will be integrated with tools to support physicians in annotations and tissue labeling of HS images.
Collapse
|
5
|
Zhang L, Huang D, Chen X, Zhu L, Chen X, Xie Z, Huang G, Gao J, Shi W, Cui G. Visible near-infrared hyperspectral imaging and supervised classification for the detection of small intestinal necrosis tissue in vivo. BIOMEDICAL OPTICS EXPRESS 2022; 13:6061-6080. [PMID: 36733734 PMCID: PMC9872898 DOI: 10.1364/boe.470202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 09/23/2022] [Accepted: 09/26/2022] [Indexed: 06/18/2023]
Abstract
Complete recognition of necrotic areas during small bowel tissue resection remains challenging due to the lack of optimal intraoperative aid identification techniques. This research utilizes hyperspectral imaging techniques to automatically distinguish normal and necrotic areas of small intestinal tissue. Sample data were obtained from the animal model of small intestinal tissue of eight Japanese large-eared white rabbits developed by experienced physicians. A spectral library of normal and necrotic regions of small intestinal tissue was created and processed using six different supervised classification algorithms. The results show that hyperspectral imaging combined with supervised classification algorithms can be a suitable technique to automatically distinguish between normal and necrotic areas of small intestinal tissue. This new technique could aid physicians in objectively identify normal and necrotic areas of small intestinal tissue.
Collapse
Affiliation(s)
- LeChao Zhang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - DanFei Huang
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - XiaoJing Chen
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - LiBin Zhu
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - XiaoQing Chen
- Pediatric General Surgery, The Second Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, 325000, China
| | - ZhongHao Xie
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuangZao Huang
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - JunZhao Gao
- College of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun, Jilin, 130000, China
- Zhongshan Research Institute, Changchun University of Science and Technology, Zhongshan, Guangdong, 528400, China
| | - Wen Shi
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| | - GuiHua Cui
- College of Electrical and Electronic Engineering, Wenzhou University, Wenzhou, Zhejiang, 325000, China
| |
Collapse
|
6
|
Ma L, Rathgeb A, Mubarak H, Tran M, Fei B. Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:056502. [PMID: 35578386 PMCID: PMC9110022 DOI: 10.1117/1.jbo.27.5.056502] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 04/28/2022] [Indexed: 06/15/2023]
Abstract
SIGNIFICANCE Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space. AIM WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics. APPROACH High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection. RESULTS Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results. CONCLUSIONS The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.
Collapse
Affiliation(s)
- Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Tianjin University, State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin, China
| | - Armand Rathgeb
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Hasan Mubarak
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Minh Tran
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- University of Texas at Dallas, Center for Imaging and Surgical Innovation, Richardson, Texas, United States
- University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
7
|
Ma L, Little JV, Chen AY, Myers L, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:046501. [PMID: 35484692 PMCID: PMC9050479 DOI: 10.1117/1.jbo.27.4.046501] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 04/05/2022] [Indexed: 06/14/2023]
Abstract
SIGNIFICANCE Automatic, fast, and accurate identification of cancer on histologic slides has many applications in oncologic pathology. AIM The purpose of this study is to investigate hyperspectral imaging (HSI) for automatic detection of head and neck cancer nuclei in histologic slides, as well as cancer region identification based on nuclei detection. APPROACH A customized hyperspectral microscopic imaging system was developed and used to scan histologic slides from 20 patients with squamous cell carcinoma (SCC). Hyperspectral images and red, green, and blue (RGB) images of the histologic slides with the same field of view were obtained and registered. A principal component analysis-based nuclei segmentation method was developed to extract nuclei patches from the hyperspectral images and the coregistered RGB images. Spectra-based support vector machine and patch-based convolutional neural networks (CNNs) were implemented for nuclei classification. The CNNs were trained with RGB patches (RGB-CNN) and hyperspectral patches (HSI-CNN) of the segmented nuclei and the utility of the extra spectral information provided by HSI was evaluated. Furthermore, cancer region identification was implemented by image-wise classification based on the percentage of cancerous nuclei detected in each image. RESULTS RGB-CNN, which mainly used the spatial information of nuclei, resulted in a 0.81 validation accuracy and 0.74 testing accuracy. HSI-CNN, which utilized the spatial and spectral features of the nuclei, showed significant improvement in classification performance and achieved 0.89 validation accuracy as well as 0.82 testing accuracy. Furthermore, the image-wise cancer region identification based on nuclei detection could generally improve the cancer detection rate. CONCLUSIONS We demonstrated that the morphological and spectral information contribute to SCC nuclei differentiation and that the spectral information within hyperspectral images could improve classification performance.
Collapse
Affiliation(s)
- Ling Ma
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- Tianjin University, State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin, China
| | - James V. Little
- Emory University School of Medicine, Department of Pathology and Laboratory Medicine, Atlanta, Georgia, United States
| | - Amy Y. Chen
- Emory University School of Medicine, Department of Otolaryngology, Atlanta, Georgia, United States
| | - Larry Myers
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Baran D. Sumer
- The University of Texas Southwestern Medical Center, Department of Otolaryngology, Dallas, Texas, United States
| | - Baowei Fei
- University of Texas at Dallas, Department of Bioengineering, Richardson, Texas, United States
- The University of Texas Southwestern Medical Center, Advanced Imaging Research Center, Dallas, Texas, United States
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, Texas, United States
| |
Collapse
|
8
|
Tumor cell identification and classification in esophageal adenocarcinoma specimens by hyperspectral imaging. Sci Rep 2022; 12:4508. [PMID: 35296685 PMCID: PMC8927097 DOI: 10.1038/s41598-022-07524-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 02/17/2022] [Indexed: 12/24/2022] Open
Abstract
Esophageal cancer is the sixth leading cause of cancer-related death worldwide. Histopathological confirmation is a key step in tumor diagnosis. Therefore, simplification in decision-making by discrimination between malignant and non-malignant cells of histological specimens can be provided by combination of new imaging technology and artificial intelligence (AI). In this work, hyperspectral imaging (HSI) data from 95 patients were used to classify three different histopathological features (squamous epithelium cells, esophageal adenocarcinoma (EAC) cells, and tumor stroma cells), based on a multi-layer perceptron with two hidden layers. We achieved an accuracy of 78% for EAC and stroma cells, and 80% for squamous epithelium. HSI combined with machine learning algorithms is a promising and innovative technique, which allows image acquisition beyond Red–Green–Blue (RGB) images. Further method validation and standardization will be necessary, before automated tumor cell identification algorithms can be used in daily clinical practice.
Collapse
|
9
|
Knospe L, Gockel I, Jansen-Winkeln B, Thieme R, Niebisch S, Moulla Y, Stelzner S, Lyros O, Diana M, Marescaux J, Chalopin C, Köhler H, Pfahl A, Maktabi M, Park JH, Yang HK. New Intraoperative Imaging Tools and Image-Guided Surgery in Gastric Cancer Surgery. Diagnostics (Basel) 2022; 12:diagnostics12020507. [PMID: 35204597 PMCID: PMC8871069 DOI: 10.3390/diagnostics12020507] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Revised: 02/06/2022] [Accepted: 02/10/2022] [Indexed: 02/05/2023] Open
Abstract
Innovations and new advancements in intraoperative real-time imaging have gained significant importance in the field of gastric cancer surgery in the recent past. Currently, the most promising procedures include indocyanine green fluorescence imaging (ICG-FI) and hyperspectral imaging or multispectral imaging (HSI, MSI). ICG-FI is utilized in a broad range of clinical applications, e.g., assessment of perfusion or lymphatic drainage, and additional implementations are currently investigated. HSI is still in the experimental phase and its value and clinical relevance require further evaluation, but initial studies have shown a successful application in perfusion assessment, and prospects concerning non-invasive tissue and tumor classification are promising. The application of machine learning and artificial intelligence technologies might enable an automatic evaluation of the acquired image data in the future. Both methods facilitate the accurate visualization of tissue characteristics that are initially indistinguishable for the human eye. By aiding surgeons in optimizing the surgical procedure, image-guided surgery can contribute to the oncologic safety and reduction of complications in gastric cancer surgery and recent advances hold promise for the application of HSI in intraoperative tissue diagnostics.
Collapse
Affiliation(s)
- Luise Knospe
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Ines Gockel
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
- Correspondence:
| | - Boris Jansen-Winkeln
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
- Department of General, Visceral and Oncological Surgery, St. Georg Hospital, 04129 Leipzig, Germany
| | - René Thieme
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Stefan Niebisch
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Yusef Moulla
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Sigmar Stelzner
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Orestis Lyros
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig AöR, 04103 Leipzig, Germany; (L.K.); (B.J.-W.); (R.T.); (S.N.); (Y.M.); (S.S.); (O.L.)
| | - Michele Diana
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.D.); (J.M.)
- ICUBE Laboratory, Photonics Instrumentation for Health, University of Strasbourg, 67400 Strasbourg, France
- Department of General, Digestive, and Endocrine Surgery, University Hospital of Strasbourg, 67091 Strasbourg, France
| | - Jacques Marescaux
- Institute for Research against Digestive Cancer (IRCAD), 67091 Strasbourg, France; (M.D.); (J.M.)
| | - Claire Chalopin
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Hannes Köhler
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Annekatrin Pfahl
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Marianne Maktabi
- Innovation Center Computer Assisted Surgery (ICCAS), Leipzig University, 04103 Leipzig, Germany; (C.C.); (H.K.); (A.P.); (M.M.)
| | - Ji-Hyeon Park
- Department of Surgery, Seoul National University Hospital, Seoul 03080, Korea; (J.-H.P.); (H.-K.Y.)
| | - Han-Kwang Yang
- Department of Surgery, Seoul National University Hospital, Seoul 03080, Korea; (J.-H.P.); (H.-K.Y.)
| |
Collapse
|
10
|
Ma L, Zhou X, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Hyperspectral Microscopic Imaging for the Detection of Head and Neck Squamous Cell Carcinoma on Histologic Slides. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2021; 11603:116030P. [PMID: 35783088 PMCID: PMC9248908 DOI: 10.1117/12.2581970] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
The purpose of this study is to investigate hyperspectral microscopic imaging and deep learning methods for automatic detection of head and neck squamous cell carcinoma (SCC) on histologic slides. Hyperspectral imaging (HSI) cubes were acquired from pathologic slides of 18 patients with SCC of the larynx, hypopharynx, and buccal mucosa. An Inception-based two-dimensional convolutional neural network (CNN) was trained and validated for the HSI data. The automatic deep learning method was tested with independent data of human patients. This study demonstrated the feasibility of using hyperspectral microscopic imaging and deep learning classification to aid pathologists in detecting SCC on histologic slides.
Collapse
Affiliation(s)
- Ling Ma
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas
- State Key Laboratory of Precision Measurement Technology and Instrument, Tianjin University
| | - Ximing Zhou
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas
| | - James V. Little
- Department of Pathology and Laboratory Medicine, Emory University
| | - Amy Y. Chen
- Department of Otolaryngology, Emory University
| | - Larry L. Myers
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baran D. Sumer
- Department of Otolaryngology, University of Texas Southwestern Medical Center
| | - Baowei Fei
- Center for Imaging and Surgical Innovation and Department of Bioengineering, University of Texas at Dallas
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center
- Department of Radiology, University of Texas Southwestern Medical Center
| |
Collapse
|
11
|
Liu K, Lin S, Zhu S, Chen Y, Yin H, Li Z, Chen Z. Hyperspectral microscopy combined with DAPI staining for the identification of hepatic carcinoma cells. BIOMEDICAL OPTICS EXPRESS 2021; 12:173-180. [PMID: 33659073 PMCID: PMC7899502 DOI: 10.1364/boe.412158] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Revised: 11/29/2020] [Accepted: 11/30/2020] [Indexed: 06/12/2023]
Abstract
In this study, the DAPI staining is firstly reported for use in the identification of hepatic carcinoma cells based on hyperspectral microscopy. Nuclei in cancer cells usually contain more aneuploidies than that in normal cells, leading to the change of DNA content. Here, we stain hepatic carcinoma tissues and normal hepatic tissues with 4',6-diamidino-2-phenylindole (DAPI) which is sensitive to the DNA content as a fluorochrome binds to DNA. Consequently, the difference in DNA content between hepatic carcinoma cells and normal hepatic cells can be identified by the fluorescent spectral characteristics. Harnessing the hyperspectral microscopy, we find that the fluorescent properties of these two kinds of cells are different not only in the intensity but also in the spectral shape. These properties are exploited to train a support vector machine (SVM) model for classifying cells. The results show that the sensitivity and specificity for the identification of 1000 hepatic carcinoma samples are 99.3% and 99.1%, respectively.
Collapse
Affiliation(s)
- Kunxing Liu
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
| | - Sifan Lin
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
| | - Siqi Zhu
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
- Guangdong Provincial Key Laboratory of Optical Fiber Sensing and Communications, Guangzhou 510632, China
| | - Yao Chen
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
| | - Hao Yin
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
| | - Zhen Li
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
- Guangdong Provincial Key Laboratory of Optical Fiber Sensing and Communications, Guangzhou 510632, China
| | - Zhenqiang Chen
- Guangdong Provincial Engineering Research Center of Crystal and Laser Technology, Guangzhou 510632, China
- Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China
- Guangdong Provincial Key Laboratory of Optical Fiber Sensing and Communications, Guangzhou 510632, China
| |
Collapse
|
12
|
Xu Z, Jiang Y, Ji J, Forsberg E, Li Y, He S. Classification, identification, and growth stage estimation of microalgae based on transmission hyperspectral microscopic imaging and machine learning. OPTICS EXPRESS 2020; 28:30686-30700. [PMID: 33115064 DOI: 10.1364/oe.406036] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
A transmission hyperspectral microscopic imager (THMI) that utilizes machine learning algorithms for hyperspectral detection of microalgae is presented. The THMI system has excellent performance with spatial and spectral resolutions of 4 µm and 3 nm, respectively. We performed hyperspectral imaging (HSI) of three species of microalgae to verify their absorption characteristics. Transmission spectra were analyzed using principal component analysis (PCA) and peak ratio algorithms for dimensionality reduction and feature extraction, and a support vector machine (SVM) model was used for classification. The average accuracy, sensitivity and specificity to distinguish one species from the other two species were found to be 94.4%, 94.4% and 97.2%, respectively. A species identification experiment for a group of mixed microalgae in solution demonstrates the usability of the classification method. Using a random forest (RF) model, the growth stage in a phaeocystis growth cycle cultivated under laboratory conditions was predicted with an accuracy of 98.1%, indicating the feasibility to evaluate the growth state of microalgae through their transmission spectra. Experimental results show that the THMI system has the capability for classification, identification and growth stage estimation of microalgae, with strong potential for in-situ marine environmental monitoring and early warning detection applications.
Collapse
|
13
|
Multi-mode Microscopic Hyperspectral Imager for the Sensing of Biological Samples. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10144876] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
In this work, we develop a multi-mode microscopic hyperspectral imager (MMHI) for the detection of biological samples in transmission imaging, reflection imaging and fluorescence mode. A hyperspectral image cube can be obtained with 5 μm spatial resolution and 3 nm spectral resolution through push-broom line scanning. To avoid possible shadows produced by the high magnification objective with a short working distance, two illumination patterns are designed to ensure the co-axiality of the illumination and detection. Three experiments for the detection of zebrafish and fingerprints and the classification of disaster-causing microalgae verify the good capability and functionality of the system. Based on the detected spectra, we can observe the impacts of β-carotene and melanin in zebrafish, hemoglobin in the fingertip, and chlorophyll in microalgae, respectively. Multi-modes can be switched freely according to the application requirement and characteristics of different samples, like transmission mode for the transparent/translucent sample, reflection mode for the opaque sample and fluorescence mode for the fluorescent sample. The MMHI system also has strong potential for the non-invasive and high-speed sensing of bio or clinical samples.
Collapse
|
14
|
Ortega S, Halicek M, Fabelo H, Camacho R, Plaza MDLL, Godtliebsen F, M. Callicó G, Fei B. Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks. SENSORS (BASEL, SWITZERLAND) 2020; 20:E1911. [PMID: 32235483 PMCID: PMC7181269 DOI: 10.3390/s20071911] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/06/2020] [Revised: 03/24/2020] [Accepted: 03/28/2020] [Indexed: 12/18/2022]
Abstract
Hyperspectral imaging (HSI) technology has demonstrated potential to provide useful information about the chemical composition of tissue and its morphological features in a single image modality. Deep learning (DL) techniques have demonstrated the ability of automatic feature extraction from data for a successful classification. In this study, we exploit HSI and DL for the automatic differentiation of glioblastoma (GB) and non-tumor tissue on hematoxylin and eosin (H&E) stained histological slides of human brain tissue. GB detection is a challenging application, showing high heterogeneity in the cellular morphology across different patients. We employed an HSI microscope, with a spectral range from 400 to 1000 nm, to collect 517 HS cubes from 13 GB patients using 20× magnification. Using a convolutional neural network (CNN), we were able to automatically detect GB within the pathological slides, achieving average sensitivity and specificity values of 88% and 77%, respectively, representing an improvement of 7% and 8% respectively, as compared to the results obtained using RGB (red, green, and blue) images. This study demonstrates that the combination of hyperspectral microscopic imaging and deep learning is a promising tool for future computational pathologies.
Collapse
Affiliation(s)
- Samuel Ortega
- Quantitative Bioimaging Laboratory, Department of Bioengineering, The University of Texas at Dallas, Richardson, TX 75080, USA;
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (H.F.); (G.M.C.)
| | - Martin Halicek
- Quantitative Bioimaging Laboratory, Department of Bioengineering, The University of Texas at Dallas, Richardson, TX 75080, USA;
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, 1841 Clifton Road NE, Atlanta, GA 30329, USA
| | - Himar Fabelo
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (H.F.); (G.M.C.)
| | - Rafael Camacho
- Department of Pathological Anatomy, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain; (R.C.); (M.d.l.L.P.)
| | - María de la Luz Plaza
- Department of Pathological Anatomy, University Hospital Doctor Negrin of Gran Canaria, Barranco de la Ballena s/n, 35010 Las Palmas de Gran Canaria, Spain; (R.C.); (M.d.l.L.P.)
| | - Fred Godtliebsen
- Department of Mathematics and Statistics, UiT The Artic, University of Norway, Hansine Hansens veg 18, 9019 Tromsø, Norway;
| | - Gustavo M. Callicó
- Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain; (H.F.); (G.M.C.)
| | - Baowei Fei
- Quantitative Bioimaging Laboratory, Department of Bioengineering, The University of Texas at Dallas, Richardson, TX 75080, USA;
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, 5323 Harry Hine Blvd, Dallas, TX 75390, USA
- Department of Radiology, University of Texas Southwestern Medical Center, 5323 Harry Hine Blvd, Dallas, TX 75390, USA
| |
Collapse
|