1
|
Bekedam NM, van Alphen MJA, de Cuba EMV, Karssemakers LHE, Karakullukcu MB, Smeele LE. Improved ground truth annotation by multimodal image registration from 3D ultrasound to histopathology for resected tongue carcinoma. Eur Arch Otorhinolaryngol 2025; 282:1399-1409. [PMID: 39347853 PMCID: PMC11890336 DOI: 10.1007/s00405-024-08979-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2024] [Accepted: 09/09/2024] [Indexed: 10/01/2024]
Abstract
OBJECTIVES This study's objectives are (1) to investigate the registration accuracy from intraoperative ultrasound (US) to histopathological images, (2) to assess the agreement and correlation between measurements in registered 3D US and histopathology, and (3) to train a nnUNet model for automatic segmentation of 3D US volumes of resected tongue specimens. METHODS Ten 3D US volumes were acquired, including the corresponding digitalized histopathological images (n = 29). Based on corresponding landmarks, the registrations between 3D US and histopathology images were calculated and evaluated using the target registration error (TRE). Tumor thickness and resection margins were measured based on three annotations: (1) manual histopathological tumor annotation (HTA), manual 3D US tumor annotation, and (2) the HTA registered in the 3D US. The agreement and correlation were computed between the measurements based on the HTA and those based on the manual US and registered HTA in US. A deep-learning model with nnUNet was trained on 151 3D US volumes. Segmentation metrics quantified the model's performance. RESULTS The median TRE was 0.42 mm. The smallest mean difference was between registered HTA in US and histopathology with 2.16 mm (95% CI - 1.31; 5.63) and a correlation of 0.924 (p < 0.001). The nnUNet predicted the tumor with a Dice similarity coefficient of 0.621, an average surface distance of 1.15 mm, and a Hausdorff distance of 3.70 mm. CONCLUSION Multimodal image registration enabled the HTA's registration in the US images and improved the agreement and correlation between the modalities. In the future, this could be used to annotate ground truth labels accurately.
Collapse
Affiliation(s)
- N M Bekedam
- Department of Head and Neck Surgery and Oncology, Netherlands Cancer Institute, Antoni Van Leeuwenhoek, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands.
- Academic Centre of Dentistry Amsterdam, Vrije Universiteit, Gustav Mahlerlaan 3004, 1081 LA, Amsterdam, The Netherlands.
| | - M J A van Alphen
- Department of Head and Neck Surgery and Oncology, Verwelius 3D Lab, Netherlands Cancer Institute, Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - E M V de Cuba
- Department of Pathology, Netherlands Cancer Institute, Antoni Van Leeuwenhoek, Amsterdam, The Netherlands
| | - L H E Karssemakers
- Department of Head and Neck Surgery and Oncology, Netherlands Cancer Institute, Antoni Van Leeuwenhoek, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands
| | - M B Karakullukcu
- Department of Head and Neck Surgery and Oncology, Netherlands Cancer Institute, Antoni Van Leeuwenhoek, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands
| | - L E Smeele
- Department of Head and Neck Surgery and Oncology, Netherlands Cancer Institute, Antoni Van Leeuwenhoek, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands
| |
Collapse
|
2
|
Schmidt B, Soerensen SJC, Bhambhvani HP, Fan RE, Bhattacharya I, Choi MH, Kunder CA, Kao C, Higgins J, Rusu M, Sonn GA. External validation of an artificial intelligence model for Gleason grading of prostate cancer on prostatectomy specimens. BJU Int 2025; 135:133-139. [PMID: 38989669 PMCID: PMC11628895 DOI: 10.1111/bju.16464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/12/2024]
Abstract
OBJECTIVES To externally validate the performance of the DeepDx Prostate artificial intelligence (AI) algorithm (Deep Bio Inc., Seoul, South Korea) for Gleason grading on whole-mount prostate histopathology, considering potential variations observed when applying AI models trained on biopsy samples to radical prostatectomy (RP) specimens due to inherent differences in tissue representation and sample size. MATERIALS AND METHODS The commercially available DeepDx Prostate AI algorithm is an automated Gleason grading system that was previously trained using 1133 prostate core biopsy images and validated on 700 biopsy images from two institutions. We assessed the AI algorithm's performance, which outputs Gleason patterns (3, 4, or 5), on 500 1-mm2 tiles created from 150 whole-mount RP specimens from a third institution. These patterns were then grouped into grade groups (GGs) for comparison with expert pathologist assessments. The reference standard was the International Society of Urological Pathology GG as established by two experienced uropathologists with a third expert to adjudicate discordant cases. We defined the main metric as the agreement with the reference standard, using Cohen's kappa. RESULTS The agreement between the two experienced pathologists in determining GGs at the tile level had a quadratically weighted Cohen's kappa of 0.94. The agreement between the AI algorithm and the reference standard in differentiating cancerous vs non-cancerous tissue had an unweighted Cohen's kappa of 0.91. Additionally, the AI algorithm's agreement with the reference standard in classifying tiles into GGs had a quadratically weighted Cohen's kappa of 0.89. In distinguishing cancerous vs non-cancerous tissue, the AI algorithm achieved a sensitivity of 0.997 and specificity of 0.88; in classifying GG ≥2 vs GG 1 and non-cancerous tissue, it demonstrated a sensitivity of 0.98 and specificity of 0.85. CONCLUSION The DeepDx Prostate AI algorithm had excellent agreement with expert uropathologists and performance in cancer identification and grading on RP specimens, despite being trained on biopsy specimens from an entirely different patient population.
Collapse
Affiliation(s)
- Bogdana Schmidt
- Division of Urology, Department of Surgery, Huntsman Cancer HospitalUniversity of UtahSalt Lake CityUTUSA
| | - Simon John Christoph Soerensen
- Department of UrologyStanford University School of MedicineStanfordCAUSA
- Department of Epidemiology and Population HealthStanford University School of MedicineStanfordCAUSA
| | - Hriday P. Bhambhvani
- Department of Urology, Weill Cornell Medical CollegeNew York‐Presbyterian HospitalNew YorkNYUSA
| | - Richard E. Fan
- Department of UrologyStanford University School of MedicineStanfordCAUSA
| | | | - Moon Hyung Choi
- Department of Radiology, College of Medicine, Eunpyeong St. Mary's HospitalThe Catholic University of KoreaSeoulKorea
| | | | - Chia‐Sui Kao
- Department of Pathology and Laboratory MedicineCleveland ClinicClevelandOHUSA
| | - John Higgins
- Department of PathologyStanford University School of MedicineStanfordCAUSA
| | - Mirabela Rusu
- Department of UrologyStanford University School of MedicineStanfordCAUSA
- Department of RadiologyStanford University School of MedicineStanfordCAUSA
- Department of Biomedical Data ScienceStanford UniversityStanfordCAUSA
| | - Geoffrey A. Sonn
- Department of UrologyStanford University School of MedicineStanfordCAUSA
- Department of RadiologyStanford University School of MedicineStanfordCAUSA
| |
Collapse
|
3
|
Miranda J, Heiselman JS, Firat C, Chakraborty J, Vanguri RS, Assuncao AN, Nincevic J, Kim TH, Rodriguez L, Urganci N, Gonen M, Garcia-Aguilar J, Gollub MJ, Shia J, Horvat N. Deformable Mapping of Rectal Cancer Whole-Mount Histology with Restaging MRI at Voxel Scale: A Feasibility Study. Radiol Imaging Cancer 2024; 6:e240073. [PMID: 39452890 PMCID: PMC11615632 DOI: 10.1148/rycan.240073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Revised: 07/24/2024] [Accepted: 09/09/2024] [Indexed: 10/26/2024]
Abstract
Purpose To develop a radiology-pathology coregistration method for 1:1 automated spatial mapping between preoperative rectal MRI and ex vivo rectal whole-mount histology (WMH). Materials and Methods This retrospective study included consecutive patients with rectal adenocarcinoma who underwent total neoadjuvant therapy followed by total mesorectal excision with preoperative rectal MRI and WMH from January 2019 to January 2022. A gastrointestinal pathologist and a radiologist established three corresponding levels for each patient at rectal MRI and WMH, subsequently delineating external and internal rectal wall contours and the tumor bed at each level and defining eight point-based landmarks. An advanced deformable image coregistration model based on the linearized iterative boundary reconstruction (LIBR) approach was compared with rigid point-based registration (PBR) and state-of-the-art deformable intensity-based multiscale spectral embedding registration (MSERg). Dice similarity coefficient (DSC), modified Hausdorff distance (MHD), and target registration error (TRE) across patients were calculated to assess the coregistration accuracy of each method. Results Eighteen patients (mean age, 54 years ± 13 [SD]; nine female) were included. LIBR demonstrated higher DSC versus PBR for external and internal rectal wall contours and tumor bed (external: 0.95 ± 0.03 vs 0.86 ± 0.04, respectively, P < .001; internal: 0.71 ± 0.21 vs 0.61 ± 0.21, P < .001; tumor bed: 0.61 ± 0.17 vs 0.52 ± 0.17, P = .001) and versus MSERg for internal rectal wall contours (0.71 ± 0.21 vs 0.63 ± 0.18, respectively; P < .001). LIBR demonstrated lower MHD versus PBR for external and internal rectal wall contours and tumor bed (external: 0.56 ± 0.25 vs 1.68 ± 0.56, respectively, P < .001; internal: 1.00 ± 0.35 vs 1.62 ± 0.59, P < .001; tumor bed: 2.45 ± 0.99 vs 2.69 ± 1.05, P = .03) and versus MSERg for internal rectal wall contours (1.00 ± 0.35 vs 1.62 ± 0.59, respectively; P < .001). LIBR demonstrated lower TRE (1.54 ± 0.39) versus PBR (2.35 ± 1.19, P = .003) and MSERg (2.36 ± 1.43, P = .03). Computation time per WMH slice for LIBR was 35.1 seconds ± 12.1. Conclusion This study demonstrates feasibility of accurate MRI-WMH coregistration using the advanced LIBR method. Keywords: MR Imaging, Abdomen/GI, Rectum, Oncology Supplemental material is available for this article. © RSNA, 2024.
Collapse
Affiliation(s)
| | | | - Canan Firat
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Jayasree Chakraborty
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Rami S. Vanguri
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Antonildes N. Assuncao
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Josip Nincevic
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Tae-Hyung Kim
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Lee Rodriguez
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Nil Urganci
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Mithat Gonen
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Julio Garcia-Aguilar
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Marc J. Gollub
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Jinru Shia
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| | - Natally Horvat
- From the Departments of Radiology (J.M., J.N., T.H.K., L.R., M.J.G.,
N.H.), Surgery (J.S.H., J.C., M.G., J.G.A.), and Pathology (C.F., N.U., J.S.),
Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10065;
Department of Radiology, University of São Paulo, São Paulo,
Brazil (J.M., A.N.A., N.H.); Department of Medicine, Division of Precision
Medicine, NYU Grossman School of Medicine, New York, NY (R.S.V.); Department of
Biomedical Engineering, Vanderbilt University, Nashville, Tenn (J.S.H.);
Research and Education Institute, Hospital Sirio-Libanes, São Paulo,
Brazil (A.N.A.); and Department of Radiology, Mayo Clinic, Rochester, Minn
(J.M., N.H.)
| |
Collapse
|
4
|
Shao W, Vesal S, Soerensen SJC, Bhattacharya I, Golestani N, Yamashita R, Kunder CA, Fan RE, Ghanouni P, Brooks JD, Sonn GA, Rusu M. RAPHIA: A deep learning pipeline for the registration of MRI and whole-mount histopathology images of the prostate. Comput Biol Med 2024; 173:108318. [PMID: 38522253 PMCID: PMC11077621 DOI: 10.1016/j.compbiomed.2024.108318] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 02/14/2024] [Accepted: 03/12/2024] [Indexed: 03/26/2024]
Abstract
Image registration can map the ground truth extent of prostate cancer from histopathology images onto MRI, facilitating the development of machine learning methods for early prostate cancer detection. Here, we present RAdiology PatHology Image Alignment (RAPHIA), an end-to-end pipeline for efficient and accurate registration of MRI and histopathology images. RAPHIA automates several time-consuming manual steps in existing approaches including prostate segmentation, estimation of the rotation angle and horizontal flipping in histopathology images, and estimation of MRI-histopathology slice correspondences. By utilizing deep learning registration networks, RAPHIA substantially reduces computational time. Furthermore, RAPHIA obviates the need for a multimodal image similarity metric by transferring histopathology image representations to MRI image representations and vice versa. With the assistance of RAPHIA, novice users achieved expert-level performance, and their mean error in estimating histopathology rotation angle was reduced by 51% (12 degrees vs 8 degrees), their mean accuracy of estimating histopathology flipping was increased by 5% (95.3% vs 100%), and their mean error in estimating MRI-histopathology slice correspondences was reduced by 45% (1.12 slices vs 0.62 slices). When compared to a recent conventional registration approach and a deep learning registration approach, RAPHIA achieved better mapping of histopathology cancer labels, with an improved mean Dice coefficient of cancer regions outlined on MRI and the deformed histopathology (0.44 vs 0.48 vs 0.50), and a reduced mean per-case processing time (51 vs 11 vs 4.5 min). The improved performance by RAPHIA allows efficient processing of large datasets for the development of machine learning models for prostate cancer detection on MRI. Our code is publicly available at: https://github.com/pimed/RAPHIA.
Collapse
Affiliation(s)
- Wei Shao
- Department of Radiology, Stanford University, Stanford, CA, 94305, United States; Department of Medicine, University of Florida, Gainesville, FL, 32610, United States.
| | - Sulaiman Vesal
- Department of Urology, Stanford University, Stanford, CA, 94305, United States
| | - Simon J C Soerensen
- Department of Urology, Stanford University, Stanford, CA, 94305, United States; Department of Epidemiology and Population Health, Stanford University, Stanford, CA, 94305, United States
| | - Indrani Bhattacharya
- Department of Radiology, Stanford University, Stanford, CA, 94305, United States
| | - Negar Golestani
- Department of Radiology, Stanford University, Stanford, CA, 94305, United States
| | - Rikiya Yamashita
- Department of Biomedical Data Science, Stanford University, Stanford, CA, 94305, United States
| | - Christian A Kunder
- Department of Pathology, Stanford University, Stanford, CA, 94305, United States
| | - Richard E Fan
- Department of Urology, Stanford University, Stanford, CA, 94305, United States
| | - Pejman Ghanouni
- Department of Radiology, Stanford University, Stanford, CA, 94305, United States
| | - James D Brooks
- Department of Urology, Stanford University, Stanford, CA, 94305, United States
| | - Geoffrey A Sonn
- Department of Radiology, Stanford University, Stanford, CA, 94305, United States; Department of Urology, Stanford University, Stanford, CA, 94305, United States
| | - Mirabela Rusu
- Department of Radiology, Stanford University, Stanford, CA, 94305, United States.
| |
Collapse
|
5
|
Li L, Shiradkar R, Tirumani SH, Bittencourt LK, Fu P, Mahran A, Buzzy C, Stricker PD, Rastinehad AR, Magi-Galluzzi C, Ponsky L, Klein E, Purysko AS, Madabhushi A. Novel radiomic analysis on bi-parametric MRI for characterizing differences between MR non-visible and visible clinically significant prostate cancer. Eur J Radiol Open 2023; 10:100496. [PMID: 37396490 PMCID: PMC10311200 DOI: 10.1016/j.ejro.2023.100496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 06/01/2023] [Accepted: 06/04/2023] [Indexed: 07/04/2023] Open
Abstract
Background around one third of clinically significant prostate cancer (CsPCa) foci are reported to be MRI non-visible (MRI─). Objective To quantify the differences between MR visible (MRI+) and MRI─ CsPCa using intra- and peri-lesional radiomic features on bi-parametric MRI (bpMRI). Methods This retrospective and multi-institutional study comprised 164 patients with pre-biopsy 3T prostate multi-parametric MRI from 2014 to 2017. The MRI─ CsPCa referred to lesions with PI-RADS v2 score < 3 but ISUP grade group > 1. Three experienced radiologists were involved in annotating lesions and PI-RADS assignment. The validation set (Dv) comprised 52 patients from a single institution, the remaining 112 patients were used for training (Dt). 200 radiomic features were extracted from intra-lesional and peri-lesional regions on bpMRI.Logistic regression with least absolute shrinkage and selection operator (LASSO) and 10-fold cross-validation was applied on Dt to identify radiomic features associated with MRI─ and MRI+ CsPCa to generate corresponding risk scores RMRI─ and RMRI+. RbpMRI was further generated by integrating RMRI─ and RMRI+. Statistical significance was determined using the Wilcoxon signed-rank test. Results Both intra-lesional and peri-lesional bpMRI Haralick and CoLlAGe radiomic features were significantly associated with MRI─ CsPCa (p < 0.05). Intra-lesional ADC Haralick and CoLlAGe radiomic features were significantly different among MRI─ and MRI+ CsPCa (p < 0.05). RbpMRI yielded the highest AUC of 0.82 (95 % CI 0.72-0.91) compared to AUCs of RMRI+ 0.76 (95 % CI 0.63-0.89), and PI-RADS 0.58 (95 % CI 0.50-0.72) on Dv. RbpMRI correctly reclassified 10 out of 14 MRI─ CsPCa on Dv. Conclusion Our preliminary results demonstrated that both intra-lesional and peri-lesional bpMRI radiomic features were significantly associated with MRI─ CsPCa. These features could assist in CsPCa identification on bpMRI.
Collapse
Affiliation(s)
- Lin Li
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, OH, USA
| | - Rakesh Shiradkar
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, OH, USA
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology
| | | | | | - Pingfu Fu
- Department of Population and Quantitative Health Sciences, Case Western Reserve University, Cleveland, OH, USA
| | - Amr Mahran
- Urology Institute, University Hospitals Cleveland Medical Center, Cleveland, OH, USA
| | - Christina Buzzy
- Center for Computational Imaging and Personalized Diagnostics, Case Western Reserve University, Cleveland, OH, USA
| | | | | | | | - Lee Ponsky
- Urology Institute, University Hospitals Cleveland Medical Center, Cleveland, OH, USA
- Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Eric Klein
- Glickman Urological and Kidney Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Andrei S. Purysko
- Glickman Urological and Kidney Institute, Cleveland Clinic, Cleveland, OH, USA
- Imaging Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology
- Atlanta Veterans Affairs Medical Center, Atlanta, GA, United States
| |
Collapse
|
6
|
Bhattacharya I, Khandwala YS, Vesal S, Shao W, Yang Q, Soerensen SJ, Fan RE, Ghanouni P, Kunder CA, Brooks JD, Hu Y, Rusu M, Sonn GA. A review of artificial intelligence in prostate cancer detection on imaging. Ther Adv Urol 2022; 14:17562872221128791. [PMID: 36249889 PMCID: PMC9554123 DOI: 10.1177/17562872221128791] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Accepted: 08/30/2022] [Indexed: 11/07/2022] Open
Abstract
A multitude of studies have explored the role of artificial intelligence (AI) in providing diagnostic support to radiologists, pathologists, and urologists in prostate cancer detection, risk-stratification, and management. This review provides a comprehensive overview of relevant literature regarding the use of AI models in (1) detecting prostate cancer on radiology images (magnetic resonance and ultrasound imaging), (2) detecting prostate cancer on histopathology images of prostate biopsy tissue, and (3) assisting in supporting tasks for prostate cancer detection (prostate gland segmentation, MRI-histopathology registration, MRI-ultrasound registration). We discuss both the potential of these AI models to assist in the clinical workflow of prostate cancer diagnosis, as well as the current limitations including variability in training data sets, algorithms, and evaluation criteria. We also discuss ongoing challenges and what is needed to bridge the gap between academic research on AI for prostate cancer and commercial solutions that improve routine clinical care.
Collapse
Affiliation(s)
- Indrani Bhattacharya
- Department of Radiology, Stanford University School of Medicine, 1201 Welch Road, Stanford, CA 94305, USA
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| | - Yash S. Khandwala
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| | - Sulaiman Vesal
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| | - Wei Shao
- Department of Radiology, Stanford University School of Medicine, Stanford, CA, USA
| | - Qianye Yang
- Centre for Medical Image Computing, University College London, London, UK
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Simon J.C. Soerensen
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
- Department of Epidemiology & Population Health, Stanford University School of Medicine, Stanford, CA, USA
| | - Richard E. Fan
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| | - Pejman Ghanouni
- Department of Radiology, Stanford University School of Medicine, Stanford, CA, USA
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| | - Christian A. Kunder
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - James D. Brooks
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| | - Yipeng Hu
- Centre for Medical Image Computing, University College London, London, UK
- Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Mirabela Rusu
- Department of Radiology, Stanford University School of Medicine, Stanford, CA, USA
| | - Geoffrey A. Sonn
- Department of Radiology, Stanford University School of Medicine, Stanford, CA, USA
- Department of Urology, Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
7
|
Selective identification and localization of indolent and aggressive prostate cancers via CorrSigNIA: an MRI-pathology correlation and deep learning framework. Med Image Anal 2022; 75:102288. [PMID: 34784540 PMCID: PMC8678366 DOI: 10.1016/j.media.2021.102288] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2020] [Revised: 09/02/2021] [Accepted: 10/20/2021] [Indexed: 01/03/2023]
Abstract
Automated methods for detecting prostate cancer and distinguishing indolent from aggressive disease on Magnetic Resonance Imaging (MRI) could assist in early diagnosis and treatment planning. Existing automated methods of prostate cancer detection mostly rely on ground truth labels with limited accuracy, ignore disease pathology characteristics observed on resected tissue, and cannot selectively identify aggressive (Gleason Pattern≥4) and indolent (Gleason Pattern=3) cancers when they co-exist in mixed lesions. In this paper, we present a radiology-pathology fusion approach, CorrSigNIA, for the selective identification and localization of indolent and aggressive prostate cancer on MRI. CorrSigNIA uses registered MRI and whole-mount histopathology images from radical prostatectomy patients to derive accurate ground truth labels and learn correlated features between radiology and pathology images. These correlated features are then used in a convolutional neural network architecture to detect and localize normal tissue, indolent cancer, and aggressive cancer on prostate MRI. CorrSigNIA was trained and validated on a dataset of 98 men, including 74 men that underwent radical prostatectomy and 24 men with normal prostate MRI. CorrSigNIA was tested on three independent test sets including 55 men that underwent radical prostatectomy, 275 men that underwent targeted biopsies, and 15 men with normal prostate MRI. CorrSigNIA achieved an accuracy of 80% in distinguishing between men with and without cancer, a lesion-level ROC-AUC of 0.81±0.31 in detecting cancers in both radical prostatectomy and biopsy cohort patients, and lesion-levels ROC-AUCs of 0.82±0.31 and 0.86±0.26 in detecting clinically significant cancers in radical prostatectomy and biopsy cohort patients respectively. CorrSigNIA consistently outperformed other methods across different evaluation metrics and cohorts. In clinical settings, CorrSigNIA may be used in prostate cancer detection as well as in selective identification of indolent and aggressive components of prostate cancer, thereby improving prostate cancer care by helping guide targeted biopsies, reducing unnecessary biopsies, and selecting and planning treatment.
Collapse
|
8
|
Zimmerman BE, Johnson SL, Odéen HA, Shea JE, Factor RE, Joshi SC, Payne AH. Histology to 3D in vivo MR registration for volumetric evaluation of MRgFUS treatment assessment biomarkers. Sci Rep 2021; 11:18923. [PMID: 34556678 PMCID: PMC8460731 DOI: 10.1038/s41598-021-97309-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Accepted: 08/24/2021] [Indexed: 11/09/2022] Open
Abstract
Advances in imaging and early cancer detection have increased interest in magnetic resonance (MR) guided focused ultrasound (MRgFUS) technologies for cancer treatment. MRgFUS ablation treatments could reduce surgical risks, preserve organ tissue and function, and improve patient quality of life. However, surgical resection and histological analysis remain the gold standard to assess cancer treatment response. For non-invasive ablation therapies such as MRgFUS, the treatment response must be determined through MR imaging biomarkers. However, current MR biomarkers are inconclusive and have not been rigorously evaluated against histology via accurate registration. Existing registration methods rely on anatomical features to directly register in vivo MR and histology. For MRgFUS applications in anatomies such as liver, kidney, or breast, anatomical features that are not caused by the treatment are often insufficient to drive direct registration. We present a novel MR to histology registration workflow that utilizes intermediate imaging and does not rely on anatomical MR features being visible in histology. The presented workflow yields an overall registration accuracy of 1.00 ± 0.13 mm. The developed registration pipeline is used to evaluate a common MRgFUS treatment assessment biomarker against histology. Evaluating MR biomarkers against histology using this registration pipeline will facilitate validating novel MRgFUS biomarkers to improve treatment assessment without surgical intervention. While the presented registration technique has been evaluated in a MRgFUS ablation treatment model, this technique could be potentially applied in any tissue to evaluate a variety of therapeutic options.
Collapse
Affiliation(s)
- Blake E Zimmerman
- Department of Biomedical Engineering, University of Utah, Salt Lake City, UT, USA. .,Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA.
| | - Sara L Johnson
- Department of Biomedical Engineering, University of Utah, Salt Lake City, UT, USA.,Utah Center for Advanced Imaging Research, University of Utah, Salt Lake City, UT, USA
| | - Henrik A Odéen
- Utah Center for Advanced Imaging Research, University of Utah, Salt Lake City, UT, USA
| | - Jill E Shea
- Department of Surgery, University of Utah, Salt Lake City, UT, USA
| | - Rachel E Factor
- Department of Pathology, University of Utah, Salt Lake City, UT, USA
| | - Sarang C Joshi
- Department of Biomedical Engineering, University of Utah, Salt Lake City, UT, USA.,Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA
| | - Allison H Payne
- Utah Center for Advanced Imaging Research, University of Utah, Salt Lake City, UT, USA
| |
Collapse
|
9
|
Sandgren K, Nilsson E, Keeratijarut Lindberg A, Strandberg S, Blomqvist L, Bergh A, Friedrich B, Axelsson J, Ögren M, Ögren M, Widmark A, Thellenberg Karlsson C, Söderkvist K, Riklund K, Jonsson J, Nyholm T. Registration of histopathology to magnetic resonance imaging of prostate cancer. Phys Imaging Radiat Oncol 2021; 18:19-25. [PMID: 34258403 PMCID: PMC8254194 DOI: 10.1016/j.phro.2021.03.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2020] [Revised: 02/16/2021] [Accepted: 03/25/2021] [Indexed: 11/30/2022] Open
Abstract
BACKGROUND AND PURPOSE The diagnostic accuracy of new imaging techniques requires validation, preferably by histopathological verification. The aim of this study was to develop and present a registration procedure between histopathology and in-vivo magnetic resonance imaging (MRI) of the prostate, to estimate its uncertainty and to evaluate the benefit of adding a contour-correcting registration. MATERIALS AND METHODS For twenty-five prostate cancer patients, planned for radical prostatectomy, a 3D-printed prostate mold based on in-vivo MRI was created and an ex-vivo MRI of the specimen, placed inside the mold, was performed. Each histopathology slice was registered to its corresponding ex-vivo MRI slice using a 2D-affine registration. The ex-vivo MRI was rigidly registered to the in-vivo MRI and the resulting transform was applied to the histopathology stack. A 2D deformable registration was used to correct for specimen distortion concerning the specimen's fit inside the mold. We estimated the spatial uncertainty by comparing positions of landmarks in the in-vivo MRI and the corresponding registered histopathology stack. RESULTS Eighty-four landmarks were identified, located in the urethra (62%), prostatic cysts (33%), and the ejaculatory ducts (5%). The median number of landmarks was 3 per patient. We showed a median in-plane error of 1.8 mm before and 1.7 mm after the contour-correcting deformable registration. In patients with extraprostatic margins, the median in-plane error improved from 2.1 mm to 1.8 mm after the contour-correcting deformable registration. CONCLUSIONS Our registration procedure accurately registers histopathology to in-vivo MRI, with low uncertainty. The contour-correcting registration was beneficial in patients with extraprostatic surgical margins.
Collapse
Affiliation(s)
- Kristina Sandgren
- Department of Radiation Sciences, Radiophysics, Umea University, Sweden
| | - Erik Nilsson
- Department of Radiation Sciences, Radiophysics, Umea University, Sweden
| | | | - Sara Strandberg
- Department of Radiation Sciences, Diagnostic Radiology, Umea University, Sweden
| | - Lennart Blomqvist
- Department of Molecular Medicine and Surgery, Karolinska Institute, Stockholm, Sweden
| | - Anders Bergh
- Department of Medical Biosciences, Pathology, Umea University, Sweden
| | - Bengt Friedrich
- Department of Surgical and Perioperative Sciences, Urology and Andrology, Umea University, Sweden
| | - Jan Axelsson
- Department of Radiation Sciences, Radiophysics, Umea University, Sweden
| | - Margareta Ögren
- Department of Radiation Sciences, Diagnostic Radiology, Umea University, Sweden
| | - Mattias Ögren
- Department of Radiation Sciences, Diagnostic Radiology, Umea University, Sweden
| | - Anders Widmark
- Department of Radiation Sciences, Oncology, Umea University, Sweden
| | | | - Karin Söderkvist
- Department of Radiation Sciences, Oncology, Umea University, Sweden
| | - Katrine Riklund
- Department of Radiation Sciences, Diagnostic Radiology, Umea University, Sweden
| | - Joakim Jonsson
- Department of Radiation Sciences, Radiophysics, Umea University, Sweden
| | - Tufve Nyholm
- Department of Radiation Sciences, Radiophysics, Umea University, Sweden
| |
Collapse
|
10
|
Liu JTC, Glaser AK, Bera K, True LD, Reder NP, Eliceiri KW, Madabhushi A. Harnessing non-destructive 3D pathology. Nat Biomed Eng 2021; 5:203-218. [PMID: 33589781 PMCID: PMC8118147 DOI: 10.1038/s41551-020-00681-x] [Citation(s) in RCA: 70] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2020] [Accepted: 12/17/2020] [Indexed: 02/08/2023]
Abstract
High-throughput methods for slide-free three-dimensional (3D) pathological analyses of whole biopsies and surgical specimens offer the promise of modernizing traditional histology workflows and delivering improvements in diagnostic performance. Advanced optical methods now enable the interrogation of orders of magnitude more tissue than previously possible, where volumetric imaging allows for enhanced quantitative analyses of cell distributions and tissue structures that are prognostic and predictive. Non-destructive imaging processes can simplify laboratory workflows, potentially reducing costs, and can ensure that samples are available for subsequent molecular assays. However, the large size of the feature-rich datasets that they generate poses challenges for data management and computer-aided analysis. In this Perspective, we provide an overview of the imaging technologies that enable 3D pathology, and the computational tools-machine learning, in particular-for image processing and interpretation. We also discuss the integration of various other diagnostic modalities with 3D pathology, along with the challenges and opportunities for clinical adoption and regulatory approval.
Collapse
Affiliation(s)
- Jonathan T C Liu
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA.
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA.
- Department of Bioengineering, University of Washington, Seattle, WA, USA.
| | - Adam K Glaser
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
| | - Kaustav Bera
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA
| | - Lawrence D True
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA
| | - Nicholas P Reder
- Department of Mechanical Engineering, University of Washington, Seattle, WA, USA
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA
| | - Kevin W Eliceiri
- Department of Medical Physics, University of Wisconsin, Madison, WI, USA.
- Department of Biomedical Engineering, University of Wisconsin, Madison, WI, USA.
- Morgridge Institute for Research, Madison, WI, USA.
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA.
- Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA.
| |
Collapse
|
11
|
Shao W, Banh L, Kunder CA, Fan RE, Soerensen SJC, Wang JB, Teslovich NC, Madhuripan N, Jawahar A, Ghanouni P, Brooks JD, Sonn GA, Rusu M. ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate. Med Image Anal 2021; 68:101919. [PMID: 33385701 PMCID: PMC7856244 DOI: 10.1016/j.media.2020.101919] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2020] [Revised: 11/18/2020] [Accepted: 11/23/2020] [Indexed: 12/21/2022]
Abstract
Magnetic resonance imaging (MRI) is an increasingly important tool for the diagnosis and treatment of prostate cancer. However, interpretation of MRI suffers from high inter-observer variability across radiologists, thereby contributing to missed clinically significant cancers, overdiagnosed low-risk cancers, and frequent false positives. Interpretation of MRI could be greatly improved by providing radiologists with an answer key that clearly shows cancer locations on MRI. Registration of histopathology images from patients who had radical prostatectomy to pre-operative MRI allows such mapping of ground truth cancer labels onto MRI. However, traditional MRI-histopathology registration approaches are computationally expensive and require careful choices of the cost function and registration hyperparameters. This paper presents ProsRegNet, a deep learning-based pipeline to accelerate and simplify MRI-histopathology image registration in prostate cancer. Our pipeline consists of image preprocessing, estimation of affine and deformable transformations by deep neural networks, and mapping cancer labels from histopathology images onto MRI using estimated transformations. We trained our neural network using MR and histopathology images of 99 patients from our internal cohort (Cohort 1) and evaluated its performance using 53 patients from three different cohorts (an additional 12 from Cohort 1 and 41 from two public cohorts). Results show that our deep learning pipeline has achieved more accurate registration results and is at least 20 times faster than a state-of-the-art registration algorithm. This important advance will provide radiologists with highly accurate prostate MRI answer keys, thereby facilitating improvements in the detection of prostate cancer on MRI. Our code is freely available at https://github.com/pimed//ProsRegNet.
Collapse
Affiliation(s)
- Wei Shao
- Department of Radiology, Stanford University, Stanford, CA 94305, USA.
| | - Linda Banh
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA
| | | | - Richard E Fan
- Department of Urology, Stanford University, Stanford, CA 94305, USA
| | | | - Jeffrey B Wang
- School of Medicine, Stanford University, Stanford, CA 94305, USA
| | | | - Nikhil Madhuripan
- Department of Radiology, University of Colorado, Aurora, CO 80045, USA
| | | | - Pejman Ghanouni
- Department of Radiology, Stanford University, Stanford, CA 94305, USA
| | - James D Brooks
- Department of Urology, Stanford University, Stanford, CA 94305, USA
| | - Geoffrey A Sonn
- Department of Radiology, Stanford University, Stanford, CA 94305, USA; Department of Urology, Stanford University, Stanford, CA 94305, USA
| | - Mirabela Rusu
- Department of Radiology, Stanford University, Stanford, CA 94305, USA.
| |
Collapse
|
12
|
Sood RR, Shao W, Kunder C, Teslovich NC, Wang JB, Soerensen SJC, Madhuripan N, Jawahar A, Brooks JD, Ghanouni P, Fan RE, Sonn GA, Rusu M. 3D Registration of pre-surgical prostate MRI and histopathology images via super-resolution volume reconstruction. Med Image Anal 2021; 69:101957. [PMID: 33550008 DOI: 10.1016/j.media.2021.101957] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Revised: 12/23/2020] [Accepted: 01/04/2021] [Indexed: 12/15/2022]
Abstract
The use of MRI for prostate cancer diagnosis and treatment is increasing rapidly. However, identifying the presence and extent of cancer on MRI remains challenging, leading to high variability in detection even among expert radiologists. Improvement in cancer detection on MRI is essential to reducing this variability and maximizing the clinical utility of MRI. To date, such improvement has been limited by the lack of accurately labeled MRI datasets. Data from patients who underwent radical prostatectomy enables the spatial alignment of digitized histopathology images of the resected prostate with corresponding pre-surgical MRI. This alignment facilitates the delineation of detailed cancer labels on MRI via the projection of cancer from histopathology images onto MRI. We introduce a framework that performs 3D registration of whole-mount histopathology images to pre-surgical MRI in three steps. First, we developed a novel multi-image super-resolution generative adversarial network (miSRGAN), which learns information useful for 3D registration by producing a reconstructed 3D MRI. Second, we trained the network to learn information between histopathology slices to facilitate the application of 3D registration methods. Third, we registered the reconstructed 3D histopathology volumes to the reconstructed 3D MRI, mapping the extent of cancer from histopathology images onto MRI without the need for slice-to-slice correspondence. When compared to interpolation methods, our super-resolution reconstruction resulted in the highest PSNR relative to clinical 3D MRI (32.15 dB vs 30.16 dB for BSpline interpolation). Moreover, the registration of 3D volumes reconstructed via super-resolution for both MRI and histopathology images showed the best alignment of cancer regions when compared to (1) the state-of-the-art RAPSODI approach, (2) volumes that were not reconstructed, or (3) volumes that were reconstructed using nearest neighbor, linear, or BSpline interpolations. The improved 3D alignment of histopathology images and MRI facilitates the projection of accurate cancer labels on MRI, allowing for the development of improved MRI interpretation schemes and machine learning models to automatically detect cancer on MRI.
Collapse
Affiliation(s)
- Rewa R Sood
- Department of Electrical Engineering, Stanford University, 350 Jane Stanford Way, Stanford, CA 94305, USA
| | - Wei Shao
- Department of Radiology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Christian Kunder
- Department of Pathology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Nikola C Teslovich
- Department of Urology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Jeffrey B Wang
- Stanford School of Medicine, 291 Campus Drive, Stanford, CA 94305, USA
| | - Simon J C Soerensen
- Department of Urology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA; Department of Urology, Aarhus University Hospital, Aarhus, Denmark
| | - Nikhil Madhuripan
- Department of Radiology, University of Colorado, Aurora, CO 80045, USA
| | | | - James D Brooks
- Department of Urology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Pejman Ghanouni
- Department of Radiology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Richard E Fan
- Department of Urology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Geoffrey A Sonn
- Department of Radiology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA; Department of Urology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA
| | - Mirabela Rusu
- Department of Radiology, Stanford University, 300 Pasteur Drive, Stanford, CA 94305, USA.
| |
Collapse
|
13
|
Alyami W, Kyme A, Bourne R. Histological Validation of MRI: A Review of Challenges in Registration of Imaging and Whole-Mount Histopathology. J Magn Reson Imaging 2020; 55:11-22. [PMID: 33128424 DOI: 10.1002/jmri.27409] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2020] [Revised: 10/07/2020] [Accepted: 10/09/2020] [Indexed: 12/20/2022] Open
Abstract
Rigorous validation with ground truth information such as histology is needed to reliably assess the current and potential value of MRI techniques to characterize tissue and identify disease-related tissue alterations. Commonly used methods that aim to directly correlate histology and MRI data generally fall short of this goal due to spatial errors that preclude direct matching. Errors result from tissue deformation, differences in spatial resolution and slice thickness, non-coplanar and/or nonintersecting plane orientations, and different image contrast mechanisms. Some of these problems arise from limitations in standard protocols for clinical tissue processing and histology-based pathology reporting, and to some extent can be addressed by modifications to standard protocols without compromising the clinical process. Typical modifications include ex vivo specimen MRI, block-face photography, addition of fiducial markers, and 3D printed molds to constrain tissue deformation and guide sectioning. This review summarizes the advantages and limitations of MRI validation techniques based on coregistration of MRI with whole-mount histology of tissue specimens. LEVEL OF EVIDENCE: 4 TECHNICAL EFFICACY STAGE: 1.
Collapse
Affiliation(s)
- Wadha Alyami
- Discipline of Medical Imaging Science, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.,Discipline of Medical Imaging Science, Faculty of Applied Medical Sciences, King Saud University, Riyadh, Saudi Arabia
| | - Andre Kyme
- School of Biomedical Engineering, Faculty of Engineering and IT, The University of Sydney, Sydney, New South Wales, Australia
| | - Roger Bourne
- Discipline of Medical Imaging Science, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
14
|
T1 and T2 MR fingerprinting measurements of prostate cancer and prostatitis correlate with deep learning-derived estimates of epithelium, lumen, and stromal composition on corresponding whole mount histopathology. Eur Radiol 2020; 31:1336-1346. [PMID: 32876839 DOI: 10.1007/s00330-020-07214-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 06/10/2020] [Accepted: 08/20/2020] [Indexed: 01/16/2023]
Abstract
OBJECTIVES To explore the associations between T1 and T2 magnetic resonance fingerprinting (MRF) measurements and corresponding tissue compartment ratios (TCRs) on whole mount histopathology of prostate cancer (PCa) and prostatitis. MATERIALS AND METHODS A retrospective, IRB-approved, HIPAA-compliant cohort consisting of 14 PCa patients who underwent 3 T multiparametric MRI along with T1 and T2 MRF maps prior to radical prostatectomy was used. Correspondences between whole mount specimens and MRI and MRF were manually established. Prostatitis, PCa, and normal peripheral zone (PZ) regions of interest (ROIs) on pathology were segmented for TCRs of epithelium, lumen, and stroma using two U-net deep learning models. Corresponding ROIs were mapped to T2-weighted MRI (T2w), apparent diffusion coefficient (ADC), and T1 and T2 MRF maps. Their correlations with TCRs were computed using Pearson's correlation coefficient (R). Statistically significant differences in means were assessed using one-way ANOVA. RESULTS Statistically significant differences (p < 0.01) in means of TCRs and T1 and T2 MRF were observed between PCa, prostatitis, and normal PZ. A negative correlation was observed between T1 and T2 MRF and epithelium (R = - 0.38, - 0.44, p < 0.05) of PCa. T1 MRF was correlated in opposite directions with stroma of PCa and prostatitis (R = 0.35, - 0.44, p < 0.05). T2 MRF was positively correlated with lumen of PCa and prostatitis (R = 0.57, 0.46, p < 0.01). Mean T2 MRF showed significant differences (p < 0.01) between PCa and prostatitis across both transition zone (TZ) and PZ, while mean T1 MRF was significant (p = 0.02) in TZ. CONCLUSION Significant associations between MRF (T1 in the TZ and T2 in the PZ) and tissue compartments on corresponding histopathology were observed. KEY POINTS • Mean T2 MRF measurements and ADC within cancerous regions of interest dropped with increasing ISUP prognostic groups (IPG). • Mean T1 and T2 MRF measurements were significantly different (p < 0.001) across IPGs, prostatitis, and normal peripheral zone (NPZ). • T2 MRF showed stronger correlations in the peripheral zone, while T1 MRF showed stronger correlations in the transition zone with histopathology for prostate cancer.
Collapse
|
15
|
An Automated Segmentation Pipeline for Intratumoural Regions in Animal Xenografts Using Machine Learning and Saturation Transfer MRI. Sci Rep 2020; 10:8063. [PMID: 32415137 PMCID: PMC7228927 DOI: 10.1038/s41598-020-64912-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2020] [Accepted: 04/24/2020] [Indexed: 11/16/2022] Open
Abstract
Saturation transfer MRI can be useful in the characterization of different tumour types. It is sensitive to tumour metabolism, microstructure, and microenvironment. This study aimed to use saturation transfer to differentiate between intratumoural regions, demarcate tumour boundaries, and reduce data acquisition times by identifying the imaging scheme with the most impact on segmentation accuracy. Saturation transfer-weighted images were acquired over a wide range of saturation amplitudes and frequency offsets along with T1 and T2 maps for 34 tumour xenografts in mice. Independent component analysis and Gaussian mixture modelling were used to segment the images and identify intratumoural regions. Comparison between the segmented regions and histopathology indicated five distinct clusters: three corresponding to intratumoural regions (active tumour, necrosis/apoptosis, and blood/edema) and two extratumoural (muscle and a mix of muscle and connective tissue). The fraction of tumour voxels segmented as necrosis/apoptosis quantitatively matched those calculated from TUNEL histopathological assays. An optimal protocol was identified providing reasonable qualitative agreement between MRI and histopathology and consisting of T1 and T2 maps and 22 magnetization transfer (MT)-weighted images. A three-image subset was identified that resulted in a greater than 90% match in positive and negative predictive value of tumour voxels compared to those found using the entire 24-image dataset. The proposed algorithm can potentially be used to develop a robust intratumoural segmentation method.
Collapse
|
16
|
Correlation between MRI phenotypes and a genomic classifier of prostate cancer: preliminary findings. Eur Radiol 2019; 29:4861-4870. [PMID: 30847589 DOI: 10.1007/s00330-019-06114-x] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Revised: 02/07/2019] [Accepted: 02/15/2019] [Indexed: 10/27/2022]
Abstract
OBJECTIVES We sought to evaluate the correlation between MRI phenotypes of prostate cancer as defined by PI-RADS v2 and the Decipher Genomic Classifier (used to estimate the risk of early metastases). METHODS This single-center, retrospective study included 72 nonconsecutive men with prostate cancer who underwent MRI before radical prostatectomy performed between April 2014 and August 2017 and whose MRI registered lesions were microdissected from radical prostatectomy specimens and then profiled using Decipher (89 lesions; 23 MRI invisible [PI-RADS v2 scores ≤ 2] and 66 MRI visible [PI-RADS v2 scores ≥ 3]). Linear regression analysis was used to assess clinicopathologic and MRI predictors of Decipher results; correlation coefficients (r) were used to quantify these associations. AUC was used to determine whether PI-RADS v2 could accurately distinguish between low-risk (Decipher score < 0.45) and intermediate-/high-risk (Decipher score ≥ 0.45) lesions. RESULTS MRI-visible lesions had higher Decipher scores than MRI-invisible lesions (mean difference 0.22; 95% CI 0.13, 0.32; p < 0.0001); most MRI-invisible lesions (82.6%) were low risk. PI-RADS v2 had moderate correlation with Decipher (r = 0.54) and had higher accuracy (AUC 0.863) than prostate cancer grade groups (AUC 0.780) in peripheral zone lesions (95% CI for difference 0.01, 0.15; p = 0.018). CONCLUSIONS MRI phenotypes of prostate cancer are positively correlated with Decipher risk groups. Although PI-RADS v2 can accurately distinguish between lesions classified by Decipher as low or intermediate/high risk, some lesions classified as intermediate/high risk by Decipher are invisible on MRI. KEY POINTS • MRI phenotypes of prostate cancer as defined by PI-RADS v2 positively correlated with a genomic classifier that estimates the risk of early metastases. • Most but not all MRI-invisible lesions had a low risk for early metastases according to the genomic classifier. • MRI could be used in conjunction with genomic assays to identify lesions that may carry biological potential for early metastases.
Collapse
|