Face recognition from research brain PET: An unexpected PET problem

Report of the Medical Image De-Identification (MIDI) Task Group -- Best Practices and Recommendations

This report addresses the technical aspects of de-identification of medical images of human subjects and biospecimens, such that re-identification risk of ethical, moral, and legal concern is sufficiently reduced to allow unrestricted public sharing for any purpose, regardless of the jurisdiction of the source and distribution sites. All medical images, regardless of the mode of acquisition, are considered, though the primary emphasis is on those with accompanying data elements, especially those encoded in formats in which the data elements are embedded, particularly Digital Imaging and Communications in Medicine (DICOM). These images include image-like objects such as Segmentations, Parametric Maps, and Radiotherapy (RT) Dose objects. The scope also includes related non-image objects, such as RT Structure Sets, Plans and Dose Volume Histograms, Structured Reports, and Presentation States. Only de-identification of publicly released data is considered, and alternative approaches to privacy preservation, such as federated learning for artificial intelligence (AI) model development, are out of scope, as are issues of privacy leakage from AI model sharing. Only technical issues of public sharing are addressed.

Summary of the National Cancer Institute 2023 Virtual Workshop on Medical Image De-identification-Part 2: Pathology Whole Slide Image De-identification, De-facing, the Role of AI in Image De-identification, and the NCI MIDI Datasets and Pipeline

De-identification of medical images intended for research is a core requirement for data sharing initiatives, particularly as the demand for data for artificial intelligence (AI) applications grows. The Center for Biomedical Informatics and Information Technology (CBIIT) of the United States National Cancer Institute (NCI) convened a two half-day virtual workshop with the intent of summarizing the state of the art in de-identification technology and processes and exploring interesting aspects of the subject. This paper summarizes the highlights of the second day of the workshop, the recordings and presentations of which are publicly available for review. The topics covered included pathology whole slide image de-identification, de-facing, the role of AI in image de-identification, and the NCI Medical Image De-Identification Initiative (MIDI) datasets and pipeline.

POINTER Study Group and for the Alzheimer's Disease Neuroimaging Initiative. Positron emission tomography harmonization in the Alzheimer's Disease Neuroimaging Initiative: A scalable and rigorous approach to multisite amyloid and tau quantification

INTRODUCTION

A key goal of the Alzheimer's Disease NeuroImaging Initiative (ADNI) positron emission tomography (PET) Core is to harmonize quantification of β-amyloid (Aβ) and tau PET image data across multiple scanners and tracers.

METHODS

We developed an analysis pipeline (Berkeley PET Imaging Pipeline, B-PIP) for ADNI Aβ and tau PET images and applied it to PET data from other multisite studies. Steps include image pre-processing, refacing, magnetic resonance imaging (MRI)/PET co-registration, visual quality control (QC), quantification of tracer uptake, and standardization of Aβ and tau standardized uptake value ratios (SUVrs) across tracers.

RESULTS

Measurements from 10,105 cross-sectional and longitudinal Aβ and tau PET scans acquired in several studies between 2010 and 2024 can be processed, harmonized, and directly merged across tracers and cohorts.

DISCUSSION

The B-PIP developed in ADNI is a scalable image harmonization approach used in several observational studies and clinical trials that facilitates rigorous Aβ and tau PET quantification and data sharing.

HIGHLIGHTS

Quantitative results from ADNI Aβ and tau PET data are generated using a rigorous, scalable image processing pipeline This pipeline has been applied to PET data from several other large, multisite studies and trials Quantitative outcomes are harmonizable across studies and are shared with the scientific community.

Re-identification of anonymised MRI head images with publicly available software: investigation of the current risk to patient privacy

Background

Facial recognition software (FRS) has historically been perceived as lacking the capability to identify individuals from cross-sectional medical images. Utilising such data for identification purposes was considered infeasible due to the substantial computational power and specialised technical expertise it would require. However, recent advancements in accessible artificial intelligence-based (AI-based) software and open-source tools have made these applications widely available and easy to use, raising new privacy concerns.

Methods

This proof-of-concept was designed as a cross-sectional study and included participants with a verified online presence. Standard magnetic resonance imaging (MRI) head scans were performed on these participants, from which three-dimensional rendering (3DR) images were created using free and publicly available software. These images were used for face searches by free and publicly available FRS. Different head orientations and hairstyles were applied to the 3DR images to assess whether non-facial features influenced the FRS results. All results were obtained between the 10th of February 2024 and the 1st of March 2024.

Findings

Face searches of 3DR images in a database containing over 800 million images from the World Wide Web (WWW) yielded correct matches for 50% of the participants in less than 10 min. The user-friendly software required minimal computational knowledge or resources, making this process broadly accessible. Modifying elements such as hairstyles or the orientation of the 3DR to better resemble actual photographs of the participants improved FRS matches.

Interpretation

Current existing FRS can swiftly and accurately identify individuals from MRI head scans. This poses a significant privacy risk for participants in enrolled clinical trials and highlights the urgent need for improved data protection measures and increased sensitivity to ensure participant confidentiality.

Funding

There was no funding source for this study.

Implementation and validation of face de-identification (de-facing) in ADNI4

INTRODUCTION

Recent technological advances have increased the risk that de-identified brain images could be re-identified from face imagery. The Alzheimer's Disease Neuroimaging Initiative (ADNI) is a leading source of publicly available de-identified brain imaging, who quickly acted to protect participants' privacy.

METHODS

An independent expert committee evaluated 11 face-deidentification ("de-facing") methods and selected four for formal testing.

RESULTS

Effects of de-facing on brain measurements were comparable across methods and sufficiently small to recommend de-facing in ADNI. The committee ultimately recommended mri_reface for advantages in reliability, and for some practical considerations. ADNI leadership approved the committee's recommendation, beginning in ADNI4.

DISCUSSION

ADNI4 de-faces all applicable brain images before subsequent pre-processing, analyses, and public release. Trained analysts inspect de-faced images to confirm complete face removal and complete non-modification of brain. This paper details the history of the algorithm selection process and extensive validation, then describes the production workflows for de-facing in ADNI.

HIGHLIGHTS

ADNI is implementing "de-facing" of MRI and PET beginning in ADNI4. "De-facing" alters face imagery in brain images to help protect privacy. Four algorithms were extensively compared for ADNI and mri_reface was chosen. Validation confirms mri_reface is robust and effective for ADNI sequences. Validation confirms mri_reface negligibly affects ADNI brain measurements.

Deep learning-based defacing tool for CT angiography: CTA-DEFACE

The growing use of artificial neural network (ANN) tools for computed tomography angiography (CTA) data analysis underscores the necessity for elevated data protection measures. We aimed to establish an automated defacing pipeline for CTA data. In this retrospective study, CTA data from multi-institutional cohorts were utilized to annotate facemasks (n = 100) and train an ANN model, subsequently tested on an external institution's dataset (n = 50) and compared to a publicly available defacing algorithm. Face detection (MTCNN) and verification (FaceNet) networks were applied to measure the similarity between the original and defaced CTA images. Dice similarity coefficient (DSC), face detection probability, and face similarity measures were calculated to evaluate model performance. The CTA-DEFACE model effectively segmented soft face tissue in CTA data achieving a DSC of 0.94 ± 0.02 (mean ± standard deviation) on the test set. Our model was benchmarked against a publicly available defacing algorithm. After applying face detection and verification networks, our model showed substantially reduced face detection probability (p < 0.001) and similarity to the original CTA image (p < 0.001). The CTA-DEFACE model enabled robust and precise defacing of CTA data. The trained network is publicly accessible at www.github.com/neuroAI-HD/CTA-DEFACE . RELEVANCE STATEMENT: The ANN model CTA-DEFACE, developed for automatic defacing of CT angiography images, achieves significantly lower face detection probabilities and greater dissimilarity from the original images compared to a publicly available model. The algorithm has been externally validated and is publicly accessible. KEY POINTS: The developed ANN model (CTA-DEFACE) automatically generates facemasks for CT angiography images. CTA-DEFACE offers superior deidentification capabilities compared to a publicly available model. By means of graphics processing unit optimization, our model ensures rapid processing of medical images. Our model underwent external validation, underscoring its reliability for real-world application.

Exploring de-anonymization risks in PET imaging: Insights from a comprehensive analysis of 853 patient scans

Due to their high resolution, anonymized CT scans can be reidentified using face recognition tools. However, little is known regarding PET deanonymization because of its lower resolution. In this study, we analysed PET/CT scans of 853 patients from a TCIA-restricted dataset (AutoPET). First, we built denoised 2D morphological reconstructions of both PET and CT scans, and then we determined how frequently a PET reconstruction could be matched to the correct CT reconstruction with no other metadata. Using the CT morphological reconstructions as ground truth allows us to frame the problem as a face recognition problem and to quantify our performance using traditional metrics (top k accuracies) without any use of patient pictures. Using our denoised PET 2D reconstructions, we achieved 72% top 10 accuracy after the realignment of all CTs in the same reference frame, and 71% top 10 accuracy after realignment and mixing within a larger face dataset of 10, 168 pictures. This highlights the need to consider face identification issues when dealing with PET imaging data.

Assessing the Impact of Defacing Algorithms on Brain Volumetry Accuracy in MRI Analyses

Background and Purpose

To ensure data privacy, the development of defacing processes, which anonymize brain images by obscuring facial features, is crucial. However, the impact of these defacing methods on brain imaging analysis poses significant concern. This study aimed to evaluate the reliability of three different defacing methods in automated brain volumetry.

Methods

Magnetic resonance imaging with three-dimensional T1 sequences was performed on ten patients diagnosed with subjective cognitive decline. Defacing was executed using mri_deface, BioImage Suite Web-based defacing, and Defacer. Brain volumes were measured employing the QBraVo program and FreeSurfer, assessing intraclass correlation coefficient (ICC) and the mean differences in brain volume measurements between the original and defaced images.

Results

The mean age of the patients was 71.10±6.17 years, with 4 (40.0%) being male. The total intracranial volume, total brain volume, and ventricle volume exhibited high ICCs across the three defacing methods and 2 volumetry analyses. All regional brain volumes showed high ICCs with all three defacing methods. Despite variations among some brain regions, no significant mean differences in regional brain volume were observed between the original and defaced images across all regions.

Conclusions

The three defacing algorithms evaluated did not significantly affect the results of image analysis for the entire brain or specific cerebral regions. These findings suggest that these algorithms can serve as robust methods for defacing in neuroimaging analysis, thereby supporting data anonymization without compromising the integrity of brain volume measurements.

Fast refacing of MR images with a generative neural network lowers re-identification risk and preserves volumetric consistency

With the rise of open data, identifiability of individuals based on 3D renderings obtained from routine structural magnetic resonance imaging (MRI) scans of the head has become a growing privacy concern. To protect subject privacy, several algorithms have been developed to de-identify imaging data using blurring, defacing or refacing. Completely removing facial structures provides the best re-identification protection but can significantly impact post-processing steps, like brain morphometry. As an alternative, refacing methods that replace individual facial structures with generic templates have a lower effect on the geometry and intensity distribution of original scans, and are able to provide more consistent post-processing results by the price of higher re-identification risk and computational complexity. In the current study, we propose a novel method for anonymized face generation for defaced 3D T1-weighted scans based on a 3D conditional generative adversarial network. To evaluate the performance of the proposed de-identification tool, a comparative study was conducted between several existing defacing and refacing tools, with two different segmentation algorithms (FAST and Morphobox). The aim was to evaluate (i) impact on brain morphometry reproducibility, (ii) re-identification risk, (iii) balance between (i) and (ii), and (iv) the processing time. The proposed method takes 9 s for face generation and is suitable for recovering consistent post-processing results after defacing.

Direct comparison between 18F-Flortaucipir tau PET and quantitative susceptibility mapping in progressive supranuclear palsy

PURPOSE

The pattern of flortaucipir tau PET uptake is topographically similar to the pattern of magnetic susceptibility in progressive supranuclear palsy (PSP); both with increased signal in subcortical structures such as the basal ganglia and midbrain, suggesting that they may be closely related. However, their relationship remains unknown since no studies have directly compared these two modalities in the same PSP cohort. We hypothesized that some flortaucipir uptake in PSP is associated with magnetic susceptibility, and hence iron deposition. The aim of this study was to evaluate the regional relationship between flortaucipir uptake and magnetic susceptibility and to examine the effects of susceptibility on flortaucipir uptake in PSP.

METHODS

Fifty PSP patients and 67 cognitively normal controls were prospectively recruited and underwent three Tesla MRI and flortaucipir tau PET scans. Quantitative susceptibility maps were reconstructed from multi-echo gradient-echo MRI images. Region of interest (ROI) analysis was performed to obtain flortaucipir and susceptibility values in the subcortical regions. Relationships between flortaucipir and susceptibility signals were evaluated using partial correlation analysis in the subcortical ROIs and voxel-based analysis in the whole brain. The effects of susceptibility on flortaucipir uptake were examined by using the framework of mediation analysis.

RESULTS

Both flortaucipir and susceptibility were greater in PSP compared to controls in the putamen, pallidum, subthalamic nucleus, red nucleus, and cerebellar dentate (p<0.05). The ROI-based and voxel-based analyses showed that these two signals were positively correlated in these five regions (r = 0.36-0.59, p<0.05). Mediation analysis showed that greater flortaucipir uptake was partially explained by susceptibility in the putamen, pallidum, subthalamic nucleus, and red nucleus, and fully explained in the cerebellar dentate.

CONCLUSIONS

These results suggest that some of the flortaucipir uptake in subcortical regions in PSP is related to iron deposition. These findings will contribute to our understanding of the mechanisms underlying flortaucipir tau PET findings in PSP and other neurodegenerative diseases.

Ethical Considerations and Fairness in the Use of Artificial Intelligence for Neuroradiology

In this review, concepts of algorithmic bias and fairness are defined qualitatively and mathematically. Illustrative examples are given of what can go wrong when unintended bias or unfairness in algorithmic development occurs. The importance of explainability, accountability, and transparency with respect to artificial intelligence algorithm development and clinical deployment is discussed. These are grounded in the concept of "primum no nocere" (first, do no harm). Steps to mitigate unfairness and bias in task definition, data collection, model definition, training, testing, deployment, and feedback are provided. Discussions on the implementation of fairness criteria that maximize benefit and minimize unfairness and harm to neuroradiology patients will be provided, including suggestions for neuroradiologists to consider as artificial intelligence algorithms gain acceptance into neuroradiology practice and become incorporated into routine clinical workflow.

Effects of de-facing software mri_reface on utility of imaging biomarkers used in Alzheimer's disease research

Brain imaging research studies increasingly use "de-facing" software to remove or replace facial imagery before public data sharing. Several works have studied the effects of de-facing software on brain imaging biomarkers by directly comparing automated measurements from unmodified vs de-faced images, but most research brain images are used in analyses of correlations with cognitive measurements or clinical statuses, and the effects of de-facing on these types of imaging-to-cognition correlations has not been measured. In this work, we focused on brain imaging measures of amyloid (A), tau (T), neurodegeneration (N), and vascular (V) measures used in Alzheimer's Disease (AD) research. We created a retrospective sample of participants from three age- and sex-matched clinical groups (cognitively unimpaired, mild cognitive impairment, and AD dementia, and we performed region- and voxel-wise analyses of: hippocampal volume (N), white matter hyperintensity volume (V), amyloid PET (A), and tau PET (T) measures, each from multiple software pipelines, on their ability to separate cognitively defined groups and their degrees of correlation with age and Clinical Dementia Rating (CDR)-Sum of Boxes (CDR-SB). We performed each of these analyses twice: once with unmodified images and once with images de-faced with leading de-facing software mri_reface, and we directly compared the findings and their statistical strengths between the original vs. the de-faced images. Analyses with original and with de-faced images had very high agreement. There were no significant differences between any voxel-wise comparisons. Among region-wise comparisons, only three out of 55 correlations were significantly different between original and de-faced images, and these were not significant after correction for multiple comparisons. Overall, the statistical power of the imaging data for AD biomarkers was almost identical between unmodified and de-faced images, and their analyses results were extremely consistent.

Facial Anonymization and Privacy Concerns in Total-Body PET/CT

Total-body PET/CT images can be rendered to produce images of a subject's face and body. In response to privacy and identifiability concerns when sharing data, we have developed and validated a workflow that obscures (defaces) a subject's face in 3-dimensional volumetric data. Methods: To validate our method, we measured facial identifiability before and after defacing images from 30 healthy subjects who were imaged with both [18F]FDG PET and CT at either 3 or 6 time points. Briefly, facial embeddings were calculated using Google's FaceNet, and an analysis of clustering was used to estimate identifiability. Results: Faces rendered from CT images were correctly matched to CT scans at other time points at a rate of 93%, which decreased to 6% after defacing. Faces rendered from PET images were correctly matched to PET images at other time points at a maximum rate of 64% and to CT images at a maximum rate of 50%, both of which decreased to 7% after defacing. We further demonstrated that defaced CT images can be used for attenuation correction during PET reconstruction, introducing a maximum bias of -3.3% in regions of the cerebral cortex nearest the face. Conclusion: We believe that the proposed method provides a baseline of anonymity and discretion when sharing image data online or between institutions and will help to facilitate collaboration and future regulatory compliance.

A face-off of MRI research sequences by their need for de-facing

It is now widely known that research brain MRI, CT, and PET images may potentially be re-identified using face recognition, and this potential can be reduced by applying face-deidentification ("de-facing") software. However, for research MRI sequences beyond T1-weighted (T1-w) and T2-FLAIR structural images, the potential for re-identification and quantitative effects of de-facing are both unknown, and the effects of de-facing T2-FLAIR are also unknown. In this work we examine these questions (where applicable) for T1-w, T2-w, T2*-w, T2-FLAIR, diffusion MRI (dMRI), functional MRI (fMRI), and arterial spin labelling (ASL) sequences. Among current-generation, vendor-product research-grade sequences, we found that 3D T1-w, T2-w, and T2-FLAIR were highly re-identifiable (96-98%). 2D T2-FLAIR and 3D multi-echo GRE (ME-GRE) were also moderately re-identifiable (44-45%), and our derived T2* from ME-GRE (comparable to a typical 2D T2*) matched at only 10%. Finally, diffusion, functional and ASL images were each minimally re-identifiable (0-8%). Applying de-facing with mri_reface version 0.3 reduced successful re-identification to ≤8%, while differential effects on popular quantitative pipelines for cortical volumes and thickness, white matter hyperintensities (WMH), and quantitative susceptibility mapping (QSM) measurements were all either comparable with or smaller than scan-rescan estimates. Consequently, high-quality de-facing software can greatly reduce the risk of re-identification for identifiable MRI sequences with only negligible effects on automated intracranial measurements. The current-generation echo-planar and spiral sequences (dMRI, fMRI, and ASL) each had minimal match rates, suggesting that they have a low risk of re-identification and can be shared without de-facing, but this conclusion should be re-evaluated if they are acquired without fat suppression, with a full-face scan coverage, or if newer developments reduce the current levels of artifacts and distortion around the face.