Toward development of a comprehensive external quality assurance program for polyfunctional intracellular cytokine staining assays

Strengthening the Application of the DAIDS GCLP Guidelines: The Implementation of an Integrated Laboratory Oversight Framework

The Division of AIDS (DAIDS) Good Clinical Laboratory Practice (GCLP) Guidelines establish a framework to guide the oversight of laboratories supporting DAIDS-sponsored clinical research or trials. Compliance with these guidelines promotes data reliability, validity, and safety of the clinical research or trial participants and laboratory staff and ensures adherence to regulatory requirements. Acknowledgment and adoption of the DAIDS GCLP Guidelines are critical in building laboratory capacity and preparedness for conducting clinical trials. In collaboration with DAIDS, laboratory experts support the implementation of the DAIDS Integrated Laboratory Oversight Framework (Framework) activities. This article describes the implementation of the GCLP Guidelines, the Framework activities, and the coordinated efforts to strengthen laboratory performance. The Framework activities include four components: Quality Assurance Oversight, GCLP Audits, GCLP Training, and Laboratory Quality Improvement. Comparison of GCLP Guidelines with other regulations or standards, including U.S. Clinical Laboratory Improvement Amendments regulation 42 CFR 493, College of American Pathologists, World Health Organization GCLP, and International Organization for Standardization, ISO 15189:2012 standards, highlighted the differences and similarities to guide integration and harmonization efforts. Processes related to the Framework activities are outlined in detail, including key data derived from the managed activities of over 175 laboratories worldwide. Via the evolution of the DAIDS GCLP Guidelines and laboratory oversight workflows, the laboratories participating in DAIDS-sponsored clinical research and trials have successfully participated in internal and external regulatory audits. The collaborative and integrated oversight approach promotes knowledge-sharing and accountability to support the implementation of the DAIDS GCLP Guidelines and compliance monitoring. Lessons learned have helped with the implementation of the DAIDS integrated laboratory oversight approach and quality oversight programs at multiple laboratories worldwide.

Improving Reliability of Immunological Assays by Defining Minimal Criteria for Cell Fitness

Human PBMC-based assays are often used as biomarkers for the diagnosis and prognosis of disease, as well as for the prediction and tracking of response to biological therapeutics. However, the development and use of PBMC-based biomarker assays is often limited by poor reproducibility. Complex immunological assays can be further complicated by variation in cell handling before analysis, especially when using cryopreserved cells. Variation in postthaw viability is further increased if PBMC isolation and cryopreservation are done more than a few hours after collection. There is currently a lack of evidence-based standards for the minimal PBMC viability or "fitness" required to ensure the integrity and reproducibility of immune cell-based assays. In this study, we use an "induced fail" approach to examine the effect of thawed human PBMC fitness on four flow cytometry-based assays. We found that cell permeability-based viability stains at the time of thawing did not accurately quantify cell fitness, whereas a combined measurement of metabolic activity and early apoptosis markers did. Investigation of the impact of different types and levels of damage on PBMC-based assays revealed that only when cells were >60-70% live and apoptosis negative did biomarker values cease to be determined by cell fitness rather than the inherent biology of the cells. These data show that, to reproducibly measure immunological biomarkers using cryopreserved PBMCs, minimal acceptable standards for cell fitness should be incorporated into the assay protocol.

Achieving intracellular cytokine staining assay concordance on two continents to assess HIV vaccine-induced T-cell responses

The HIV Vaccine Trials Network (HVTN) conducts clinical trials on 4 continents in pursuit of a safe and effective HIV vaccine. Cellular immune responses to vaccination that define vaccine immunogenicity and/or immune correlates of protection can be measured using multiparameter intracellular cytokine staining (ICS) assays. The HVTN cellular immunology laboratory, located in Seattle, WA, conducts ICS assays for vaccine trials according to Good Clinical Laboratory Practices (GCLP). In 2013, the HVTN established a second GCLP compliant cellular immunology laboratory in Cape Town, South Africa to assess vaccine immunogenicity for HVTN trials conducted on the African continent. To ensure ICS readouts in the 2 laboratories were directly comparable, we conducted concordance testing using PBMC from healthy controls and vaccine trial participants. Despite standardized procedures and instrumentation, shared quality control measures and quality assurance oversight, several factors impacted our ability to obtain close agreement in T-cell responses measured in the 2 laboratories. One of these was the type of fetal bovine serum (FBS) used in the assay, which impacted lymphocyte cell viability and background responses. In addition, the differences in supernatant removal technique also significantly affected our ability to detect positive responses to vaccine antigens. Standardization of these factors allowed us to achieve and maintain ICS assay concordance across the 2 laboratories over multiple years, accelerating our efforts to evaluate HIV vaccines. The insights gained in this process are valuable for assay transfer efforts by groups of investigators that need to directly compare data generated in different laboratories around the globe.

FlowKit: A Python Toolkit for Integrated Manual and Automated Cytometry Analysis Workflows

An important challenge for primary or secondary analysis of cytometry data is how to facilitate productive collaboration between domain and quantitative experts. Domain experts in cytometry laboratories and core facilities increasingly recognize the need for automated workflows in the face of increasing data complexity, but by and large, still conduct all analysis using traditional applications, predominantly FlowJo. To a large extent, this cuts domain experts off from the rapidly growing library of Single Cell Data Science algorithms available, curtailing the potential contributions of these experts to the validation and interpretation of results. To address this challenge, we developed FlowKit, a Gating-ML 2.0-compliant Python package that can read and write FCS files and FlowJo workspaces. We present examples of the use of FlowKit for constructing reporting and analysis workflows, including round-tripping results to and from FlowJo for joint analysis by both domain and quantitative experts.

A Cytometrist's Guide to Coordinating and Performing Effective COVID-19 Research

Cytometry is playing a crucial role in addressing the COVID-19 pandemic. In this commentary-written by a variety of stakeholders in the cytometry, immunology, and infectious disease communities-we review cytometry's role in the COVID-19 response and discuss workflow issues critical to planning and executing effective research in this emerging field. We discuss sample procurement and processing, biosafety, technology options, data sharing, and the translation of research findings into clinical environments. © 2020 International Society for Advancement of Cytometry.

Gating Harmonization Guidelines for Intracellular Cytokine Staining Validated in Second International Multiconsortia Proficiency Panel Conducted by Cancer Immunotherapy Consortium (CIC/CRI)

Results from the first gating proficiency panel of intracellular cytokine staining (ICS) highlighted the value of using a consensus gating approach to reduce the variability across laboratories in reported %CD8⁺ or %CD4⁺ cytokine-positive cells. Based on the data analysis from the first proficiency panel, harmonization guidelines for a consensus gating protocol were proposed. To validate the recommendations from the first panel and to examine factors that were not included in the first panel, a second ICS gating proficiency panel was organized. All participants analyzed the same set of Flow Cytometry Standard (FCS) files using their own gating protocol. An optional learning module was provided to demonstrate how to apply the previously established gating recommendations and harmonization guidelines to actual ICS data files. Eighty-three participants took part in this proficiency panel. The results from this proficiency panel confirmed the harmonization guidelines from the first panel. These recommendations addressed the (1) placement of the cytokine-positive gate, (2) identification of CD4⁺ CD8⁺ double-positive T cells, (3) placement of lymphocyte gate, (4) inclusion of dim cells, (5) gate uniformity, and (6) proper adjustment of the biexponential scaling. In addition, based on the results of this proficiency gating panel, two new recommendations were added to expand the harmonization guidelines: (1) inclusion of dump channel marker to gate all live and dump negative cells and (2) backgating to confirm the correct placement of gates across all populations. © 2020 International Society for Advancement of Cytometry.

Immunophenotyping of Human Regulatory T Cells

Regulatory T cells, also known as Tregs, play a pivotal role in maintaining homeostasis of the immune system and self-tolerance. Tregs express CD3, CD4, CD25, and FOXP3 but lack CD127. CD4 and CD3 identify helper T lymphocytes, of which Tregs are a subset. CD25 is IL-2Rα, an essential activation marker that is expressed in high levels on Tregs. FOXP3 is the canonical transcription factor, important in the development, maintenance, and identification of Tregs. CD127 is IL-7 receptor, expressed inversely with suppression, and is therefore downregulated on Tregs. Flow cytometry is a powerful tool that is capable of simultaneously measuring Tregs along with several markers associated with subpopulations of Tregs, activation, maturation, proliferation, and surrogates of functional suppression. This chapter describes a multicolor flow cytometry-based approach to measure human Tregs, including details for surface staining, fixation/permeabilization, intracellular/intranuclear staining, acquisition of samples on a flow cytometer, plus analysis and interpretation of resulting FCS files.

Immune Monitoring for CMV in Transplantation

PURPOSE OF REVIEW

Immune monitoring to determine when and how the recovery of cytomegalovirus (CMV)-specific T-cells occurs post-transplantation may help clinicians to risk stratify individuals at risk of complications from CMV. We aimed to review all recent clinical studies using CMV immune monitoring in the pre- and post-transplant setting including the use of recently developed standardized assays (Quantiferon-CMV and the CMV ELISPOT) to better understand in whom, when, and how immune monitoring is best used.

RECENT FINDINGS

Pre-transplant assessment of CMV immunity in solid-organ transplant recipients where CMV seropositive recipients had undetectable cell-mediated responses despite past immunity has shown that they are at a much higher risk of developing CMV reactivation. Post-transplant CMV immune monitoring can guide (shorten or prolong) the duration of antiviral prophylaxis, identify recipients at risk of post-prophylaxis CMV disease, and predict recurrent CMV reactivation. Thus, CMV immune monitoring, in addition to current clinical and DNA-based monitoring for CMV, has the potential to be incorporated into routine clinical care to better improve CMV management in both the stem and solid-organ transplant population.

Variability of the IFN-γ ELISpot assay in the context of proficiency testing and bridging studies

Assays that assess cellular mediated immune responses performed under Good Clinical Laboratory Practice (GCLP) guidelines are required to provide specific and reproducible results. Defined validation procedures are required to establish the Standard Operating Procedure (SOP), include pass and fail criteria, as well as implement positivity criteria. However, little to no guidance is provided on how to perform longitudinal assessment of the key reagents utilized in the assay. Through the External Quality Assurance Program Oversight Laboratory (EQAPOL), an Interferon-gamma (IFN-γ) Enzyme-linked immunosorbent spot (ELISpot) assay proficiency testing program is administered. A limit of acceptable within site variability was estimated after six rounds of proficiency testing (PT). Previously, a PT send-out specific within site variability limit was calculated based on the dispersion (variance/mean) of the nine replicate wells of data. Now an overall 'dispersion limit' for the ELISpot PT program within site variability has been calculated as a dispersion of 3.3. The utility of this metric was assessed using a control sample to calculate the within (precision) and between (accuracy) experiment variability to determine if the dispersion limit could be applied to bridging studies (studies that assess lot-to-lot variations of key reagents) for comparing the accuracy of results with new lots to results with old lots. Finally, simulations were conducted to explore how this dispersion limit could provide guidance in the number of replicate wells needed for within and between experiment variability and the appropriate donor reactivity (number of antigen-specific cells) to be used for the evaluation of new reagents. Our bridging study simulations indicate using a minimum of six replicate wells of a control donor sample with reactivity of at least 150 spot forming cells per well is optimal. To determine significant lot-to-lot variations use the 3.3 dispersion limit for between and within experiment variability.

Analysis of cell-mediated immune responses in support of dengue vaccine development efforts

Dengue vaccine development has made significant strides, but a better understanding of how vaccine-induced immune responses correlate with vaccine efficacy can greatly accelerate development, testing, and deployment as well as ameliorate potential risks and safety concerns. Advances in basic immunology knowledge and techniques have already improved our understanding of cell-mediated immunity of natural dengue virus infection and vaccination. We conclude that the evidence base is adequate to argue for inclusion of assessments of cell-mediated immunity as part of clinical trials of dengue vaccines, although further research to identify useful correlates of protective immunity is needed.

Managing Multi-center Flow Cytometry Data for Immune Monitoring

With the recent results of promising cancer vaccines and immunotherapy1–5, immune monitoring has become increasingly relevant for measuring treatment-induced effects on T cells, and an essential tool for shedding light on the mechanisms responsible for a successful treatment. Flow cytometry is the canonical multi-parameter assay for the fine characterization of single cells in solution, and is ubiquitously used in pre-clinical tumor immunology and in cancer immunotherapy trials. Current state-of-the-art polychromatic flow cytometry involves multi-step, multi-reagent assays followed by sample acquisition on sophisticated instruments capable of capturing up to 20 parameters per cell at a rate of tens of thousands of cells per second. Given the complexity of flow cytometry assays, reproducibility is a major concern, especially for multi-center studies. A promising approach for improving reproducibility is the use of automated analysis borrowing from statistics, machine learning and information visualization21–23, as these methods directly address the subjectivity, operator-dependence, labor-intensive and low fidelity of manual analysis. However, it is quite time-consuming to investigate and test new automated analysis techniques on large data sets without some centralized information management system. For large-scale automated analysis to be practical, the presence of consistent and high-quality data linked to the raw FCS files is indispensable. In particular, the use of machine-readable standard vocabularies to characterize channel metadata is essential when constructing analytic pipelines to avoid errors in processing, analysis and interpretation of results. For automation, this high-quality metadata needs to be programmatically accessible, implying the need for a consistent Application Programming Interface (API). In this manuscript, we propose that upfront time spent normalizing flow cytometry data to conform to carefully designed data models enables automated analysis, potentially saving time in the long run. The ReFlow informatics framework was developed to address these data management challenges.

Introduction to a Special Issue of the Journal of Immunological Methods: Building global resource programs to support HIV/AIDS clinical trial studies

This Special Issue of the Journal of Immunological Methods includes 16 manuscripts describing quality assurance activities related to virologic and immunologic monitoring of six global laboratory resource programs that support international HIV/AIDS clinical trial studies: Collaboration for AIDS Vaccine Discovery (CAVD); Center for HIV/AIDS Vaccine Immunology (CHAVI); External Quality Assurance Program Oversight Laboratory (EQAPOL); HIV Vaccine Trial Network (HVTN); International AIDS Vaccine Initiative (IAVI); and Immunology Quality Assessment (IQA). The reports from these programs address the many components required to develop comprehensive quality control activities and subsequent quality assurance programs for immune monitoring in global clinical trials including: all aspects of processing, storing, and quality assessment of PBMC preparations used ubiquitously in HIV clinical trials, the development and optimization of assays for CD8 HIV responses and HIV neutralization, a comprehensive global HIV virus repository, and reports on the development and execution of novel external proficiency testing programs for immunophenotyping, intracellular cytokine staining, ELISPOT and luminex based cytokine measurements. In addition, there are articles describing the implementation of Good Clinical Laboratory Practices (GCLP) in a large quality assurance laboratory, the development of statistical methods specific for external proficiency testing assessment, a discussion on the ability to set objective thresholds for measuring rare events by flow cytometry, and finally, a manuscript which addresses a framework for the structured reporting of T cell immune function based assays. It is anticipated that this series of manuscripts covering a wide range of quality assurance activities associated with the conduct of global clinical trials will provide a resource for individuals and programs involved in improving the harmonization, standardization, accuracy, and sensitivity of virologic and immunologic testing.