Unsupervised Quality Assessment of Mass Spectrometry Proteomics Experiments by Multivariate Quality Control Metrics

iDIA-QC: AI-empowered data-independent acquisition mass spectrometry-based quality control

Quality control (QC) in mass spectrometry (MS)-based proteomics is mainly based on data-dependent acquisition (DDA) analysis of standard samples. Here, we collect 2754 files acquired by data independent acquisition (DIA) and paired 2638 DDA files from mouse liver digests using 21 mass spectrometers across nine laboratories over 31 months. Our data demonstrate that DIA-based LC-MS/MS-related consensus QC metrics exhibit higher sensitivity compared to DDA-based QC metrics in detecting changes in LC-MS status. We then prioritize 15 metrics and invite 21 experts to manually assess the quality of 2754 DIA files based on those metrics. We develop an AI model for DIA-based QC using 2110 training files. It achieves AUCs of 0.91 (LC) and 0.97 (MS) in the first validation dataset (n = 528), and 0.78 (LC) and 0.94 (MS) in an independent validation dataset (n = 116). Finally, we develop an offline software called iDIA-QC for convenient adoption of this methodology.

Quality Assurance for Multiplex Quantitative Clinical Chemistry Proteomics in Large Clinical Trials

BACKGROUND

To evaluate the clinical performance and effectiveness of a multiplex apolipoprotein panel in the context of cardiovascular precision diagnostics, clinical samples of patients with recent acute coronary syndrome in the ODYSSEY OUTCOMES trial were measured by quantitative clinical chemistry proteomics (qCCP). The ISO15189-accredited laboratory setting, including the total testing process (TTP), served as a foundation for this study. Consequently, tailored quality assurance measures needed to be designed and implemented to suit the demands of a multiplex LC-MS/MS test.

METHODS

Nine serum apolipoproteins were measured in 23 376 samples with a laboratory-developed multiplex apolipoprotein test on 4 Agilent 6495 LC-MS/MS systems. A fit-for-purpose process was designed with tailored additions enhancing the accredited laboratory infrastructure and the TTP. Quality assurance was organized in 3 steps: system suitability testing (SST), internal quality control (IQC) evaluation with adjusted Westgard rules to fit a multiplex test, and interpeptide agreement analysis. Data was semi-automatically evaluated with a custom R script.

RESULTS

LC-MS/MS analyses were performed with the following between-run CVs: for apolipoprotein (Apo) (a) 6.2%, Apo A-I 2.3%, Apo A-II 2.1%, Apo A-IV 2.9%, Apo B 1.9%, Apo C-I 3.3%, Apo C-II 3.3%, Apo C-III 2.7%, and for Apo E 3.3% and an average interpeptide agreement Pearson r of 0.981.

CONCLUSIONS

This is the first study of its kind in which qCCP was performed at this scale. This research successfully demonstrates the feasibility of high-throughput LC-MS/MS applications in large clinical trials. ClinicalTrials.gov Registration Number: NCT01663402.

A Framework for Quality Control in Quantitative Proteomics

A thorough evaluation of the quality, reproducibility, and variability of bottom-up proteomics data is necessary at every stage of a workflow, from planning to analysis. We share vignettes applying adaptable quality control (QC) measures to assess sample preparation, system function, and quantitative analysis. System suitability samples are repeatedly measured longitudinally with targeted methods, and we share examples where they are used on three instrument platforms to identify severe system failures and track function over months to years. Internal QCs incorporated at the protein and peptide levels allow our team to assess sample preparation issues and to differentiate system failures from sample-specific issues. External QC samples prepared alongside our experimental samples are used to verify the consistency and quantitative potential of our results during batch correction and normalization before assessing biological phenotypes. We combine these controls with rapid analysis (Skyline), longitudinal QC metrics (AutoQC), and server-based data deposition (PanoramaWeb). We propose that this integrated approach to QC is a useful starting point for groups to facilitate rapid quality control assessment to ensure that valuable instrument time is used to collect the best quality data possible. Data are available on Panorama Public and ProteomeXchange under the identifier PXD051318.

A framework for quality control in quantitative proteomics

A thorough evaluation of the quality, reproducibility, and variability of bottom-up proteomics data is necessary at every stage of a workflow from planning to analysis. We share vignettes applying adaptable quality control (QC) measures to assess sample preparation, system function, and quantitative analysis. System suitability samples are repeatedly measured longitudinally with targeted methods, and we share examples where they are used on three instrument platforms to identify severe system failures and track function over months to years. Internal QCs incorporated at protein and peptide-level allow our team to assess sample preparation issues and to differentiate system failures from sample-specific issues. External QC samples prepared alongside our experimental samples are used to verify the consistency and quantitative potential of our results during batch correction and normalization before assessing biological phenotypes. We combine these controls with rapid analysis (Skyline), longitudinal QC metrics (AutoQC), and server-based data deposition (PanoramaWeb). We propose that this integrated approach to QC is a useful starting point for groups to facilitate rapid quality control assessment to ensure that valuable instrument time is used to collect the best quality data possible. Data are available on Panorama Public and on ProteomeXchange under the identifier PXD051318.

Observations from the Proteomics Bench

Many challenges in proteomics result from the high-throughput nature of the experiments. This paper first presents pre-analytical problems, which still occur, although the call for standardization in omics has been ongoing for many years. This article also discusses aspects that affect bioinformatic analysis based on three sets of reference data measured with different orbitrap instruments. Despite continuous advances in mass spectrometer technology as well as analysis software, data-set-wise quality control is still necessary, and decoy-based estimation, although challenged by modern instruments, should be utilized. We draw attention to the fact that numerous young researchers perceive proteomics as a mature, readily applicable technology. However, it is important to emphasize that the maximum potential of the technology can only be realized by an educated handling of its limitations.

Amantadine Toxicity in Apostichopus japonicus Revealed by Proteomics

Amantadine exposure can alter biological processes in sea cucumbers, which are an economically important seafood in China. In this study, amantadine toxicity in Apostichopus japonicus was analyzed by oxidative stress and histopathological methods. Quantitative tandem mass tag labeling was used to examine changes in protein contents and metabolic pathways in A. japonicus intestinal tissues after exposure to 100 µg/L amantadine for 96 h. Catalase activity significantly increased from days 1 to 3 of exposure, but it decreased on day 4. Superoxide dismutase and glutathione activities were inhibited throughout the exposure period. Malondialdehyde contents increased on days 1 and 4 but decreased on days 2 and 3. Proteomics analysis revealed 111 differentially expressed proteins in the intestines of A. japonicus after amantadine exposure compared with the control group. An analysis of the involved metabolic pathways showed that the glycolytic and glycogenic pathways may have increased energy production and conversion in A. japonicus after amantadine exposure. The NF-κB, TNF, and IL-17 pathways were likely induced by amantadine exposure, thereby activating NF-κB and triggering intestinal inflammation and apoptosis. Amino acid metabolism analysis showed that the leucine and isoleucine degradation pathways and the phenylalanine metabolic pathway inhibited protein synthesis and growth in A. japonicus. This study investigated the regulatory response mechanisms in A. japonicus intestinal tissues after exposure to amantadine, providing a theoretical basis for further research on amantadine toxicity.

Interrogating Fractionation and Other Sources of Variability in Shotgun Proteomes Using Quality Metrics

The increasing amount of publicly available proteomics data creates opportunities for data scientists to investigate quality metrics in novel ways. QuaMeter IDFree is used to generate quality metrics from 665 RAW files and 97 WIFF files representing publicly available "shotgun" mass spectrometry datasets. These experiments are selected to represent Mycobacterium tuberculosis lysates, mouse MDSCs, and exosomes derived from human cell lines. Machine learning techniques are demonstrated to detect outliers within experiments and it is shown that quality metrics may be used to distinguish sources of variability among these experiments. In particular, the findings demonstrate that according to nested ANOVA performed on an SDS-PAGE shotgun principal component analysis, runs of fractions from the same gel regions cluster together rather than technical replicates, close temporal proximity, or even biological samples. This indicates that the individual fraction may have had a higher impact on the quality metrics than other factors. In addition, sample type, instrument type, mass analyzer, fragmentation technique, and digestion enzyme are identified as sources of variability. From a quality control perspective, the importance of study design and in particular, the run order, is illustrated in seeking ways to limit the impact of technical variability.

Molecular networks in Network Medicine: Development and applications

Network Medicine applies network science approaches to investigate disease pathogenesis. Many different analytical methods have been used to infer relevant molecular networks, including protein-protein interaction networks, correlation-based networks, gene regulatory networks, and Bayesian networks. Network Medicine applies these integrated approaches to Omics Big Data (including genetics, epigenetics, transcriptomics, metabolomics, and proteomics) using computational biology tools and, thereby, has the potential to provide improvements in the diagnosis, prognosis, and treatment of complex diseases. We discuss briefly the types of molecular data that are used in molecular network analyses, survey the analytical methods for inferring molecular networks, and review efforts to validate and visualize molecular networks. Successful applications of molecular network analysis have been reported in pulmonary arterial hypertension, coronary heart disease, diabetes mellitus, chronic lung diseases, and drug development. Important knowledge gaps in Network Medicine include incompleteness of the molecular interactome, challenges in identifying key genes within genetic association regions, and limited applications to human diseases. This article is categorized under: Models of Systems Properties and Processes > Mechanistic Models Translational, Genomic, and Systems Medicine > Translational Medicine Analytical and Computational Methods > Analytical Methods Analytical and Computational Methods > Computational Methods.

Quantitative Proteomics Data in the Public Domain: Challenges and Opportunities

Mass spectrometry based proteomics is no longer only a qualitative discipline, and can be successfully employed to obtain a truly multidimensional view of the proteome. In particular, systematic protein expression profiling is now a routine part of many studies in the field and beyond. The large growth in the number of quantitative studies is accompanied by a trend to share publicly the associated analysis results and the underlying raw data. This trend, established and strongly supported by public repositories such as the PRIDE database at the European Bioinformatics Institute, opens up enormous possibilities to explore the data beyond the original publications, for instance by reusing, reanalyzing, and performing different flavors of meta-analysis studies. To help researchers and scientists realize about this potential, here we describe the mainstream public proteomics resources containing quantitative proteomics data, including the processed analysis results and/or the underlying raw data. We then present and discuss the most important points to consider when attempting to (re)use proteomics data in the public domain. We conclude by highlighting potential pitfalls of (re)using quantitative data and discuss some of our own experiences in this context.

Quality control in mass spectrometry-based proteomics

Mass spectrometry is a highly complex analytical technique and mass spectrometry-based proteomics experiments can be subject to a large variability, which forms an obstacle to obtaining accurate and reproducible results. Therefore, a comprehensive and systematic approach to quality control is an essential requirement to inspire confidence in the generated results. A typical mass spectrometry experiment consists of multiple different phases including the sample preparation, liquid chromatography, mass spectrometry, and bioinformatics stages. We review potential sources of variability that can impact the results of a mass spectrometry experiment occurring in all of these steps, and we discuss how to monitor and remedy the negative influences on the experimental results. Furthermore, we describe how specialized quality control samples of varying sample complexity can be incorporated into the experimental workflow and how they can be used to rigorously assess detailed aspects of the instrument performance.

Quality Control Analysis in Real-time (QC-ART): A Tool for Real-time Quality Control Assessment of Mass Spectrometry-based Proteomics Data

Liquid chromatography-mass spectrometry (LC-MS)-based proteomics studies of large sample cohorts can easily require from months to years to complete. Acquiring consistent, high-quality data in such large-scale studies is challenging because of normal variations in instrumentation performance over time, as well as artifacts introduced by the samples themselves, such as those because of collection, storage and processing. Existing quality control methods for proteomics data primarily focus on post-hoc analysis to remove low-quality data that would degrade downstream statistics; they are not designed to evaluate the data in near real-time, which would allow for interventions as soon as deviations in data quality are detected. In addition to flagging analyses that demonstrate outlier behavior, evaluating how the data structure changes over time can aide in understanding typical instrument performance or identify issues such as a degradation in data quality because of the need for instrument cleaning and/or re-calibration. To address this gap for proteomics, we developed Quality Control Analysis in Real-Time (QC-ART), a tool for evaluating data as they are acquired to dynamically flag potential issues with instrument performance or sample quality. QC-ART has similar accuracy as standard post-hoc analysis methods with the additional benefit of real-time analysis. We demonstrate the utility and performance of QC-ART in identifying deviations in data quality because of both instrument and sample issues in near real-time for LC-MS-based plasma proteomics analyses of a sample subset of The Environmental Determinants of Diabetes in the Young cohort. We also present a case where QC-ART facilitated the identification of oxidative modifications, which are often underappreciated in proteomic experiments.

QCloud: A cloud-based quality control system for mass spectrometry-based proteomics laboratories

The increasing number of biomedical and translational applications in mass spectrometry-based proteomics poses new analytical challenges and raises the need for automated quality control systems. Despite previous efforts to set standard file formats, data processing workflows and key evaluation parameters for quality control, automated quality control systems are not yet widespread among proteomics laboratories, which limits the acquisition of high-quality results, inter-laboratory comparisons and the assessment of variability of instrumental platforms. Here we present QCloud, a cloud-based system to support proteomics laboratories in daily quality assessment using a user-friendly interface, easy setup, automated data processing and archiving, and unbiased instrument evaluation. QCloud supports the most common targeted and untargeted proteomics workflows, it accepts data formats from different vendors and it enables the annotation of acquired data and reporting incidences. A complete version of the QCloud system has successfully been developed and it is now open to the proteomics community (http://qcloud.crg.eu). QCloud system is an open source project, publicly available under a Creative Commons License Attribution-ShareAlike 4.0.

The Human Proteome Organization-Proteomics Standards Initiative Quality Control Working Group: Making Quality Control More Accessible for Biological Mass Spectrometry

To have confidence in results acquired during biological mass spectrometry experiments, a systematic approach to quality control is of vital importance. Nonetheless, until now, only scattered initiatives have been undertaken to this end, and these individual efforts have often not been complementary. To address this issue, the Human Proteome Organization-Proteomics Standards Initiative has established a new working group on quality control at its meeting in the spring of 2016. The goal of this working group is to provide a unifying framework for quality control data. The initial focus will be on providing a community-driven standardized file format for quality control. For this purpose, the previously proposed qcML format will be adapted to support a variety of use cases for both proteomics and metabolomics applications, and it will be established as an official PSI format. An important consideration is to avoid enforcing restrictive requirements on quality control but instead provide the basic technical necessities required to support extensive quality control for any type of mass spectrometry-based workflow. We want to emphasize that this is an open community effort, and we seek participation from all scientists with an interest in this field.