1
|
Masca NGD, Hensor EMA, Cornelius VR, Buffa FM, Marriott HM, Eales JM, Messenger MP, Anderson AE, Boot C, Bunce C, Goldin RD, Harris J, Hinchliffe RF, Junaid H, Kingston S, Martin-Ruiz C, Nelson CP, Peacock J, Seed PT, Shinkins B, Staples KJ, Toombs J, Wright AKA, Teare MD. RIPOSTE: a framework for improving the design and analysis of laboratory-based research. eLife 2015; 4:e05519. [PMID: 25951517 PMCID: PMC4461852 DOI: 10.7554/elife.05519] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2014] [Accepted: 05/01/2015] [Indexed: 12/17/2022] Open
Abstract
Lack of reproducibility is an ongoing problem in some areas of the biomedical sciences. Poor experimental design and a failure to engage with experienced statisticians at key stages in the design and analysis of experiments are two factors that contribute to this problem. The RIPOSTE (Reducing IrreProducibility in labOratory STudiEs) framework has been developed to support early and regular discussions between scientists and statisticians in order to improve the design, conduct and analysis of laboratory studies and, therefore, to reduce irreproducibility. This framework is intended for use during the early stages of a research project, when specific questions or hypotheses are proposed. The essential points within the framework are explained and illustrated using three examples (a medical equipment test, a macrophage study and a gene expression study). Sound study design minimises the possibility of bias being introduced into experiments and leads to higher quality research with more reproducible results.
Collapse
Affiliation(s)
- Nicholas GD Masca
- Cardiovascular Biomedical Research Unit, University of Leicester, Leicester, United Kingdom
| | - Elizabeth MA Hensor
- Leeds Institute of Rheumatic and Musculoskeletal Medicine, University of Leeds, Leeds, United Kingdom; Leeds Institute of Rheumatic and Musculoskeletal Medicine, NIHR Leeds Musculoskeletal Biomedical Research Unit, Leeds, United Kingdom
| | - Victoria R Cornelius
- Department of Primary Care and Public Health Sciences, King's College London, London, United Kingdom
| | - Francesca M Buffa
- Applied Computational Genomics, University of Oxford, Oxford, United Kingdom
| | - Helen M Marriott
- Department of Infection and Immunity, University of Sheffield, Sheffield, United Kingdom; The Florey Institute, University of Sheffield, Sheffield, United Kingdom
| | - James M Eales
- Department of Cardiovascular Sciences, University of Leicester, Leicester, United Kingdom
| | - Michael P Messenger
- NIHR Diagnostic Evidence Co-Operative Leeds, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom
| | - Amy E Anderson
- Musculoskeletal Research Group, Institute of Cellular Medicine, University of Newcastle, Newcastle, United Kingdom
| | - Chris Boot
- Newcastle Hospitals NHS Trust, Newcastle, United Kingdom
| | - Catey Bunce
- NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, United Kingdom; London School of Hygiene and Tropical Medicine, London, United Kingdom
| | - Robert D Goldin
- Centre for Pathology, Imperial College, London, United Kingdom
| | - Jessica Harris
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, United Kingdom
| | - Rod F Hinchliffe
- Department of Paediatric Haematology, Sheffield Children's NHS Foundation Trust, Sheffield, United Kingdom
| | - Hiba Junaid
- Royal London Hospital, London, United Kingdom
| | - Shaun Kingston
- Respiratory Biomedical Research Unit, Royal Brompton and Harefield NHS Trust, London, United Kingdom
| | - Carmen Martin-Ruiz
- Institute for Ageing and Health, Newcastle University, Newcastle, United Kingdom
| | - Christopher P Nelson
- Department of Cardiovascular Sciences, NIHR Leicester Cardiovascular Biomedical Research Unit, University of Leicester, Leicester, United Kingdom
| | - Janet Peacock
- Division of Health and Social Care Research, Kings College London, London, United Kingdom; NIHR Biomedical Research Centre at Guy's and St Thomas' NHS Foundation, London, United Kingdom
| | - Paul T Seed
- Division of Women's Health, King's College London, London, United Kingdom
| | - Bethany Shinkins
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom
| | - Karl J Staples
- Clinical and Experimental Sciences, University of Southampton and NIHR Southampton Respiratory Biomedical Research Unit, Southampton General Hospital, Southampton, United Kingdom
| | - Jamie Toombs
- Department of Molecular Neuroscience, Institute of Neurology, University College London, London, United Kingdom
| | - Adam KA Wright
- Institute of Lung Health, Respiratory Biomedical Unit, University Hospitals of Leicester NHS Trust, Leicester, United Kingdom
| | - M Dawn Teare
- Sheffield School of Health and Related Research, University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|
2
|
Chen C, Grennan K, Badner J, Zhang D, Gershon E, Jin L, Liu C. Removing batch effects in analysis of expression microarray data: an evaluation of six batch adjustment methods. PLoS One 2011; 6:e17238. [PMID: 21386892 PMCID: PMC3046121 DOI: 10.1371/journal.pone.0017238] [Citation(s) in RCA: 323] [Impact Index Per Article: 24.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2010] [Accepted: 01/24/2011] [Indexed: 01/07/2023] Open
Abstract
The expression microarray is a frequently used approach to study gene expression on a genome-wide scale. However, the data produced by the thousands of microarray studies published annually are confounded by “batch effects,” the systematic error introduced when samples are processed in multiple batches. Although batch effects can be reduced by careful experimental design, they cannot be eliminated unless the whole study is done in a single batch. A number of programs are now available to adjust microarray data for batch effects prior to analysis. We systematically evaluated six of these programs using multiple measures of precision, accuracy and overall performance. ComBat, an Empirical Bayes method, outperformed the other five programs by most metrics. We also showed that it is essential to standardize expression data at the probe level when testing for correlation of expression profiles, due to a sizeable probe effect in microarray data that can inflate the correlation among replicates and unrelated samples.
Collapse
Affiliation(s)
- Chao Chen
- National Ministry of Education Key Laboratory of Contemporary Anthropology, Fudan University, Shanghai, People's Republic of China
- Department of Psychiatry, University of Chicago, Chicago, Illinois, United States of America
| | - Kay Grennan
- Department of Psychiatry, University of Chicago, Chicago, Illinois, United States of America
| | - Judith Badner
- Department of Psychiatry, University of Chicago, Chicago, Illinois, United States of America
| | - Dandan Zhang
- Department of Pathology, Zhejiang University, Hangzhou, People's Republic of China
| | - Elliot Gershon
- Department of Psychiatry, University of Chicago, Chicago, Illinois, United States of America
| | - Li Jin
- National Ministry of Education Key Laboratory of Contemporary Anthropology, Fudan University, Shanghai, People's Republic of China
| | - Chunyu Liu
- Department of Psychiatry, University of Chicago, Chicago, Illinois, United States of America
- * E-mail:
| |
Collapse
|
3
|
Wurch LL, Haley ST, Orchard ED, Gobler CJ, Dyhrman ST. Nutrient-regulated transcriptional responses in the brown tide-forming alga Aureococcus anophagefferens. Environ Microbiol 2011; 13:468-81. [PMID: 20880332 PMCID: PMC3282463 DOI: 10.1111/j.1462-2920.2010.02351.x] [Citation(s) in RCA: 47] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
Long-SAGE (serial analysis of gene expression) was used to profile the transcriptome of the brown tide-forming alga, Aureococcus anophagefferens, under nutrient replete (control), and nitrogen (N) and phosphorus (P) deficiency to understand how this organism responds at the transcriptional level to varying nutrient conditions. This approach has aided A. anophagefferens genome annotation efforts and identified a suite of genes upregulated by N and P deficiency, some of which have known roles in nutrient metabolism. Genes upregulated under N deficiency include an ammonium transporter, an acetamidase/formamidase and two peptidases. This suggests an ability to utilize reduced N compounds and dissolved organic nitrogen, supporting the hypothesized importance of these N sources in A. anophagefferens bloom formation. There are also a broad suite of P-regulated genes, including an alkaline phosphatase, and two 5'-nucleotidases, suggesting A. anophagefferens may use dissolved organic phosphorus under low phosphate conditions. These N- and P-regulated genes may be important targets for exploring nutrient controls on bloom formation in field populations.
Collapse
Affiliation(s)
- Louie L. Wurch
- MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering, Cambridge, MA 02139
| | - Sheean T. Haley
- Woods Hole Oceanographic Institution, Biology Department, Woods Hole, MA 02543
| | - Elizabeth D. Orchard
- MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering, Cambridge, MA 02139
| | - Christopher J. Gobler
- Stony Brook University, School of Marine and Atmospheric Sciences, Stony Brook, NY 11794
| | - Sonya T. Dyhrman
- Woods Hole Oceanographic Institution, Biology Department, Woods Hole, MA 02543
| |
Collapse
|
4
|
Abstract
We present a practical guide to some of the issues involved in comparing or integrating different microarray studies. We discuss the influence that various factors have on the agreement between studies, such as different technologies and platforms, statistical analysis criteria, protocols, and lab variability. We discuss methods to carry out or refine such comparisons, and detail several common pitfalls to avoid. Finally, we illustrate these ideas with an example case.
Collapse
Affiliation(s)
- Mayte Suárez-Fariñas
- Center for Studies in Physics and Biology, The Rockefeller University, New York, NY, USA
| | | |
Collapse
|
5
|
Heijne WHM, Kienhuis AS, van Ommen B, Stierum RH, Groten JP. Systems toxicology: applications of toxicogenomics, transcriptomics, proteomics and metabolomics in toxicology. Expert Rev Proteomics 2006; 2:767-80. [PMID: 16209655 DOI: 10.1586/14789450.2.5.767] [Citation(s) in RCA: 119] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Toxicogenomics can facilitate the identification and characterization of toxicity, as illustrated in this review. Toxicogenomics, the application of the functional genomics technologies (transcriptomics, proteomics and metabolomics) in toxicology enables the study of adverse effects of xenobiotic substances in relation to structure and activity of the genome. The advantages and limitations of the different technologies are evaluated, and the prospects for integration of the technologies into a systems biology or systems toxicology approach are discussed. Applications of toxicogenomics in various laboratories around the world show that the crucial steps and sequence of events at the molecular level can be studied to provide detailed insights into mechanisms of toxic action. Toxicogenomics allowed for more sensitive and earlier detection of adverse effects in (animal) toxicity studies. Furthermore, the effects of exposure to mixtures could be studied in more detail. This review argues that in the (near) future, human health risk assessment will truly benefit from toxicogenomics (systems toxicology).
Collapse
|
6
|
Gong P, Harbers GM, Grainger DW. Multi-technique Comparison of Immobilized and Hybridized Oligonucleotide Surface Density on Commercial Amine-Reactive Microarray Slides. Anal Chem 2006; 78:2342-51. [PMID: 16579618 DOI: 10.1021/ac051812m] [Citation(s) in RCA: 62] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
To establish a quantitative, corroborative understanding of observed correlations between immobilized probe DNA density on microarray surfaces and target hybridization efficiency in biological samples, we have characterized amine-terminated, single-stranded DNA probes attached to amine-reactive commercial microarray slides and complementary DNA target hybridization using fluorescence imaging, X-ray photoelectron spectroscopy (XPS) and 32P-radiometric assays. Importantly, we have reproduced DNA probe microarray immobilization densities in macroscopic spotted dimensions using high ionic strength, high-concentration DNA probe solutions to permit direct XPS surface analysis of DNA surface chemistry with good reliability and reproducibility. Target capture hybridization efficiency with complementary DNA exhibited an optimum value at intermediate DNA probe immobilization densities. The macroscopic array model provides a new platform for the study of DNA surface chemistry using highly sensitive, quantitative surface analytical techniques (e.g., XPS, ToF-SIMS). Sensitive 32P-DNA radiometric density measurements were calibrated with more routine XPS DNA signals, facilitating future routine DNA density determinations without the use of a hazardous radioactive assay. The objective is to provide new insight into different surface chemistry influences on immobilized DNA probe environments that affect target capture efficiency from solution to improve microarray assay performance.
Collapse
Affiliation(s)
- Ping Gong
- Department of Chemistry, Colorado State University, Fort Collins, Colorado 80523-1872, USA
| | | | | |
Collapse
|
7
|
Suárez-Fariñas M, Noggle S, Heke M, Hemmati-Brivanlou A, Magnasco MO. Comparing independent microarray studies: the case of human embryonic stem cells. BMC Genomics 2005; 6:99. [PMID: 16042783 PMCID: PMC1183205 DOI: 10.1186/1471-2164-6-99] [Citation(s) in RCA: 36] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2005] [Accepted: 07/22/2005] [Indexed: 11/10/2022] Open
Abstract
Background Microarray studies of the same phenomenon in different labs often appear at variance because the published lists of regulated transcripts have disproportionately small intersections. We demonstrate that comparing studies by intersecting lists in this manner is methodologically flawed by reanalyzing three studies of the molecular signature of "stemness" in human embryonic stem cells. There are only 7 genes common to all three published lists, suggesting disagreement. Results Carefully reanalyzing all three together from the raw data we detect 111 genes upregulated and 95 downregulated in all three studies. The upregulated list was subject to rtRTPCR analysis and 75% of the genes were confirmed. Conclusion Our findings show that the three studies have a substantial core of common genes, which is missed if only the published lists are examined. Combined analysis of multiple experiments can be a powerful way to distil coherent conclusions.
Collapse
Affiliation(s)
- Mayte Suárez-Fariñas
- Center for Studies in Physics and Biology, The Rockefeller University. 1230 York Ave, Box 212, New York, NY 10021, U.S.A
| | - Scott Noggle
- Department Laboratory of Molecular Vertebrate Embryology, Camridge, The Rockefeller University. 1230 York Ave, Box 32, New York, NY 10021, U.S.A
| | - Michael Heke
- Department Laboratory of Molecular Vertebrate Embryology, Camridge, The Rockefeller University. 1230 York Ave, Box 32, New York, NY 10021, U.S.A
| | - Ali Hemmati-Brivanlou
- Department Laboratory of Molecular Vertebrate Embryology, Camridge, The Rockefeller University. 1230 York Ave, Box 32, New York, NY 10021, U.S.A
| | - Marcelo O Magnasco
- Center for Studies in Physics and Biology, The Rockefeller University. 1230 York Ave, Box 212, New York, NY 10021, U.S.A
| |
Collapse
|