Assessing the quality of published genetic association studies in meta-analyses: the quality of genetic studies (Q-Genie) tool

Background

Advances in genomics technology have led to a dramatic increase in the number of published genetic association studies. Systematic reviews and meta-analyses are a common method of synthesizing findings and providing reliable estimates of the effect of a genetic variant on a trait of interest. However, summary estimates are subject to bias due to the varying methodological quality of individual studies. We embarked on an effort to develop and evaluate a tool that assesses the quality of published genetic association studies. Performance characteristics (i.e. validity, reliability, and item discrimination) were evaluated using a sample of thirty studies randomly selected from a previously conducted systematic review.

Results

The tool demonstrates excellent psychometric properties and generates a quality score for each study with corresponding ratings of ‘low’, ‘moderate’, or ‘high’ quality. We applied our tool to a published systematic review to exclude studies of low quality, and found a decrease in heterogeneity and an increase in precision of summary estimates.

Conclusion

This tool can be used in systematic reviews to inform the selection of studies for inclusion, to conduct sensitivity analyses, and to perform meta-regressions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12863-015-0211-2) contains supplementary material, which is available to authorized users.

Severe bleeding events in adults and children with primary immune thrombocytopenia: a systematic review

BACKGROUND

The burden of severe bleeding in adults and children with immune thrombocytopenia (ITP) has not been established.

OBJECTIVES

To describe the frequency and severity of bleeding events in patients with ITP, and the methods used to measure bleeding in ITP studies.

PATIENTS/METHODS

We performed a systematic review of all prospective ITP studies that enrolled 20 or more patients. Two reviewers searched Medline, Embase, CINAHL and the Cochrane registry up to May 2014. Overall weighted proportions were estimated using a random effects model. Measurement properties of bleeding assessment tools were evaluated.

RESULTS

We identified 118 studies that reported bleeding (n = 10 908 patients). Weighted proportions for intracerebral hemorrhage (ICH) were 1.4% for adults (95% confidence interval [CI], 0.9-2.1%) and 0.4% for children (95% CI, 0.2-0.7%; P < 0.01), most of whom had chronic ITP. The weighted proportion for severe (non-ICH) bleeding was 9.6% for adults (95% CI, 4.1-17.1%) and 20.2% for children (95% CI, 10.0-32.9%; P < 0.01) with newly-diagnosed or chronic ITP. Methods of reporting and definitions of severe bleeding were highly variable in primary studies. Two bleeding assessment tools (Buchanan 2002 for children; Page 2007 for adults) demonstrated adequate inter-rater reliability and validity in independent assessments.

CONCLUSIONS

ICH was more common in adults and tended to occur during chronic ITP; other severe bleeds were more common in children and occurred at all stages of disease. Reporting of non-ICH bleeding was variable across studies. Further attention to ITP-specific bleeding measurement in clinical trials is needed to improve standardization of this important outcome for patients.

Ab initio calculation of shocked xenon reflectivity

Reflectivity of shocked compressed xenon plasma is calculated within the framework of the density functional theory approach. Dependencies on the frequency of incident radiation and on the plasma density are analyzed. The Fresnel formula for the reflectivity is used. The longitudinal expression in the long-wavelength limit is applied for the calculation of the imaginary part of the dielectric function. The real part of the dielectric function is calculated by means of the Kramers-Kronig transformation. The results are compared with experimental data. The approach for the calculation of plasma frequency is developed.

Systematic review of the limited evidence base for treatments of Eustachian tube dysfunction: a health technology assessment

BACKGROUND

The Health Technology Assessment programme commissioned a wide-ranging review of treatments for adult Eustachian tube dysfunction. Treatments range from advice and observation and pharmacological treatments to surgical options.

OBJECTIVE

(i) To assess the evidence for interventions for adults with a clinical diagnosis of Eustachian tube dysfunction and (ii) to identify priorities for future research.

TYPE OF REVIEW

Systematic review (PROSPERO registration CRD42012003035) adhering to PRISMA guidance.

SEARCH

An extensive search of 15 databases including MEDLINE, EMBASE and CENTRAL (up to October 2012).

EVALUATION METHOD

Controlled and uncontrolled studies of interventions for adult Eustachian tube dysfunction were included. Because of insufficient data, the protocol was amended to also include controlled studies with mixed adult/child populations. Risk of bias was assessed. Narrative synthesis was employed due to high clinical heterogeneity.

RESULTS

Interventions assessed were pharmacological treatments [two randomised controlled trials (RCTs), one controlled non-randomised trial (CCT), 159 patients]; mechanical pressure equalisation devices (one randomised controlled trial, one CCT, 48 patients); and surgery, including laser tuboplasty (seven case series, 192 patients), balloon dilatation (three case series, 103 patients), myringotomy without grommet insertion (two case series, 121 patients), transtubal steroids (one case series, 11 patients) and laser coagulation (one retrospective controlled study, 40 patients). All studies had high risk of bias except two pharmacological trials; one had low risk and one unclear risk. No evidence was found for many treatments. The single low risk of bias RCT (n = 91; 67% adults) showed no effect of nasal steroids and favoured placebo for improved middle ear function (RR 1.20, 95% CI 0.91-1.58) and symptoms (P = 0.07). Other studies showed improvements in middle ear function for mechanical devices, antihistamine/ephedrine and nasal decongestant, but they had significant methodological weaknesses including insufficient length of follow-up. None of the surgical studies were adequately controlled, and many reported high levels of co-intervention. Therefore, observed benefits for tuboplasty and balloon dilatation in symptoms, middle ear function or hearing could not be reliably attributed to the interventions assessed. There was variability in definitions of the condition.

CONCLUSION

Eustachian tube dysfunction is a poorly defined condition. Due to the limited and poor-quality evidence, it is inappropriate to make conclusions on the effectiveness of any intervention; the evidence base is insufficient to guide recommendations for a trial of any particular intervention. Consensus on diagnostic criteria for Eustachian tube dysfunction is required to inform inclusion criteria of future trials.

Research challenges in digital education

Simulation and other forms of digital learning will occupy a place of increasing prominence in medical education in the future. However, to maximally use the potential of these media, we must go beyond a research agenda dictated by a 'Does it work?' question to one driven by careful analysis of the nature of the task to be learned and its relation to the characteristics of the technology. Secondly, we must change the focus from the characteristics of individual devices to a broader approach to design of a digital curriculum based on current understanding of the nature of human learning.

Omalizumab for the treatment of severe persistent allergic asthma: a systematic review and economic evaluation

BACKGROUND

Allergic asthma is a long-term disorder of the airways resulting from overexpression of immunoglobulin E (IgE) in response to environmental allergens. Patients with poorly controlled asthma are at high risk of exacerbations requiring additional treatment, including hospitalisations. Severe exacerbations are potentially life threatening. Guidelines identify five treatment steps for both adults and children. Omalizumab (Xolair(®)) is a recombinant DNA-derived humanised monoclonal antibody indicated as an add-on therapy in patients aged ≥ 6 years with severe persistent allergic asthma uncontrolled at treatment step 4 or 5.

OBJECTIVE

To determine the clinical effectiveness, safety and cost-effectiveness of omalizumab, as an add-on therapy to standard care, within its licensed indication, compared with standard therapy alone for the treatment of severe persistent allergic asthma in adults and adolescents aged ≥ 12 years and children aged 6-11 years.

DATA SOURCES

Eleven electronic databases (including MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials) and additional sources including regulatory agency reports were searched from inception to October 2011. Additional data sources include: the manufacturer's submission (MS); two previous National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) submissions; and existing reviews on the safety of omalizumab and oral corticosteroids (OCSs).

REVIEW METHODS

Systematic reviews of the clinical effectiveness and cost-effectiveness evidence for omalizumab were performed. The primary outcome was number of clinically significant (CS) exacerbations. Other outcomes included asthma symptoms, unscheduled health-care use, asthma-related mortality, OCS use and health-related quality of life (HRQoL). Because of methodological and clinical heterogeneity between trials, a narrative synthesis was applied. Pragmatic reviews with best evidence syntheses were used to assess adverse events of omalizumab and OCSs. The cost-effectiveness of omalizumab was assessed from the perspective of the UK NHS in the two separate populations: adults and adolescents, and children, using a cohort Markov model. Costs and outcomes were discounted at 3.5% per annum. Results are presented for additional subgroup populations: (1) hospitalised for asthma in the previous year, (2) adults and adolescents on maintenance OCSs and (3) three or more exacerbations in the previous year.

RESULTS

Eleven randomised controlled trials (RCTs) and 13 observational studies were identified, including four RCTs/subgroups in the adult licensed population and one subgroup in children. A minority of patients were on maintenance OCSs. No evidence comparing omalizumab with OCSs was identified. Omalizumab significantly reduced the incidence of CS exacerbations in both adults and children [adults: INvestigatioN of Omalizumab in seVere Asthma Trial (INNOVATE): rate ratio 0.74; 95% CI 0.55 to 1.00; children IA-05 EUP (the a priori subgroup of patients who met the European Medicines Agency license criteria) 0.66; 95% CI 0.44 to 1.00]. Significant benefits were observed for a range of other outcomes in adults. Subgroup evidence showed benefits in adults on maintenance OCSs. Evidence for an OCS-sparing effect of omalizumab was limited but consistent. Omalizumab is available as 75 mg and 150 mg prefilled syringes at prices of £128.07 and £256.15 respectively. The incremental cost-effectiveness ratio (ICER) for adults and adolescents is £83,822 per quality-adjusted life-year (QALY) gained, whereas the ICER for children is £78,009 per QALY gained. The results are similar for the subgroup population of ≥ 3 exacerbations in the previous year, whereas the ICER for the other subgroup populations are lower; £46,431 for the hospitalisation subgroup in adults and adolescents, £44,142 for the hospitalisation subgroup in children and £50,181 for the maintenance OCS subgroup.

CONCLUSION

Omalizumab reduces the incidence of CS exacerbations in adults and children, with benefits on other outcomes in adults. Limited, underpowered subgroup evidence exists that omalizumab reduces exacerbations and OCS requirements in adults on OCSs. Evidence in children is weaker and more uncertain. The ICERs are above conventional NHS thresholds of cost-effectiveness. The key drivers of cost-effectiveness are asthma-related mortality risk and, to a lesser extent, HRQoL improvement and OCS-related adverse effects. An adequately powered double-blind RCT in both adults and children on maintenance OCSs and an individual patient data meta-analysis of existing trials should be considered. A registry of all patients on omalizumab should be established.

STUDY REGISTRATION

The study was registered as PROSPERO CRD42011001625.

FUNDING

This report was commissioned by the National Institute for Health Research Health Technology Assessment programme on behalf of NICE as project number HTA 10/128/01.

The estimation of the rates of lead exchange between body compartments of smelter employees

The overwhelming proportion of the mass of lead (Pb) is stored in bone and the residence time of Pb in bone is much longer than that in other tissues. Hence, in a metabolic model that we used to solve the differential equations governing the transfer of lead between body compartments, three main compartments are involved: blood (as a transfer compartment), cortical bone (tibia), and trabecular bone (calcaneus). There is a bidirectional connection between blood and the other two compartments. A grid search chi-squared minimization method was used to estimate the initial values of lead transfer rate values from tibia (λTB) and calcaneus (λCB) to blood of 209 smelter employees whose bone lead measurements are available from 1994, 1999, and 2008, and their blood lead level from 1967 onwards (depending on exposure history from once per month to once per year), and then the initial values of kinematic parameters were used to develop multivariate models in order to express λTB and λCB as a function of employment time, age, body lead contents and their interaction. We observed a significant decrease in the transfer rate of lead from bone to blood with increasing body lead contents. The model was tested by calculating the bone lead concentration in 1999 and 2008, and by comparing those values with the measured ones. A good agreement was found between the calculated and measured tibia/calcaneus lead values. Also, we found that the transfer rate of lead from tibia to blood can be expressed solely as a function of cumulative blood lead index.

Conscious versus unconscious thinking in the medical domain: the deliberation-without-attention effect examined

Previous studies have shown that with important decisions, unconscious thought has surprisingly led to better choices than conscious thought. The present study challenges this so-called 'deliberation-without-attention effect' in the medical domain. In a computerized study, physicians and medical students were asked, after either conscious or unconscious thought, to estimate the 5-year survival probabilities of four fictitious patients with varying medical characteristics. We assumed that experienced physicians would outperform students as a result of their superior knowledge. The central question was whether unconscious thought in this task would lead to better performance in experts or novices, in line with the deliberation-without-attention effect. We created four fictitious male 60-year-old patients, each of whom with signs and symptoms related to likely prognosis, from 12 (Complex) or 4 (Simple) categories. This manipulation resulted in objectively different life expectancies for these patients. Participants (86 experienced physicians and 57 medical students) were randomly allocated to the Simple or Complex condition. Statements were randomly presented for 8 s. Next, each participant assessed the life expectancies after either conscious or unconscious thought. As expected, experienced physicians were better in assessing life expectancies than medical students. No significant differences were found in performance between conscious and unconscious thought, nor did we detect a significant interaction between expertise level and mode of thought. In a medical decision task, unconscious thought did not lead to better performance of experienced physicians or medical students than conscious thought. Our findings do not support the deliberation-without-attention effect.

The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses

OBJECTIVE

We evaluated the inter-rater reliability (IRR) of assessing the quality of evidence (QoE) using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach.

STUDY DESIGN AND SETTING

On completing two training exercises, participants worked independently as individual raters to assess the QoE of 16 outcomes. After recording their initial impression using a global rating, raters graded the QoE following the GRADE approach. Subsequently, randomly paired raters submitted a consensus rating.

RESULTS

The IRR without using the GRADE approach for two individual raters was 0.31 (95% confidence interval [95% CI] = 0.21-0.42) among Health Research Methodology students (n = 10) and 0.27 (95% CI = 0.19-0.37) among the GRADE working group members (n = 15). The corresponding IRR of the GRADE approach in assessing the QoE was significantly higher, that is, 0.66 (95% CI = 0.56-0.75) and 0.72 (95% CI = 0.61-0.79), respectively. The IRR further increased for three (0.80 [95% CI = 0.73-0.86] and 0.74 [95% CI = 0.65-0.81]) or four raters (0.84 [95% CI = 0.78-0.89] and 0.79 [95% CI = 0.71-0.85]). The IRR did not improve when QoE was assessed through a consensus rating.

CONCLUSION

Our findings suggest that trained individuals using the GRADE approach improves reliability in comparison to intuitive judgments about the QoE and that two individual raters can reliably assess the QoE using the GRADE system.

Historical factors influencing medical education research productivity

The author explores the reasons why both Canada and the Netherlands have a productivity in medical education research that is disproportionate to their size. He suggests that the historical introduction of problem-based learning early in both countries by McMaster and Maastricht, respectively, with a commitment for programmatic and broader research in both institutions has left this legacy.