Cost-Effectiveness of Routine Monitoring of Long-Term Conditions in Primary Care: Informing Decision Modelling with a Systematic Review in Hypertension, Type 2 Diabetes and Chronic Kidney Disease

BACKGROUND

Long-term conditions (LTCs) are major public health problems with a considerable health-related and economic burden. Modelling is key in assessing costs and benefits of different disease management strategies, including routine monitoring, in the conditions of hypertension, type 2 diabetes mellitus (T2DM) and chronic kidney disease (CKD) in primary care.

OBJECTIVE

This review aimed to identify published model-based cost-effectiveness studies of routine laboratory testing strategies in these LTCs to inform a model evaluating the cost effectiveness of testing strategies in the UK.

METHODS

We searched the Medline and Embase databases from inception to July 2023; the National Institute for Health and Care Institute (NICE) website was also searched. Studies were included if they were model-based economic evaluations, evaluated testing strategies, assessed regular testing, and considered adults aged >16 years. Studies identified were summarised by testing strategies, model type, structure, inputs, assessment of uncertainty, and conclusions drawn.

RESULTS

Five studies were included in the review, i.e. Markov (n = 3) and microsimulation (n = 2) models. Models were applied within T2DM (n = 2), hypertension (n = 1), T2DM/hypertension (n = 1) and CKD (n = 1). Comorbidity between all three LTCs was modelled to varying extents. All studies used a lifetime horizon, except for a 10-year horizon T2DM model, and all used quality-adjusted life-years as the effectiveness outcome, except a TD2M model that used glycaemic control. No studies explicitly provided a rationale for their selected modelling approach. UK models were available for diabetes and CKD, but these compared only a limited set of routine monitoring tests and frequencies.

CONCLUSIONS

There were few studies comparing routine testing strategies in the UK, indicating a need to develop a novel model in all three LTCs. Justification for the modelling technique of the identified studies was lacking. Markov and microsimulation models, with and without comorbidities, were used; however, the findings of this review can provide data sources and inform modelling approaches for evaluating the cost effectiveness of testing strategies in all three LTCs.

Identifying the Optimum Strategy for Identifying Adults and Children With Celiac Disease: A Cost-Effectiveness and Value of Information Analysis

OBJECTIVES

Celiac disease (CD) is thought to affect around 1% of people in the United Kingdom, but only approximately 30% are diagnosed. The aim of this work was to assess the cost-effectiveness of strategies for identifying adults and children with CD in terms of who to test and which tests to use.

METHODS

A decision tree and Markov model were used to describe testing strategies and model long-term consequences of CD. The analysis compared a selection of pre-test probabilities of CD above which patients should be screened, as well as the use of different serological tests, with or without genetic testing. Value of information analysis was used to prioritize parameters for future research.

RESULTS

Using serological testing alone in adults, immunoglobulin A (IgA) tissue transglutaminase (tTG) at a 1% pre-test probability (equivalent to population screening) was most cost-effective. If combining serological testing with genetic testing, human leukocyte antigen combined with IgA tTG at a 5% pre-test probability was most cost-effective. In children, the most cost-effective strategy was a 10% pre-test probability with human leukocyte antigen plus IgA tTG. Value of information analysis highlighted the probability of late diagnosis of CD and the accuracy of serological tests as important parameters. The analysis also suggested prioritizing research in adult women over adult men or children.

CONCLUSIONS

For adults, these cost-effectiveness results suggest UK National Screening Committee Criteria for population-based screening for CD should be explored. Substantial uncertainty in the results indicate a high value in conducting further research.

Risk of bias assessments in individual participant data meta-analyses of test accuracy and prediction models: a review shows improvements are needed

OBJECTIVES

Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement.

STUDY DESIGN AND SETTING

We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs.

RESULTS

Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD.

CONCLUSION

Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.

Exploration of reasons for primary care testing (the Why Test study): a UK-wide audit using the Primary care Academic CollaboraTive

BACKGROUND

Rates of blood testing have increased over the past two decades. Reasons for testing cannot easily be extracted from electronic health record databases.

AIM

To explore who requests blood tests and why, and what the outcomes of testing are in UK primary care.

DESIGN AND SETTING

A retrospective audit of electronic health records in general practices in England, Wales, Scotland, and Northern Ireland was undertaken.

METHOD

Fifty-seven clinicians from the Primary care Academic CollaboraTive (PACT) each reviewed the electronic health records of 50 patients who had blood tests in April 2021. Anonymised data were extracted including patient characteristics, who requested the tests, reasons for testing, test results, and outcomes of testing.

RESULTS

Data were collected from 2572 patients across 57 GP practices. The commonest reasons for testing in primary care were investigation of symptoms (43.2%), monitoring of existing disease (30.1%), monitoring of existing medications (10.1%), and follow up of previous abnormalities (6.8%); patient requested testing was rare in this study (1.5%). Abnormal and borderline results were common, with 26.6% of patients having completely normal test results. Around one-quarter of tests were thought to be partially or fully unnecessary when reviewed retrospectively by a clinical colleague. Overall, 6.2% of tests in primary care led to a new diagnosis or confirmation of a diagnosis.

CONCLUSION

The utilisation of a national collaborative model (PACT) has enabled a unique exploration of the rationale and outcomes of blood testing in primary care, highlighting areas for future research and optimisation.

Variation in laboratory testing for patients with long-term conditions: a longitudinal cohort study in UK primary care

BACKGROUND

Use of laboratory testing has increased in the UK over the past few decades, with considerable geographical variation.

AIM

To evaluate what laboratory tests are used to monitor people with hypertension, type 2 (T2) diabetes, or chronic kidney disease (CKD) and assess variation in test use in UK primary care.

DESIGN & SETTING

Longitudinal cohort study of people registered with UK general practices between June 2013 and May 2018 and previously diagnosed with hypertension, T2 diabetes, or CKD.

METHOD

Clinical Practice Research Datalink (CPRD) primary care data linked to ethnic group and deprivation was used to examine testing rates over time, by GP practice, age, sex, ethnic group, and socioeconomic deprivation, with age-sex standardisation.

RESULTS

Nearly 1 million patients were included, and more than 27 million tests. The most ordered tests were for renal function (1463 per 1000 person-years), liver function (1063 per 1000 person-years), and full blood count (FBC; 996 per 1000 person-years). There was evidence of undertesting (compared with current guidelines) for HbA1c and albumin:creatinine ratio (ACR) or microalbumin, and potential overtesting of lipids, FBC, liver function, and thyroid function. Some GP practices had up to 27 times higher testing rates than others (HbA1c testing among patients with CKD).

CONCLUSION

Testing rates are no longer increasing, but they are not always within the guidelines for monitoring long-term conditions (LTCs). There was considerable variation by GP practice, indicating uncertainty over the most appropriate testing frequencies for different conditions. Standardising the monitoring of LTCs based on the latest evidence would provide greater consistency of access to monitoring tests.

Defining the optimum strategy for identifying adults and children with coeliac disease: systematic review and economic modelling

BACKGROUND

Coeliac disease is an autoimmune disorder triggered by ingesting gluten. It affects approximately 1% of the UK population, but only one in three people is thought to have a diagnosis. Untreated coeliac disease may lead to malnutrition, anaemia, osteoporosis and lymphoma.

OBJECTIVES

The objectives were to define at-risk groups and determine the cost-effectiveness of active case-finding strategies in primary care.

DESIGN

(1) Systematic review of the accuracy of potential diagnostic indicators for coeliac disease. (2) Routine data analysis to develop prediction models for identification of people who may benefit from testing for coeliac disease. (3) Systematic review of the accuracy of diagnostic tests for coeliac disease. (4) Systematic review of the accuracy of genetic tests for coeliac disease (literature search conducted in April 2021). (5) Online survey to identify diagnostic thresholds for testing, starting treatment and referral for biopsy. (6) Economic modelling to identify the cost-effectiveness of different active case-finding strategies, informed by the findings from previous objectives.

DATA SOURCES

For the first systematic review, the following databases were searched from 1997 to April 2021: MEDLINE® (National Library of Medicine, Bethesda, MD, USA), Embase® (Elsevier, Amsterdam, the Netherlands), Cochrane Library, Web of Science™ (Clarivate™, Philadelphia, PA, USA), the World Health Organization International Clinical Trials Registry Platform ( WHO ICTRP ) and the National Institutes of Health Clinical Trials database. For the second systematic review, the following databases were searched from January 1990 to August 2020: MEDLINE, Embase, Cochrane Library, Web of Science, Kleijnen Systematic Reviews ( KSR ) Evidence, WHO ICTRP and the National Institutes of Health Clinical Trials database. For prediction model development, Clinical Practice Research Datalink GOLD, Clinical Practice Research Datalink Aurum and a subcohort of the Avon Longitudinal Study of Parents and Children were used; for estimates for the economic models, Clinical Practice Research Datalink Aurum was used.

REVIEW METHODS

For review 1, cohort and case-control studies reporting on a diagnostic indicator in a population with and a population without coeliac disease were eligible. For review 2, diagnostic cohort studies including patients presenting with coeliac disease symptoms who were tested with serological tests for coeliac disease and underwent a duodenal biopsy as reference standard were eligible. In both reviews, risk of bias was assessed using the quality assessment of diagnostic accuracy studies 2 tool. Bivariate random-effects meta-analyses were fitted, in which binomial likelihoods for the numbers of true positives and true negatives were assumed.

RESULTS

People with dermatitis herpetiformis, a family history of coeliac disease, migraine, anaemia, type 1 diabetes, osteoporosis or chronic liver disease are 1.5-2 times more likely than the general population to have coeliac disease; individual gastrointestinal symptoms were not useful for identifying coeliac disease. For children, women and men, prediction models included 24, 24 and 21 indicators of coeliac disease, respectively. The models showed good discrimination between patients with and patients without coeliac disease, but performed less well when externally validated. Serological tests were found to have good diagnostic accuracy for coeliac disease. Immunoglobulin A tissue transglutaminase had the highest sensitivity and endomysial antibody the highest specificity. There was little improvement when tests were used in combination. Survey respondents (n = 472) wanted to be 66% certain of the diagnosis from a blood test before starting a gluten-free diet if symptomatic, and 90% certain if asymptomatic. Cost-effectiveness analyses found that, among adults, and using serological testing alone, immunoglobulin A tissue transglutaminase was most cost-effective at a 1% pre-test probability (equivalent to population screening). Strategies using immunoglobulin A endomysial antibody plus human leucocyte antigen or human leucocyte antigen plus immunoglobulin A tissue transglutaminase with any pre-test probability had similar cost-effectiveness results, which were also similar to the cost-effectiveness results of immunoglobulin A tissue transglutaminase at a 1% pre-test probability. The most practical alternative for implementation within the NHS is likely to be a combination of human leucocyte antigen and immunoglobulin A tissue transglutaminase testing among those with a pre-test probability above 1.5%. Among children, the most cost-effective strategy was a 10% pre-test probability with human leucocyte antigen plus immunoglobulin A tissue transglutaminase, but there was uncertainty around the most cost-effective pre-test probability. There was substantial uncertainty in economic model results, which means that there would be great value in conducting further research.

LIMITATIONS

The interpretation of meta-analyses was limited by the substantial heterogeneity between the included studies, and most included studies were judged to be at high risk of bias. The main limitations of the prediction models were that we were restricted to diagnostic indicators that were recorded by general practitioners and that, because coeliac disease is underdiagnosed, it is also under-reported in health-care data. The cost-effectiveness model is a simplification of coeliac disease and modelled an average cohort rather than individuals. Evidence was weak on the probability of routine coeliac disease diagnosis, the accuracy of serological and genetic tests and the utility of a gluten-free diet.

CONCLUSIONS

Population screening with immunoglobulin A tissue transglutaminase (1% pre-test probability) and of immunoglobulin A endomysial antibody followed by human leucocyte antigen testing or human leucocyte antigen testing followed by immunoglobulin A tissue transglutaminase with any pre-test probability appear to have similar cost-effectiveness results. As decisions to implement population screening cannot be made based on our economic analysis alone, and given the practical challenges of identifying patients with higher pre-test probabilities, we recommend that human leucocyte antigen combined with immunoglobulin A tissue transglutaminase testing should be considered for adults with at least a 1.5% pre-test probability of coeliac disease, equivalent to having at least one predictor. A more targeted strategy of 10% pre-test probability is recommended for children (e.g. children with anaemia).

FUTURE WORK

Future work should consider whether or not population-based screening for coeliac disease could meet the UK National Screening Committee criteria and whether or not it necessitates a long-term randomised controlled trial of screening strategies. Large prospective cohort studies in which all participants receive accurate tests for coeliac disease are needed.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42019115506 and CRD42020170766.

FUNDING

This project was funded by the National Institute for Health and Care Research ( NIHR ) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 44. See the NIHR Journals Library website for further project information.

Blood tests in primary care: A qualitative study of communication and decision-making between doctors and patients

Objective

Blood tests are commonly used in primary care as a tool to aid diagnosis, and to offer reassurance and validation for patients. If doctors and patients do not have a shared understanding of the reasons for testing and the meaning of results, these aims may not be fulfilled. Shared decision‐making is widely advocated; yet, most research focusses on treatment decisions rather than diagnostic decisions. The aim of this study was to explore communication and decision‐making around diagnostic blood tests in primary care.

Methods

Qualitative interviews were undertaken with patients and clinicians in UK primary care. Patients were interviewed at the time of blood testing, with a follow‐up interview after they received test results. Interviews with clinicians who requested the tests provided paired data to compare clinicians' and patients' expectations, experiences and understandings of tests. Interviews were analysed thematically using inductive and deductive coding.

Results

A total of 80 interviews with 28 patients and 19 doctors were completed. We identified a mismatch in expectations and understanding of tests, which led to downstream consequences including frustration, anxiety and uncertainty for patients. There was no evidence of shared decision‐making in consultations preceding the decision to test. Doctors adopted a paternalistic approach, believing that they were protecting patients from anxiety.

Conclusion

Patients were not able to develop informed preferences and did not perceive that choice is possible in decisions about testing, because they did not have sufficient information and a shared understanding of tests. A lack of shared understanding at the point of decision‐making led to downstream consequences when test results did not fulfil patients' expectations. Although shared decision‐making is recommended as best practice, it does not reflect the reality of doctors' and patients' accounts of testing; a broader model of shared understanding seems to be more relevant to the complexity of primary care diagnosis.

Patient or Public Contribution

A patient and public involvement group comprising five participants with lived experience of blood testing in primary care met regularly during the study. They contributed to the development of the research objectives, planning recruitment methods, reviewing patient information leaflets and topic guides and also contributed to discussion of emerging themes at an early stage in the analysis process.

'I guess I'll wait to hear'- communication of blood test results in primary care a qualitative study

BACKGROUND

Rates of blood testing in primary care are rising. Communicating blood test results generates significant workload for patients, GPs, and practice staff.

AIM

To explore GPs' and patients' experience of systems of blood test communication.

DESIGN AND SETTING

Qualitative interviews with patients and GPs in UK primary care in both urban and rural practices in the West of England.

METHOD

A total of 28 patients and 19 GPs from six practices were recruited, with a range of socioeconomic and demographic characteristics. Patients were interviewed at two time points: a) at or soon after their blood test and b) after they had received their test results. The GPs who requested the tests were also interviewed (they could complete a maximum of two interviews about different patients). Eighty qualitative interviews were undertaken; 54 patient interviews and 26 GP interviews.

RESULTS

Methods of test result communication varied between doctors and were based on habits, unwritten heuristics, and personal preferences rather than protocols. Doctors expected patients to know how to access their test results. In contrast, patients were often uncertain and used guesswork to decide when and how to access their tests. Patients and doctors generally assumed that the other party would make contact, with potential implications for patient safety. Text messaging and online methods of communication have benefits, but were perceived by some patients as 'flippant' or 'confusing'. Delays and difficulties obtaining and interpreting test results can lead to anxiety and frustration for patients.

CONCLUSION

Current systems of test result communication are complex and confusing, and mostly based on habits and routines rather than clear protocols. This has important implications for patient-centred care and patient safety.

Systematic review with meta-analysis: the accuracy of serological tests to support the diagnosis of coeliac disease

BACKGROUND

There is growing support for a biopsy avoidant approach to diagnose coeliac disease in both children and adults, using a serological diagnosis instead.

AIMS

To assess the diagnostic accuracy of serological tests for coeliac disease in adults and children.

METHODS

Seven electronic databases were searched between January 1990 and August 2020. Eligible diagnostic studies evaluated the accuracy of serological tests for coeliac disease against duodenal biopsy. Risk of bias assessment was performed using QUADAS-2. Bivariate random-effects meta-analyses were used to estimate serology sensitivity and specificity at the most commonly reported thresholds.

RESULTS

113 studies (n = 28,338) were included, all in secondary care populations. A subset of studies were included in meta-analyses due to variations in diagnostic thresholds. Summary sensitivity and specificity of immunoglobulin A (IgA) anti-tissue transglutaminase were 90.7% (95% confidence interval: 87.3%, 93.2%) and 87.4% (84.4%, 90.0%) in adults (5 studies) and 97.7% (91.0%, 99.4%) and 70.2% (39.3%, 89.6%) in children (6 studies); and of IgA endomysial antibodies were 88.0% (75.2%, 94.7%) and 99.6% (92.3%, 100%) in adults (5 studies) and 94.5% (88.9%, 97.3%) and 93.8% (85.2%, 97.5%) in children (5 studies).

CONCLUSIONS

Anti-tissue transglutaminase sensitivity appears to be sufficient to rule out coeliac disease in children. The high specificity of endomysial antibody in adults supports its use to rule in coeliac disease. This evidence underpins the current development of clinical guidelines for a serological diagnosis of coeliac disease. Studies in primary care are needed to evaluate serological testing strategies in this setting.

QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies

Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.

Agreement was moderate between data-based and opinion-based assessments of biases affecting randomized trials within meta-analyses

BACKGROUND AND OBJECTIVE

Randomized trials included in meta-analyses are often affected by bias caused by methodological flaws or limitations, but the degree of bias is unknown. Two proposed methods adjust the trial results for bias using empirical evidence from published meta-epidemiological studies or expert opinion.

METHODS

We investigated agreement between data-based and opinion-based approaches to assessing bias in each of four domains: sequence generation, allocation concealment, blinding, and incomplete outcome data. From each sampled meta-analysis, a pair of trials with the highest and lowest empirical model-based bias estimates was selected. Independent assessors were asked which trial within each pair was judged more biased on the basis of detailed trial design summaries.

RESULTS

Assessors judged trials to be equally biased in 68% of pairs evaluated. When assessors judged one trial as more biased, the proportion of judgments agreeing with the model-based ranking was highest for allocation concealment (79%) and blinding (79%) and lower for sequence generation (59%) and incomplete outcome data (56%).

CONCLUSION

Most trial pairs found to be discrepant empirically were judged to be equally biased by assessors. We found moderate agreement between opinion and data-based evidence in pairs where assessors ranked one trial as more biased.

PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

Quality of family relationships and outcomes of dementia: a systematic review

OBJECTIVES

To evaluate the association between the quality of relationship between a person with dementia and their family carer and outcomes for the person with dementia.

DESIGN

Systematic review.

ELIGIBILITY CRITERIA

Cohort studies of people with clinically diagnosed dementia and their main carers. Exposures of interest were any elements of relationship quality, for example, attachment style, expressed emotion and coping style. Our primary outcome was institutionalisation, and secondary outcomes were hospitalisation, death, quality of life and behavioural and psychiatric symptoms of dementia ('challenging behaviour').

DATA SOURCES

MEDLINE, Embase, Web of Science, PsycInfo, the Cochrane Library and Opengrey were searched from inception to May 2017.

STUDY APPRAISAL AND SYNTHESIS METHODS

The Newcastle-Ottawa Scale was used to assess risk of bias. A narrative synthesis of results was performed due to differences between studies.

RESULTS

Twenty studies were included. None of the studies controlled for all prespecified confounding factors (age, gender, socioeconomic status and severity of dementia). Reporting of results was inadequate with many studies simply reporting whether associations were 'statistically significant' without providing effect size estimates or CIs. There was a suggestion of an association between relationship factors and global challenging behaviour. All studies evaluating global challenging behaviour provided statistical evidence of an association (most P values below 0.02). There was no consistent evidence for an association for any other outcome assessed.

CONCLUSIONS

There is currently no strong or consistent evidence on the effects of relationship factors on institutionalisation, hospitalisation, death or quality of life for people with dementia. There was a suggestion of an association between relationship factors and challenging behaviour, although the evidence for this was weak. To improve our ability to support those with dementia and their families, further robust studies are needed.

PROSPERO REGISTRATION NUMBER

CRD42015020518.

How well do health professionals interpret diagnostic information? A systematic review

OBJECTIVE

To evaluate whether clinicians differ in how they evaluate and interpret diagnostic test information.

DESIGN

Systematic review.

DATA SOURCES

MEDLINE, EMBASE and PsycINFO from inception to September 2013; bibliographies of retrieved studies, experts and citation search of key included studies.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Primary studies that provided information on the accuracy of any diagnostic test (eg, sensitivity, specificity, likelihood ratios) to health professionals and that reported outcomes relating to their understanding of information on or implications of test accuracy.

RESULTS

We included 24 studies. 6 assessed ability to define accuracy metrics: health professionals were less likely to identify the correct definition of likelihood ratios than of sensitivity and specificity. -25 studies assessed Bayesian reasoning. Most assessed the influence of a positive test result on the probability of disease: they generally found health professionals' estimation of post-test probability to be poor, with a tendency to overestimation. 3 studies found that approaches based on likelihood ratios resulted in more accurate estimates of post-test probability than approaches based on estimates of sensitivity and specificity alone, while 3 found less accurate estimates. 5 studies found that presenting natural frequencies rather than probabilities improved post-test probability estimation and speed of calculations.

CONCLUSIONS

Commonly used measures of test accuracy are poorly understood by health professionals. Reporting test accuracy using natural frequencies and visual aids may facilitate improved understanding and better estimation of the post-test probability of disease.

Cannabinoids for Medical Use: A Systematic Review and Meta-analysis

IMPORTANCE

Cannabis and cannabinoid drugs are widely used to treat disease or alleviate symptoms, but their efficacy for specific indications is not clear.

OBJECTIVE

To conduct a systematic review of the benefits and adverse events (AEs) of cannabinoids.

DATA SOURCES

Twenty-eight databases from inception to April 2015.

STUDY SELECTION

Randomized clinical trials of cannabinoids for the following indications: nausea and vomiting due to chemotherapy, appetite stimulation in HIV/AIDS, chronic pain, spasticity due to multiple sclerosis or paraplegia, depression, anxiety disorder, sleep disorder, psychosis, glaucoma, or Tourette syndrome.

DATA EXTRACTION AND SYNTHESIS

Study quality was assessed using the Cochrane risk of bias tool. All review stages were conducted independently by 2 reviewers. Where possible, data were pooled using random-effects meta-analysis.

MAIN OUTCOMES AND MEASURES

Patient-relevant/disease-specific outcomes, activities of daily living, quality of life, global impression of change, and AEs.

RESULTS

A total of 79 trials (6462 participants) were included; 4 were judged at low risk of bias. Most trials showed improvement in symptoms associated with cannabinoids but these associations did not reach statistical significance in all trials. Compared with placebo, cannabinoids were associated with a greater average number of patients showing a complete nausea and vomiting response (47% vs 20%; odds ratio [OR], 3.82 [95% CI, 1.55-9.42]; 3 trials), reduction in pain (37% vs 31%; OR, 1.41 [95% CI, 0.99-2.00]; 8 trials), a greater average reduction in numerical rating scale pain assessment (on a 0-10-point scale; weighted mean difference [WMD], -0.46 [95% CI, -0.80 to -0.11]; 6 trials), and average reduction in the Ashworth spasticity scale (WMD, -0.36 [95% CI, -0.69 to -0.05]; 7 trials). There was an increased risk of short-term AEs with cannabinoids, including serious AEs. Common AEs included dizziness, dry mouth, nausea, fatigue, somnolence, euphoria, vomiting, disorientation, drowsiness, confusion, loss of balance, and hallucination.

CONCLUSIONS AND RELEVANCE

There was moderate-quality evidence to support the use of cannabinoids for the treatment of chronic pain and spasticity. There was low-quality evidence suggesting that cannabinoids were associated with improvements in nausea and vomiting due to chemotherapy, weight gain in HIV infection, sleep disorders, and Tourette syndrome. Cannabinoids were associated with an increased risk of short-term AEs.

Using clinical practice variations as a method for commissioners and clinicians to identify and prioritise opportunities for disinvestment in health care: a cross-sectional study, systematic reviews and qualitative study

Background

NHS expenditure has stagnated since the economic crisis of 2007, resulting in financial pressures. One response is for policy-makers to regulate use of existing health-care technologies and disinvest from inefficiently used health technologies. A key challenge to disinvestment is to identify existing health technologies with uncertain cost-effectiveness.

Objectives

We aimed to explore if geographical variation in procedure rates is a marker of clinical uncertainty and might be used by local commissioners to identify procedures that are potential candidates for disinvestment. We also explore obstacles and solutions to local commissioners achieving disinvestment, and patient and clinician perspectives on regulating access to procedures.

Methods

We used Hospital Episode Statistics to measure geographical variation in procedure rates from 2007/8 to 2011/12. Expected procedure numbers for each primary care trust (PCT) were calculated adjusting for proxies of need. Random effects Poisson regression quantified the residual inter-PCT procedure rate variability. We benchmarked local procedure rates in two PCTs against national rates. We conducted rapid systematic reviews of two high-use procedures selected by the PCTs [carpal tunnel release (CTR) and laser capsulotomy], searching bibliographical databases to identify systematic reviews and randomised controlled trials (RCTs). We conducted non-participant overt observations of commissioning meetings and semistructured interviews with stakeholders about disinvestment in general and with clinicians and patients about one disinvestment case study. Transcripts were analysed thematically using constant comparison methods derived from grounded theory.

Results

There was large inter-PCT variability in procedure rates for many common NHS procedures. Variation in procedure rates was highest where the diffusion or discontinuance was rapidly evolving and where substitute procedures were available, suggesting that variation is a proxy for clinical uncertainty about appropriate use. In both PCTs we identified procedures where high local use might represent an opportunity for disinvestment. However, there were barriers to achieving disinvestment in both procedure case studies. RCTs comparing CTR with conservative care indicated that surgery was clinically effective and cost-effective on average but provided limited evidence on patient subgroups to inform commissioning criteria and achieve savings. We found no RCTs of laser capsulotomy. The apparently high rate of capsulotomy was probably due to the coding inaccuracy; some savings might be achieved by greater use of outpatient procedures. Commissioning meetings were dominated by new funding requests. Benchmarking did not appear to be routinely carried out because of capacity issues and concerns about data reliability. Perceived barriers to disinvestment included lack of collaboration, central support and tools for disinvestment. Clinicians felt threshold criteria had little impact on their practice and that prior approval systems would not be cost-effective. Most patients were unaware of rationing.

Conclusions

Policy-makers could use geographical variation as a starting point to identify procedures where health technology reassessment or RCTs might be needed to inform policy. Commissioners can use benchmarking to identify procedures with high local use, possibly indicating overtreatment. However, coding inconsistency and limited evidence are major barriers to achieving disinvestment through benchmarking. Increased central support for commissioners to tackle disinvestment is needed, including tools, accurate data and relevant evidence. Early engagement with patients and clinicians is essential for successful local disinvestment.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Systematic review and meta-analysis of the current evidence on the duration of protection by bacillus Calmette-Guérin vaccination against tuberculosis

BACKGROUND

Recent evidence suggests that the duration of protection by bacillus Calmette-Guérin (BCG) may exceed previous estimates with potential implications for estimating clinical and cost-efficacy.

OBJECTIVES

To estimate the protection and duration of protection provided by BCG vaccination against tuberculosis, explore how this protection changes with time since vaccination, and examine the reasons behind the variation in protection and the rate of waning of protection.

DATA SOURCES

Electronic databases including MEDLINE, Excerpta Medica Database (EMBASE), Cochrane Databases, NHS Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE), Web of Knowledge, Biosciences Information Service (BIOSIS), Latin American and Caribbean Health Sciences Literature (LILACs), MEDCARIB Database, Cumulative Index to Nursing and Allied Health Literature (CINAHL) were searched from inception to May 2009. Index to Theses, System for Information on Grey Literature in Europe (SIGLE), Centre for Agricultural Bioscience International (CABI) Abstracts, Scopus, Article First, Academic Complete, Africa-Wide Information, Google Scholar, Global Health, British National Bibliography for Report Literature, and clinical trial registration websites were searched from inception to October 2009.

REVIEW METHODS

Electronic databases searches, screening of identified studies, data extraction and analysis were undertaken. Meta-analysis was used to present numerical and graphical summaries of clinical efficacy and efficacy by time since vaccination. Evidence of heterogeneity was assessed using the tau-squared statistic. Meta-regression allowed the investigation of observed heterogeneity. Factors investigated included BCG strain, latitude, stringency of pre-BCG vaccination tuberculin testing, age at vaccination, site of disease, study design and vulnerability to biases. Rate of waning of protection was estimated using the ratio of the measure of efficacy after 10 years compared with the efficacy in the first 10 years of a study.

RESULTS

Study selection. A total of 21,030 references were identified, providing data on 132 studies after abstract and full-text review. Efficacy. Protection against pulmonary tuberculosis in adults is variable, ranging from substantial protection in the UK MRC trial {rate ratio 0.22 [95% confidence interval (CI) 0.16 to 0.31]}, to absence of clinically important benefit, as in the large Chingleput trial [rate ratio 1.05 (95% CI 0.88 to 1.25)] and greater in latitudes further away from the equator. BCG vaccination efficacy was usually high, and varied little by form of disease (with higher protection against meningeal and miliary tuberculosis) or study design when BCG vaccination was given only to infants or to children after strict screening for tuberculin sensitivity. High levels of protection against death were observed from both trials and observational studies. The observed protective effect of BCG vaccination did not differ by the strain of BCG vaccine used in trials.

DURATION

Reviewed studies showed that BCG vaccination protects against pulmonary and extrapulmonary tuberculosis for up to 10 years. Most studies either did not follow up participants for long enough or had very few cases after 15 years. This should not be taken to indicate an absence of effect: five studies (one trial and four observational studies) provided evidence of measurable protection at least 15 years after vaccination. Efficacy declined with time. The rate of decline was variable, with faster decline in latitudes further from the equator and in situations where BCG vaccination was given to tuberculin-sensitive participants after stringent tuberculin testing.

LIMITATIONS

The main limitation of this review relates to quality of included trials, most of which were conducted before current standards for reporting were formulated. In addition, data were lacking in some areas and the review had to rely on evidence from observational studies.

CONCLUSIONS

BCG vaccination protection against tuberculosis varies between populations, to an extent that cannot be attributed to chance alone. Failure to exclude those already sensitised to mycobacteria and study latitude closer to the equator were associated with lower efficacy. These factors explained most of the observed variation. There is good evidence that BCG vaccination protection declines with time and that protection can last for up to 10 years. Data on protection beyond 15 years are limited; however, a small number of trials and observational studies suggest that BCG vaccination may protect for longer. Further studies are required to investigate the duration of protection by BCG vaccination.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Protection by BCG vaccine against tuberculosis: a systematic review of randomized controlled trials

BACKGROUND

Randomized trials assessing BCG vaccine protection against tuberculosis have widely varying results, for reasons that are not well understood.

METHODS

We examined associations of trial setting and design with BCG efficacy against pulmonary and miliary or meningeal tuberculosis by conducting a systematic review, meta-analyses, and meta-regression.

RESULTS

We identified 18 trials reporting pulmonary tuberculosis and 6 reporting miliary or meningeal tuberculosis. Univariable meta-regression indicated efficacy against pulmonary tuberculosis varied according to 3 characteristics. Protection appeared greatest in children stringently tuberculin tested, to try to exclude prior infection with Mycobacterium tuberculosis or sensitization to environmental mycobacteria (rate ratio [RR], 0.26; 95% confidence interval [CI], .18-.37), or infants (RR, 0.41; 95% CI, .29-.58). Protection was weaker in children not stringently tested (RR, 0.59; 95% CI, .35-1.01) and older individuals stringently or not stringently tested (RR, 0.88; 95% CI, .59-1.31 and RR, 0.81; 95% CI, .55-1.22, respectively). Protection was higher in trials further from the equator where environmental mycobacteria are less and with lower risk of diagnostic detection bias. These associations were attenuated in a multivariable model, but each had an independent effect. There was no evidence that efficacy was associated with BCG strain. Protection against meningeal and miliary tuberculosis was also high in infants (RR, 0.1; 95% CI, .01-.77) and children stringently tuberculin tested (RR, 0.08; 95% CI, .03-.25).

CONCLUSIONS

Absence of prior M. tuberculosis infection or sensitization with environmental mycobacteria is associated with higher efficacy of BCG against pulmonary tuberculosis and possibly against miliary and meningeal tuberculosis. Evaluations of new tuberculosis vaccines should account for the possibility that prior infection may mask or block their effects.

A systematic review classifies sources of bias and variation in diagnostic test accuracy studies

OBJECTIVE

To classify the sources of bias and variation and to provide an updated summary of the evidence of the effects of each source of bias and variation.

STUDY DESIGN AND SETTING

We conducted a systematic review of studies of any design with the main objective of addressing bias or variation in the results of diagnostic accuracy studies. We searched MEDLINE, EMBASE, BIOSIS, the Cochrane Methodology Register, and Database of Abstracts of Reviews of Effects (DARE) from 2001 to October 2011. Citation searches based on three key papers were conducted, and studies from our previous review (search to 2001) were eligible. One reviewer extracted data on the study design, objective, sources of bias and/or variation, and results. A second reviewer checked the extraction.

RESULTS

We summarized the number of studies providing evidence of an effect arising from each source of bias and variation on the estimates of sensitivity, specificity, and overall accuracy.

CONCLUSIONS

We found consistent evidence for the effects of case-control design, observer variability, availability of clinical information, reference standard, partial and differential verification bias, demographic features, and disease prevalence and severity. Effects were generally stronger for sensitivity than for specificity. Evidence for other sources of bias and variation was limited.

QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies

In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk of bias, and the first 3 domains are also assessed in terms of concerns regarding applicability. Signalling questions are included to help judge risk of bias. The QUADAS-2 tool is applied in 4 phases: summarize the review question, tailor the tool and produce review-specific guidance, construct a flow diagram for the primary study, and judge bias and applicability. This tool will allow for more transparent rating of bias and applicability of primary diagnostic accuracy studies.

Systematic review: accuracy of anti-citrullinated Peptide antibodies for diagnosing rheumatoid arthritis

BACKGROUND

Early recognition and treatment of rheumatoid arthritis is important to prevent irreversible joint damage. Anti-citrullinated peptide antibodies (ACPA) have been suggested for early diagnosis.

PURPOSE

To compare the accuracy of ACPA and rheumatoid factor in diagnosing rheumatoid arthritis in patients with early symptoms of the disease.

DATA SOURCES

10 medical databases from inception to September 2009, with no language or publication restrictions, and references of included studies.

STUDY SELECTION

Two independent reviewers screened searches. Full articles were assessed by one reviewer and checked by a second reviewer to identify studies that reported 2 x 2 data on ACPA for the diagnosis of rheumatoid arthritis (by 1987 American College of Rheumatology criteria).

DATA EXTRACTION

One reviewer abstracted data on patient characteristics, ACPA details, and 2 x 2 data and assessed study quality by using the QUADAS tool. A second reviewer checked extractions.

DATA SYNTHESIS

151 studies were included, with considerable heterogeneity in sensitivity (range, 12% to 93%) and specificity (range, 63% to 100%). In cohort studies that investigated second-generation anti-cyclic citrullinated peptide antibodies (anti-CCP2) in patients with early rheumatoid arthritis (<2 years), summary sensitivity and specificity were 57% (95% CI, 51% to 63%) and 96% (CI, 93% to 97%), respectively. Case-control and cross-sectional studies and studies of patients with established rheumatoid arthritis all overestimated sensitivity. Anti-CCP2 had greater specificity than rheumatoid factor (96% vs. 86%), with similar sensitivity. Evidence was insufficient to ascertain whether the combination of anti-CCP2 and rheumatoid factor provides additional benefit over anti-CCP2 alone.

LIMITATIONS

Most studies used a diagnostic case-control design, which overestimated sensitivity. Items relating to study quality were rarely reported. Publication bias could not be assessed.

CONCLUSION

Anti-CCP2 should be included in the work-up of patients with early symptoms of rheumatoid arthritis.

Graphical presentation of diagnostic information

BACKGROUND

Graphical displays of results allow researchers to summarise and communicate the key findings of their study. Diagnostic information should be presented in an easily interpretable way, which conveys both test characteristics (diagnostic accuracy) and the potential for use in clinical practice (predictive value).

METHODS

We discuss the types of graphical display commonly encountered in primary diagnostic accuracy studies and systematic reviews of such studies, and systematically review the use of graphical displays in recent diagnostic primary studies and systematic reviews.

RESULTS

We identified 57 primary studies and 49 systematic reviews. Fifty-six percent of primary studies and 53% of systematic reviews used graphical displays to present results. Dot-plot or box-and- whisker plots were the most commonly used graph in primary studies and were included in 22 (39%) studies. ROC plots were the most common type of plot included in systematic reviews and were included in 22 (45%) reviews. One primary study and five systematic reviews included a probability-modifying plot.

CONCLUSION

Graphical displays are currently underused in primary diagnostic accuracy studies and systematic reviews of such studies. Diagnostic accuracy studies need to include multiple types of graphic in order to provide both a detailed overview of the results (diagnostic accuracy) and to communicate information that can be used to inform clinical practice (predictive value). Work is required to improve graphical displays, to better communicate the utility of a test in clinical practice and the implications of test results for individual patients.

Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies

Background

A quality assessment tool for diagnostic accuracy studies, named QUADAS, has recently been developed. Although QUADAS has been used in several systematic reviews, it has not been formally validated. The objective was to evaluate the validity and usefulness of QUADAS.

Methods

Three reviewers independently rated the quality of 30 studies using QUADAS. We assessed the proportion of agreements between each reviewer and the final consensus rating. This was done for all QUADAS items combined and for each individual item. Twenty reviewers who had used QUADAS in their reviews completed a short structured questionnaire on their experience of QUADAS.

Results

Over all items, the agreements between each reviewer and the final consensus rating were 91%, 90% and 85%. The results for individual QUADAS items varied between 50% and 100% with a median value of 90%. Items related to uninterpretable test results and withdrawals led to the most disagreements. The feedback on the content of the tool was generally positive with only small numbers of reviewers reporting problems with coverage, ease of use, clarity of instructions and validity.

Conclusion

Major modifications to the content of QUADAS itself are not necessary. The evaluation highlighted particular difficulties in scoring the items on uninterpretable results and withdrawals. Revised guidelines for scoring these items are proposed. It is essential that reviewers tailor guidelines for scoring items to their review, and ensure that all reviewers are clear on how to score studies. Reviewers should consider whether all QUADAS items are relevant to their review, and whether additional quality items should be assessed as part of their review.

How does study quality affect the results of a diagnostic meta-analysis?

BACKGROUND

The use of systematic literature review to inform evidence based practice in diagnostics is rapidly expanding. Although the primary diagnostic literature is extensive, studies are often of low methodological quality or poorly reported. There has been no rigorously evaluated, evidence based tool to assess the methodological quality of diagnostic studies. The primary objective of this study was to determine the extent to which variations in the quality of primary studies impact the results of a diagnostic meta-analysis and whether this differs with diagnostic test type. A secondary objective was to contribute to the evaluation of QUADAS, an evidence-based tool for the assessment of quality in diagnostic accuracy studies.

METHODS

This study was conducted as part of large systematic review of tests used in the diagnosis and further investigation of urinary tract infection (UTI) in children. All studies included in this review were assessed using QUADAS, an evidence-based tool for the assessment of quality in systematic reviews of diagnostic accuracy studies. The impact of individual components of QUADAS on a summary measure of diagnostic accuracy was investigated using regression analysis. The review divided the diagnosis and further investigation of UTI into the following three clinical stages: diagnosis of UTI, localisation of infection, and further investigation of the UTI. Each stage used different types of diagnostic test, which were considered to involve different quality concerns.

RESULTS

Many of the studies included in our review were poorly reported. The proportion of QUADAS items fulfilled was similar for studies in different sections of the review. However, as might be expected, the individual items fulfilled differed between the three clinical stages. Regression analysis found that different items showed a strong association with test performance for the different tests evaluated. These differences were observed both within and between the three clinical stages assessed by the review. The results of regression analyses were also affected by whether or not a weighting (by sample size) was applied. Our analysis was severely limited by the completeness of reporting and the differences between the index tests evaluated and the reference standards used to confirm diagnoses in the primary studies. Few tests were evaluated by sufficient studies to allow meaningful use of meta-analytic pooling and investigation of heterogeneity. This meant that further analysis to investigate heterogeneity could only be undertaken using a subset of studies, and that the findings are open to various interpretations.

CONCLUSION

Further work is needed to investigate the influence of methodological quality on the results of diagnostic meta-analyses. Large data sets of well-reported primary studies are needed to address this question. Without significant improvements in the completeness of reporting of primary studies, progress in this area will be limited.

Further investigation of confirmed urinary tract infection (UTI) in children under five years: a systematic review

BACKGROUND

Further investigation of confirmed UTI in children aims to prevent renal scarring and future complications.

METHODS

We conducted a systematic review to determine the most effective approach to the further investigation of confirmed urinary tract infection (UTI) in children under five years of age.

RESULTS

73 studies were included. Many studies had methodological limitations or were poorly reported. Effectiveness of further investigations: One study found that routine imaging did not lead to a reduction in recurrent UTIs or renal scarring. Diagnostic accuracy: The studies do not support the use of less invasive tests such as ultrasound as an alternative to renal scintigraphy, either to rule out infection of the upper urinary tract (LR- = 0.57, 95%CI: 0.47, 0.68) and thus to exclude patients from further investigation or to detect renal scarring (LR+ = 3.5, 95% CI: 2.5, 4.8). None of the tests investigated can accurately predict the development of renal scarring. The available evidence supports the consideration of contrast-enhanced ultrasound techniques for detecting vesico-ureteric reflux (VUR), as an alternative to micturating cystourethrography (MCUG) (LR+ = 14.1, 95% CI: 9.5, 20.8; LR- = 0.20, 95%CI: 0.13, 0.29); these techniques have the advantage of not requiring exposure to ionising radiation.

CONCLUSION

There is no evidence to support the clinical effectiveness of routine investigation of children with confirmed UTI. Primary research on the effectiveness, in terms of improved patient outcome, of testing at all stages in the investigation of confirmed urinary tract infection is urgently required.

Systematic review of water fluoridation

OBJECTIVE

To review the safety and efficacy of fluoridation of drinking water.

DESIGN

Search of 25 electronic databases and world wide web. Relevant journals hand searched; further information requested from authors. Inclusion criteria were a predefined hierarchy of evidence and objectives. Study validity was assessed with checklists. Two reviewers independently screened sources, extracted data, and assessed validity.

MAIN OUTCOME MEASURES

Decayed, missing, and filled primary/permanent teeth. Proportion of children without caries. Measure of effect was the difference in change in prevalence of caries from baseline to final examination in fluoridated compared with control areas. For potential adverse effects, all outcomes reported were used.

RESULTS

214 studies were included. The quality of studies was low to moderate. Water fluoridation was associated with an increased proportion of children without caries and a reduction in the number of teeth affected by caries. The range (median) of mean differences in the proportion of children without caries was -5.0% to 64% (14.6%). The range (median) of mean change in decayed, missing, and filled primary/permanent teeth was 0.5 to 4.4 (2.25) teeth. A dose-dependent increase in dental fluorosis was found. At a fluoride level of 1 ppm an estimated 12.5% (95% confidence interval 7.0% to 21.5%) of exposed people would have fluorosis that they would find aesthetically concerning.

CONCLUSIONS

The evidence of a beneficial reduction in caries should be considered together with the increased prevalence of dental fluorosis. There was no clear evidence of other potential adverse effects.