Measuring clinical performance using routinely collected clinical data

Using primary care databases for addiction research: An introduction and overview of strengths and weaknesses

Primary care databases extract and combine routine data from the electronic patient records of various participating practices on a regular basis. These databases can be used for innovative and relevant addiction research, but such use requires a thorough understanding of how data were originally collected and how they need to be processed and statistically analysed to produce sound scientific evidence. The aims of this paper are therefore to (1) make a case for why primary care databases should be considered more frequently for addiction research; (2) provide an overview of how primary care databases are constructed; (3) highlight important methodological and statistical strengths and weaknesses of using primary care databases for research; and (4) give practical advice about how a researcher can get access to databases. Three major primary care databases from the UK serve as examples: Clinical Practice Research Datalink (CPRD), The Health Improvement Network (THIN), and QResearch.

Impact of the introduction and withdrawal of financial incentives on the delivery of alcohol screening and brief advice in English primary health care: an interrupted time-series analysis

AIM

To evaluate the impact of the introduction and withdrawal of financial incentives on alcohol screening and brief advice delivery in English primary care.

DESIGN

Interrupted time-series using data from The Health Improvement Network (THIN) database. Data were split into three periods: (1) before the introduction of financial incentives (1 January 2006-31 March 2008); (2) during the implementation of financial incentives (1 April 2008-31 March 2015); and (3) after the withdrawal of financial incentives (1 April 2015-31 December 2016). Segmented regression models were fitted, with slope and step change coefficients at both intervention points.

SETTING

England.

PARTICIPANTS

Newly registered patients (16+) in 500 primary care practices for 2006-16 (n = 4 278 723).

MEASUREMENTS

The outcome measures were percentage of patients each month who: (1) were screened for alcohol use; (2) screened positive for higher-risk drinking; and (3) were reported as having received brief advice on alcohol consumption.

FINDINGS

There was no significant change in the percentage of newly registered patients who were screened for alcohol use when financial incentives were introduced. However, the percentage fell (P < 0.001) immediately when incentives were withdrawn, and fell by a further 2.96 [95% confidence interval (CI) = 2.21-3.70] patients per 1000 each month thereafter. After the introduction of incentives, there was an immediate increase of 9.05 (95% CI = 3.87-14.23) per 1000 patients screening positive for higher-risk drinking, but no significant further change over time. Withdrawal of financial incentives was associated with an immediate fall in screen-positive rates of 29.96 (95% CI = 19.56-40.35) per 1000 patients, followed by a rise each month thereafter of 2.14 (95% CI = 1.51-2.77) per 1000. Screen-positive patients recorded as receiving alcohol brief advice increased by 20.15 (95% CI = 12.30-28.00) per 1000 following the introduction of financial incentives, and continued to increase by 0.39 (95% CI = 0.26-0.53) per 1000 monthly until withdrawal. At this point, delivery of brief advice fell by 18.33 (95% CI = 11.97-24.69) per 1000 patients and continued to fall by a further 0.70 (95% CI = 0.28-1.12) per 1000 per month.

CONCLUSIONS

Removing a financial incentive for alcohol prevention in English primary care was associated with an immediate and sustained reduction in the rate of screening for alcohol use and brief advice provision. This contrasts with no, or limited, increase in screening and brief advice delivery rates following the introduction of the scheme.

Development and validation of a novel computer-aided score to predict the risk of in-hospital mortality for acutely ill medical admissions in two acute hospitals using their first electronically recorded blood test results and vital signs: a cross-sectional study

OBJECTIVES

There are no established mortality risk equations specifically for emergency medical patients who are admitted to a general hospital ward. Such risk equations may be useful in supporting the clinical decision-making process. We aim to develop and externally validate a computer-aided risk of mortality (CARM) score by combining the first electronically recorded vital signs and blood test results for emergency medical admissions.

DESIGN

Logistic regression model development and external validation study.

SETTING

Two acute hospitals (Northern Lincolnshire and Goole NHS Foundation Trust Hospital (NH)-model development data; York Hospital (YH)-external validation data).

PARTICIPANTS

Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic National Early Warning Score(s) and blood test results recorded on admission.

RESULTS

The risk of in-hospital mortality following emergency medical admission was 5.7% (NH: 1766/30 996) and 6.5% (YH: 1703/26 247). The C-statistic for the CARM score in NH was 0.87 (95% CI 0.86 to 0.88) and was similar in an external hospital setting YH (0.86, 95% CI 0.85 to 0.87) and the calibration slope included 1 (0.97, 95% CI 0.94 to 1.00).

CONCLUSIONS

We have developed a novel, externally validated CARM score with good performance characteristics for estimating the risk of in-hospital mortality following an emergency medical admission using the patient's first, electronically recorded, vital signs and blood test results. Since the CARM score places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Index blood tests and national early warning scores within 24 hours of emergency admission can predict the risk of in-hospital mortality: a model development and validation study

BACKGROUND

We explored the use of routine blood tests and national early warning scores (NEWS) reported within ±24 hours of admission to predict in-hospital mortality in emergency admissions, using empirical decision Tree models because they are intuitive and may ultimately be used to support clinical decision making.

METHODOLOGY

A retrospective analysis of adult emergency admissions to a large acute hospital during April 2009 to March 2010 in the West Midlands, England, with a full set of index blood tests results (albumin, creatinine, haemoglobin, potassium, sodium, urea, white cell count and an index NEWS undertaken within ±24 hours of admission). We developed a Tree model by randomly splitting the admissions into a training (50%) and validation dataset (50%) and assessed its accuracy using the concordance (c-) statistic. Emergency admissions (about 30%) did not have a full set of index blood tests and/or NEWS and so were not included in our analysis.

RESULTS

There were 23248 emergency admissions with a full set of blood tests and NEWS with an in-hospital mortality of 5.69%. The Tree model identified age, NEWS, albumin, sodium, white cell count and urea as significant (p<0.001) predictors of death, which described 17 homogeneous subgroups of admissions with mortality ranging from 0.2% to 60%. The c-statistic for the training model was 0.864 (95%CI 0.852 to 0.87) and when applied to the testing data set this was 0.853 (95%CI 0.840 to 0.866).

CONCLUSIONS

An easy to interpret validated risk adjustment Tree model using blood test and NEWS taken within ±24 hours of admission provides good discrimination and offers a novel approach to risk adjustment which may potentially support clinical decision making. Given the nature of the clinical data, the results are likely to be generalisable but further research is required to investigate this promising approach.

Comparison of different methods of risk stratification in urgent and emergency surgery

Background

The aim was to compare a number of risk scoring systems prospectively in a cohort of patients who underwent non-elective surgery.

Methods

This was a cohort study of 2349 consecutive patients who had urgent or emergency surgery in a district general hospital in the UK. All patients were scored prospectively using the Revised Goldman Cardiac Risk Index (RGCRI), Portsmouth modification of the Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity (P-POSSUM), Surgical Risk Score (SRS) and Biochemistry and Haematology Outcome Models (BHOM). Actual 30-day and 1-year survival rates were compared with the predicted outcomes using receiver–operator characteristic (ROC) curves and Hosmer–Lemeshow analysis.

Results

Some 141 patients (6·0 per cent) died within 30 days of operation. This increased to 254 (10·8 per cent) by 1 year. The area under the ROC curve for death within 30 days was 0·90 for P-POSSUM, 0·85 for SRS, 0·84 for BHOM and 0·73 for RGCRI. Only the first three risk scores were able to discriminate accurately within the groups (area under ROC curve over 0·8), with no significant variation between expected and observed mortality rates confirmed by Hosmer–Lemeshow analysis. Similar results were found for the ability of each score to predict outcome at 1 year.

Conclusion

P-POSSUM, SRS and BHOM scoring systems were all able to predict outcome after emergency and urgent surgery, but the SRS had the advantage of ease of calculation. BHOM requires only the most commonly available blood test data and the computer holding these data can easily perform the calculation.d.