The hidden cost of clinical audit: a questionnaire study of NHS staff

Machine Learning Analysis to Identify Data Entry Errors in Prehospital Patient Care Reports: A Case Study of a National Out-of-Hospital Cardiac Arrest Registry

Background: The objective of this study was to develop and validate machine learning models for data entry error detection in a national out-of-hospital cardiac arrest (OHCA) prehospital patient care report database.Methods: Adult OHCAs of presumed cardiac etiology were included. Data entry errors were defined as discrepancies between the coded data and the free-text note documenting the intervention or event; for example, information that was recorded as "absent" in the coded data but "present" in the free-text note. Machine learning models using the extreme gradient boosting, logistic regression, extreme gradient boosting outlier detection, and K-nearest neighbor outlier detection algorithms for error detection within nine core variables were developed and then validated for each variable.Results: Among 12,100 OHCAs, the proportion of cases with at least one error type was 16.2%. The area under the receiver operating characteristic curve (AUC) of the best-performing model (model with the highest AUC for each outcome variable) was 0.71-0.95. Machine learning models detected errors most efficiently for outcome place and initial rhythm errors; 82.6% of place errors and 93.8% of initial rhythm errors could be detected while checking 11 and 35% of data, respectively, compared to the strategy of checking all data.Conclusion: Machine learning models can detect data entry errors in care reports of emergency medical services (EMS) clinicians with acceptable performance and likely can improve the efficiency of the process of data quality control. EMS organizations that provide more prehospital interventions for OHCA patients could have higher error rates and may benefit from the adoption of error-detection models.

Income, Relative Deprivation and the Self-Rated Health of Older People in Urban and Rural China

Background: Substantial evidence indicated that absolute income is directly associated with health. Few studies have, however, examined if relative income may be equally associated with health. This study aimed to investigate the association between absolute income/relative deprivation (RD) and self-rated health (SRH). We also investigated whether the urban-rural difference was existing in these associations.

Methods: Using cross-sectional data of 7,070 participants in the Shandong Family Health Service Survey of older people, this study applied binary logistic model and semi-parametric model to estimate the effect of absolute income and relative deprivation on SRH of older people. The Kakwani Index was used as a measure of relative deprivation at the individual level.

Results: Absolute income has a significant positive effect on the SRH among both urban and rural older people. When considered RD as a variable, both absolute income and RD have negative significant effects on SRH among all older people. In addition, the negative effect of RD on rural elderly is more pronounced than that of urban older populations. Semi-parametric regression results show that there was a complex non-linear relationship between income and SRH. Psychological distress substantially attenuated the association between relative deprivation and SRH.

Conclusions: Relative deprivation is negatively associated with self-rated health in both urban and rural older people after controlling the absolute income. RD may partly explain the association between income inequality and worse health status. Compared with the urban elderly, the effect of income-based relative deprivation on SRH was more pronounced among the rural elderly, and more care should be given to the lower income and rural older populations.

Improving the reporting of orthodontic clinical audits: an evaluation

Aims The aim of this study was to evaluate the reporting of orthodontic audits published between 2013-2019 following the introduction of a submission template in 2015.Methods An audit reporting checklist was developed, with each audit independently assessed by two assessors. Based on the previous quality checklist, an overall score of 4 or less represented poor reporting, 5-8 fair reporting and 9 or greater good reporting. All data variables were collected in a pre-piloted Excel data collection sheet.Results One hundred and fifty-nine audits were identified. A range of reporting scores were evident. The overall mean score was 10.1 (SD 1.5). Reporting scores showed improvement during the study timeframe, with a general increase in scores evident from 2015. Higher scores were achieved by multi-cycle audits (coefficient [coef]: 2.0, 95% CI: 1.38, 2.62, p <0.001). Lower scores were achieved by partial audits (coef: -1.8, 95% CI: -2.23, -1.36, p <0.001), but scores increased every year (coef: 0.2, 95% CI: 0.12, 0.27, p <0.001).Conclusions The reporting of orthodontic audits is rated as good, with yearly improvement in scores evident. The introduction of a submission template had a positive effect on the reporting of audits. Recommendations to further improve the quality of audits are outlined.

'How many audits do you really need?': Learnings from 5-years of peripheral intravenous catheter audits

BACKGROUND

Peripheral intravenous catheters (PIVCs) are medical devices used to administer intravenous therapy but can be complicated by soft tissue or bloodstream infection. Monitoring PIVC safety and quality through clinical auditing supports quality infection prevention however is labour intensive. We sought to determine the optimal patient 'number' for clinical audits to inform evidence-based surveillance.

METHODS

We studied a dataset of cross-sectional PIVC clinical audits collected over five years (2015-2019) in a large Australian metropolitan hospital. Audits included adult medical, surgical, women's, cancer, emergency and critical care patients, with audit sizes of 69-220 PIVCs. The primary outcome was PIVC complications for one or more patient reported symptom/auditor observed sign of infection or other complications. Complication prevalence and 95% confidence interval (CI) were calculated. We modelled scenarios of low (10%), medium (20%) and high (50%) prevalence estimates against audit sizes of 20, 50, 100, 150, 200, 250, and 300. This was used to develop a decision-making tool to guide audit size.

RESULTS

Of 2274 PIVCs evaluated, 475 (21%) had a complication. Complication prevalence per round varied from 7.8% (95% CI, 4.2-12.9) to 39% (95% CI, 32.0-46.4). Precision improved with larger audit size and lower complication rates. However, precision was not meaningfully improved by auditing >150 patients at a complication rate of 20% (95% CI 13.9%-27.3%), nor >200 patients at a complication rate of 50% (95% CI 42.9%-57.1%).

CONCLUSION

Audit sizes should be 100 to 250 PIVCs per audit round depending on complication prevalence.

Developing and Testing a Local Expert-Based Reading Process for Use to Examine Discrepancies Between Guidelines and Current Clinical Practices

The use of relevant guidelines is critical in psychiatric clinical practice to ensure the homogeneity of the global care provided. Consequently, it is important to identify whether they are utilized successfully and, if not, why. This would enable pragmatic solutions to be agreed to improve the organization of care and the removal of any barriers to the guidelines' implementation. The first step in this process, before any exploration of the limitations of the guidelines themselves, involves a determination of whether they are actually applied in clinical practice. We therefore evaluated discrepancies between the guidelines relating to patients with borderline personality disorder and current practices in the psychiatric Emergency Department at Toulouse University Hospital. This was achieved using a reading process involving a panel of eight local experts who analyzed relevant medical files extracted from a database. They were guided by, and instructed to answer, six standardized questions in relation to each file to determine the method's feasibility. A total of 333 files were analyzed to determine whether, in the local experts' judgment, the care provided reflected current guidance. This reading process revealed substantial agreement (0.85%; Fleiss Kappa -0.69), which is a promising outcome and suggests that such methods could be used in future protocols. Moreover, the process is practical and reliable and requires very few materials.

I've got 99 problems but a phone ain't one: Electronic and mobile health in low and middle income countries

Mobile technology is very prevalent in Kenya-mobile phone penetration is at 88% and mobile data subscriptions form 99% of all internet subscriptions. While there is great potential for such ubiquitous technology to revolutionise access and quality of healthcare in low-resource settings, there have been few successes at scale. Implementations of electronic health (e-Health) and mobile health (m-Health) technologies in countries like Kenya are yet to tackle human resource constraints or the political, ethical and financial considerations of such technologies. We outline recent innovations that could improve access and quality while considering the costs of healthcare. One is an attempt to create a scalable clinical decision support system by engaging a global network of specialist doctors and reversing some of the damaging effects of medical brain drain. The other efficiently extracts digital information from paper-based records using low-cost and locally produced tools such as rubber stamps to improve adherence to clinical practice guidelines. By bringing down the costs of remote consultations and clinical audit, respectively, these projects offer the potential for clinics in resource-limited settings to deliver high-quality care. This paper makes a case for continued and increased investment in social enterprises that bridge academia, public and private sectors to deliver sustainable and scalable e-Health and m-Health solutions.

Feasibility of using routinely collected inpatient data to monitor quality and inform choice: a case study using the UK inflammatory bowel disease audit

OBJECTIVE

To assess the utility and cost of using routinely collected inpatient data for large-scale audit.

DESIGN

Comparison of audit data items collected nationally in a designed audit of inflammatory bowel disease (UK IBD audit) with routinely collected inpatient data; surveys of audit sites to compare costs.

SETTING

National Health Service hospitals across England, Wales and Northern Ireland that participated in the UK IBD audit.

PATIENTS

Patients in the UK IBD audit.

INTERVENTIONS

None.

MAIN OUTCOME MEASURES

Percentage agreement between designed audit data items collected for the UK IBD audit and routine inpatient data items; costs of conducting the designed UK IBD audit and the routine data audit.

RESULTS

There were very high matching rates between the designed audit data and routine data for a small subset of basic important information collected in the UK IBD audit, including mortality; major surgery; dates of admission, surgery, discharge and death; principal diagnoses; and sociodemographic patient characteristics. There were lower matching rates for other items, including source of admission, primary reason for admission, most comorbidities, colonoscopy and sigmoidoscopy. Routine data did not cover most detailed information collected in the UK IBD audit. Using routine data was much less costly than collecting designed audit data.

CONCLUSION

Although valuable for large population-based studies, and less costly than designed data, routine inpatient data are not suitable for the evaluation of individual patient care within a designed audit.

Attitudes to peer review as a competence assurance structure — results of a survey of Irish physicians

BACKGROUND

The Medical Council in Ireland is currently implementing Competence Assurance Structures (CAS). Peer review has been proposed as a tool to measure physician competence.

AIMS

To assess the attitudes of physicians working in the Irish healthcare system to a peer review programme of competence assurance prior to its implementation.

METHODS

A postal survey was sent to all physicians in the Irish Medical Directory in November 2003. Nine questions were asked to gauge attitudes to peer review as a CA tool. The returned questionnaires were collated and data extracted based on responses.

RESULTS

The response rate was 67%. The majority of respondents (92%) felt peer review would inform competence assurance in Ireland. Most physicians who were surveyed felt an on-site assessment (88%) every 5 years (87%) was the preferred method. Over 30% responded that there should be a financial incentive for completing a review, and 70% would pay to be assessed. The UK model of competence assurance was the model most physicians preferred for the Irish setting (42%).

CONCLUSION

The majority of physicians practising in Ireland would favour a peer review system of competence assurance. The financial implications, and structure, of such a system would need to be explored prior to implementation.

Audits, errors and the misplace of clinical indicators: revisiting the Quality in Australian Health Care Study

Publication of the Quality in Australian Health Care Study in 1995 represented a defining moment for Australian health care providers. The high incidence and cost of preventable adverse events underscored a need for defined process, error recognition and audit cycle. Despite this, surgical audit has continued to emphasize clinical indicators relevant to technical performance. The greatest burden of preventable error can be traced to deficiencies in the process by which management expectations are supported. Recognizing this, the focus of clinical audit must be expanded. In particular, outcome assessment should be routine rather than sporadic, and should broadly encompass safety, effectiveness and efficiency. Devolving this responsibility to paraclinical groups is in itself insufficient. Quality and safety cannot be adequately addressed unless surgeons actively participate in audit cycle. Failure to meet this challenge in a transparent and timely manner potentially undermines the future of professional autonomy.