Changing the measure of quality in the NHS: from purchasing activity to purchasing protocols

The design, characteristics and predictors of mortality in the North of England Cellulitis Treatment Assessment (NECTA)

AIMS

Cellulitis is a common cause of acute medical admissions in UK hospitals. The factors that determine susceptibility to an acute admission or to mortality following hospital admission are poorly defined.

METHODS

We studied a retrospective cohort of 568 patients with a diagnosis of cellulitis between 1 January 2001 and 31 December 2003 in the north-east of England to see whether we could determine these factors. We collected data on the factors that were associated with acute hospital admissions and survival. We used a primary end-point of deaths within 1 year of admission for cellulitis.

RESULTS

The characteristics that identified patients at high risk of mortality were present in 39.9% of the cohort studied. The four most common of these characteristics were lower limb oedema 30.1% (95% CI: -26.0 to 34.1), ulceration 24% (95% CI: -20.2 to 27.8), previous myocardial infarction (MI) 19.9% (95% CI: -16.3 to 23.4) and blunt injury 18.7% (95% CI: -15.3 to 22.2). Significant predictors of mortality were: patient's age (p < 0.001), presence of penetrating injury (p < 0.001), previous MI (p < 0.001), presence of liver disease (p = 0.003), presence of lower limb oedema (p = 0.01) and long-term use of drugs that caused sodium and water retention (p < 0.001). Treatment with i.v. flucloxacillin was found to be a significant predictor of survival (odds ratio = 3.43, z =3.42. p < 0.001) at 360 days.

CONCLUSION

Our results show that cellulitis as a cause of an acute medical admission may present with a variety of clinical features. Some of these clinical features can be used to predict mortality within 360 days of an acute hospital admission.

Stroke care: how do we measure quality?

The purpose of this paper is to review the methods used to measure quality of stroke care. Relevant articles were searched for on Medline using the following key words: stroke, quality, outcome of care, process of care, structure of care. Articles that examined how to measure the quality of stroke care and that examined difficulties in the measurement of care outcomes, processes, and structures were selected. Selected articles were reviewed to summarise methods used to measure quality of stroke care and the primary outcome measures of the studies were extracted. Conclusions were drawn about the best ways to measure the quality of stroke care. Practical problems in using outcome measures to monitor quality of care include the consequences of case mix and difficulties in risk adjustment. Clinicians may use process measures to understand differences in outcome. Once a process of care has been linked to an outcome measure, this care process should be measured. The national sentinel audit for stroke is an audit tool used to examine the quality of the processes of stroke care.

Evidence-based medicine and hypertension

BACKGROUND

Evidence-based medicine (EBM) has been propagated as a revolutionary development which will improve the quality of clinical decision-making and guideline development Historically it follows an early 19th-century French attempt to introduce mathematical analysis into clinical practice. This met with resistance from both clinicians and scientists and was only accepted in more recent times with the development of clinical epidemiology and clinical trials.

NATURE OF EBM

EMB claims to utilize the best available evidence to reach scientific conclusions, rejecting the appeal to expert authority. This involves a hierarchy of sources which places large controlled trials at the apex. Less value is attributed to arguments from clinical observation or pathophysiology. Systematic reviews and meta-analyses of trials therefore provide the strongest evidence for clinical decisions.

THE CONCEPT OF EVIDENCE

The approach advocated in EBM is an over-simplification of the process of clinical thinking which involves interpretation and synthesis of relevant evidence from all sources and extrapolation to the clinical situation. In this process, there is no hierarchy of evidence. The relative value given to any particular evidence depends more upon its relevance and persuasiveness than the category to which it belongs. Discussion and debate amongst informed 'experts' is an integral feature of this process at each stage.

IMPACT OF EBM

Although advocates of EBM acknowledge the contribution of all forms of evidence, the differential value attached to different sources has led to naïve and simplistic attempts to omit the traditional processes of interpretation, synthesis and extrapolation and to draw wide-ranging conclusions from trial data without adequate scientific discussion.

The role of outputs and outcomes in purchaser accountability: reflecting on New Zealand experiences

Recent reforms in a number of countries' health systems have led to the separation of funder, purchaser and provider roles and the strengthening of funders' and purchasers' positions relative to providers. One of the aims of such reforms is to improve accountability. This paper reports on experiences in New Zealand where, in addition to improving the accountability of providers, purchaser accountability has also been a key policy issue. Attempts have been made in New Zealand to develop a funder-purchaser accountability framework based on a mix of outcomes, outputs and inputs. This paper discusses the roles that each might play in contracts and accountability relationships between funders and purchasers. The paper concludes that holding purchasers accountable for outcomes is likely to prove difficult and controversial, because of problems of attribution and because New Zealand funders in recent years have played an important role in determining the priority outputs and inputs which must be purchased. The paper suggests that accountability is more appropriate at the output and process level, in addition to holding purchasers accountable for the ways in which they make decisions and undertake contracting roles. Holding purchasers accountable for purchasing outputs and processes, however, requires greater commitment on the part of the funder to setting priorities more clearly; specifying the range and level of outputs to be purchased and the terms of access to those services; and funding services to this level. The international attention currently being paid to the development of practice guidelines and priority criteria also suggests that holding purchasers accountable for a form of inputs may become an increasingly common practice in future. From 1 July 1998, New Zealand will introduce a priority criteria system for determining access to elective surgery; accountability is thus becoming focused on inputs in the form of patient characteristics. This approach will greatly assist in promoting accountability.

A protocol for the pharmacologic treatment of major depression. A field test of a potential prototype

BACKGROUND

Much attention is being given to developing clinical practice guidelines for management of mental health disorders. The aim of this study was to field test a prototype protocol for the pharmacologic treatment of Major Depression.

METHOD

The protocol consisted of four, six week, treatment phases with critical choices in therapy defined by scores on the MADRS (Montgomery Asberg Depression Rating Scale). Observational data as collected on the behaviour of the protocol in terms of relevance, acceptability, ease of use and effectiveness.

RESULTS

Effectiveness of the protocol was good for those patients who were retained within it, with three quarters of them attaining remission. However more than half of all patients dropped out-non attendance and adverse events being the most common reasons for this.

CONCLUSION

The protocol for the treatment of Major Depression appeared relevant, easy to use and potentially effective.

LIMITATION

Problems with non-adherence by both doctors and patients posed major challenges to the protocol's design. Such difficulties demonstrate the need to field test any proposed design as preconceptions about a protocol's performance may be misplaced.

CLINICAL RELEVANCE

The protocol tested represents progress towards the goal of developing optimal strategies for the use of pharmacotherapeutic agents in the treatment of depression.

Continuity of information for breast cancer patients: the development, use and evaluation of a multidisciplinary care-protocol

The multidisciplinary nature of much patient-care may lead to gaps in the continuity of information which they receive, as well as to different care-professionals giving them contradictory information. As a counter-measure, a protocol has been developed which integrates medical, nursing, and a variety of extramural events and activities into a comprehensive description of 15 'moments' in the care of breast cancer surgery-patients. Among innovations, the protocol includes information about psychosocial guidance following diagnosis, and about the discharge procedure and contact with fellow-sufferers. The protocol was implemented in Rotterdam in 1994, in two hospitals and in the community; and evaluated formatively on the basis of reactions from 53 patients and 81 care-professionals. Both groups found its form and content to be successful and informative.

Defining core services: New Zealand experiences

A key aspect of the New Zealand health reforms was the proposed development of an explicit core of services to which all New Zealanders would have access. A range of approaches has been taken by the government, its advisers, purchasers and providers to describe sets of services to which New Zealanders are to have access. The development of an explicit core aims to promote equity of access to services, to ensure that those services available are those that are the most cost-effective and the services New Zealanders feel to be the most important, and to clarify entitlements to publicly funded health care. This paper describes the current approaches that are being used to define core services in New Zealand, discusses the reasons behind some of the choices made and notes some key issues for further policy debate.

Doctors and change

The introduction of clinical care pathways into one acute hospital trust provided the opportunity to investigate reasons why doctors behave the way they do, and to identify receptive and non-receptive factors for change. A multi-method research strategy was used to obtain primary data, both quantitative and qualitative material, namely responses to a questionnaire and interviews. The questionnaire was distributed to 70 consultants from a variety of medical and surgical specialties, and 42 questionnaires were returned (60 per cent response rate). Reports some of the findings and shows that consultant behaviour and the factors that influence that behaviour are affected by their own characteristics, specifically age and specialty, although all specialties agreed that non-financial incentives would influence their behaviour. Characteristics of opinion leaders were also found to be specialty-specific in their degree of influence. Concludes that the multi-method research approach was effective in identifying factors which influence consultant behaviour, and further studies should be carried out in this area.

Outcomes and process in genetic counselling

Although it may be simple to evaluate some elements of clinical genetics, it is difficult to evaluate genetic counselling. We review previous studies of the outcomes of genetic counselling; although the methods used may be valid in research studies, there are practical and ethical difficulties in applying them to the measurement of clinical effectiveness in standard practice. No simple measures of outcomes would be suitable. Research evidence will be helpful in deciding what services it is appropriate to offer, and the quality of a service can then be assured by assessing the quality of the clinical process in three ways: 1) adherence to agreed protocols and standards of care; 2) peer review and audit of clinical activity; and 3) ongoing review of the satisfaction of clients and referring physicians with the service. The assessment of client satisfaction will need to be a sophisticated form of retrospective satisfaction with the service provided, and such a scheme has yet to be fully developed.

Improving the quality of health care through contracting: a study of health authority practice

OBJECTIVES

To investigate approaches of district health authorities to quality in contracting.

DESIGN

Descriptive survey.

SETTING

All district health authorities in one health region of England in a National Health Service accounting year.

MATERIAL

129 quality specifications used in contracting for services in six specialties (eight general quality specifications and 121 service specific quality specifications)

MAIN MEASURES

Evaluation of the use of quality specifications; their scope and content in relation to established criteria of healthcare quality.

RESULTS

Most district health authorities developed quality specifications which would be applicable to their local hospital. When purchasing care outside their boundaries they adopted the quality specifications developed by other health authorities. The service specific quality specifications were more limited in scope than the general quality specifications. The quality of clinical care was referred to in 75% of general and 43% of service specific quality specifications. Both types of specification considered quality issues in superficial and broad terms only. Established features of quality improvement were rarely included. Prerequisites to ensure provider accountability and satisfactory delivery of service specifications were not routinely included in contracts.

CONCLUSION

Quality specifications within service contracts are commonly used by health authorities. This study shows that their use of this approach to quality improvement is inconsistent and unlikely to achieve desired quality goals. Continued reliance on the current approach is holding back a more fundamental debate on how to create effective management of quality improvement through the interaction between purchasers and providers of health care.

Central dimensions of clinical practice evaluation: efficiency, appropriateness and effectiveness--II

That a treatment selected for a given condition works, or that it works better than alternative treatments, or that it was selected because it works as well as but is cheaper than alternative treatments, should be of pivotal concern to clinicians and is of central concern to patients and to health care managers. Attempts to address these concerns have resulted in what is now widely termed the 'effectiveness movement'. The protagonists of the movement have been concerned to create a culture of evaluation and inquiry within which the formulation of evidence-based clinical guidelines and their introduction into routine practice have played a prominent part. The need to ensure cost effectiveness of clinical intervention has been at least as emphasized as the need to ensure the clinical effectiveness of health care interventions. Although cost-effectiveness analyses are now an indispensable feature of practice guideline formulation and treatment evaluation, few studies have examined any deterioration in patient outcome associated with successful cost containment. An adequate understanding of the concept of clinical effectiveness and the associated aims of the 'effectiveness movement' is central to an understanding of the future nature and extent of health service provision, not simply in the UK but also internationally. Having examined the concepts of efficiency and appropriateness previously (O'Neill, Miles & Polychronis 1996, Journal of Evaluation in Clinical Practice 2, 13-27) we move in this second of two articles to a detailed explanation of the concept of effectiveness, and to an examination of the derivation and use of clinical practice guideline, concluding with a consideration of the role of practice guidelines in ensuring the cost effectiveness of health care intervention. The reservation is expressed that a 'guidelines culture', when established, will be manipulated by health care commissioners for largely political purposes, creating a systematic bias in the purchasing process that will actively disadvantage a range of patient groups.

Quality of mental health service care: the forgotten pathway from process to outcome

The validity of the concept of outcome depends on a relationship between routine treatment and later health status. Outcome evaluations and audits are very rare in psychiatry. A substantial expansion in epidemiologically based, naturalistic, observational, process-outcome data collection in routine psychiatric practice is essential in order to identify treatment allocation biases and other reasons for unexpected outcomes. Identified causes of undertreatment should lead to locally agreed detailed clinical guidelines. Experimental evaluation should take place in routine clinical practice settings, with change in both process and outcome as the objective. Ultimately, the results of both experimental and observational outcome studies on representative service users should converge, permitting outcomes to be the ultimate arbitrator of quality.

Guidelines, enthusiasms, uncertainty, and the limits to purchasing

Recently government ministers have set out their vision of the future of purchasing. Ineffective treatments will be discarded and purchasing will be based on guidelines or protocols rather than activity. But have the advocates of this approach considered all the issues? This paper examines the challenges of balancing the desire for protocol based uniformity with the needs of individual patients, explores the extent to which existing purchasing structures can support this process, and questions whether such moves will actually lead to reduced costs. In each case it is concluded that oversimplistic analyses are likely to be misleading and that much of the current debate fails to recognise the complexity of health care.

Quality: link with effectiveness

In summary, though the notion of "quality of care" has become fashionable, most of the focus has been on initiatives such as the patient's charter, waiting times, quality of the physical environment, patient centredness in outcomes measurement, etc. Nevertheless, at the heart of quality must be the effectiveness and cost effectiveness of interventions. Without ensuring that health technologies are effective and are delivered appropriately then many of the other dimensions of quality may simply be window dressing. Substantial variations in the rates of procedures, the way in which similar patients are treated, and the degree to which professionals often ignore the best scientific evidence have all been well documented. The NHS needs methods for ensuring that the effectiveness dimension of quality is brought to the fore and becomes a routine part of quality assessment and activity. Clinical autonomy can no longer be an excuse for inappropriate care. The challenge for the future is twofold: to increase the amount of health technology assessment carried out and to develop methods of ensuring that health care converges with this best practice--that is, the promotion of evidence based practice. By introducing evidence based clinical guidelines and associated utilisation review and persuading purchasers to "purchase protocols" rather than just procedures the effectiveness dimension may become more routine, but it will require a radical rethink of the type of data collected and the way in which the purchaser provider split is managed.

Effect of guidelines on management of head injury on record keeping and decision making in accident and emergency departments

OBJECTIVE

To compare record keeping and decision making in accident and emergency departments before and after distribution of guidelines on head injury management as indices of implementation.

DESIGN

Before (1987) and after (1990) study of accident and emergency medical records.

SETTING

Two accident and emergency departments in England.

PATIENTS

1144 adult patients with head injury in department 1 (533 in 1987, 613 in 1990) and 734 in department 2 (370, 364 respectively).

MAIN MEASURES

Recording of relevant symptoms and signs as determined in the guidelines; presence of, indications for, and rates and appropriateness of skull x ray examination and admission.

RESULTS

The median number of guidelines variables recorded for all study periods ranged from 7 to 9 out of a possible maximum of 27. For key decision making variables the presence or absence of penetrating injury was least likely to be recorded (< or = 1%) and that of loss of consciousness most likely (> or = 75%). Altogether, the proportion of patients receiving skull x ray examination or admitted varied from 25%-60% and 7%-23% respectively; overall, 69% (1280/1856) and 64% (1177/1851) of patients were managed appropriately. However, no consistent change occurred in the departments between the study periods. For instance, in department 1 the proportion of appropriate x ray examinations rose significantly after distribution of the guidelines (from 61% (202/330) to 73% (305/417)) and appropriate decisions on whether to x ray or not also rose (from 65% (340/522) to 72% (435/608)). There was no significant change in department 2, although the proportion of appropriate admissions fell (from 33% (55/166) to 15% (19/130)).

CONCLUSIONS

Recording practice and decision making were variable and had not consistently improved after dissemination of the guidelines. Strategies are required to ensure effective implementation of guidelines.

Guidelines, protocols and outcomes

Clinical guidelines are being widely advocated. Discusses the agenda for outcomes in a context of both purchasing and providing and from user and clinical perspectives. Debates the question of whether practitioners will change their behaviour and act on research evidence.