Reducing variability in treatment decision-making: effectiveness of educating clinicians about uncertainty

How do clinicians rate patient's performance status using the ECOG performance scale? A mixed-methods exploration of variability in decision-making in oncology

Background

Medical decisions made by oncology clinicians have serious implications, even when made collaboratively with the patient. Clinicians often use the Eastern Clinical Oncology Group (ECOG) performance status (PS) scores to help them make treatment-related decisions.

Methods

The current study explores the variability of the ECOG score when applied to 12 predetermined specially designed clinical case vignettes presented to a group of oncology clinicians (n = 72). The quantitative analysis included evaluation of variability of ECOG PS scores and exploration of rater and patient-related factors which may influence the final ECOG rating. In-depth interviews were conducted with oncology clinicians to ascertain factors that they felt were important while making treatment-related decisions. Basic and global themes were generated following qualitative data analysis.

Results

Quantitative results showed that there was poor agreement in ECOG rating between raters. Overall concordance with the gold standard rating ranged between 19.4% and 56.9% for the vignettes. Moreover, patients deemed to have socially desirable qualities (p < 0.004) were rated to have better PS and women patients (p < 0.004) to have worse PS. Clinicians having international work experience had increased concordance with ECOG PS rating. Qualitative results showed that ‘perceived socio-economic background of the patient’, ‘age of the patient’, ‘patient’s and family’s preferences’ and ‘past treatment response’ were the major themes highlighted by respondents that influenced the treatment-related decisions made by clinicians.

Conclusion

There is considerable variability in ECOG PS determined by clinicians. Decision-making in oncology is complex, multifactorial and is influenced by rater and patient-related factors.

Description of comprehensive dental services supported by the Medicare Chronic Disease Dental Scheme in the first 23 months of operation

OBJECTIVE

Australia's Medicare universal insurance system has supported comprehensive dental service through the Chronic Disease Dental Scheme (CDDS) since November 2007. Public debate opposing CDDS includes claims of over-servicing, calls for expansion to universal eligibility, and government threat of closure. Here we examine CDDS services over the first 23 months of operation.

METHODS

CDDS statistics on patient age, gender and item numbers claimed from November 2007 to December 2009 from Medicare were subjected to analysis.

RESULTS

The distribution of 404,768 total CDDS patients varied across Australia from 3.6% of the population in NSW to 0.07% in NT, while uptake increased over time. The average patient had 7.58 dental treatments, and the most common were: direct restorations (2.27), preventive and periodontal services (1.46), diagnostic services (1.43), extractions (0.77), and new dentures (0.53). Crown and bridgework appeared over-represented (0.48).

CONCLUSION

Although data do suggest over-servicing in crown and bridgework, there also appears to be significant community need for the CDDS.

IMPLICATION

Clear guidelines for dental clinical diagnosis and treatment planning, as well as a pre-approval process for crown and bridgework is suggested to improve the CDDS, and this could form the basis for expansion to universal eligibility for dental Medicare.

Caries management: technical solutions to biological problems or evidence-based care?

Caries-related clinical decision-making remains a centrepiece of clinical dentistry. However, the oral disease patterns are rapidly changing towards the better among major sections of the populations, most notably in the high-income countries. The caries decline is well documented among children and younger adults, and is gradually seen to trickle into middle and old age as well. Although it is tempting for the dental profession to take the credit for this development, the evidence points in a different direction. The major contribution of dentistry seems primarily related to changes in the treatment philosophies towards a less interventionist approach. This review aims to spur a further change in the diagnostic and treatment criteria used in the management of dental caries for the benefit of the oral health status of our patients. We must come to terms with the fact that our traditional core skills, our manual dexterity and technical competence, have less to offer to oral health than we have been accustomed to think. The dental schools and the professional dental organizations must carry the responsibility for promoting the necessary changes in the caries related clinical decision-making strategies to allow practicing dentists to provide appropriate oral health care to our populations.

Dental caries paradigms in diagnosis and diagnostic research

In this article, the fundamentals of caries diagnosis are reviewed from the three component perspectives, namely the strategy, the logics, and the tactics. Strategy concerns the objectives of the diagnostic process (i.e. why we diagnose caries). The logics describe how we assemble and evaluate the information collected and how this leads to an assessment of diagnostic value. Finally, tactics are about how we collect the information necessary to arrive at a correct diagnosis. We argue that the hitherto-dominant essentialistic caries paradigm should be replaced by a nominalistic caries concept. This allows us to circumvent the problem of a lack of a caries gold standard and to proceed in caries-diagnostic research to find the diagnostic methods that result in the best health outcomes for our patients. We also demonstrate the limitations of the medical model when attempting to understand caries diagnosis, and adhere to the Bader & Shugars caries script model. It is concluded that the current caries profile, characterized by lower prevalence and extent, and slower progression, has increased the need for an academic strengthening of the dental curriculum with respect to diagnostic reasoning and clinical decision-making processes.

The influence of providing a clinical practice guideline on dental students' decision making

The aim of this study was to assess the effect of the provision of a clinical practice guideline (CPG) on dental students' decisions to remove asymptomatic, impacted lower third molars. All dental students, who in 2001 were in the 3rd, 4th or 5th (final) year of their study at the Nijmegen College of Dental Sciences, were invited to participate. A pre-test-post-test control group design was used. Given 36 patient cases, all dental students were asked to assess the need for removal of asymptomatic, impacted lower third molars. All pre-test respondents were randomly allocated to the control or intervention group. After the provision of a CPG to the intervention group, both groups were asked to assess the same cases again. Frequencies of decisions to remove the third molars were calculated. Chi-square tests and anova were used to test the influence of study year and gender on the drop-out rate and on the effect of the provision of a CPG on students' treatment decisions. The decrease in indications to remove third molars by the intervention group was statistically significant (P < 0.05). In the control group, no significant decrease was observed. It was concluded that the provision of a CPG significantly influences dental students' decision making about treatment in a third-molar decision task. Students who used the CPG showed more guideline-conformed decision making.

Effect of selected literature on dentists' decisions to remove asymptomatic, impacted lower third molars

The aim of this study was to assess the effect of studying selected literature on dentists' decisions to remove asymptomatic, impacted lower third molars. A pre-test-post-test control group design was used. Given 36 patient cases, two groups of 16 general dental practitioners each were asked to assess the need for removal of asymptomatic impacted lower third molars. The cases were classified by three parameters: 'position of the third molar', 'impaction type', and 'patient age'. After studying selected literature on this subject by the intervention group, both groups were asked to assess the same cases again. Frequencies of decisions to remove the third molars were calculated. For each participant, tables were composed by crosstabulating the indication to remove a third molar with each of the three parameters. T-tests were used to test the significance of the difference between pre-test and post-test decisions. The overall number of indications to remove asymptomatic, impacted lower third molars decreased by 37% in the intervention group. In the control group, the difference between pre- and post-test was not statistically significant. It was concluded that the provision of selected literature significantly influences treatment decision making by dentists in a third molar decision task.

Evaluation of computer aided learning in developing clinical decision-making skills

AIM

The aim of this study was to determine whether an educational intervention delivered by a computer aided learning package improved the sensitivity and specificity of dentists' restorative treatment decisions.

METHOD

The study was a randomised controlled study using a Solomon three-group design. Ninety-five dentists were randomly allocated to the three study groups. One group of dentists read the radiographs pre and post an educational intervention, a second group read the radiographs once, after the intervention, and a third group read the radiographs twice, but received no intervention. On each occasion the dentists read 24 surfaces on each of 15 radiographs and made 360 decisions on how certain they were about restoring the tooth surface. Comparisons of mean sensitivity, specificity and areas under ROC curves were made within and between the study groups. Kappa values were used to assess changes in the level of agreement between dentists.

RESULTS

There were no significant changes in sensitivity, specificity or area under ROC curves caused by the intervention. There was no evidence that the level of agreement between the dentists improved after the intervention.

CONCLUSION

A computer aided learning package had no effect on dentists' treatment decision-making behaviour.