Measuring doctors' self-insight into their treatment decisions

Digital Identity: The effect of trust and reputation information on user judgement in the Sharing Economy

The Sharing Economy (SE) is a growing ecosystem focusing on peer-to-peer enterprise. In the SE the information available to assist individuals (users) in making decisions focuses predominantly on community-generated trust and reputation information. However, how such information impacts user judgement is still being understood. To explore such effects, we constructed an artificial SE accommodation platform where we varied the elements related to hosts' digital identity, measuring users' perceptions and decisions to interact. Across three studies, we find that trust and reputation information increases not only the users' perceived trustworthiness, credibility, and sociability of hosts, but also the propensity to rent a private room in their home. This effect is seen when providing users both with complete profiles and profiles with partial user-selected information. Closer investigations reveal that three elements relating to the host's digital identity are sufficient to produce such positive perceptions and increased rental decisions, regardless of which three elements are presented. Our findings have relevant implications for human judgment and privacy in the SE, and question its current culture of ever increasing information-sharing.

Prior belief and polarity in multicue learning

We report two experiments in which participants are trained using a multicue probability learning (MCPL) task, which attempts to simulate the acquisition of expert judgement by experience in the real world. Participants were asked to predict performance in certain occupations given a profile of personality test results with trial-by-trial outcome feedback. Only some cues were relevant, and the polarity of the cues (positive or negative predictors) was unspecified. In addition, 25% of random noise was added to the feedback to simulate real world uncertainty. The main factor of interest was that the role of prior belief (determined in a separate study of stereotypes) interfered with the learning process. Experiment 1 failed to find any influence of prior belief in the cues that were irrelevant to the criterion being trained. However, in Experiment 2 people learned to use the relevant cues better when their effect conformed with rather than conflicted with prior belief. Both experiments showed strong effects of cue polarity, with positive predictors much more easily learned. The results are discussed with reference to the cognitive processes involved in MCPL and closely related tasks.

Studying Clinical Reasoning, Part 2: Applying Social Judgement Theory

Part 1 of this paper (Harries and Harries 2001) examined the reasoning studies of the 1980s and 1990s and critiqued the ethnographic and information-processing approaches, based on stated information use. The need for an approach that acknowledged the intuitive nature of experienced thinkers' reasoning was identified.

Part 2 describes such an approach ± social judgement theory ± and presents a pilot application in occupational therapy research. The method used is judgement analysis. The issue under study is that of prioritisation policies in community mental health work. The results present the prioritisation policies of four occupational therapists in relation to managing community mental health referrals.

Evaluation of breakfast cereals with the current nutrition facts panel (NFP) and the Food and Drug Administration’s NFP proposal

Objective

To compare judgements of nutrition and judgement accuracy when evaluating cereals with the current US Food and Drug Administration (FDA) nutrition facts panel (NFP) and two new proposed NFP based on FDA guidelines.

Design

A between-subjects design randomly assigned participants to three NFP conditions (current NFP label and two modified NFP based on FDA proposals). Participants viewed breakfast cereals, and rated each on nutritional quality and on the likelihood of purchasing and consuming it. Participants provided demographic information and responses to questionnaires assessing nutrition/obesity knowledge, concern for healthy eating and nutrient importance.

Setting

USA.

Subjects

Two hundred and thirteen adults who completed an online survey (66·2 % female, mean age 37·31 (sd 12·56) years).

Results

Judged nutrition quality of cereals was positively correlated with protein, fibre and potassium and negatively correlated with sugars and sodium. This pattern appeared when using the current NFP or the modified versions. Highlighted nutrients in modified NFP formats did not affect their perceived importance. Accuracy of the nutrition quality judgements was measured in relationship to an objectively defined nutrition score, NuVal®. Nutrition judgement accuracy was highest under the current NFP (Spearman’s ρ=0·76 for the current NFP; 0·64 and 0·72 for the other formats). Regression analysis showed that nutrition judgement accuracy increased significantly (adjusted R2=0·13) with obesity knowledge (β=0·27), age (β=0·15) and current NFP (β=0·13).

Conclusions

The current NFP is equally or more effective in conveying nutritional information compared with NFP formats based on the FDA proposal.

Unconscious influences on decision making: a critical review

To what extent do we know our own minds when making decisions? Variants of this question have preoccupied researchers in a wide range of domains, from mainstream experimental psychology (cognition, perception, social behavior) to cognitive neuroscience and behavioral economics. A pervasive view places a heavy explanatory burden on an intelligent cognitive unconscious, with many theories assigning causally effective roles to unconscious influences. This article presents a novel framework for evaluating these claims and reviews evidence from three major bodies of research in which unconscious factors have been studied: multiple-cue judgment, deliberation without attention, and decisions under uncertainty. Studies of priming (subliminal and primes-to-behavior) and the role of awareness in movement and perception (e.g., timing of willed actions, blindsight) are also given brief consideration. The review highlights that inadequate procedures for assessing awareness, failures to consider artifactual explanations of "landmark" results, and a tendency to uncritically accept conclusions that fit with our intuitions have all contributed to unconscious influences being ascribed inflated and erroneous explanatory power in theories of decision making. The review concludes by recommending that future research should focus on tasks in which participants' attention is diverted away from the experimenter's hypothesis, rather than the highly reflective tasks that are currently often employed.

Context, as well as inputs, shape decisions, but are people aware of it?

Even if people are experts at understanding how various input cues landed them at a particular decision (something the authors refer to as cue utilization), they may still fail to appreciate how context influences the weight given to those input variables. We review evidence suggesting that people are unaware of contextual influences on their decisions.

Perceptual analysis of the reading of dermatopathology virtual slides by pathology residents

CONTEXT

The process by which pathologists arrive at a given diagnosis-a combination of their slide exploration strategy, perceptual information gathering, and cognitive decision making-has not been thoroughly explored, and many questions remain unanswered.

OBJECTIVE

To determine how pathology residents learn to diagnose inflammatory skin dermatoses, we contrasted the slide exploration strategy, perceptual capture of relevant histopathologic findings, and cognitive integration of identified features between 2 groups of residents, those who had and those who had not undergone their dermatopathology rotation.

DESIGN

Residents read a case set of 20 virtual slides (10 depicting nodular and diffuse dermatitis and 10 depicting subepidermal vesicular dermatitis), using an in-house-developed interface. We recorded residents' reports of diagnostic findings, conjectured diagnostic hypotheses, and final (or differential) diagnosis for each case, and time stamped each interaction with the interface. We created search maps of residents' slide exploration strategy.

RESULTS

No statistically significant differences were observed between the resident groups in the number of correctly or incorrectly reported diagnostic findings, but residents with dermatopathology training generated significantly more correct hypotheses (mean improvement of 88.5%) and correct diagnoses (70% of all correct diagnoses).

CONCLUSIONS

Two types of slide exploration strategy were identified for both groups: (1) a focused and efficient search, observed when the final diagnosis was correct; and (2) a more dispersed, time-consuming strategy, observed when the final diagnosis was incorrect. This difference was statistically significant, and it suggests that initial interpretation of a slide may bias further slide exploration.

Using the method of judgement analysis to address variations in diagnostic decision making

Background

Heart failure is not a clear-cut diagnosis but a complex clinical syndrome with consequent diagnostic uncertainty. Judgment analysis is a method to help clinical teams to understand how they make complex decisions. The method of judgment analysis was used to determine the factors that influence clinicians' diagnostic decisions about heart failure.

Methods

Three consultants, three middle grade doctors, and two junior doctors each evaluated 45 patient scenarios. The main outcomes were: clinicians' decisions whether or not to make a diagnosis of suspected heart failure; the relative importance of key factors within and between clinician groups in making these decisions, and the acceptability of the scenarios.

Results

The method was able to discriminate between important and unimportant factors in clinicians' diagnostic decisions. Junior and consultant physicians tended to use patient information similarly, although junior doctors placed particular weight on the chest X-Ray. Middle-grade doctors tended to use information differently but their diagnostic decisions agreed with consultants more frequently (k = 0.47) than junior doctors and consultants (k = 0.23), or middle grade and junior grade doctors (k = 0.10).

Conclusions

Judgment analysis is a potentially valuable method to assess influences upon diagnostic decisions, helping clinicians to manage the quality assurance process through evaluation of care and continuing professional development.

The role of computerized decision support in reducing errors in selecting medicines for prescription: narrative review

This narrative review includes a summary of research examining prescribing errors, prescription decision making and the role computerized decision support plays in this decision-making process. A reduction in medication prescribing errors, specifically a reduction in the selection of inappropriate medications, is expected to result from the implementation of an effective computerized decision support system. Previous research has investigated the impact of the implementation of electronic systems on medication errors more broadly. This review examines the specific characteristics of decision support systems that may contribute to fewer knowledge-based mistakes in prescribing, and critically appraises the large volume of information available on the decision-making process of selecting medicines for prescription. The results highlight a need for work investigating what decision strategies are used by doctors with different levels of expertise in the prescribing of medications. The nature of the relationship between decision support and decision performance is not well understood and future research is needed to determine the mechanisms by which computerized decision support influences medication selection.

The Judgment of Future Suicide-Related Behavior

Background: Judging whether a youth is at risk for suicide-related behavior (SRB) is considered an extremely challenging task. There are only few studies of helpline counselors, and little is known about their ability to accurately determine the level of risk for SRB. Aims: To examine whether helpline counselors can agree on judgments of risk for SRB, and whether their judgments are consistent with youths’ actual behavior in a 6-month period following intake. Methods: 34 helpline counselors, recruited from three helplines, were studied. Information was collected on their judgments of risk for SRB for each of 45 youths over a 6-month period following initial intake. Results: Contrary to expectations, the counselors had a high rate of agreement (k = .56), and their risk judgments could be used quite successfully (80.0% correct classification) in identifying youths who later engaged in SRB. Conclusions: Unlike most other groups represented in the decision-making literature, helpline counselors agree and are accurate in their judgments of risk for SRB. Our findings suggest that it might be beneficial to apply some of the procedures used to train helpline clinicians to other types of clinicians. Further studies of helpline clinicians are suggested.

Evaluation of nurses' self-insight into their pain assessment and treatment decisions

Research generally indicates that providers demonstrate modest insight into their clinical decision processes. In a previous study utilizing virtual human (VH) technology, we found that patient demographic characteristics and facial expressions of pain were statistically significant predictors of many nurses' pain-related decisions. The current study examined the correspondence between the statistically identified and self-reported influences of contextual information on pain-related decisions. Fifty-four nurses viewed vignettes containing a video of a VH patient and text describing a postsurgical context. VH sex, race, age, and facial expression varied across vignettes. Participants made pain-assessment and treatment decisions on visual analogue scales. Participants subsequently indicated the information they relied on when making decisions. None of the participants reported using VH sex, race, or age in their decision process. Statistical modeling indicated that 28 to 54% of participants (depending on the decision) used VH demographic cues. 76% of participants demonstrated concordance between their reported and actual use of the VH facial expression cue. Vital signs, text-based clinical summary, and VH movement were also reported as influential factors. These data suggest that biases may be prominent in practitioner decision-making about pain, but that providers have minimal awareness of and/or a lack of willingness to acknowledge this bias.

PERSPECTIVE

The current study highlights the complexity of provider decision-making about pain management. The VH technology could be used in future research and education applications aimed at improving the care of all persons in pain.

Occupational therapists’ decision-making in the management of clients with upper limb hypertonicity

This study applied social judgement theory to the exploration of factors influencing occupational therapists' decision-making when they manage upper limb hypertonicity in clients with cerebral palsy. To achieve this goal, both objective and stated policies were investigated. This quantitative study drew information from a literature review and a survey with experienced occupational therapists to identify 12 factors that could influence decision-making. Based on these 12 factors 110 case vignettes of individuals with cerebral palsy and upper limb hypertonicity were generated. Intervention decisions were elicited from 18 experienced occupational therapists for each of the 110 case vignettes. Therapists were also asked to rank the factors in order of perceived importance. Occupational therapists generally used severity of spasticity, wrist and finger posture, and client and family background information to guide their clinical intervention choices. However, therapists demonstrated poor insight into the nature of their decision-making processes. This was highlighted in the disparity between their stated and objective policies. These findings have implications for both the professional development of therapists and the training of students.

What do the theories of Egon Brunswik have to say to medical education?

Every day physicians make judgments about patient management and diagnosis based on less than perfect information from many different sources. Judgment and decision-making research has taught us a great deal about such decisions, but these insights rarely find their way into the medical curriculum. One productive line of investigation in the study of judgment and decision making has followed the insights and theories developed by the psychologist, Egon Brunswik. His theories are becoming increasingly relevant to modern judgment problems. In this paper, I outline Brunswik's theories, trace their development over the last 50 years and speculate on what role they should play in medical education.

Trust in motives, trust in competence: Separate factors determining the effectiveness of risk communication

According to Siegrist, Earle and Gutscher’s (2003) model of risk communication, the effect of advice about risk on an agent’s behavior depends on the agent’s trust in the competence of the advisor and on their trust in the motives of the advisor. Trust in competence depends on how good the advice received from the source has been in the past. Trust in motives depends on how similar the agent assesses the advisor’s values to be to their own. We show that past quality of advice and degree of similarity between advisors’ and judges’ values have separate (non-interacting) effects on two types of agent behavior: the degree of trust expressed in a source (stated trust) and the weight given to the source’s advice (revealed trust). These findings support Siegrist et al.’s model. We also found that revealed trust was affected more than stated trust by differences in advisor quality. It is not clear how this finding should be accommodated within Siegrist et al.’s (2003) model.

A review of research into the development of radiologic expertise: implications for computer-based training

RATIONALE AND OBJECTIVES

Studies of radiologic error reveal high levels of variation between radiologists. Although it is known that experts outperform novices, we have only limited knowledge about radiologic expertise and how it is acquired.

MATERIALS AND METHODS

This review identifies three areas of research: studies of the impact of experience and related factors on the accuracy of decision-making; studies of the organization of expert knowledge; and studies of radiologists' perceptual processes.

RESULTS AND CONCLUSION

Interpreting evidence from these three paradigms in the light of recent research into perceptual learning and studies of the visual pathway has a number of conclusions for the training of radiologists, particularly for the design of computer-based learning programs that are able to illustrate the similarities and differences between diagnoses, to give access to large numbers of cases and to help identify weaknesses in the way trainees build up a global representation from fixated regions.

A note on determining the number of cues used in judgment analysis studies: The issue of type II error

Many judgment analysis studies employ multiple regression procedures to estimate the importance of cues. Some studies test the significance of regression coefficients in order to decide whether or not specific cues are attended to by the judge or decision maker. This practice is dubious because it ignores type II error. The purposes of this note are (1) to draw attention to this issue, specifically as it appears in studies of self-insight, (2) to illustrate the problem with examples from the judgment literature, and (3) to provide a simple method for calculating post-hoc power in regression analyses in order to facilitate the reporting of type II errors when regression models are used.

Insight and strategy in multiple-cue learning

In multiple-cue learning (also known as probabilistic category learning) people acquire information about cue-outcome relations and combine these into predictions or judgments. Previous researchers claimed that people can achieve high levels of performance without explicit knowledge of the task structure or insight into their own judgment policies. It has also been argued that people use a variety of suboptimal strategies to solve such tasks. In three experiments the authors reexamined these conclusions by introducing novel measures of task knowledge and self-insight and using "rolling regression" methods to analyze individual learning. Participants successfully learned a four-cue probabilistic environment and showed accurate knowledge of both the task structure and their own judgment processes. Learning analyses suggested that the apparent use of suboptimal strategies emerges from the incremental tracking of statistical contingencies in the environment.

Assessing the Likelihood of an Important Clinical Outcome: New Insights from a Comparison of Clinical and Actuarial Judgment

Purpose. To assess and rank the performance of different methods of predicting the probability of death following a specified surgical procedure. Method. Actuarial estimates of the probability of early mortality for 40 patients were derived from 2 sources: a large published surgical series and a smaller series from the center where surgery was performed. Surgeons and cardiologists also provided probability estimates for these patients. Results. Estimates derived from the published literature were too optimistic and did not differentiate between patients more, or less, likely to die (i.e., failed to discriminate). Doctors’ judgments were unbiased but failed to discriminate. Local actuarial estimates (influenced by only 1 or 2 variables) were unbiased, did discriminate, but exhibited more random variation. Conclusions. The preferred source of estimates depends upon which aspect of accuracy is of greatest importance. Differences in patient selection and error in the identification of risk factors mean that published results will not always appropriately predict surgical risk at other institutions. Risk stratification may be more robust when based on a small set of cross-validated predictors rather than a larger set of predictors that includes some whose reliability has not been confirmed.

Perceived barriers for treatment of chronic heart failure in general practice; are they affecting performance?

BACKGROUND

The aim of this study is to determine to what extent barriers perceived by general practitioners (GPs) for prescribing angiotensin-converting enzyme inhibitors (ACE-I) in chronic heart failure (CHF) patients are related to underuse and underdosing of these drugs in actual practice.

METHODS

Barriers were assessed with a semi-structured questionnaire. Prescribing data were extracted from GPs' computerised medical records for a random sample of their CHF patients. Relations between barriers and prescribing behaviour were assessed by means of Spearman rank correlation and multivariate regression modelling.

RESULTS

GPs prescribed ACE-I to 45% of their patients and had previously initiated such treatment in an additional 3.5%, in an average standardised dose of 13.5 mg. They perceived a median of four barriers in prescribing ACE-I or optimising ACE-I dose. Many GPs found it difficult to change treatment initiated by a cardiologist. Furthermore, initiating ACE-I in patients already using a diuretic or stable on their current medication was perceived as barrier. Titrating the ACE-I dose was seen as difficult by more than half of the GPs. No significant relationships could be found between the barriers perceived and actual ACE-I prescribing. Regarding ACE-I dosing, the few GPs who did not agree that the ACE-I should be as high as possible prescribed higher ACE-I doses.

CONCLUSION

Variation between GPs in prescribing ACE-I for CHF cannot be explained by differences in the barriers they perceive. Tailor-made interventions targeting only those doctors that perceive a specific barrier will therefore not be an efficient approach to improve quality of care.

Explicit and implicit processes in multicue judgment

In two experiments, a multicue probability learning task was used to train participants in relating judgments to a criterion, on the basis of several cues that could or could not be relevant. The outcome feedback had 25% added noise to simulate real-world experience-based learning. Judgmental strategies acquired were measured by individual multiple linear regression analyses of a test phase (with no feedback) and were compared with self-ratings of cue relevance. In a third experiment, participants were instructed explicitly on cue relevance, with no training phase. The pattern of results suggested that both implicit and explicit cognitive processes influenced judgments and that they may have been sensitive to different task manipulations in the learning phase. On more complex tasks, despite weak explicit learning, explicit processes continued to influence judgments, producing a decrement in performance. These findings explain why studies of expert judgment often show only moderate levels of self-insight, since people have only partial access to the processes determining their judgments.

The value of clinical judgement analysis for improving the quality of doctors' prescribing decisions

BACKGROUND

Many initiatives are taken to improve prescribing decisions. Educational strategies for doctors have been effective in at least 50% of cases. Some reflection on one's own performance seems to be a common feature of the most effective strategies. So far, such reflections have mainly focused on the observed outcomes of the doctors' decisions, i.e. on what doctors do in practice. Studies in other fields have shown that another form of feedback based on the analysis of judgements may be useful as well.

OBJECTIVES

The objectives of the study were to discuss the principles underlying clinical judgement analysis, give examples of its use in the medical context, and discuss its potential for improving prescribing decisions.

RESULTS

Clinical judgement analysis can look behind the outcome of a decision to the underlying decision process. Carefully constructed or selected case material is required for this analysis. Combining feedback on outcomes with feedback based on clinical judgement analysis offers doctors insight both in what they do, and why or when they do it. It may reveal determinants of decision making which are not available through unaided introspection. Interventions using this combination of feedback for improving doctors' prescribing behaviour have been (partly) successful in 4 cases and unsuccessful in one case.

CONCLUSIONS

Clinical judgement analysis gives doctors a structured reflection on the decision-making policy, and can help them to improve their future decisions. It may be especially useful for groups of doctors who try to work towards a consensus policy. The approach is not very helpful when simple decision rules are appropriate.

Taking advice, using information and knowing what you are doing

Subjective descriptions of judgment policies have been found to be imperfect. This could be because subjective weights are obtained on just a single occasion after all judgments have been completed. It could also be because people have tended to state their perception of an ideal way of responding rather than their perception of how they actually responded. Finally, it could be because they experience difficulty in relating variation in stimulus dimensions to variation in quite different response dimensions. In our task, people made sales forecasts on the basis of four pieces of information. They also stated the weight they placed on each one and the weight they should have placed on it. The means of weights stated on each trial were more appropriate than those stated at the end of all trials. Stated actual weights were very similar to stated ideal weights. Weights were more appropriate when forecasts and cues varied along the same dimension than when they did not. Thus, our results are consistent with the view that all three factors affect people's ability to provide subjective descriptions of their judgment policies.