A decision analytic method for scoring performance on computer-based patient simulations

As computer based clinical case simulations become increasingly popular for training and evaluating clinicians, approaches are needed to evaluate a trainee's or examinee's solution of the simulated cases. We developed a decision analytic approach to scoring performance on computerized patient case simulations. We developed decision models for computerized patient case simulations in four specific domains in the field of infectious disease. The decision models were represented as influence diagrams. A single decision node represents the possible diagnoses the user may make. One chance node represents a probability distribution over the set of competing diagnoses in the simulations. The value node contains the utilities associated with all possible combinations of diagnosis and disease. All relevant data that the user may request from the simulation are represented as chance nodes with arcs to or from the diagnosis node and/or each other. Probabilities in the decision model were derived from the literature, where available, or expert opinion. Utilities were assessed by standard gamble from clinical experts. The process of solving computer based patient simulations involves repeated cycles of requesting data (history, physical examination or laboratory) and receiving these data from the simulations. Each time the user requests clinical data from the simulation, the influence diagram is evaluated with and without an arc from the corresponding chance node to the decision node. The difference in expected utility between the two solutions of the influence diagram represents the expected value of information (VOI) from the requested clinical datum. The ratio of the expected VOI from the data requested and the expected value of perfect information about the diagnosis is a normative measure of the quality of each of the user's data requests. This approach provides a continuous measure of the quality of the user's data requests in a way that is sensitive to the previous data collected. The score distinguishes serious from minor misdiagnoses. And the same influence diagram can be used to evaluate performance on multiple simulations in the same clinical domain.

Cost and Appropriateness of Treating Plasmodium falciparum Infections in the United States

Background: Although approximately 1000 U.S. citizens per year are reported to the Centers for Disease Control and Prevention as having acquired malaria infection during foreign travel, little information exists with regard to the cost and appropriateness of malaria therapy received in the United States. Methods: Data on treatment of U.S. citizens reported to the Centers for Disease Control and Prevention (CDC) as having acquired Plasmodium falciparum malaria in 1988-1989 while traveling in subSaharan Africa were collected by phone interview. These data were used to derive a relative index of illness severity, to estimate the costs of malaria-specific therapy, and to assess adherence to existing therapy recommendations. All monetary values throughout this study will be expressed in U.S. dollars. Results: Of 142 patients, 110 (77%) were classified as having mild, 21 (15%) as having moderate, and 11 (8%) as having severe infections. Two (1.4%) deaths were reported. Overall, the mean (6 standard deviation) cost of treatment per case was $2743.51 (6 8416.82; range $191.75 to $79,801.73). Estimated with relation to severity, the median cost for treatment per case was $467.54 for mild, $2701.16 for moderate, and $12,515.52 for severe infections. Forty-two (30%) of these patients had at least one element of therapy that was inconsistent with recommendations current at the time of the study; 27(19%) received chloroquine; 12 (9%) received primaquine unnecessarily; eight (6%) received inappropriate dosages of pyrimethamine/sulfadoxine (Fansidar)*; and three (2%) received potentially inappropriate dosing regimens of quinine. Conclusions: The relatively low fatality rate, and the fact that 70% of patients received appropriate therapy suggests that the overall standard of care for what is a relatively infrequent disease in the United States is good. However, because of rapidly changing drug resistance patterns, both physicians and travelers need to remain informed to avoid the costs and risks of this potentially severe, but easily preventable infectious disease. (J Travel Med 2:16-21, 1995)