Measurement of clinical reflective capacity early in training as a predictor of clinical reasoning performance at the end of residency: an experimental study on the script concordance test

BACKGROUND

The script concordance (SC) test was conceived to measure knowledge organization, the presence of links between items of knowledge which allow for interpretation of data in clinical decision making situations. Earlier studies have shown that the SC test has good psychometric qualities and overcomes some of the limitations of simulation clinical testing. This study explores the predictive validity of the test.

OBJECTIVES

To verify whether scores obtained by students at the end of clerkship predict their clinical reasoning performance at the end of residency.

DESIGN

Comparison of scores obtained on a SC test taken at the end of clerkship with those obtained 2 years later at the end of residency on two clinical reasoning assessments of known validity, called the short-answer management problems (SAMPs) and the simulated office orals (SOOs), and an objective structured clinical examination (OSCE) aimed at assessing hands-on skills and clinical reasoning. Data were treated by Pearson correlation analysis.

SUBJECTS AND SETTING

A cohort of 24 students from a medical school in Quebec was followed up to the end of their residency in family medicine, completed in several schools across Quebec.

RESULTS

The observed Pearson correlation coefficients of the SC test were statistically significant (0.451, P=0.013; 0.447; P=0.015) when compared with the SAMPs and the SOOs, respectively. They were not statistically significant (0.340, P=0.052) when compared with the OSCE.

CONCLUSION

The authors assumed that the richness of knowledge organization, as indicated by SC test scores, would predict part of the performance on the measures of clinical reasoning (SAMP and SOO), but would predict less well performance on the OSCE which measures both clinical skills and clinical reasoning. Data found in the study are coherent with this hypothesis. This is evidence in favour of the construct validity of the SC test. It also indicates that, in the context of certification assessment, if a candidate has shown good organization of clinical knowledge at an early point in training, it can be expected that he/she will show good organization at subsequent measurements of this kind of knowledge. This appears to be true even if the later measures bear on a wider clinical domain.

Low plasma leptin concentration and low rates of fat oxidation in weight-stable post-obese subjects

OBJECTIVE

A low resting metabolic rate for a given body size and composition, a low rate of fat oxidation, low levels of physical activity, and low plasma leptin concentrations are all risk factors for body weight gain. The aim of the present investigation was to compare resting metabolic rate (RMR), respiratory quotient (RQ), levels of physical activity, and plasma leptin concentrations in eight post-obese adults (2 males and 6 females; 48.9 +/- 12.2 years; body mass index [BMI]: 24.5 +/- 1.0 kg/m2; body fat 33 +/- 5%; mean +/- SD) who lost 27.1 +/- 21.3 kg (16 to 79 kg) and had maintained this weight loss for > or =2 months (2 to 9 months) to eight age- and BMI-matched control never-obese subjects (1 male and 7 females; 49.1 +/- 5.2 years; BMI 24.4 +/- 1.0 kg/m2; body fat 33 +/- 7%).

RESEARCH METHODS AND PROCEDURES

Following 3 days of weight maintenance diet (50% carbohydrate and 30% fat), RMR and RQ were measured after a 10-hour fast using indirect calorimetry and plasma leptin concentrations were measured using radioimmunoassay. Levels of physical activity were estimated using an accelerometer over a 48-hour period in free living conditions.

RESULTS

After adjustment for fat mass and fat-free mass, post-obese subjects had, compared with controls, similar levels of physical activity (4185 +/- 205 vs. 4295 +/- 204 counts) and similar RMR (1383 +/- 268 vs. 1430 +/- 104 kcal/day) but higher RQ (0.86 +/- 0.04 vs. 0.81 +/- 0.03, p < 0.05). Leptin concentration correlated positively with percent body fat (r = 0.57, p < 0.05) and, after adjusting for fat mass and fat-free mass, was lower in post-obese than in control subjects (4.5 +/- 2.1 vs. 11.6 +/- 7.9 ng/mL, p < 0.05).

DISCUSSION

The low fat oxidation and low plasma leptin concentrations observed in post-obese individuals may, in part, explain their propensity to relapse.

The Script Concordance test: a tool to assess the reflective clinician

BACKGROUND

The Script Concordance (SC) test is a new assessment tool. It is designed to probe whether knowledge of examinees is efficiently organized for clinical actions. That kind of organization of knowledge is named a script. The SC test places examinees in written, but authentic, clinical situations in which they must interpret data to make decisions.

PURPOSE

The SC test is designed to measure the degree of concordance that exists between examinees' scripts and scripts of a panel of experts. The objective of this article is to provide interested educators with the practical "how to" information needed to build and use an SC test.

METHODS

The theoretical background of the SC test is described. The principles of construction of an SC test are presented, including the writing of clinical cases, the choice of item format, the validation of the test, and the elaboration of the scoring system.

RESULTS

A series of studies have shown that the SC test has interesting psychometric properties, in terms of reliability, face validity, and construct validity. Results from these studies are succinctly presented and commented.

CONCLUSION

The SC test is a simple and direct approach to testing organization and use of knowledge. It has the strong advantage for a testing method of being relatively easy to construct and use and to be machine-scorable. It can be either paper- or computer-based and can be used in undergraduate, postgraduate, or continuing medical education.

Association between licensing examination scores and resource use and quality of care in primary care practice

CONTEXT

Clinical competence is a determinant of the quality of care delivered, and may be associated with use of health care resources by primary care physicians. Clinical competence is assumed to be assessed by licensing examinations, yet there is a paucity of information on whether scores achieved predict subsequent practice.

OBJECTIVE

To determine if licensing examination scores were associated with selected aspects of quality of care and resource use in initial primary care practice.

DESIGN

Prospective cohort study of recently licensed family physicians, followed up for the first 18 months of practice.

SETTING

The Quebec health care system.

PARTICIPANTS

A total of 614 family physicians who passed the licensing examination between 1991 and 1993 and entered fee-for-service practice in Quebec.

MAIN OUTCOME MEASURES

All patients seen by physicians were identified by the universal health insurance board and all health services provided to these patients were retrieved for the 18 months prior to (baseline) and after (follow-up) the physicians' entry into practice. Medical service and prescription claims files were used to measure rates of resource use (specialty consultation, symptom-relief prescribing compared with disease-specific prescribing) and quality of care (inappropriate prescribing, mammography screening). Baseline data were used to adjust for differences in practice population.

RESULTS

Study physicians saw a total of 1116389 patients, of whom 113535 (10.2%) were elderly and 83391 (7.5%) were women aged 50 to 69 years. Physicians with higher licensing examination scores referred more of their patients for consultation (3.8/1000 patients per SD increase in score; 95% confidence interval [CI], 1.2-7.0; P = .005), prescribed to elderly patients fewer inappropriate medications (-2.7/1000 patients per SD increase in score; 95% CI, -4.8 to -0.7; P=.009) and more disease-specific medications relative to symptom-relief medications (3.9/1000 patients per SD increase in score; 95% CI, 0.3 to 7.4; P= .03), and referred more women aged 50 to 69 years (6.6/1000 patients per SD increase in score; 95% CI, 1.2-11.9; P = .02) for mammography screening. If patients of physicians with the lowest scores had experienced the same rates of consultation, prescribing, and screening as patients of physicians with the highest scores, an additional 3027 patients would have been referred, 179 fewer elderly patients would have been prescribed symptom-relief medication, 912 more elderly patients would have been prescribed disease-specific medication, 189 fewer patients would have received inappropriate medication, and 121 more women would have received mammography screening.

CONCLUSIONS

Licensing examination scores are significant predictors of consultation, prescribing, and mammography screening rates in initial primary care practice.

The Diagnosis Script Questionnaire: A New Tool to Assess a Specific Dimension of Clinical Competence

The Diagnosis Script Questionnaire (DSQ) assesses a specific skill of clinical competence: the ability to weigh collected information in light of entertained hypotheses. The questionnaire presents a clinical vignette for which several hypotheses are relevant. The model of the questions is: if you are thinking of hypothesis A and you find sign Z, what is the effect on your hypothesis? Answers are placed on a 7-point Likert scale, with values ranging from "it can only be this hypothesis" to "this hypothesis is definitely rejected." The scoring process is innovative and reflects the variability of answers among experts.The questionnaire was administered in gynecology-obstetrics; 103 respondents, divided into three groups, 15 faculty, 12 residents, and 76 clerkship students volunteered. Mean global scores were 45.3 for faculty, 40.5 for residents, and 35.8 for students. The differences between the three groups were statistically significant with Welch ANOVA (p < 0.001). The Bonferroni post-hoc correction however indicated that the only significant difference was between student and faculty groups (p < 0.001). Cronbach's alpha was 0.822 for the total group; for the student and resident groups, 0.794 and 0.812 respectively. The proportion of the total variance explained by the interaction items/participants as estimated by generalizability was 42.1%, 65.4% and 73.4% for the faculty, resident and student groups respectively.Results agree with the theories of development of clinical competence which states that knowledge structures specifically adapted to diagnostic tasks appear with clinical experience. This new assessment tool appears promising and warrants future development.

When to recommend compulsory versus optional CME programs? A study to establish criteria

When should remedial continuing medical education (CME) be compulsory for family physicians? When should it be optional? Should it be structured or not? In 1993-1994, the authors addressed this need for criteria by conducting a study that used reports on 14 physicians who had undergone a structured oral interview (SOI) at the College of Physicians of Quebec. (The SOI is a day-long encounter during which two specially trained physician-interviewers present a physician with standardized clinical cases that focus on ten specific aspects of a family physician's competence.) The 14 SOI reports were reviewed by 12 external physician-judges in an attempt to see how consistently they could link the ten aspects of competence, as shown in the reports, to five particular types of recommended remedial CME programs (the strictest being "compulsory program with suspended license" and the most lenient being "simple suggestions for improvement"). There was substantial agreement among the judges when choosing between compulsory and optional programs (kappa = 0.63, p < .05). The main criteria used when recommending an optional program were overall strengths and the quality of clinical reasoning. The same two criteria were also used for recommending a compulsory program, but the judges also considered three additional factors: the physician's ability to recognize his or her limits and how he or she handled referrals and prescribed medications. Many of the criteria used by the judges were based on unique information that came out of observations and interactions during the SOIs, such as quality of argumentation, sustaining a train of thought, sense for the case as a whole, or awareness of one's limits. Finally, the external judges corroborated the decisions previously made by the College of Physicians of Quebec concerning the appropriate CME programs for the 14 physicians.

Content validation of key features on a national examination of clinical decision-making skills

Key features (KFs) represent the critical, or essential, steps in the identification and management of a clinical problem. KFs for 59 clinical problems were defined by members of a test committee for the Medical Council of Canada as part of their efforts to create a more valid written examination of clinical decision-making skills for the Canadian Qualifying Examination in Medicine. In order to evaluate the content validity of KFs that the test committee had defined for the examination, 99 physicians from outside the committee, who came from clerkship programs at all 16 of Canada's medical schools, participated in three studies conducted in 1991. The first study was retrospective and was designed to find the degree of agreement or disagreement that the outside physicians had with the KFs already defined for each problem by the committee members. The second study was prospective and was to compare the KFs generated de novo by the participants with those already defined by the committee members. The third study was to gather the outside physicians' opinions of the frequencies with which graduating students in Canada are exposed to the 59 problems used in the retrospective and prospective studies. Almost all the KFs defined by the test committee were corroborated by the outside physicians, 92% in the retrospective study and 94% in the prospective one.(ABSTRACT TRUNCATED AT 250 WORDS)

Prophylactic maltose tetrapalmitate and bacillus Calmette-Guerin immunotherapy of recurrent superficial bladder tumors: preliminary report

We divided randomly into 3 groups 47 patients with recurrent superficial transitional cell carcinoma of the bladder: group 1-15 controls who underwent transurethral resection only, group 2-17 patients who underwent transurethral resection and bacillus Calmette-Guerin therapy, and group 3-15 patients treated by transurethral resection and maltose tetrapalmitate. Mean followup was 22.93 months for the controls, 28.0 months for group 2 and 24.4 months for group 3. The recurrence rate per 100 patient-months was 11.34 in the controls, 7.4 in group 2 and 7.19 in group 3, and the recurrence index per month was 0.113, 0.070 and 0.072, respectively. The recurrence rate and recurrence index per month were significantly decreased in the treated groups compared to the controls (p less than 0.005). There was no significant difference between the bacillus Calmette-Guerin and maltose tetrapalmitate groups. Invasive carcinoma developed in 60 per cent of the controls, 29.4 per cent of group 2 and 20 per cent of group 3. Invasive carcinoma required cystectomy or definitive radiotherapy. Bacillus Calmette-Guerin caused irritation of the bladder mucosa, while maltose tetrapalmitate did not have any side effects.

Mitogenic, immunoadjuvancy, and genetic studies on fatty acyl maltose

Three synthetic glycolipids, maltose tetrapalmitate (MTP), maltose hexastearate (MHS), and maltose hexalinoleate (MHL) prepared as nontoxic lipid A analogs, and Escherichia coli lipopolysaccharide (LPS) were assayed for their mitogenic activity using spleen lymphocytes in nine inbred mouse strains and three F1 hybrids. The MTP and LPS were also assayed for their ability to enhance plaque-forming cell (PFC) responses using sheep red blood cells as the antigen in the same inbred mouse strains and F1 hybrids, The mitogenic activity of synthetic glyco-lipids was several fold lower than that of LPS and MHL was inferior to MTP and MHS. DBA/2J was the most responsive strain for MTP and DBA/1J and C3H/HeJ the least. The mitogenic activity of MTP was generally in agreement with the PFC response stimulation by it. Low-dose cyclophosphamide treatment of mice synergized MTP for PFC response augmentation. Genetic studies on MTP mitogenicity revealed that 90% of responder DBA/2J X nonresponder C3H/HeJ F1 hybrids had intermediate mitogenic activity. Among F2, 73% had intermediate-high activity and 27% were nonmitogenic. Among F1 X C3H/HeJ backcrosses 11% had high, 56% intermediate, and 33% had no mitogenic activity, whereas, for the F1 X DBA/2J backcross, 14% had high, 36% intermediate, and 50% low or negligible activity. The data favored a single gene for MTP activation of immune cells.

Combined radiotherapy, chemotherapy, and maltose tetrapalmitate immunotherapy in the treatment of 4'dimethylaminoazobenzene-induced liver cancer

Therapeutic effects of radiotherapy (R), chemotherapy, and maltose tetrapalmitate (MTP) immunotherapy alone and in combinations were tried against 4' dimethylaminoazobenzene (DAB) induced primary liver cancer in Wistar rats in three separate protocols. Rats were fed a low protein synthetic diet containing 0.06% DAB for 90-120 days. Around 90 days, liver cancers developed in all the animals. In the first protocol, animals were either left untreated or treated with cyclophosphamide (Cy), MTP (i.p. or oral) and Cy plus oral MTP. Rats in the MTP (i.p.) group maintained a steady liver weight but neither Cy nor Cy + MTP influenced the survival time or liver weight. In the second protocol, R as well as a 3-drug combination at 2 dose levels were tried alone and with MTP before or soon after cessation of DAB feeding. Survival times were decreased by R and chemotherapy due to combined toxicities of DAB and treatments and were partially restored by MTP. In the third protocol, MTP, R, and Cy were each tried alone and in combinations, 21 days after cessation of 100-day DAB feeding. Increase in survivals were obtained by each treatment, although tumor weight was best controlled by triple R+ Cy + MTP combination.

Ganglioside patterns and phenotypic characteristics in a normal variant and a transformed back variant of a simian virus 40-induced hamster tumor cell line

Ganglioside patterns of a cloned Simian virus 40- (SV40) induced hamster tumor cell (Cl(2)TSV(5)-S), its normal variant (Cl(2)TSV(5)-R) which are Cl(2)TSV(5)-S gradually adapted to grow in the presence of 2 microg/ml actinomycin D and exhibit certain normal phenotypic characteristics, and its back variant (Cl(2)TSV(5)-RR), which are Cl(2)TSV(5)-R cells grown in the absence of actinomycin D for more than 60 passages and which exhibit greater phenotypic similarity to Cl(2)TSV(5)-S cells, have been analyzed. All three cell lines contain N(acetylneuraminyl) galactosylglycosyl ceramide (hematoside, GM(3)), N-acetylgalactosaminyl (N-acetylneuraminyl) galactosylglucosyl ceramide (GM(2)), and a higher ganglioside tentatively identified as disialohematoside. However, Cl(2)TSV(5)-R have more GM(2) than Cl(2)TSV(5)-S whereas Cl(2)TSV(5)-RR contain an intermediate amount of GM(2). The amount of GM(2) is correlative with the activity of UDP-N-acetylgalactosamine: hematoside N-acetylgalactosaminyl transferase in the extract of the three cell lines and with their agglutination by wheat germ agglutinin.

Growth of normal and transformed rat embryo fibroblasts: effects of glycolipids from Salmonella minnesota R mutants

Addition of glycolipids obtained from Salmonella minnesota R mutants to normal, spontaneously transformed, and SV40-transformed rat embryo fibroblasts in culture results in an inhibition of growth of transformed cells but not of normal cells. In the presence of the glycolipid with the smallest carbohydrate chain length, spontaneously transformed cells stop growing when they reach confluency. Inhibition of growth of transformed cells is inversely related to the chain length of the core sugars. Glycolipid mR595 is shown to bind with the cell membrane of transformed cells and elicits an augmentation in the intracellular level of cyclic AMP. Normal cells bind relatively less glycolipid mR595 and show a lower percent of increase in cyclic AMP due to glycolipid mR595 than do transformed cells.