[Script concordance test for knowledge assessment of lung cancer screening]

BACKGROUND

Annual screening for lung cancer using low-dose CT-scans is associated with decreased mortality. A survey conducted in Rhône-Alpes area in France found that clinicians need education and information on this topic. Script concordance tests (SCT) are a tool for assessing clinical reasoning in situations of uncertainty. They have not previously been used in France in the context of continuing medical education.

METHOD

We created a questionnaire with 5 multiple-choice questions (MCQ) and two SCT scenarios. The questionnaire was sent to all clinicians and residents who are members of French-Speaking Respiratory Society or the French Young Pulmonologist Association.

RESULTS

One hundred and ninety answers were analyzed. Seventy percent stated that decreasing mortality was the best criterion for assessing the effectiveness of a cancer screening policy, and 75% that low-dose CT scan was the best test to achieve this in lung cancer screening. Forty-five percent knew the eligibility criteria of the population, and 62% that low-dose CT scan should be performed annually. Participation in tumor boards and certification in oncology were significantly associated with a better score at MCQ and SCT. SCT and MCQ scores were significantly correlated (Spearman's Rho 0.339; P<0.0001).

CONCLUSION

SCT are feasible by electronic survey and seem relevant. Improving knowledge of clinicians on lung cancer screening is still critical.

Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test?

PURPOSE

The script concordance test (SCT) assesses clinical reasoning in the context of uncertainty. Because there is no single correct answer, scoring is based on a comparison of answers provided by examinees with those provided by members of a panel of reference made up of experienced practitioners. This study aims to determine how many members are needed on the panel to obtain reliable scores to compare against the scores of examinees.

METHODS

A group of 80 residents were tested on 73 items (Cronbach's alpha: 0.76). A total of 38 family doctors made up the pool of experienced practitioners, from which 1000 random panels of reference of increasing sizes (5, 10, 15, 20, 25 and 30) were generated with a resampling procedure. Residents' scores were computed for each panel sample. Units of analysis were means of residents' score, test reliability coefficient and correlation coefficient between scores obtained with a given panel of reference versus the scores obtained with the full panel of 38. Statistics were averaged across the 1000 samples for each panel size for the mean and test reliability computations, and across 100 samples for the correlation computation.

RESULTS

For sample variability, there was a 3-fold increase in standard deviation of means between a sample panel size of 5 (SD=1.57) and a panel size of 30 (SD=0.50). For reliability, there was a large difference in precision between a panel size of 5 (0.62) and a panel size of 10 (0.70). When the panel size was over 20, the gain became negligible (0.74 for 20 and 0.76 for 38). For correlation, the mean correlation coefficient values were 0.90 with 5 panel members, 0.95 with 10 members and 0.98 with 20 members.

CONCLUSION

Any number over 10 is associated with acceptable reliability and good correlation between the samples versus the full panel of 38. For high stake examinations, using a panel of 20 members is recommended. Recruiting more than 20 panel members shows only a marginal benefit in terms of psychometric properties.

[Self-assessment for neurosurgery residents by script concordance test (SCT). The process of test elaboration]

BACKGROUND

The script concordance test (SCT) is a new tool to assess clinical reasoning in a context of uncertainty. It was chosen for computer-based self-assessment training for neurosurgery residents.

PURPOSE

The aim of this work is to describe the construction of a first series of questions and provide teachers practical information needed to build a SCT.

METHOD

The theorical background of the SCT is explained. We review the steps we followed when writing of a series of more than two hundred questions.

RESULTS

We present the different steps of our work, including writing clinical cases, validation of questions and elaboration of scoring systems. We explain the main difficulties we encountered.

CONCLUSION

The script concordance test is an interesting easy-to-construct assessment tool useful for evaluating clinical competence. This work summarizes our experience in building a SCT, which should be helpful for teachers interested in this new assessment tool, especially in neurosurgery.

[Selection and training in Canadian ophthalmology residency. A concept to be meditated?]

The authors describe in detail the modalities for the selection of students applying to the ophthalmology programs in the various Canadian universities; the university of Montréal will serve as a model. The characteristics of their 5-year training assured by numerous teachers and short training rotations as well as by a compulsory research project are explained. They highlight the meticulous organization and the compulsory constraints for students, teachers and universities. In absence of evaluation criteria on the efficiency of interventionist training compared to less detailed apprenticeship-program training such as that dispensed in France, they note the differences between the two systems, a necessary precursor to the standardization of training programs which would allow more complete recognition of degrees in the context of a globalization of professional activities.

[Evaluation of clinical reasoning in urology: contribution of the Script Concordance Test]

OBJECTIVE

The Script Concordance test is designed to measure knowledge organization, to allow interpretation of data in clinical decision making. The objective of this study was to assess the value of this new written evaluation instrument to evaluate clinical reasoning in urology.

MATERIAL AND METHOD

An 80-item examination was completed by a group of medical students (n = 15), a group of urology interns (n = 11), a group of registrars-assistants (n = 7), and a group of experienced urologists (n = 10). The scores obtained were compared by analysis of variance. The reliability of the test was studied by calculating Cronbach's coefficient alpha.

RESULTS

The mean score was 46.95 +/- 6.80 for students, 56.18 +/- 1.73 for interns, 66.27 +/- 4.92 for registrars and 63.38 +/- 2.19 for urologists. The differences observed between the scores for students, interns and urologists were significant. The reliability coefficient was 0.79 for the entire examination.

CONCLUSIONS

This test is able to discriminate various levels of experience in urology. It proposes a simple and direct approach to evaluation of knowledge organization. Further studies are necessary to confirm the contribution of this test to the strategy of evaluation of the clinical skills in urology.

Measurement of clinical reflective capacity early in training as a predictor of clinical reasoning performance at the end of residency: an experimental study on the script concordance test

BACKGROUND

The script concordance (SC) test was conceived to measure knowledge organization, the presence of links between items of knowledge which allow for interpretation of data in clinical decision making situations. Earlier studies have shown that the SC test has good psychometric qualities and overcomes some of the limitations of simulation clinical testing. This study explores the predictive validity of the test.

OBJECTIVES

To verify whether scores obtained by students at the end of clerkship predict their clinical reasoning performance at the end of residency.

DESIGN

Comparison of scores obtained on a SC test taken at the end of clerkship with those obtained 2 years later at the end of residency on two clinical reasoning assessments of known validity, called the short-answer management problems (SAMPs) and the simulated office orals (SOOs), and an objective structured clinical examination (OSCE) aimed at assessing hands-on skills and clinical reasoning. Data were treated by Pearson correlation analysis.

SUBJECTS AND SETTING

A cohort of 24 students from a medical school in Quebec was followed up to the end of their residency in family medicine, completed in several schools across Quebec.

RESULTS

The observed Pearson correlation coefficients of the SC test were statistically significant (0.451, P=0.013; 0.447; P=0.015) when compared with the SAMPs and the SOOs, respectively. They were not statistically significant (0.340, P=0.052) when compared with the OSCE.

CONCLUSION

The authors assumed that the richness of knowledge organization, as indicated by SC test scores, would predict part of the performance on the measures of clinical reasoning (SAMP and SOO), but would predict less well performance on the OSCE which measures both clinical skills and clinical reasoning. Data found in the study are coherent with this hypothesis. This is evidence in favour of the construct validity of the SC test. It also indicates that, in the context of certification assessment, if a candidate has shown good organization of clinical knowledge at an early point in training, it can be expected that he/she will show good organization at subsequent measurements of this kind of knowledge. This appears to be true even if the later measures bear on a wider clinical domain.

[Consultation skills in urology: a model for teaching and evaluation]

OBJECTIVE

Consultation activity is an important aspect of urological practice, but the specific teaching of this activity is underdeveloped. The objective of this study was to establish a list of consultancy skills as a basis for a planned and structured approach to teaching and evaluation of consultancy skills in urology.

MATERIAL AND METHOD

Two-step qualitative protocol: 1) Establishment of an initial list of skills based on data of the literature; 2) Submission of this list to a series of "focus groups" (urologists, interns, referring physicians) in order to validate and progressively refine the model.

RESULTS

The items identified were classified into 3 distinct lists: 1) theoretical knowledge; 2) technical skills specific to urology, predominantly performed in the consulting setting, 3) interpersonal skills exclusively concerning the consultant-referring physician relationship.

CONCLUSIONS

The consensual specification of these skills can be used to objectively define teaching and evaluation strategies for urology consultancy skills.

Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning instruction and research

Medical diagnosis is a categorization task that allows physicians to make predictions about features of clinical situations and to determine appropriate course of action. The script concept, which first arose in cognitive psychology, provides a theoretical framework to explain how medical diagnostic knowledge can be structured for diagnostic problem solving. The main characteristics of the script concept are pre-stored knowledge, values acceptable or not acceptable for each illness attribute, and default values. Scripts are networks of knowledge adapted to goals of clinical tasks. The authors describe how scripts are used in diagnostic tasks, how the script concept fits within the clinical reasoning literature, how it contrasts with competing theories of clinical reasoning, how educators can help students build and refine scripts, and how scripts can be used to assess clinical competence.

The Script Concordance test: a tool to assess the reflective clinician

BACKGROUND

The Script Concordance (SC) test is a new assessment tool. It is designed to probe whether knowledge of examinees is efficiently organized for clinical actions. That kind of organization of knowledge is named a script. The SC test places examinees in written, but authentic, clinical situations in which they must interpret data to make decisions.

PURPOSE

The SC test is designed to measure the degree of concordance that exists between examinees' scripts and scripts of a panel of experts. The objective of this article is to provide interested educators with the practical "how to" information needed to build and use an SC test.

METHODS

The theoretical background of the SC test is described. The principles of construction of an SC test are presented, including the writing of clinical cases, the choice of item format, the validation of the test, and the elaboration of the scoring system.

RESULTS

A series of studies have shown that the SC test has interesting psychometric properties, in terms of reliability, face validity, and construct validity. Results from these studies are succinctly presented and commented.

CONCLUSION

The SC test is a simple and direct approach to testing organization and use of knowledge. It has the strong advantage for a testing method of being relatively easy to construct and use and to be machine-scorable. It can be either paper- or computer-based and can be used in undergraduate, postgraduate, or continuing medical education.

[Teaching resources of a hospital Gynecology-Obstetrics service. Review of the literature and practical applications]

OBJECTIVES

To summarize the methods encountered in a gynecological department for teaching medical students.

STUDY

Review of the Medline literature underlying the benefits and disadvantages of each method using the issues of the modern theories of teaching.

RESULTS

All the methods are helpful for learning, with different and complementary objectives. Students can constitute a set of skills using a teaching program containing clear objectives and evaluation on which the future medical practice will be based.

CONCLUSION

Students have immediate benefits from an active clinical learning involving them and are prepared to the Continued Medical Education.

[Apprenticeship in radiology at the University Hospital Center in Grenoble: research on the most effective methods]

From analyses of strengths and weaknesses of the apprenticeship of radiology in France, of existing pedagogic methods, of time constraints within clinical settings and of scientific theories of teaching and learning, the authors define objectives that teaching should target and propose a new method of instruction problem and practise-based. The method appears applicable in French radiology departments. It should forster early acquisition by residents of professional attitudes and way of thinking, and make them active self-directed learners.

Consultation skills in radiology: a qualitative study

OBJECTIVE

Consultation is an important part of radiologic practice. Recently, many concerns have been expressed about the lack of emphasis on consultation skills as part of the training requirements for radiologists. In order to improve the teaching and assessment of trainees, we designed a qualitative study to specify the consultation skills expected from a trainee by the end of residency.

METHODS

Three successive focus groups were held with professors of radiology, residents in radiology, and referring physicians. Participants were asked to identify the various competencies required from radiologic consultants (1) spontaneously, (2) by reacting to a previously generated list, and (3) by reacting to common clinical problems encountered in radiology.

RESULTS

Consultation skills thus identified were organized in a framework consisting of 2 groups: observable skills and standards of practice. Observable skills were subdivided into 5 "problem-setting" skills and 6 "results-management" skills. Seven qualities and attitudes identified were combined under the rubric "standards of practice."

CONCLUSION

The specification of these abilities and competencies through a consensus process should permit the teaching of these skills and their evaluation through objective, structured clinical examinations.

The Diagnosis Script Questionnaire: A New Tool to Assess a Specific Dimension of Clinical Competence

The Diagnosis Script Questionnaire (DSQ) assesses a specific skill of clinical competence: the ability to weigh collected information in light of entertained hypotheses. The questionnaire presents a clinical vignette for which several hypotheses are relevant. The model of the questions is: if you are thinking of hypothesis A and you find sign Z, what is the effect on your hypothesis? Answers are placed on a 7-point Likert scale, with values ranging from "it can only be this hypothesis" to "this hypothesis is definitely rejected." The scoring process is innovative and reflects the variability of answers among experts.The questionnaire was administered in gynecology-obstetrics; 103 respondents, divided into three groups, 15 faculty, 12 residents, and 76 clerkship students volunteered. Mean global scores were 45.3 for faculty, 40.5 for residents, and 35.8 for students. The differences between the three groups were statistically significant with Welch ANOVA (p < 0.001). The Bonferroni post-hoc correction however indicated that the only significant difference was between student and faculty groups (p < 0.001). Cronbach's alpha was 0.822 for the total group; for the student and resident groups, 0.794 and 0.812 respectively. The proportion of the total variance explained by the interaction items/participants as estimated by generalizability was 42.1%, 65.4% and 73.4% for the faculty, resident and student groups respectively.Results agree with the theories of development of clinical competence which states that knowledge structures specifically adapted to diagnostic tasks appear with clinical experience. This new assessment tool appears promising and warrants future development.

[Evaluation of clinical competence in urology: innovative approach based on performance observation]

OBJECTIVE

The authors present a pilot project for the evaluation of clinical skills in urology using a method of evaluation based on observation of real performance.

MATERIAL AND METHODS

An objective and structured clinical examination (OSCE) applied to urology was developed according to a precise predetermined design: 1) Identification of the objectives to be evaluated. 2) Choice of sampling of clinical situations representative of routine urological practice. 3) Construction, on the basis of these cases, of physician-patient interaction stations and question stations, with, for each case, weighting of objectives, construction of observation grids and writing of instructions for candidates, simulated patients and observers.

RESULTS

An OSCE circuit of 10 clinical cases and 16 stations was constructed. The main poles of activity and urological settings were represented. Objective complementary investigations, diagnosis and treatment received the highest weightings. The reliability coefficient, the content validity and the construct validity will be verified on the basis of the overall score obtained by candidate.

CONCLUSION

Establishment of the score and the expected results in terms of reliability, validity, and feasibility are discussed. The psychometric qualities of the OSCE have been demonstrated. Although a single instrument is not sufficient, it is currently the measuring tool which most closely approximates ideal evaluation of clinical skills. If the feasibility of this method is confirmed, this pilot project in urology could provide a new approach to evaluation strategies and could participate in the current reflection concerning urology training.

Congenital fistula of the apex of the pyriform sinus: an overlooked phenomenon of debatable origin

Fistula of the pyriform sinus apex is an often overlooked entity which generally manifests itself as acute suppurative thyroiditis or recurrent deep neck abscesses in children or young adults. Two cases are reported. Arguments in favor of a fourth pharyngeal pouch origin are stressed and the surgical management is described.

Assessment of laryngeal cancer: CT scan versus endoscopy

In order to assess the value of the systematic use of CT scan in the workup for laryngeal cancer, a retrospective study was conducted on 66 consecutive previously untreated cases of laryngeal cancer. Endoscopic and CT scan findings were systematically compared. The areas of particular difficulty in CT scan assessment are described. CT scan alone understaged laryngeal cancers in 10.6% of cases, all of them being superficial spreading tumors within the larynx or in the juxtalaryngeal areas. CT scan worsened the staging in 22.7% of cases, all of them being deep invasions overlooked by endoscopy. CT was most useful in lesions initially classified as T2 and T3, which included all those reclassified by CT. None of the T1 lesions was upgraded after CT, and systematic use of CT for this stage is not warranted. CT scan workup changed the therapeutic attitude in 10 of the 66 patients (15.1%), causing a switch to conservative surgery in seven patients and total laryngectomy with radiotherapy for the three others. CT was also valuable in choosing the most suitable technique for conservation surgery.

Assessment of the cricoarytenoid joint: high-resolution CT scan study with histo-anatomical correlation

Eight normal larynges from autopsy cases were CT scanned and 1.5 mm contiguous slices in the cricoarytenoid joint region were taken. The larynges were then sectioned and stained for histological correlation. Different degrees of calcification, ossification and bone marrow cavitation of the cricoid and arytenoid cartilages were observed. This explained the variety of presentations of these structures on CT films. For example, the cricoarytenoid articular interspace was not always visible due to the saddle-shaped configuration of the joint superimposing cartilages on axial views. Features of normal CA joints on CT scan are discussed.

Pleomorphic lipoma of the tongue: case report and literature review

A case of pleomorphic lipoma of the tongue in a 60-year-old woman is reported. Recently described by Enzinger (1977), pleomorphic lipoma is a rare benign tumor, occurring mostly in elderly males. The subcutaneous tissues of the neck and back are the most common sites involved. Differentiation from some histological variants of liposarcoma remains difficult. Complete surgical excision is the treatment of choice. To our knowledge, this is the first case of pleomorphic lipoma of the tongue reported in the English literature.

Cricoarytenoiditis: CT assessment in rheumatoid arthritis

The cricoarytenoid (CA) joint is a true diarthrodial joint that can be affected by rheumatoid disease. Its strategic location in the airway anatomy makes its evaluation of clinical importance. Direct fiberoptic laryngoscopy (DFL) and high-resolution computerized tomography (HRCT) were used to assess the larynx in 32 rheumatoid patients. Abnormalities were seen in 75% of patients at endoscopic examination. HRCT studies showed abnormalities in 72%. Erosion-luxation of the CA joint and surrounding soft-tissue swelling can be demonstrated on HRCT scans. A radiologic grading of the rheumatoid larynx is proposed, stressing that accurate evaluation of the larynx should be part of the diagnostic evaluation of every rheumatoid arthritic patient, given the high frequency of occurrence of rheumatoid laryngitis.

Cricoarytenoiditis in rheumatoid arthritis: comparison of fibrolaryngoscopic and high resolution computerized tomographic findings

Cricoarytenoid involvement in rheumatoid arthritis has been assessed in 32 patients by combining clinical evaluation, direct fiberoptic laryngoscopy (DFL), and high resolution computerized tomography (CT). Symptoms were inconstant, variable, and rarely prominent. Absence of symptoms does not exclude an involvement. One case presented as an acute respiratory distress. The study showed an unexpected frequency of involvement with 75% of abnormal DFL and 72% of abnormal CT. Semiology of laryngoscopy and CT are described.

Carcinoid tumors of the larynx. A case study

Carcinoid tumors of the larynx are very rare: only 11 cases have been described in the available literature. We report another case, and discuss the nature of the cells of origin and the relationship between these tumors and another rare laryngeal tumor, the oat cell carcinoma.