Junior Radiologists' Forum (JRF): National trainee survey

Quality of radiology training and role of Royal Australian and New Zealand College of Radiology in supporting radiology trainees in NSW: Results of the first radiology trainee survey

INTRODUCTION

The Royal Australian and New Zealand College radiology training programme is a 5-year programme with a vast curriculum including reporting and research requirements. Undertaking training can be stressful for trainees who must balance their educational needs and work responsibilities. We undertook the first independent survey of New South Wales (NSW) radiology trainees to evaluate their perceptions about the quality of their training.

METHODS

Focus groups with trainees from multiple NSW training sites were conducted to construct a survey which was then distributed to all NSW Radiology trainees (n = 118). Data from the survey were analysed, and factors correlating to the overall satisfaction with the programme were explored using Spearman's correlation.

RESULTS

Survey response rate was 70.3%. Eighty-nine per cent of trainees were satisfied with their career choice, and 73% were satisfied with the training programme. Majority felt they had a good exposure to cases, modalities and access to resources to complete their training. Trainee satisfaction significantly correlated with a supportive work environment (r_s  = 0.83, P < 0.0001), which involved supportive consultants (r_s  = 0.75, P < 0.0001), good peer support (r_s  = 0.60, P < 0.0001) and their training site respecting work/life balance (r_s  = 0.62, P < 0.0001). As trainees progressed through the training programme, they became less satisfied, with trainees in years 3 and above being the most dissatisfied.

CONCLUSION

NSW radiology trainees are generally satisfied with their training programme and career choices. Trainee satisfaction correlated most strongly with supportive work environment, good consultant support, peer relationships and good work/life balance; satisfaction decreased for senior trainees.

Development of competence in volumetric image interpretation in radiology residents

BACKGROUND

During residency, radiology residents learn to interpret volumetric radiological images. The development of their competence for volumetric image interpretation, as opposed to 2D image reading, is not completely understood. The purpose of the present study was to investigate how competence for volumetric image interpretation develops in radiology residents and how this compares with competence development for 2D image interpretation, by studying resident scores on image-based items in digital radiology tests.

METHODS

We reviewed resident scores on volumetric and 2D image-based test items in 9 consecutive semi-annual digital radiology tests that were carried out from November 2013 to April 2018. We assessed percentage-correct sum scores for all test items about volumetric images and for all test items about 2D images in each test as well as for all residents across the 9 tests (i.e. 4.5 years of test materials). We used a paired t-test to analyze whether scores differed between volumetric and 2D image-based test items in individual residents in postgraduate year (PGY) 0-5, subdivided in 10 half-year phases (PGY 0-0.5, 0.5-1.0, 1.0-1.5 et cetera).

RESULTS

The percentage-correct scores on volumetric and 2D image-based items showed a comparable trend of development, increasing in the first half of residency and flattening off in the second half. Chance-corrected scores were generally lower in volumetric than in 2D items (on average 1-5% points). In PGY 1.5-4.5, this score difference was statistically significant (p-values ranging from 0.02 to < 0.001), with the largest difference found in PGY 2.5 (mean: 5% points; 95% CI: -7.3 - -3.4). At the end of training in PGY 5, there was no statistically significant score difference between both item types.

CONCLUSIONS

The development of competence in volumetric image interpretation fits a similar curvilinear growth curve during radiology residency as 2D image interpretation competence in digital radiology tests. Although residents performed significantly lower on volumetric than 2D items in PGY 1.5-4.5, we consider the magnitude of this difference as relatively small for our educational setting and we suggest that throughout radiology training there are no relevant differences in the development of both types of competences, as investigated by digital radiology tests.

Determinants of Difficulty and Discriminating Power of Image-based Test Items in Postgraduate Radiological Examinations

RATIONALE AND OBJECTIVES

The psychometric characteristics of image-based test items in radiological written examinations are not well known. In this study, we explored difficulty and discriminating power of these test items in postgraduate radiological digital examinations.

MATERIALS AND METHODS

We reviewed test items of seven Dutch Radiology Progress Tests (DRPTs) that were taken from October 2013 to April 2017. The DRPT is a semiannual formative examination, required for all Dutch radiology residents. We assessed several stimulus and response characteristics of test items. The response format of test items included true or false, single right multiple choice with 2, 3, 4, or ≥5 answer options, pick-N multiple-choice, drag-and-drop, and long-list-menu formats. We calculated item P values and item-rest-correlation (Rir) values to assess difficulty and discriminating power. We performed linear regression analysis in image-based test items to investigate whether P and Rir values were significantly related to stimulus and response characteristics. Also, we compared psychometric indices between image-based test items and text-alone items.

RESULTS

P and Rir values of image-based items (n = 369) were significantly related to the type of response format (P < .001), and not to which of the seven DRPTs the item was obtained from, radiological subspecialty domain, nonvolumetric or volumetric character of images, or context-rich or context-free character of the stimulus. When accounted for type of response format, difficulty and discriminating power of image-based items did not differ significantly from text-alone items (n = 881). Test items with a relatively large number of answer options were generally more difficult, and discriminated better among high- and low-performing candidates.

CONCLUSION

In postgraduate radiological written examinations, difficulty and discriminating power of image-based test items are related to the type of response format and are comparable to those of text-alone items. We recommend a response format with a relatively large number of answer options to optimize psychometric indices of radiological image-based test items.

Fourteen years of progress testing in radiology residency training: experiences from The Netherlands

Objectives

To describe the development of the Dutch Radiology Progress Test (DRPT) for knowledge testing in radiology residency training in The Netherlands from its start in 2003 up to 2016.

Methods

We reviewed all DRPTs conducted since 2003. We assessed key changes and events in the test throughout the years, as well as resident participation and dispensation for the DRPT, test reliability and discriminative power of test items.

Results

The DRPT has been conducted semi-annually since 2003, except for 2015 when one digital DRPT failed. Key changes in these years were improvements in test analysis and feedback, test digitalization (2013) and inclusion of test items on nuclear medicine (2016). From 2003 to 2016, resident dispensation rates increased (Pearson’s correlation coefficient 0.74, P-value <0.01) to maximally 16 %. Cronbach´s alpha for test reliability varied between 0.83 and 0.93. The percentage of DRPT test items with negative item-rest-correlations, indicating relatively poor discriminative power, varied between 4 % and 11 %.

Conclusions

Progress testing has proven feasible and sustainable in Dutch radiology residency training, keeping up with innovations in the radiological profession. Test reliability and discriminative power of test items have remained fair over the years, while resident dispensation rates have increased.

Key Points

• Progress testing allows for monitoring knowledge development from novice to senior trainee.

• In postgraduate medical training, progress testing is used infrequently.

• Progress testing is feasible and sustainable in radiology residency training.