Impact of inpatient caseload, emergency department duties, and online learning resource on General Medicine In-Training Examination scores in Japan

Hospital Use of a Web-Based Clinical Knowledge Support System and In-Training Examination Performance Among Postgraduate Resident Physicians in Japan: Nationwide Observational Study

Background

The relationship between educational outcomes and the use of web-based clinical knowledge support systems in teaching hospitals remains unknown in Japan. A previous study on this topic could have been affected by recall bias because of the use of a self-reported questionnaire.

Objective

We aimed to explore the relationship between the use of the Wolters Kluwer UpToDate clinical knowledge support system in teaching hospitals and residents' General Medicine In-Training Examination (GM-ITE) scores. In this study, we objectively evaluated the relationship between the total number of UpToDate hospital use logs and the GM-ITE scores.

Methods

This nationwide cross-sectional study included postgraduate year-1 and -2 residents who had taken the examination in the 2020 academic year. Hospital-level information was obtained from published web pages, and UpToDate hospital use logs were provided by Wolters Kluwer. We evaluated the relationship between the total number of UpToDate hospital use logs and residents' GM-ITE scores. We analyzed 215 teaching hospitals with at least 5 GM-ITE examinees and hospital use logs from 2017 to 2019.

Results

The study population consisted of 3013 residents from 215 teaching hospitals with at least 5 GM-ITE examinees and web-based resource use log data from 2017 to 2019. High-use hospital residents had significantly higher GM-ITE scores than low-use hospital residents (mean 26.9, SD 2.0 vs mean 26.2, SD 2.3; P=.009; Cohen d=0.35, 95% CI 0.08-0.62). The GM-ITE scores were significantly correlated with the total number of hospital use logs (Pearson r=0.28; P<.001). The multilevel analysis revealed a positive association between the total number of logs divided by the number of hospital physicians and the GM-ITE scores (estimated coefficient=0.36, 95% CI 0.14-0.59; P=.001).

Conclusions

The findings suggest that the development of residents' clinical reasoning abilities through UpToDate is associated with high GM-ITE scores. Thus, higher use of UpToDate may lead physicians and residents in high-use hospitals to increase the implementation of evidence-based medicine, leading to high educational outcomes.

Distribution of internal medicine rotations among resident physicians in Japan: a nationwide, multicenter, cross-sectional study

BACKGROUND

In Japan, postgraduate clinical training encompasses a 2-year residency program, including at least 24 weeks of internal medicine (IM) rotations. However, the fragmented structure of these rotations can compromise the training's quality and depth. For example, a resident might spend only a few weeks in cardiology before moving to endocrinology, without sufficient time to deepen their understanding or have clinical experience. This study examined current patterns and lengths of IM rotations within the Japanese postgraduate medical system. It scrutinized the piecemeal approach-whereby residents may engage in multiple short-term stints across various subspecialties without an overarching, integrated experience-and explored potential consequences for their clinical education.

METHODS

This nationwide, multicenter, cross-sectional study used data from self-reported questionnaires completed by participants in the 2022 General Medicine In-Training Examination (GM-ITE). Data of 1,393 postgraduate year (PGY) one and two resident physicians who participated in the GM-ITE were included. We examined the IM rotation duration and number of IM subspecialties chosen by resident physicians during a 2-year rotation.

RESULTS

Approximately half of the participants chose IM rotation periods of 32-40 weeks. A significant proportion of participants rotated in 5-7 internal medicine departments throughout the observation period. Notable variations in the distribution of rotations were observed, characterized by a common pattern where resident physicians typically spend 4 weeks in each department before moving to the next. This 4-week rotation is incrementally repeated across different subspecialties without a longer, continuous period in any single area. Notably, 39.7% of participants did not undertake general internal medicine rotations. These results suggest a narrowed exposure to medical conditions and patient care practices.

CONCLUSIONS

Our study highlights the need to address the fragmented structure of IM rotations in Japan. We suggest that short, specialized learning periods may limit the opportunity to gain broad in-depth knowledge and practical experience. To improve the efficacy of postgraduate clinical education, we recommend fostering more sustained and comprehensive learning experiences.

Development of a Clinical Simulation Video to Evaluate Multiple Domains of Clinical Competence: Cross-Sectional Study

BACKGROUND

Medical students in Japan undergo a 2-year postgraduate residency program to acquire clinical knowledge and general medical skills. The General Medicine In-Training Examination (GM-ITE) assesses postgraduate residents' clinical knowledge. A clinical simulation video (CSV) may assess learners' interpersonal abilities.

OBJECTIVE

This study aimed to evaluate the relationship between GM-ITE scores and resident physicians' diagnostic skills by having them watch a CSV and to explore resident physicians' perceptions of the CSV's realism, educational value, and impact on their motivation to learn.

METHODS

The participants included 56 postgraduate medical residents who took the GM-ITE between January 21 and January 28, 2021; watched the CSV; and then provided a diagnosis. The CSV and GM-ITE scores were compared, and the validity of the simulations was examined using discrimination indices, wherein ≥0.20 indicated high discriminatory power and >0.40 indicated a very good measure of the subject's qualifications. Additionally, we administered an anonymous questionnaire to ascertain participants' views on the realism and educational value of the CSV and its impact on their motivation to learn.

RESULTS

Of the 56 participants, 6 (11%) provided the correct diagnosis, and all were from the second postgraduate year. All domains indicated high discriminatory power. The (anonymous) follow-up responses indicated that the CSV format was more suitable than the conventional GM-ITE for assessing clinical competence. The anonymous survey revealed that 12 (52%) participants found the CSV format more suitable than the GM-ITE for assessing clinical competence, 18 (78%) affirmed the realism of the video simulation, and 17 (74%) indicated that the experience increased their motivation to learn.

CONCLUSIONS

The findings indicated that CSV modules simulating real-world clinical examinations were successful in assessing examinees' clinical competence across multiple domains. The study demonstrated that the CSV not only augmented the assessment of diagnostic skills but also positively impacted learners' motivation, suggesting a multifaceted role for simulation in medical education.

Association between mentorship and mental health among junior residents: A nationwide cross-sectional study in Japan

Background

Mentorship is a dynamic, reciprocal relationship in which an advanced careerist (mentor) encourages the growth of a novice (mentee). Mentorship may protect the mental health of residents at risk for depression and burnout, yet despite its frequent use and known benefits, limited reports exist regarding the prevalence and mental effects of mentorship on residents in Japan.

Methods

We conducted a cross-sectional study involving postgraduate year 1 and 2 (PGY-1 and PGY-2) residents in Japan who took the General Medicine In-Training Examination (GM-ITE) at the end of the 2021 academic year. Data on mentorship were collected using surveys administered immediately following GM-ITE completion. The primary outcome was the Patient Health Questionaire-2 (PHQ-2), which consisted depressed mood and loss of interest. A positive response for either item indicated PHQ-2 positive. We examined associations between self-reported mentorship and PHQ-2 by multi-level analysis.

Results

Of 4929 residents, 3266 (66.3%) residents reported having at least one mentor. Compared to residents without any mentor, those with a mentor were associated with a lower likelihood of a positive PHQ-2 response (adjusted odds ratio [aOR] 0.75; 95% confidence interval [95% CI] 0.65-0.86). Mentor characteristic significantly associated with negative PHQ-2 response was a formal mentor (aOR; 0.68; 95% CI 0.55-0.84).

Conclusions

A mentor-based support system was positively associated with residents' mental health. Further research is needed to determine the quality of mentorship during clinical residency in Japan.

Association between regional quota program in medical schools and practical clinical competency based on General Medicine In-Training Examination score: a nationwide cross-sectional study of resident physicians in Japan

PURPOSE

A regional quota program (RQP) was introduced in Japan to ameliorate the urban-rural imbalance of physicians. Despite concerns about the low learning abilities of RQP graduates, the relationship between the RQP and practical clinical competency after initiating clinical residency has not been evaluated.

METHODS

We conducted a nationwide cross-sectional study to assess the association between the RQP and practical clinical competency based on General Medicine In-Training Examination (GM-ITE) scores. We compared the overall and category GM-ITE results between RQP graduates and other resident physicians. The relationship between the RQP and scores was examined using multilevel linear regression analysis.

RESULTS

There were 4978 other resident physicians and 1119 RQP graduates out of 6097 participants from 593 training hospitals. Being younger; preferring internal, general, or emergency medicine; managing fewer inpatients; and having fewer ER shifts were all characteristics of RQP graduates. In multilevel multivariable linear regression analysis, there was no significant association between RQP graduates and total GM-ITE scores (coefficient: 0.26; 95% confidence interval: -0.09, 0.61; P = .15). The associations of RQP graduates with GM-ITE scores in each category and specialty were not clinically relevant. However, in the same multivariable model, the analysis did reveal that total GM-ITE scores demonstrated strong positive associations with younger age and GM preference, both of which were significantly common in RQP graduates.

CONCLUSION

Practical clinical competency evaluated based on the GM-ITE score showed no clinically relevant differences between RQP graduates and other resident physicians. Key messages What is already known on this topic Many countries offer unique admission processes to medical schools and special undergraduate programs to increase the supply of physicians in rural areas. Concerns have been raised about the motivation, learning capabilities, and academic performance of the program graduates. What this study adds This nationwide cross-sectional study in Japan revealed clinical competency based on the scores from the General Medicine In-Training Examination showed no clinically relevant differences between graduates of regional quota programs and other resident physicians. How this study might affect research, practice, or policy The study provides evidence to support the Japanese regional quota program from the perspective of clinical competency after initiating clinical practice.

Association between prolonged weekly duty hours and self-study time among residents: a cross-sectional study

PURPOSE

In 2024, the Japanese government will enforce a maximum 80-hour weekly duty hours (DHs) regulation for medical residents. Although this reduction in weekly DHs could increase the self-study time (SST) of these residents, the relationship between these two variables remains unclear. The aim of the study was to investigate the relationship between the SST and DHs of residents in Japan.

METHODS

In this nationwide cross-sectional study, the subjects were candidates of the General Medicine In-Training Examination in the 2020 academic year. We administered questionnaires and categorically asked questions regarding daily SST and weekly DHs during the training period. To account for hospital variability, proportional odds regression models with generalized estimating equations were used to analyse the association between SST and DHs.

RESULTS

Of the surveyed 6117 residents, 32.0% were female, 49.1% were postgraduate year-1 residents, 83.8% were affiliated with community hospitals, and 19.9% worked for ≥80 hours/week. Multivariable analysis revealed that residents working ≥80 hours/week spent more time on self-study than those working 60-70 hours/week. Conversely, residents who worked <50 hours/week spent less time on self-study than those who worked 60-70 hours/week. The factors associated with longer SST were sex, postgraduate year, career aspiration for internal medicine, affiliation with community hospitals, academic involvement, and well-being.

CONCLUSION

Residents with long DHs had longer SSTs than residents with short DHs. Future DH restrictions may not increase but rather decrease resident SST. Effective measures to encourage self-study are required, as DH restrictions may shorten SST.

Medical resident's pursuing specialty and differences in clinical proficiency among medical residents in Japan: a nationwide cross-sectional study

IMPORTANCE

Standardized examinations assess both learners and training programs within the medical training system in Japan. However, it is unknown if there is an association between clinical proficiency as assessed by the General Medicine In-Training Examination (GM-ITE) and pursuing specialty.

OBJECTIVE

To determine the relative achievement of fundamental skills as assessed by the standardized GM-ITE based on pursuing career specialty among residents in the Japanese training system.

DESIGN

Nationwide cross-sectional study.

SETTING

Medical residents in Japan who attempted the GM-ITE in their first or second year were surveyed.

PARTICIPANTS

A total of 4,363 postgraduate years 1 and 2 residents who completed the GM-ITE were surveyed between January 18 and March 31, 2021.

MAIN MEASURES

GM-ITE total scores and individual scores in each of four domains assessing clinical knowledge: 1) medical interview and professionalism, 2) symptomatology and clinical reasoning, 3) physical examination and treatment, and 4) detailed disease knowledge.

RESULTS

When compared to the most pursued specialty, internal medicine, only those residents who chose general medicine achieved higher GM-ITE scores (coefficient 1.38, 95% CI 0.08 to 2.68, p = 0.038). Conversely, the nine specialties and "Other/Not decided" groups scored significantly lower. Higher scores were noted among residents entering general medicine, emergency medicine, and internal medicine and among those who trained in community hospitals with higher numbers of beds, were more advanced in their training, spent more time working and studying, and cared for a moderate but not an extreme number of patients at a time.

CONCLUSIONS

Levels of basic skill achievement differed depending on respective chosen future specialties among residents in Japan. Scores were higher among those pursuing careers in general medical fields and lower among those pursuing highly specialized careers. Residents in training programs devoid of specialty-specific competition may not possess the same motivations as those in competitive systems.

Impact of duty hours on competency-related knowledge acquisition among community hospital residents

Background

The effect of duty hour (DH) restrictions on postgraduate residents' acquisition of clinical competencies is unclear. We evaluated the relationship between DHs and competency-related knowledge acquisition using the General Medicine In-training Examination (GM-ITE).

Methods

We conducted a multicenter, cross-sectional study of community hospital residents among 2019 GM-ITE examinees. Self-reported average DHs per week were classified into five DH categories and the competency domains were classified into four areas: symptomatology and clinical reasoning (CR), physical examination and clinical procedure (PP), medical interview and professionalism (MP), and disease knowledge (DK). The association between these scores and DHs was examined using random-intercept linear models with and without adjustment for confounding factors.

Results

We included 4753 participants in the analyses. Of these, 31% were women, and 49.1% were in the postgraduate year (PGY) 2. Mean CR and MP scores were lower among residents in Category 1 (<50 h) than in residents in Category 3 (≥60 and <70 h; reference group). Mean DK scores were lower among residents in Categories 1 and 2 (≥50 and <60 h) than in the reference group. PGY-2 residents in Categories 1 and 2 had lower CR scores than those in Category 3; however, PGY-1 residents in Category 5 showed higher scores.

Conclusions

The relationship between DHs and each competency area is not strictly linear. The acquisition of knowledge of physical examination and clinical procedures skills in particular may not be related to DHs.

Impact of the resident duty hours on in-training examination score: A nationwide study in Japan

PURPOSE

The relationship between duty hours (DH) and the performance of postgraduate residents is needed to establish appropriate DH limits. This study explores their relationship using the General Medicine In-training Examination (GM-ITE).

MATERIALS AND METHODS

In this cross-sectional study, GM-ITE examinees of 2019 had participated. We analyzed data from the examination and questionnaire, including DH per week (eight categories). We examined the association between DH and GM-ITE score, using random-intercept linear models with and without adjustments.

RESULTS

Five thousand five hundred and ninety-three participants (50.7% PGY-1, 31.6% female, 10.0% university hospitals) were included. Mean GM-ITE scores were lower among residents in Category 2 (45-50 h; mean score difference, -1.05; p < 0.001) and Category 4 (55-60 h; -0.63; p = 0.008) compared with residents in Category 5 (60-65 h; Reference). PGY-2 residents in Categories 2-4 had lower GM-ITE scores compared to those in Category 5. University residents in Category 1 and Category 5 showed a large mean difference (-3.43; p = 0.01).

CONCLUSIONS

DH <60-65 h per week was independently associated with lower resident performance, but more DH did not improve performance. DH of 60-65 h per week may be the optimal balance for a resident's education and well-being.

Validation of the General Medicine in-Training Examination Using the Professional and Linguistic Assessments Board Examination Among Postgraduate Residents in Japan

Purpose

In Japan, the General Medicine In-training Examination (GM-ITE) was developed by a non-profit organization in 2012. The GM-ITE aimed to assess the general clinical knowledge among residents and to improve the training programs; however, it has not been sufficiently validated and is not used for high-stake decision-making. This study examined the association between GM-ITE and another test measure, the Professional and Linguistic Assessments Board (PLAB) 1 examination.

Methods

Ninety-seven residents who completed the GM-ITE in fiscal year 2019 were recruited and took the PLAB 1 examination in Japanese. The association between two tests was assessed using the Pearson product-moment statistics. The discrimination indexes were also assessed for each question.

Results

A total of 91 residents at 17 teaching hospitals were finally included in the analysis, of whom 69 (75.8%) were women and 59 (64.8%) were postgraduate second year residents. All the participants were affiliated with community hospitals. Positive correlations were demonstrated between the GM-ITE and the PLAB scores (r = 0.58, p < 0.001). The correlations between the PLAB score and the scores in GM-ITE categories were as follows: symptomatology/clinical reasoning (r = 0.54, p < 0.001), physical examination/procedure (r = 0.38, p < 0.001), medical interview/professionalism (r = 0.25, p < 0.001), and disease knowledge (r = 0.36, p < 0.001). The mean discrimination index of each question of the GM-ITE (mean ± SD; 0.23 ± 0.15) was higher than that of the PLAB (0.16 ± 0.16; p = 0.004).

Conclusion

This study demonstrates incremental validity evidence of the GM-ITE to assess the clinical knowledge acquisition. The results indicate that GM-ITE can be widely used to improve resident education in Japan.

The impact of autopsy participation on clinical residency

Background

Autopsy has had an essential role in ensuring the quality of education and medical care. However, its role in clinical residency has not been clarified. This study assessed actual autopsy circumstances during clinical residency and evaluated the association between autopsy and clinical knowledge.

Methods

We conducted a cross‐sectional study involving postgraduate second year residents in Japan who took the General Medicine In‐Training Examination in 2019. We modeled the General Medicine In‐Training Examination scores of the residents to examine their association with autopsy experiences and the number of autopsy experiences to assess its predictors.

Results

Of 2715 postgraduate second year residents, 353 (13.8%) had no autopsy participation, and 1015 (39.7%) had only one experience. Although autopsy participation was not related to the mean General Medicine In‐Training Examination score, the residents' clinicopathological conference participation, self‐study for more than 60 min per day, and wish to be pathologists were significantly associated with autopsy experiences. They experienced more autopsies when they belonged to small‐sized hospitals in rural areas performing many autopsies.

Conclusion

We reported the current status of autopsy in clinical residency and showed that more than half of the residents experienced no or only one autopsy. General Medicine In‐Training Examination scores were not correlated with the number of autopsy experiences.

Difference in the general medicine in-training examination score between community-based hospitals and university hospitals: a cross-sectional study based on 15,188 Japanese resident physicians

BACKGROUND

The general medicine in-training examination (GM-ITE) is designed to objectively evaluate the postgraduate clinical competencies (PGY) 1 and 2 residents in Japan. Although the total GM-ITE scores tended to be lower in PGY-1 and PGY-2 residents in university hospitals than those in community-based hospitals, the most divergent areas of essential clinical competencies have not yet been revealed.

METHODS

We conducted a nationwide, multicenter, cross-sectional study in Japan, using the GM-ITE to compare university and community-based hospitals in the four areas of basic clinical knowledge". Specifically, "medical interview and professionalism," "symptomatology and clinical reasoning," "physical examination and clinical procedures," and "disease knowledge" were assessed.

RESULTS

We found no significant difference in "medical interview and professionalism" scores between the community-based and university hospital residents. However, significant differences were found in the remaining three areas. A 1.28-point difference (95% confidence interval: 0.96-1.59) in "physical examination and clinical procedures" in PGY-1 residents was found; this area alone accounts for approximately half of the difference in total score.

CONCLUSIONS

The standardization of junior residency programs and the general clinical education programs in Japan should be promoted and will improve the overall training that our residents receive. This is especially needed in categories where university hospitals have low scores, such as "physical examination and clinical procedures."

Impact of general medicine rotation training on the in-training examination scores of 11, 244 Japanese resident physicians: a Nationwide multi-center cross-sectional study

BACKGROUND

Although general medicine (GM) faculty in Japanese medical schools have an important role in educating medical students, the importance of residents' rotation training in GM in postgraduate education has not been sufficiently recognized in Japan. To evaluate the relationship between the rotation of resident physicians in the GM department and their In-Training Examination score.

METHODS

This study is a nationwide multi-center cross-sectional study in Japan. Participants of this study are Japanese junior resident physicians [postgraduate year (PGY)-1 and PGY-2] who took the General Medicine In-Training Examination (GM-ITE) in fiscal years 2016 to 2018 at least once (n = 11,244). The numbers of participating hospitals in the GM-ITE were 381, 459, and 503 in 2016, 2017, and 2018.The GM-ITE score consisted of four categories (medical interview/professionalism, symptomatology/clinical reasoning, physical examination/procedure, and disease knowledge). We evaluated relationship between educational environment (including hospital information) and the GM-ITE score.

RESULTS

A total of 4464 (39.7%) residents experienced GM department rotation training. Residents who rotated had higher total scores than residents who did not rotate (38.1 ± 12.1, 36.8 ± 11.7, and 36.5 ± 11.5 for residents who experienced GM rotation training, those who did not experience this training in hospitals with a GM department, and those who did not experience GM rotation training in hospitals without a GM department, p = 0.0038). The association between GM rotation and competency remained after multivariable adjustment in the multilevel model: the score difference between GM rotation training residents and non-GM rotation residents in hospitals without a GM department was estimated as 1.18 (standard error, 0.30, p = 0.0001), which was approximately half of the standard deviation of random effects due to hospital variation (estimated as 2.00).

CONCLUSIONS

GM rotation training improved the GM-ITE score of residents and should be considered mandatory for junior residents in Japan.

Awareness of Diagnostic Error among Japanese Residents: a Nationwide Study

BACKGROUND

Residents' understanding of diagnostic error may differ between countries. We sought to explore the relationship between diagnostic error knowledge and self-study, clinical knowledge, and experience.

METHODS

Our nationwide study involved postgraduate year 1 and 2 (PGY-1 and -2) Japanese residents. The Diagnostic Error Knowledge Assessment Test (D-KAT) and General Medicine In-Training Examination (GM-ITE) were administered at the end of the 2014 academic year. D-KAT scores were compared with the benchmark scores of US residents. Associations between D-KAT score and gender, PGY, emergency department (ED) rotations per month, mean number of inpatients handled at any given time, and mean daily minutes of self-study were also analyzed, both with and without adjusting for GM-ITE scores. Student's t test was used for comparisons with linear mixed models and structural equation models (SEM) to explore associations with D-KAT or GM-ITE scores.

RESULTS

The mean D-KAT score among Japanese PGY-2 residents was significantly lower than that of their US PGY-2 counterparts (6.2 vs. 8.3, p < 0.001). GM-ITE scores correlated with ED rotations (≥6 rotations: 2.14; 0.16-4.13; p = 0.03), inpatient caseloads (5-9 patients: 1.79; 0.82-2.76; p < 0.001), and average daily minutes of self-study (≥91 min: 2.05; 0.56-3.53; p = 0.01). SEM revealed that D-KAT scores were directly associated with GM-ITE scores (ß = 0.37, 95% CI: 0.34-0.41) and indirectly associated with ED rotations (ß = 0.06, 95% CI: 0.02-0.10), inpatient caseload (ß = 0.04, 95% CI: 0.003-0.08), and average daily minutes of study (ß = 0.13, 95% CI: 0.09-0.17).

CONCLUSIONS

Knowledge regarding diagnostic error among Japanese residents was poor compared with that among US residents. D-KAT scores correlated strongly with GM-ITE scores, and the latter scores were positively associated with a greater number of ED rotations, larger caseload (though only up to 15 patients), and more time spent studying.

Effectiveness of a clinical knowledge support system for reducing diagnostic errors in outpatient care in Japan: A retrospective study

Clinical evidence has indicated the effectiveness of computer-based systems for preventing and reducing diagnostic errors. Our study aimed to evaluate the effectiveness of UpToDate, a computer-based clinical knowledge management system, for reducing diagnostic errors. We retrospectively identified 100 patients who visited an outpatient department in a community-based hospital from July 2014 to June 2015. Fifty patients (exposure group) were seen by UpToDate-equipped physicians and another 50 (control group) were seen by UpToDate-unequipped physicians. We extracted data on patient sex, age, primary diagnosis, and case difficulty that could potentially affect diagnostic outcomes. We compared the two groups regarding diagnostic error rate and performed logistic regression analysis to analyze the concurrent effects of various factors affecting diagnostic error. The diagnostic error rate was 2% in the exposure group, while the error rate was 24% in the control group. Multivariate logistic regression analysis showed that error rate reduction was significantly associated with exposure to UpToDate with an odds ratio of 15.21 (95% CI 1.86-124.36). Our results demonstrated the effectiveness of UpToDate for the prevention and reduction of diagnostic error.