Can USMLE and COMLEX-USA Scores Predict At-Risk Emergency Medicine Residents' Performance on In-Training Examinations?

PURPOSE

The United States Medical Licensing Examination (USMLE) and Comprehensive Osteopathic Medical Licensing Examination (COMLEX) scores are standard methods used to determine residency candidates' medical knowledge. The authors were interested in using the USMLE and COMLEX part 2 scores in our emergency medicine (EM) residency program to identify at-risk residents who may have difficulty on the in-training exam (ITE) and to determine the cutoff values under which an intern could be given an individualized study plan to ensure medical knowledge competency.

METHODS

The authors abstracted the USMLE and COMLEX part 2 scores and the American Board of Emergency Medicine (ABEM) ITE scores for a cohort of first-year EM residents graduating years 2010-2022, converting raw scores to percentiles, and compared part 2 and ABEM ITE scores with Pearson's correlation, a Bland-Altman analysis of bias and 95% limits of agreement, and ROC analysis to determine optimal the cut-off values for predicting ABEM ITE < 50th percentile and the estimated test characteristics.

RESULTS

Scores were available for 152 residents, including 93 USMLE and 88 COMLEX exams. The correlations between part 2 scores and ABEM ITE were r = 0.36 (95%CI: 0.17, 0.52; p < 0.001) for USMLE and r = 0.50 (95%CI: 0.33, 0.64; p < 0.001) for COMLEX. Bias and limits of agreement for both part 2 scores were -14 ± 63% for USMLE and 13 ± 50% for COMLEX in predicting the ABEM ITE scores. USMLE < 37th percentile and COMLEX < 53rd percentile identified 42% (N = 39) and 27% (N = 24) of EM residents, respectively, as at risk, with a sensitivity of 61% and 49% and specificity of 71% and 92%, respectively.

CONCLUSION

USMLE and COMLEX part 2 scores have a very limited role in identifying those at risk of low ITE performance, suggesting that other factors should be considered to identify interns in need of medical knowledge remediation.

A Comparison Between Ultrasound Pachymetry and CASIA2 (Anterior-Segment Optical Coherence Tomography) in the Measurement of Central Corneal Thickness

Background and objective Due in part to its effect on intraocular pressure (IOP) measurements, the assessment of central corneal thickness (CCT) is recognized as an essential part of the initial glaucoma assessment. The most widely utilized clinical technique to measure CCT is ultrasound pachymetry (USP). In recent years, many dedicated anterior-segment optical coherence tomography scanners (AS-OCTs) have been developed. Previous studies have compared CCT measurements between USP and various AS-OCTs. This study aimed to assess the degree of agreement between USP and CASIA2 (Tomey Corporation, Nagoya, Japan), a second-generation swept-source AS-OCT developed in Japan. Methodology The data on CCT screening measurements of 156 eyes (88 patients) performed over a period of three months, from January to March 2020, on glaucoma patients attending the Royal Hallamshire Hospital (RHH) in Sheffield, UK were collected retrospectively and statistically analyzed. Results The average age of the 88 patients included in the study was 66 years (range: 20-86 years). Our findings show that when compared to CASIA2 measurements, USP measurement of the CCT resulted in significantly thicker values (paired t-test: t=23.15,p<2.2 x 10^-16). The average difference between the two methods was 19.98 ± 10.78 μm. It is hypothesized that this difference may be due in part to inaccurate probe placement during ultrasound probe measurement, resulting in thicker CCT values. Conclusion The observed difference may be clinically significant as it could induce clinical discrepancy in terms of perceived glaucoma risk in patients. Therefore, USP and CASIA2 should not be used interchangeably, and clinicians should take into account the significant difference between these methods.

Agreement between blood pressure from research study visits versus electronic medical records and associations with hypertensive disorder diagnoses in pregnant women with overweight/obesity

OBJECTIVES

Blood pressure (BP) abstracted from electronic medical records (EMR) is moderately correlated to BP in nonpregnant adults with limited agreement. Little is known about the agreement of research versus EMR BP measured during pregnancy or associations of EMR BP with hypertensive disorder of pregnancy (HDP) diagnoses.

METHODS

BP was measured according to guidelines at in-person research study visits in 214 women with prepregnancy overweight or obesity (44.4% African American, mean age = 29.8 ± 4.8 years) at weeks 16 and 32 of pregnancy. Clinic BP readings that occurred within 1 week of the study visits were abstracted from the EMR. We assessed correlations between sources using Pearson's coefficients; the agreement was evaluated with Bland-Altman plots. We compared differences in the proportion of women with an HDP diagnosis in the EMR between women with versus without a hypertensive EMR BP measurement.

RESULTS

SBP and DBP from study visits and the EMR were modestly moderately correlated at both time points; 0.20 < r < 0.44; P < 0.05 for all. The average mean difference was 10.5 mmHg for SBP and <1 mmHg for DBP in early and 7.3 mmHg for SBP and -1.7 mmHg for DBP in late pregnancy. Women with at least one hypertensive BP reading in the EMR were more likely to have an HDP diagnosis recorded in the EMR; 43.5 versus 3.3%; P < 0.01.

CONCLUSION

EMR SBP was higher but moderately correlated with research quality BP in early and late pregnancy. Women with a hypertensive EMR BP measurement were more likely to have an HDP diagnosis in the EMR.

Is Sitting Always Inactive and Standing Always Active? A Simultaneous Free-Living activPal and ActiGraph Analysis

Sedentary Behavior (SB), defined as sitting with minimal physical activity, is an emergent public health topic. However, the measurement of SB considers either posture (e.g., activPal) or physical activity (e.g., ActiGraph), and thus neglects either active sitting or inactive standing. The aim of this study was to determine the true amount of active sitting and inactive standing in daily life, and to analyze by how much these behaviors falsify the single sensors' sedentary estimates. Sedentary time of 100 office workers estimated with activPal and ActiGraph was therefore compared with Bland-Altman statistics to a combined sensor analysis, the posture and physical activity index (POPAI). POPAI classified each activPal sitting and standing event into inactive or active using the ActiGraph counts. Participants spent 45.0% [32.2%-59.1%] of the waking hours inactive sitting (equal to SB), 13.7% [7.8%-21.6%] active sitting, and 12.0% [5.7%-24.1%] inactive standing (mean [5th-95th percentile]). The activPal overestimated sedentary time by 30.3% [12.3%-48.4%] and the ActiGraph by 22.5% [3.2%-41.8%] (bias [95% limit-of-agreement]). The results showed that sitting is not always inactive, and standing is not always active. Caution should therefore be paid when interpreting the activPal (ignoring active sitting) and ActiGraph (ignoring inactive standing) measured time as SB.

Performance characteristics of the measurement of gastric volume using single photon emission computed tomography

BACKGROUND

A non-invasive single photon emission computed tomography (SPECT) method was developed and validated to measure gastric volumes (GV). The natural variation in gastric volume responses and performance characteristics of SPECT imaging are unclear.

METHODS

The primary aim of this study was to assess the performance characteristics of SPECT imaging by estimating the inter-individual coefficients of variation (COV) in fasting and postprandial GV in 433 participants in prior research studies over the last decade, and the intra-individual COV in all volunteers who had undergone at least two studies. The secondary aim was to assess the relationship of gender, BMI and age on GV.

KEY RESULTS

The COV(INTER) for all subjects in the study (n = 433) was 32.6% fasting, 16.0% fed, and 19.0%Δ fed - fasting. The COV(INTRA) for 47 subjects with repeat estimates of gastric volume was 37.0% fasting, 17.6% fed, and 22.0%Δ fed - fasting. COV(INTRA) was stable over time interval from 2 to 60 months. There were no significant differences by gender or subgroups. Mean fed and gastric accommodation volumes were associated with age and BMI but the magnitude of variation attributable was <5%.

CONCLUSIONS & INFERENCES

COV(INTRA) and COV(INTER) of GV by SPECT are very similar, and there is a small effect of age and BMI. These data are important for planning future studies of GV and further validate SPECT for studies of gastric motility disorders and obesity.

Invalidity of one actigraphy brand for identifying sleep and wake among infants

STUDY OBJECTIVES

Few commercially available brands of actigraphs (ACT) have been subjected to rigorous validation with infant participants. The purpose of this study was to examine the agreement between concurrent polysomnography (PSG) and one brand of ACT (AW-64, Mitter Co. Inc.) using appropriate statistical techniques among a sample of healthy infants.

METHODS

Twenty-two healthy infants (14.1+/-0.6 months) had one night of ankle ACT recording during research PSG at Kosair Children's Hospital Sleep Research Center in Louisville, Kentucky. Macroanalyses were conducted using the Bland-Altman concordance technique to assess agreement between total sleep time (TST) and wake after sleep onset (WASO) simultaneously measured by PSG and ACT, using two ACT algorithm settings. Microanalyses were also calculated to examine sensitivity, specificity, and accuracy of ACT within each PSG-identified sleep state. Correlations were calculated between PSG-identified arousals and the discrepancies between ACT and PSG.

RESULTS

The Bland-Altman concordance technique revealed that ACT underestimated TST by 72.25 (SD=61.48) minutes and by > or = 60 min among 54.55% of infants. Furthermore, ACT overestimated WASO by 13.85 (SD=30.94) minutes and by > or = 30 min among 40.91% of infants. Sensitivity, specificity, and accuracy analyses revealed that ACT adequately identified sleep, but poorly identified wake. PSG and ACT discrepancies were positively associated with PSG-identified arousals (r=.45).

CONCLUSIONS

Improved device and/or software development is needed before the AW-64 can be considered a valid method for identifying infant sleep and wake.