Building large-scale registries from unstructured clinical notes using a low-resource natural language processing pipeline

Building clinical registries is an important step in clinical research and improvement of patient care quality. Natural Language Processing (NLP) methods have shown promising results in extracting valuable information from unstructured clinical notes. However, the structure and nature of clinical notes are very different from regular text that state-of-the-art NLP models are trained and tested on, and they have their own set of challenges. In this study, we propose Sentence Extractor with Keywords (SE-K), an efficient and interpretable classification approach for extracting information from clinical notes and show that it outperforms more computationally expensive methods in text classification. Following the Institutional Review Board (IRB) approval, we used SE-K and two embedding based NLP approaches (Sentence Extractor with Embeddings (SE-E) and Bidirectional Encoder Representations from Transformers (BERT)) to develop comprehensive registry of anterior cruciate ligament surgeries from 20 years of unstructured clinical data at a multi-site tertiary-care regional children's hospital. The low-resource approach (SE-K) had better performance (average AUROC of 0.94 ± 0.04) than the embedding-based approaches (SE-E: 0.93 ± 0.04 and BERT: 0.87 ± 0.09) for out of sample validation, in addition to minimum performance drop between test and out-of-sample validation. Moreover, the SE-K approach was at least six times faster (on CPU) than SE-E (on CPU) and BERT (on GPU) and provides interpretability. Our proposed approach, SE-K, can be effectively used to extract relevant variables from clinic notes to build large-scale registries, with consistently better performance compared to the more resource-intensive approaches (e.g., BERT). Such approaches can facilitate information extraction from unstructured notes for registry building, quality improvement and adverse event monitoring.

Telehealth Potential in Pediatric Orthopaedics and Sports Medicine Care is Comparable to In-Person Care But Disparities Remain

BACKGROUND

Understanding the challenges and potential of telehealth visits (THVs) in a large population can inform future practice and policy discussion for pediatric orthopaedic and sports medicine (OSM) care. We comprehensively assess telehealth challenges and potential in a large pediatric OSM population based on access, visit completion, patient satisfaction, and technological challenges.

METHODS

Demographics, address, insurance, visit information, patient feedback, experience with video visits, and technical challenges of all 2019 to 2020 visits at our hospital were assessed (3,278,006 visits). We evaluated the differences in rate of telehealth utilization, rate of patient adherence, disparities in care access and patient satisfaction, and technological issues.

RESULTS

Compared with in-person prepandemic visits, THVs had lower ratios of non-White patients (by 5.8%; P<0.001), Hispanic patients (by 2.8%; P<0.001) and patients with public insurance (by 1.8%; P<0.001), and a higher mean distance between the patient's residence and clinic (by 18.8 miles; P<0.001). There were minimal differences in median household income (average $2297 less in THV; P<0.001) and social vulnerability index (average 0.01 points lower in THV; P<0.001) between groups. THVs had comparable patient satisfaction to in-person visits. Non-White patients, Hispanics, and those with public insurance had lower ratings for both in-person visits and THVs and had more technical difficulties during their THV.

CONCLUSIONS

Telehealth is a viable method of care for a range of pediatric OSM conditions, providing a similar quality of care as in-person visits with a greater geographic reach. However, in its current format, reduced disparities were not observed in pediatric OSM THVs.

LEVEL OF EVIDENCE

Level III.

Systematic evaluation of common natural language processing techniques to codify clinical notes

Proper codification of medical diagnoses and procedures is essential for optimized health care management, quality improvement, research, and reimbursement tasks within large healthcare systems. Assignment of diagnostic or procedure codes is a tedious manual process, often prone to human error. Natural Language Processing (NLP) has been suggested to facilitate this manual codification process. Yet, little is known on best practices to utilize NLP for such applications. With Large Language Models (LLMs) becoming more ubiquitous in daily life, it is critical to remember, not every task requires that level of resource and effort. Here we comprehensively assessed the performance of common NLP techniques to predict current procedural terminology (CPT) from operative notes. CPT codes are commonly used to track surgical procedures and interventions and are the primary means for reimbursement. Our analysis of 100 most common musculoskeletal CPT codes suggest that traditional approaches can outperform more resource intensive approaches like BERT significantly (P-value = 4.4e-17) with average AUROC of 0.96 and accuracy of 0.97, in addition to providing interpretability which can be very helpful and even crucial in the clinical domain. We also proposed a complexity measure to quantify the complexity of a classification task and how this measure could influence the effect of dataset size on model's performance. Finally, we provide preliminary evidence that NLP can help minimize the codification error, including mislabeling due to human error.

Prevalence and Predictors of Concomitant Meniscal Surgery During Pediatric and Adolescent ACL Reconstruction: Analysis of 4729 Patients Over 20 Years at a Tertiary-Care Regional Children's Hospital

Background

The rate of concomitant meniscal procedures performed in conjunction with anterior cruciate ligament (ACL) reconstruction is increasing. Few studies have examined these procedures in high-risk pediatric cohorts.

Hypotheses

That (1) the rates of meniscal repair compared with meniscectomy would increase throughout the study period and (2) patient-related factors would be able to predict the type of meniscal operation, which would differ according to age.

Study Design

Cohort study (prevalence); Level of evidence, 2.

Methods

Natural language processing was used to extract clinical variables from notes of patients who underwent ACL reconstruction between 2000 and 2020 at a single institution. Patients were stratified to pediatric (5-13 years) and adolescent (14-19 years) cohorts. Linear regression was used to evaluate changes in the prevalence of concomitant meniscal surgery during the study period. Logistic regression was used to determine predictors of the need for and type of meniscal procedure.

Results

Of 4729 patients (mean age, 16 ± 2 years; 54.7% female) identified, 2458 patients (52%) underwent concomitant meniscal procedures (55% repair rate). The prevalence of lateral meniscal (LM) procedures increased in both pediatric and adolescent cohorts, whereas the prevalence of medial meniscal (MM) repair increased in the adolescent cohort (P = .02). In the adolescent cohort, older age was predictive of concomitant medial meniscectomy (P = .031). In the pediatric cohort, female sex was predictive of concomitant MM surgery and of undergoing lateral meniscectomy versus repair (P≤ .029). Female sex was associated with decreased odds of concomitant LM surgery in both cohorts (P≤ .018). Revision ACLR was predictive of concomitant MM surgery and of meniscectomy (medial and lateral) in the adolescent cohort (P < .001). Higher body mass index was associated with increased odds of undergoing medial meniscectomy versus repair in the pediatric cohort (P = .03).

Conclusion

More than half of the young patients who underwent ACLR had meniscal pathology warranting surgical intervention. The prevalence of MM repair compared with meniscectomy in adolescents increased throughout the study period. Patients who underwent revision ACLR were more likely to undergo concomitant meniscal surgeries, which were more often meniscectomy. Female sex had mixed effects in both the pediatric and adolescent cohorts.

Prevalence and Predictors of Concomitant Meniscal and Ligamentous Injuries Associated With ACL Surgery: An Analysis of 20 Years of ACL Reconstruction at a Tertiary Care Children's Hospital

BACKGROUND

There is an increasing rate of procedures being performed for concomitant injuries during anterior cruciate ligament (ACL) surgery. Few studies have examined risk factors for these associated injuries in young patients.

HYPOTHESIS

There are patient-related factors predictive of concomitant knee pathology that differ between age-based cohorts.

STUDY DESIGN

Cross-sectional study; Level of evidence, 3.

METHODS

Natural language processing was used to extract clinical variables from available notes of patients undergoing ACL surgery between 2000 and 2020 at a single institution (5174 ACL surgeries; mean age, 17 ± 4 years; 53.1% female; accuracy, >98%). Patients were stratified to pediatric (5-13 years), adolescent (14-19 years), and young adult (20-35 years) cohorts. Logistic regression was used to determine predictors of concomitant injury to the menisci, medial collateral ligament (MCL), posterolateral corner (PLC), and posterior cruciate ligament (PCL).

RESULTS

Between 2000 and 2020, 54% of pediatric, 71% of adolescent, and 70% of adult patients had ≥1 concomitant soft tissue injury. In children and adolescents, increased age was consistently predictive of sustaining a concomitant injury (P < .02). Female children had increased odds of concomitant medial meniscal injury, while female adults had decreased odds (P≤ .046). Adolescent and adult female patients had decreased odds of concomitant lateral meniscal injury (P≤ .027). Female children had increased odds of injury to the MCL (P = .015), whereas female children and adolescents had decreased odds of PCL injury (P≤ .044). Adolescents undergoing revision ACL surgery had increased odds of meniscal injury (P≤ .001) and decreased odds of concomitant MCL injury (P = .028). Increased body mass index (BMI) was associated with increased odds of concomitant medial meniscal injury in all cohorts (P≤ .041), lateral meniscal injury in adults (P = .045), and PLC injury in children (P = .016). Contact injuries were associated with increased odds of MCL injury in adolescents (P = .017) and PLC injury in adolescents and adults (P < .014).

CONCLUSION

These findings support the hypothesis, as there were multiple factors that significantly affected the risk of concomitant injuries that differed between cohorts. Increased age, BMI, and contact injury history were generally associated with increased odds of sustaining a concomitant injury, whereas female sex and revision ACL surgery had mixed effects. Further studies are essential to investigate the sex-based differences in risk for concomitant injuries and to develop tailored treatment plans that minimize the risk of secondary ACL injury.

Disparities in cannabis use and documentation in electronic health records among children and young adults

The legalizations of medical and recreational cannabis have generated a great deal of interest in studying the health impacts of cannabis products. Despite increases in cannabis use, its documentation during clinical visits is not yet mainstream. This lack of information hampers efforts to study cannabis's effects on health outcomes. A clear and in-depth understanding of current trends in cannabis use documentation is necessary to develop proper guidelines to screen and document cannabis use. Here we have developed and used a natural language processing pipeline to evaluate the trends and disparities in cannabis documentation. The pipeline includes a screening step to identify clinical notes with cannabis use documentation which is then fed into a BERT-based classifier to confirm positive use. This pipeline is applied to more than 23 million notes from a large cohort of 370,087 patients seen in a high-volume multi-site pediatric and young adult clinic over a period of 21 years. Our findings show a very low but growing rate of cannabis use documentation (<2%) in electronic health records with significant demographic and socioeconomic disparities in both documentation and positive use, which requires further attention.

Cognitive Digital Biomarkers from Automated Transcription of Spoken Language

BACKGROUND

Although patients with Alzheimer's disease and other cognitive-related neurodegenerative disorders may benefit from early detection, development of a reliable diagnostic test has remained elusive. The penetration of digital voice-recording technologies and multiple cognitive processes deployed when constructing spoken responses might offer an opportunity to predict cognitive status.

OBJECTIVE

To determine whether cognitive status might be predicted from voice recordings of neuropsychological testing.

DESIGN

Comparison of acoustic and (para)linguistic variables from low-quality automated transcriptions of neuropsychological testing (n = 200) versus variables from high-quality manual transcriptions (n = 127). We trained a logistic regression classifier to predict cognitive status, which was tested against actual diagnoses.

SETTING

Observational cohort study.

PARTICIPANTS

146 participants in the Framingham Heart Study.

MEASUREMENTS

Acoustic and either paralinguistic variables (e.g., speaking time) from automated transcriptions or linguistic variables (e.g., phrase complexity) from manual transcriptions.

RESULTS

Models based on demographic features alone were not robust (area under the receiver-operator characteristic curve [AUROC] 0.60). Addition of clinical and standard acoustic features boosted the AUROC to 0.81. Additional inclusion of transcription-related features yielded an AUROC of 0.90.

CONCLUSIONS

The use of voice-based digital biomarkers derived from automated processing methods, combined with standard patient screening, might constitute a scalable way to enable early detection of dementia.