Disease-modifying therapy use and health resource utilisation associated with multiple sclerosis over time: A retrospective cohort study from Alberta, Canada

BACKGROUND

Estimating multiple sclerosis (MS) prevalence and incidence, and assessing the utilisation of disease-modifying therapies (DMTs) and healthcare resources over time is critical to understanding the evolution of disease burden and impacts of therapies upon the healthcare system.

METHODS

A retrospective population-based study was used to determine MS prevalence and incidence (2003-2019), and describe utilisation of DMTs (2009-2019) and healthcare resources (1998-2019) among people living with MS (pwMS) using administrative data in Alberta.

RESULTS

Prevalence increased from 259 (95% confidence interval [CI]: 253-265) to 310 (95% CI: 304, 315) cases per 100,000 population, and incidence decreased from 21.2 (95% CI: 19.6-22.8) to 12.7 (95% CI: 11.7-13.8) cases per 100,000 population. The proportion of pwMS who received ≥1 DMT dispensation increased (24% to 31% annually); use of older platform injection therapies decreased, and newer oral-based, induction, and highly-effective therapies increased. The proportion of pwMS who had at least one MS-related physician, ambulatory, or tertiary clinic visits increased, and emergency department visits and hospitalizations decreased.

CONCLUSIONS

Alberta has one of the highest rates of MS globally. The proportion of pwMS who received DMTs and had outpatient visits increased, while acute care visits decreased over time. The landscape of MS care appears to be rapidly evolving in response to changes in disease burden and new highly-effective therapies.

Strategies for working across Canadian practice-based research and learning networks (PBRLNs) in primary care: focus on frailty

Background

Practice based research and learning networks (PBRLNs) are groups of learning communities that focus on improving delivery and quality of care. Accurate data from primary care electronic medical records (EMRs) is crucial in forming the backbone for PBRLNs. The purpose of this work is to: (1) report on descriptive findings from recent frailty work, (2) describe strategies for working across PBRLNs in primary care, and (3) provide lessons learned for engaging PBRLNs.

Methods

We carried out a participatory based descriptive study that engaged five different PBRLNs. We collected Clinical Frailty Scale scores from a sample of participating physicians within each PBRLN. Descriptive statistics were used to analyze frailty scores and patients’ associated risk factors and demographics. We used the Consolidated Framework for Implementation Research to inform thematic analysis of qualitative data (meeting minutes, notes, and conversations with co-investigators of each network) in recognizing challenges of working across networks.

Results

One hundred nine physicians participated in collecting CFS scores across the five provinces (n = 5466). Percentages of frail (11-17%) and not frail (82-91%) patients were similar in all networks, except Ontario who had a higher percentage of frail patients (25%). The majority of frail patients were female (65%) and had a significantly higher prevalence of hypertension, dementia, and depression. Frail patients had more prescribed medications and numbers of healthcare encounters. There were several noteworthy challenges experienced throughout the research process related to differences across provinces in the areas of: numbers of stakeholders/staff involved and thus levels of burden, recruitment strategies, data collection strategies, enhancing engagement, and timelines.

Discussion

Lessons learned throughout this multi-jurisdictional work included: the need for continuity in ethics, regular team meetings, enhancing levels of engagement with stakeholders, the need for structural support and recognizing differences in data sharing across provinces.

Conclusion

The differences noted across CPCSSN networks in our frailty study highlight the challenges of multi-jurisdictional work across provinces and the need for consistent and collaborative healthcare planning efforts.

Machine learning for identification of frailty in Canadian primary care practices

Introduction

Frailty is a medical syndrome, commonly affecting people aged 65 years and over and is characterized by a greater risk of adverse outcomes following illness or injury. Electronic medical records contain a large amount of longitudinal data that can be used for primary care research. Machine learning can fully utilize this wide breadth of data for the detection of diseases and syndromes. The creation of a frailty case definition using machine learning may facilitate early intervention, inform advanced screening tests, and allow for surveillance.

Objectives

The objective of this study was to develop a validated case definition of frailty for the primary care context, using machine learning.

Methods

Physicians participating in the Canadian Primary Care Sentinel Surveillance Network across Canada were asked to retrospectively identify the level of frailty present in a sample of their own patients (total n = 5,466), collected from 2015–2019. Frailty levels were dichotomized using a cut-off of 5. Extracted features included previously prescribed medications, billing codes, and other routinely collected primary care data. We used eight supervised machine learning algorithms, with performance assessed using a hold-out test set. A balanced training dataset was also created by oversampling. Sensitivity analyses considered two alternative dichotomization cut-offs. Model performance was evaluated using area under the receiver-operating characteristic curve, F1, accuracy, sensitivity, specificity, negative predictive value and positive predictive value.

Results

The prevalence of frailty within our sample was 18.4%. Of the eight models developed to identify frail patients, an XGBoost model achieved the highest sensitivity (78.14%) and specificity (74.41%). The balanced training dataset did not improve classification performance. Sensitivity analyses did not show improved performance for cut-offs other than 5.

Conclusion

Supervised machine learning was able to create well performing classification models for frailty. Future research is needed to assess frailty inter-rater reliability, and link multiple data sources for frailty identification.

Developing and Validating a Primary Care EMR-based Frailty Definition using Machine Learning

Introduction

Individuals who have been identified as frail have an increased state of vulnerability, often leading to adverse health events, increased health spending, and potentially detrimental outcomes.

Objective

The objective of this work is to develop and validate a case definition for frailty that can be used in a primary care electronic medical record database.

Methods

This is a cross-sectional validation study using data from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) in Southern Alberta. 52 CPCSSN sentinels assessed a random sample of their own patients using the Rockwood Clinical Frailty scale, resulting in a total of 875 patients to be used as reference standard. Patients must be over the age of 65 and have had a clinic visit within the last 24 months. The case definition for frailty was developed using machine learning methods using CPCSSN records for the 875 patients.

Results

Of the 875 patients, 155 (17.7%) were frail and 720 (84.2%) were not frail. Validation metrics of the case definition were: sensitivity and specificity of 0.28, 95% CI (0.21 to 0.36) and 0.94, 95% CI (0.93 to 0.96), respectively; PPV and NPV of 0.53, 95% CI (0.42 to 0.64) and 0.86, 95% CI (0.83 to 0.88), respectively.

Conclusions

The low sensitivity and specificity results could be because frailty as a construct remains under-developed and relatively poorly understood due to its complex nature. These results contribute to the literature by demonstrating that case definitions for frailty require expert consensus and potentially more sophisticated algorithms to be successful.

Developing a Primary Care EMR-based Frailty Definition using Machine Learning

IntroductionFrailty is a geriatric syndrome that is predictive of heightened vulnerability for disability, hospitalization, and mortality. Annually an estimated 250,000 frail Canadians die, and this estimate is expected to double in the next 40 years, as Canadians grow older. Currently there is no single accepted clinical definition of frailty. Objectives and ApproachThe objective of this study was to develop an operational definition of frailty using machine learning that can be applied to a primary care electronic medical record (EMR) database. The Canadian Primary Care Sentinel Surveillance Network (CPCSSN) is a pan-Canadian network of primary care practices that collect de-identified patient information (such as encounter diagnoses, health conditions, and laboratory data) from EMRs. 780 patients from CPCSSN have were randomly selected and assessed by physicians using the Rockwood Clinical Frailty Scale (as frail or not frail), and their clinical characteristics from CPCSSN used to develop the definition using machine-learning. ResultsA total of 8,044 clinical features were extracted from these tables: billing, problem list, encounter diagnosis, labs, medications and referrals. A chi-squared automatic interaction detector (CHAID) approach was selected as the best approach. The bootstrapping process used a cost matrix that prioritized high sensitivity and positive predictive value. 10-fold cross validation was used for validity measures. Key features factored into the algorithm included: diagnosis of dementia (ICD-9 code 290), medications furosemide and vitamins, and use of key word “obstruction” within the billing table. The validation measures with 95% confidence intervals are as follows: sensitivity of 28% (95% CI: 21% to 36%), specificity of 94% (95% CI: 93% to 96%), positive predictive value of 53% (95% CI: 42% to 64%), negative predictive value of 86% (95% CI: 83% to 88%). Conclusion/ImplicationsNo other primary care specific frailty screening tools have sufficient validity. These results suggest heterogeneous diseases require clearly defined features and potentially more sophisticated algorithms to account for heterogeneity. Further research utilizing continuous features and continuous frailty scores may be more suitable in the creation of a case detection algorithm.