Using Serial Trichotomization With Common Cognitive Tests to Screen for Fitness to Drive

Serial Trichotomization to Determine Fitness to Drive: Results From a Cohort of Clients Referred to a Neurology Program

IMPORTANCE

Determining cognitive fitness to drive is challenging. A previous study used serial trichotomization with five cognitive tests to determine whether drivers should continue driving, undergo further evaluation, or stop driving.

OBJECTIVE

To examine agreement between serial trichotomization and fitness-to-drive determinations made by occupational therapists.

DESIGN

Drivers referred for cognitive screens completed all tests used in the previous study. Occupational therapists provided fitness-to-drive recommendations (safe, indeterminate, or unsafe) using all clinical information available. We examined the agreement between the tests' results (using cut points from the previous study) and occupational therapists' recommendations.

SETTING

Outpatient neurology program at a chronic care and rehabilitation hospital.

PARTICIPANTS

279 clients (M age  = 66.35 yr; SD = 13.25).

OUTCOMES AND MEASURES

Tests included the Trail Making Tests A and B, the Clock Drawing Test (CDT), the Montreal Cognitive Assessment, and the Motor-Free Visual Perception Test, using a road test as the gold standard. The previous study used dual cut points with 100% sensitivity and specificity to reduce false positives and false negatives.

RESULTS

Weighted κs ranged from .03 (95% confidence interval [CI] [-.01, .08]) for the CDT to .54 (95% CI [.46, .62]) for the Trail Making Test, Part B. Although the agreement between serial trichotomization and the final recommendations was moderate (κ = .59; 95% CI [.50, .67]), serial trichotomization appeared useful for identifying unsafe drivers.

CONCLUSIONS AND RELEVANCE

These results remind us of the variability inherent in stand-alone cognitive tests, even within a serial trichotomization framework, and the importance of clinical judgement and road tests in decision making about driving. Plain-Language Summary: It can be challenging for occupational therapists to accurately determine a client's cognitive fitness to drive. Many occupational therapists lack the time, have limited training, or do not have access to comprehensive driving evaluation tools. A serial testing approach can support occupational therapists in assessing a client's cognitive fitness to drive. This study used an approach based on a series of five cognitive tests to determine whether a client should continue driving, undergo further evaluation, or stop driving. The series of tests were used to classify drivers as safe, indeterminate, or unsafe. In principle, a driver would take the second test only if the driver was classified as indeterminate on the basis of first test, and so on. By applying the tests in sequence, few drivers should remain classified as indeterminate at the end of the series of tests. This serial approach has the potential to streamline the decision-making process for occupational therapists by classifying the more extreme unsafe cases while still providing an accurate assessment of cognitive fitness to drive.

A Three-Tiered Comprehensive Driving Evaluation Integrating a Driving Simulator Test for Drivers with Borderline Cognitive Fitness-to-Drive: Proof of Concept

Comprehensive driving evaluations for older adults with cognitive impairment are time-consuming, expensive, and involve risk. To minimize these challenges, we evaluated a three-tiered driving evaluation process incorporating cognitive tests (Step 1), a driving simulator test (Step 2), and a road test (Step 3). Participants in this study were referred to a driving assessment center for concerns about cognitive fitness-to-drive. Each participant completed all three evaluation steps. Their fitness-to-drive was determined independently by an occupational therapist and an experienced driving evaluator with a driver instructor background. Our main objective was to examine the agreement between the occupational therapist's determination of fitness-to-drive after each step and the driving evaluator's determination of fitness-to-drive after the road test. As a secondary objective, the occupational therapist's confidence in their determinations was also examined. Results showed agreement for 38.8% of participants after Step 1, 46.5% after Step 2, and 92.3% after Step 3. The mean occupational therapist's confidence rating in their determination (scale of 0 to 100; higher is better) was 36.15 after Step 1, 49.54 after Step 2, and 90.54 after Step 3. All drivers deemed to have passed the evaluation had been identified as such after the driving simulator test. These results suggest that the best agreement between the occupational therapist and the driving evaluator was reached after the final step. However, the results also indicate that for some participants, a road test may not be required following a driving simulator test. Eliminating the road test in some instances may create efficiencies and reduce cost and risk while maintaining accurate determinations of fitness-to-drive.

A Bayesian finite mixture model approach to evaluate dichotomization method for correlated ELISA tests

In diagnostic accuracy studies, a commonly employed approach involves dichotomizing continuous data and subsequently analyzing them using a Bayesian latent class model (BLCM), often relying on binomial or multinomial distributions, rather than preserving their continuous nature. However, this procedure can inadvertently lead to less reliable outcomes due to the inherent loss of information when converting the original continuous measurements into binary values. Through comprehensive simulations, we demonstrated the limitations and disadvantages of dichotomizing continuous biomarkers from two correlated tests. Our findings highlighted notable disparities between the true values and the model estimates as a result of dichotomization. We discovered the crucial significance of selecting a reference test with high diagnostic accuracy in test evaluation in order to obtain reliable estimates of test accuracy and prevalences. Our study served as a call to action for veterinary researchers to exercise caution when utilizing dichotomization.

Clinical Tests Predicting On-Road Performance in Older Drivers with Cognitive Impairment

Background. The Trail Making Test Part B (Trails B) and Useful Field of View® (UFOV) can predict on-road outcomes in drivers with cognitive impairment (CI); however, studies have not included drivers referred for comprehensive driving evaluations (CDEs), who typically have more severe CI. Purpose. We determined the predictive ability of Trails B and UFOV on pass/fail on-road outcomes in drivers with CI (Montreal Cognitive Assessment <26) referred for CDEs. Method. Retrospective data collection from two driving assessments centers (N  =  100, mean age  =  76.2  ±  8.8 years). Findings. The Trails B (area under the curve [AUC]  =  .70) and UFOV subtests 2 (AUC  =  .73) and 3 (AUC  =  .76) predicted pass/fail outcomes. A cut-point ≥467 ms on UFOV subtest 3 better-predicted pass/fail outcomes with 78.9% sensitivity and 73.5% specificity. In comparison, a cut-point ≥3.58 min on Trails B had lower sensitivity (73.7%) and specificity (61.8%). Implications. The UFOV subtest 3 may be more useful than the Trails B for predicting pass/fail outcomes in drivers with more severe CI referred for CDEs.

Further psychometric evaluation of the Structured Multidisciplinary Work Evaluation Tool (SMET) questionnaire: Practical implications in healthcare settings

BACKGROUND

Scientific research has identified a lack of psychometrically well-tested methods for evaluation of the work environment in healthcare settings. The Structured Multidisciplinary Work Evaluation Tool (SMET) questionnaire has been evaluated and has shown good content validity, as well as intra-rater and test-retest reliability. There are, however, still unknowns regarding the psychometric properties. If the SMET questionnaire is to be used in practical occupational health service (OHS) work and scientific research in healthcare settings, further psychometric evaluation is needed.

OBJECTIVE

The aim of this study was to gain further understanding of the psychometric properties of the SMET questionnaire when used in research and clinical OHS practice in healthcare settings.

METHODS

The psychometric evaluation was conducted using classical test theory (Cronbach's alpha, explorative factor analysis) and Rasch analysis (measurement targeting, category threshold order, person separation index) on data previously collected in development projects within the healthcare sector.

RESULTS

The results support the use of the SMET questionnaire as a psychometrically well-tested method for evaluation of the work environment in healthcare settings. They support the use of the initial 1-10 scale since all 10 steps are used. The results also support the trichotomization procedure since the trichotomized scale captures the construct of the work environment with good measurement targeting and good category threshold order.

CONCLUSION

The results of this study support the use of the SMET questionnaire as a psychometrically well-tested method for a broad multifactorial evaluation of the work environment in healthcare settings.

Establishing the Predictive Validity of the ScanCourse for Assessing On-Road Driving Performance

IMPORTANCE

Scanning the environment is critical for driving safety. The ScanCourse is a functional assessment that assesses a person's ability to scan the environment for visual information while in motion. Measurement properties for the ScanCourse have been reported; however, its predictive validity is unknown.

OBJECTIVE

To determine the predictive validity of the ScanCourse for on-road driving performance and establish clinical cutoff scores.

DESIGN

Retrospective chart reviews were conducted over a 6-mo period.

SETTING

Four Canadian driver rehabilitation programs.

PARTICIPANTS

Charts from patients with neurological or vision conditions were eligible if they contained ScanCourse and on-road driving evaluation results between September 1, 2008, and August 30, 2018. Three hundred twenty-five charts were included for analysis.

OUTCOMES AND MEASURES

Area under the curve (AUC) analysis was used to determine the predictive validity of ScanCourse scores for on-road outcomes; cutoff scores were established by optimizing sensitivity and specificity.

RESULTS

The ScanCourse had an AUC of .702. The optimal cutoff score was 18/20 with a sensitivity of 76.7% and a specificity of 47.1%.

CONCLUSIONS AND RELEVANCE

Assessing the scanning abilities of at-risk drivers who intend to return to driving after sustaining an injury can help identify safety risks and inform interventions. The ScanCourse was found to have acceptable discriminatory ability for on-road driving performance. This study provides evidence supporting its continued use as a screening tool to assess driver fitness with an identified optimal cutoff score for clinical use.

WHAT THIS ARTICLE ADDS

Measuring the predictive ability of the ScanCourse assessment in relation to on-road driving performance provides occupational therapists with an evidence-based clinical tool to assist with screening fitness to drive among at-risk people.

Using Serial Trichotomization with Neuropsychological Measures to Inform Clinical Decisions on Fitness-to-Drive among Older Adults with Cognitive Impairment

Decisions related to driving safety and when to cease driving are complex and costly. There is an interest in developing an off-road driving test utilizing neuropsychological tests that could help assess fitness-to-drive. Serial trichotomization has demonstrated potential as it yields 100% sensitivity and specificity in retrospective test samples. The purpose of this study was to test serial trichotomization using four common neuropsychological tests (Trail Making Test Part A and B, Clock Drawing Test, and Modified Mini-Mental State Examination). Test scores from 105 patients who were seen in a memory clinic were abstracted. After applying the model, participants were classified as unfit, fit, or requiring further testing, 38.1%, 25.8%, and 36.1%, respectively. This study provides further evidence that trichotomization can facilitate the assessment of fitness-to-drive.

Evaluating the Measurement Properties of the ScanCourse, a Dual-Task Assessment of Visual Scanning

IMPORTANCE

The ScanCourse is used by occupational therapists to evaluate visual scanning ability during locomotion. Its measurement properties have not been examined.

OBJECTIVE

To assess the interrater reliability, test-retest reliability, and construct validity of the ScanCourse.

DESIGN

This study involved data collection at two time points. To assess test-retest reliability, the ScanCourse was administered twice within a 2-week period. To assess interrater reliability, a second rater was present for one session. To assess level of agreement, a Bland-Altman plot was created. To assess absolute reliability, the standard error of measurement was calculated. To evaluate construct validity, the results of the ScanCourse were compared with results of the Bells Test and Trail Making Test A and B.

SETTING

Rehabilitation hospital.

PARTICIPANTS

Forty-one patients with neurological impairments.

OUTCOMES AND MEASURES

The ScanCourse (participants identify numbered cards placed on both sides of a hallway at various heights during locomotion).

RESULTS

The ScanCourse was found to have excellent interrater reliability (intraclass correlation coefficient [ICC] [1,1] = .998; 95% confidence interval [CI] [.996-.999]), test-retest reliability (ICC [1,1] = .912; 95% CI [.811-.959]), a high level of agreement, and a low standard error of measurement (.503), and it was found to be significantly correlated with Trails A (r_s = -.436, p = .009) and B (r_s = -.364, p = .029).

CONCLUSIONS AND RELEVANCE

The assessment was found to have strong measurement properties, and it is therefore an appropriate tool for assessing dual-task visual scanning among those with neurological impairments.

WHAT THIS ARTICLE ADDS

This research demonstrates that the ScanCourse is reliable between raters and over time and that scores on the measure vary as anticipated with scores on a related measure, which provides evidence of its validity. These findings support its use in practice.

Evaluating the Measurement Properties of the ScanCourse, a Dual-Task Assessment of Visual Scanning

IMPORTANCE

The ScanCourse is used by occupational therapists to evaluate visual scanning ability during locomotion. Its measurement properties have not been examined.

OBJECTIVE

To assess the interrater reliability, test-retest reliability, and construct validity of the ScanCourse.

DESIGN

This study involved data collection at two time points. To assess test-retest reliability, the ScanCourse was administered twice within a 2-week period. To assess interrater reliability, a second rater was present for one session. To assess level of agreement, a Bland-Altman plot was created. To assess absolute reliability, the standard error of measurement was calculated. To evaluate construct validity, the results of the ScanCourse were compared with results of the Bells Test and Trail Making Test A and B.

SETTING

Rehabilitation hospital.

PARTICIPANTS

Forty-one patients with neurological impairments. Outcomes and Measures: The ScanCourse (participants identify numbered cards placed on both sides of a hallway at various heights during locomotion).

RESULTS

The ScanCourse was found to have excellent interrater reliability (intraclass correlation coefficient [ICC] [1,1] = .998; 95% confidence interval [CI] [.996-.999]), test-retest reliability (ICC [1,1] = .912; 95% CI [.811-.959]), a high level of agreement, and a low standard error of measurement (.503), and it was found to be significantly correlated with Trails A (rs = -.436, p = .009) and B (rs = -.364, p = .029).

CONCLUSIONS AND RELEVANCE

The assessment was found to have strong measurement properties, and it is therefore an appropriate tool for assessing dual-task visual scanning among those with neurological impairments. What This Article Adds: This research demonstrates that the ScanCourse is reliable between raters and over time and that scores on the measure vary as anticipated with scores on a related measure, which provides evidence of its validity. These findings support its use in practice.

State of the Journal, 2018

The American Journal of Occupational Therapy (AJOT) continues to be the most highly ranked occupational therapy journal as measured by its Journal Citation Reports impact factor (its 5-yr impact factor is now 3.325). AJOT’s goal is to remain occupational therapy’s leading research journal with disciplinary and interdisciplinary impact. AJOT instituted measures starting in January 2018 to address increasing concerns about research reproducibility and study reporting methods. Starting in 2019, AJOT will be distributed online only and will offer an annual “Best of” print compendium to AOTA members.

Naturalistic Driving: A Framework and Advances in Using Big Data

Driving is an activity that facilitates physical, cognitive, and social stimulation in older adults, ultimately leading to better physical and cognitive health. However, aging is associated with declines in vision, physical health, and cognitive health, all of which can affect driving ability. One way of assessing driving ability is with the use of sensors in the older adult’s own vehicle. This paper provides a framework for driving assessment and addresses how naturalistic driving studies can assist in such assessments. The framework includes driving characteristics (how much driving, speed, position, type of road), actions and reactions (lane changes, intersections, passing, merging, traffic lights, pedestrians, other vehicles), destinations (variety and distance, sequencing and route planning), and driving conditions (time of day and season). Data from a subset of Ottawa drivers from the Candrive study is used to illustrate the use of naturalistic driving data. Challenges in using naturalistic driving big data and the changing technology in vehicles are discussed.

Transportation and Aging: An Updated Research Agenda for Advancing Safe Mobility

This article discusses what is currently known about three important topics related to older driver safety and mobility: screening and evaluation, education and training interventions, and in-vehicle technology. Progress is being made to improve the safe mobility of older adults in these key areas; however, significant research gaps remain. This article advances the state of knowledge by identifying these gaps, and proposing further research topics will improve the lives of older adults. In addition, we discuss several themes that emerged from the review, including the need for multidisciplinary, community-wide solutions; large-scale, longitudinal studies; improved education/training for both older adults themselves and the variety of stakeholders involved in older adult transportation; and programs and interventions that are flexible and responsive to individual needs and differences.