A tablet-based multi-dimensional drawing system can effectively distinguish patients with amnestic MCI from healthy individuals

The population with dementia is expected to rise to 152 million in 2050 due to the aging population worldwide. Therefore, it is significant to identify and intervene in the early stage of dementia. The Rey-Osterreth complex figure (ROCF) test is a visuospatial test scale. Its scoring methods are numerous, time-consuming, and inconsistent, which is unsuitable for wide application as required by the high number of people at risk. Therefore, there is an urgent need for a rapid, objective, and sensitive digital scoring method to detect cognitive dysfunction in the early stage accurately. This study aims to clarify the organizational strategy of aMCI patients to draw complex figures through a multi-dimensional digital evaluation system. At the same time, a rapid, objective, and sensitive digital scoring method is established to replace traditional scoring. The data of 64 subjects (38 aMCI patients and 26 NC individuals) were analyzed in this study. All subjects completed the tablet's Geriatric Complex Figure (GCF) test, including copying, 3-min recall, and 20-min delayed recall, and also underwent a standardized neuropsychological test battery and classic ROCF test. Digital GCF (dGCF) variables and conventional GCF (cGCF) scores were input into the forward stepwise logistic regression model to construct classification models. Finally, ROC curves were made to visualize the difference in the diagnostic value of dGCF variables vs. cGCF scores in categorizing the diagnostic groups. In 20-min delayed recall, aMCI patients' time in air and pause time were longer than NC individuals. Patients with aMCI had more short strokes and poorer ability of detail integration (all p < 0.05). The diagnostic sensitivity of dGCF variables for aMCI patients was 89.47%, slightly higher than cGCF scores (sensitivity: 84.21%). The diagnostic accuracy of both was comparable (dGCF: 70.3%; cGCF: 73.4%). Moreover, combining dGCF variables and cGCF scores could significantly improve the diagnostic accuracy and specificity (accuracy: 78.1%, specificity: 84.62%). At the same time, we construct the regression equations of the two models. Our study shows that dGCF equipment can quantitatively evaluate drawing performance, and its performance is comparable to the time-consuming cGCF score. The regression equation of the model we constructed can well identify patients with aMCI in clinical application. We believe this new technique can be a highly effective screening tool for patients with MCI.

Automating Rey Complex Figure Test scoring using a deep learning-based approach: a potential large-scale screening tool for cognitive decline

BACKGROUND

The Rey Complex Figure Test (RCFT) has been widely used to evaluate the neurocognitive functions in various clinical groups with a broad range of ages. However, despite its usefulness, the scoring method is as complex as the figure. Such a complicated scoring system can lead to the risk of reducing the extent of agreement among raters. Although several attempts have been made to use RCFT in clinical settings in a digitalized format, little attention has been given to develop direct automatic scoring that is comparable to experienced psychologists. Therefore, we aimed to develop an artificial intelligence (AI) scoring system for RCFT using a deep learning (DL) algorithm and confirmed its validity.

METHODS

A total of 6680 subjects were enrolled in the Gwangju Alzheimer's and Related Dementia cohort registry, Korea, from January 2015 to June 2021. We obtained 20,040 scanned images using three images per subject (copy, immediate recall, and delayed recall) and scores rated by 32 experienced psychologists. We trained the automated scoring system using the DenseNet architecture. To increase the model performance, we improved the quality of training data by re-examining some images with poor results (mean absolute error (MAE) [Formula: see text] 5 [points]) and re-trained our model. Finally, we conducted an external validation with 150 images scored by five experienced psychologists.

RESULTS

For fivefold cross-validation, our first model obtained MAE = 1.24 [points] and R-squared ([Formula: see text]) = 0.977. However, after evaluating and updating the model, the performance of the final model was improved (MAE = 0.95 [points], [Formula: see text] = 0.986). Predicted scores among cognitively normal, mild cognitive impairment, and dementia were significantly different. For the 150 independent test sets, the MAE and [Formula: see text] between AI and average scores by five human experts were 0.64 [points] and 0.994, respectively.

CONCLUSION

We concluded that there was no fundamental difference between the rating scores of experienced psychologists and those of our AI scoring system. We expect that our AI psychologist will be able to contribute to screen the early stages of Alzheimer's disease pathology in medical checkup centers or large-scale community-based research institutes in a faster and cost-effective way.

Comparison of RCF Scoring System to Clinical Decision for the Rey Complex Figure Using Machine-Learning Algorithm

BACKGROUND AND PURPOSE

Interpreting the Rey complex figure (RCF) requires a standard RCF scoring system and clinical decision by clinicians. The interpretation of RCF using clinical decision by clinicians might not be accurate in the diagnosing of mild cognitive impairment (MCI) or dementia patients in comparison with the RCF scoring system. For this reason, a machine-learning algorithm was used to demonstrate that scoring RCF using clinical decision is not as accurate as of the RCF scoring system in predicting MCI or mild dementia patients from normal subjects.

METHODS

The RCF dataset consisted of 2,232 subjects with formal neuropsychological assessments. The RCF dataset was classified into 2 datasets. The first dataset was to compare normal vs. abnormal and the second dataset was to compare normal vs. MCI vs. mild dementia. Models were trained using a convolutional neural network for machine learning. Receiver operating characteristic curves were used to compare the sensitivity, specificity, and area under the curve (AUC) of models.

RESULTS

The trained model's accuracy for predicting cognitive states was 96% with the first dataset (normal vs. abnormal) and 88% with the second dataset (normal vs. MCI vs. mild dementia). The model had a sensitivity of 85% for detecting abnormal with an AUC of 0.847 with the first dataset. It had a sensitivity of 78% for detecting MCI or mild dementia with an AUC of 0.778 with the second dataset.

CONCLUSIONS

Based on this study, the RCF scoring system has the potential to present more accurate criteria than the clinical decision for distinguishing cognitive impairment among patients.

Overview of the Complex Figure Test and Its Clinical Application in Neuropsychiatric Disorders, Including Copying and Recall

The Rey–Osterrieth Complex Figure (ROCF) test is a commonly used neuropsychological assessment tool. It is widely used to assess the visuo-constructional ability and visual memory of neuropsychiatric disorders, including copying and recall tests. By drawing the complex figure, the functional decline of a patient in multiple cognitive dimensions can be assessed, including attention and concentration, fine-motor coordination, visuospatial perception, non-verbal memory, planning and organization, and spatial orientation. This review first describes the different versions and scoring methods of ROCF. It then reviews the application of ROCF in the assessment of visuo-constructional ability in patients with dementia, other brain diseases, and psychiatric disorders. Finally, based on the scoring method of the digital system, future research hopes to develop a new digital ROCF scoring method combined with machine learning algorithms to standardize clinical practice and explore the characteristic neuropsychological structure information of different disorders.

Automatic, Qualitative Scoring of the Interlocking Pentagon Drawing Test (PDT) based on U-Net and Mobile Sensor Data

We implemented a mobile phone application of the pentagon drawing test (PDT), called mPDT, with a novel, automatic, and qualitative scoring method for the application based on U-Net (a convolutional network for biomedical image segmentation) coupled with mobile sensor data obtained with the mPDT. For the scoring protocol, the U-Net was trained with 199 PDT hand-drawn images of 512 × 512 resolution obtained via the mPDT in order to generate a trained model, Deep5, for segmenting a drawn right or left pentagon. The U-Net was also trained with 199 images of 512 × 512 resolution to attain the trained model, DeepLock, for segmenting an interlocking figure. Here, the epochs were iterated until the accuracy was greater than 98% and saturated. The mobile senor data primarily consisted of x and y coordinates, timestamps, and touch-events of all the samples with a 20 ms sampling period. The velocities were then calculated using the primary sensor data. With Deep5, DeepLock, and the sensor data, four parameters were extracted. These included the number of angles (0–4 points), distance/intersection between the two drawn figures (0–4 points), closure/opening of the drawn figure contours (0–2 points), and tremors detected (0–1 points). The parameters gave a scaling of 11 points in total. The performance evaluation for the mPDT included 230 images from subjects and their associated sensor data. The results of the performance test indicated, respectively, a sensitivity, specificity, accuracy, and precision of 97.53%, 92.62%, 94.35%, and 87.78% for the number of angles parameter; 93.10%, 97.90%, 96.09%, and 96.43% for the distance/intersection parameter; 94.03%, 90.63%, 92.61%, and 93.33% for the closure/opening parameter; and 100.00%, 100.00%, 100.00%, and 100.00% for the detected tremor parameter. These results suggest that the mPDT is very robust in differentiating dementia disease subtypes and is able to contribute to clinical practice and field studies.