Using artificial intelligence to optimize delivery of weight loss treatment: Protocol for an efficacy and cost-effectiveness trial

Gold standard behavioral weight loss (BWL) is limited by the availability of expert clinicians and high cost of delivery. The artificial intelligence (AI) technique of reinforcement learning (RL) is an optimization solution that tracks outcomes associated with specific actions and, over time, learns which actions yield a desired outcome. RL is increasingly utilized to optimize medical treatments (e.g., chemotherapy dosages), and has very recently started to be utilized by behavioral treatments. For example, we previously demonstrated that RL successfully optimized BWL by dynamically choosing between treatments of varying cost/intensity each week for each participant based on automatic monitoring of digital data (e.g., weight change). In that preliminary work, participants randomized to the AI condition required one-third the amount of coaching contact as those randomized to the gold standard condition but had nearly identical weight losses. The current protocol extends our pilot work and will be the first full-scale randomized controlled trial of a RL system for weight control. The primary aim is to evaluate the hypothesis that a RL-based 12-month BWL program will produce non-inferior weight losses to standard BWL treatment, but at lower costs. Secondary aims include testing mechanistic targets (calorie intake, physical activity) and predictors (depression, binge eating). As such, adults with overweight/obesity (N = 336) will be randomized to either a gold standard condition (12 months of weekly BWL groups) or AI-optimized weekly interventions that represent a combination of expert-led group, expert-led call, paraprofessional-led call, and automated message). Participants will be assessed at 0, 1, 6 and 12 months.

Examining virtual driving test performance and its relationship to individuals with HIV-associated neurocognitive disorders

Significance

Existing screening tools for HIV-associated neurocognitive disorders (HAND) are often clinically impractical for detecting milder forms of impairment. The formal diagnosis of HAND requires an assessment of both cognition and impairment in activities of daily living (ADL). To address the critical need for identifying patients who may have disability associated with HAND, we implemented a low-cost screening tool, the Virtual Driving Test (VDT) platform, in a vulnerable cohort of people with HIV (PWH). The VDT presents an opportunity to cost-effectively screen for milder forms of impairment while providing practical guidance for a cognitively demanding ADL.

Objectives

We aimed to: (1) evaluate whether VDT performance variables were associated with a HAND diagnosis and if so; (2) systematically identify a manageable subset of variables for use in a future screening model for HAND. As a secondary objective, we examined the relative associations of identified variables with impairment within the individual domains used to diagnose HAND.

Methods

In a cross-sectional design, 62 PWH were recruited from an established HIV cohort and completed a comprehensive neuropsychological assessment (CNPA), followed by a self-directed VDT. Dichotomized diagnoses of HAND-specific impairment and impairment within each of the seven CNPA domains were ascertained. A systematic variable selection process was used to reduce the large amount of VDT data generated, to a smaller subset of VDT variables, estimated to be associated with HAND. In addition, we examined associations between the identified variables and impairment within each of the CNPA domains.

Results

More than half of the participants (N = 35) had a confirmed presence of HAND. A subset of twenty VDT performance variables was isolated and then ranked by the strength of its estimated associations with HAND. In addition, several variables within the final subset had statistically significant associations with impairment in motor function, executive function, and attention and working memory, consistent with previous research.

Conclusion

We identified a subset of VDT performance variables that are associated with HAND and assess relevant functional abilities among individuals with HAND. Additional research is required to develop and validate a predictive HAND screening model incorporating this subset.

Simulator Pre-Screening of Underprepared Drivers Prior to Licensing On-Road Examination: Clustering of Virtual Driving Test Time Series Data

BACKGROUND

A large Midwestern state commissioned a virtual driving test (VDT) to assess driving skills preparedness before the on-road examination (ORE). Since July 2017, a pilot deployment of the VDT in state licensing centers (VDT pilot) has collected both VDT and ORE data from new license applicants with the aim of creating a scoring algorithm that could predict those who were underprepared.

OBJECTIVE

Leveraging data collected from the VDT pilot, this study aimed to develop and conduct an initial evaluation of a novel machine learning (ML)-based classifier using limited domain knowledge and minimal feature engineering to reliably predict applicant pass/fail on the ORE. Such methods, if proven useful, could be applicable to the classification of other time series data collected within medical and other settings.

METHODS

We analyzed an initial dataset that comprised 4308 drivers who completed both the VDT and the ORE, in which 1096 (25.4%) drivers went on to fail the ORE. We studied 2 different approaches to constructing feature sets to use as input to ML algorithms: the standard method of reducing the time series data to a set of manually defined variables that summarize driving behavior and a novel approach using time series clustering. We then fed these representations into different ML algorithms to compare their ability to predict a driver's ORE outcome (pass/fail).

RESULTS

The new method using time series clustering performed similarly compared with the standard method in terms of overall accuracy for predicting pass or fail outcome (76.1% vs 76.2%) and area under the curve (0.656 vs 0.682). However, the time series clustering slightly outperformed the standard method in differentially predicting failure on the ORE. The novel clustering method yielded a risk ratio for failure of 3.07 (95% CI 2.75-3.43), whereas the standard variables method yielded a risk ratio for failure of 2.68 (95% CI 2.41-2.99). In addition, the time series clustering method with logistic regression produced the lowest ratio of false alarms (those who were predicted to fail but went on to pass the ORE; 27.2%).

CONCLUSIONS

Our results provide initial evidence that the clustering method is useful for feature construction in classification tasks involving time series data when resources are limited to create multiple, domain-relevant variables.

Can the artificial intelligence technique of reinforcement learning use continuously-monitored digital data to optimize treatment for weight loss?

Behavioral weight loss (WL) trials show that, on average, participants regain lost weight unless provided long-term, intensive-and thus costly-intervention. Optimization solutions have shown mixed success. The artificial intelligence principle of "reinforcement learning" (RL) offers a new and more sophisticated form of optimization in which the intensity of each individual's intervention is continuously adjusted depending on patterns of response. In this pilot, we evaluated the feasibility and acceptability of a RL-based WL intervention, and whether optimization would achieve equivalent benefit at a reduced cost compared to a non-optimized intensive intervention. Participants (n = 52) completed a 1-month, group-based in-person behavioral WL intervention and then (in Phase II) were randomly assigned to receive 3 months of twice-weekly remote interventions that were non-optimized (NO; 10-min phone calls) or optimized (a combination of phone calls, text exchanges, and automated messages selected by an algorithm). The Individually-Optimized (IO) and Group-Optimized (GO) algorithms selected interventions based on past performance of each intervention for each participant, and for each group member that fit into a fixed amount of time (e.g., 1 h), respectively. Results indicated that the system was feasible to deploy and acceptable to participants and coaches. As hypothesized, we were able to achieve equivalent Phase II weight losses (NO = 4.42%, IO = 4.56%, GO = 4.39%) at roughly one-third the cost (1.73 and 1.77 coaching hours/participant for IO and GO, versus 4.38 for NO), indicating strong promise for a RL system approach to weight loss and maintenance.

The 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment

The 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2017) was held at the Snowbird Ski and Summer Resort in Little Cottonwod Canyon in the Wasatch Range of the Rock Mountains near Salt Lake County, Utah. Along with the main conference presentations, the meeting included two tutorials, three workshops, and invited keynotes. This report summarizes the main conference. It also includes contributions from the organizers of the three workshops.

The First microRTS Artificial Intelligence Competition

This article presents the results of the first edition of the microRTS (μRTS) AI competition, which was hosted by the IEEE Computational Intelligence in Games (CIG) 2017 conference. The goal of the competition is to spur research on AI techniques for real-time strategy (RTS) games. In this first edition, the competition received three submissions, focusing on address- ing problems such as balancing long-term and short-term search, the use of machine learning to learn how to play against certain opponents, and finally, dealing with partial observability in RTS games.

Reports on the 2017 AAAI Spring Symposium Series

The Association for the Advancement of Artificial Intelligence, in cooperation with Stanford University's Department of Computer Science, presented the 2017 Spring Symposium Series, held Monday through Wednesday, March 27–29, 2017 on the campus of Stanford University. The eight symposia held were Artificial Intelligence for the Social Good (SS-17-01); Computational Construction Grammar and Natural Language Understanding (SS-17-02); Computational Context: Why It's Important, What It Means, and Can It Be Computed? (SS-17-03); Designing the User Experience of Machine Learning Systems (SS-17-04); Interactive Multisensory Object Perception for Embodied Agents (SS-17-05); Learning from Observation of Humans (SS-17-06); Science of Intelligence: Computational Principles of Natural and Artificial Intelligence (SS-17-07); and Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing (SS-17-08). This report, compiled from organizers of the symposia, summarizes the research that took place.

Combinatorial Multi-armed Bandits for Real-Time Strategy Games

Games with large branching factors pose a significant challenge for game tree search algorithms. In this paper, we address this problem with a sampling strategy for Monte Carlo Tree Search (MCTS) algorithms called "naive sampling", based on a variant of the Multi-armed Bandit problem called "Combinatorial Multi-armed Bandits" (CMAB). We analyze the theoretical properties of several variants of naive sampling, and empirically compare it against the other existing strategies in the literature for CMABs. We then evaluate these strategies in the context of real-time strategy (RTS) games, a genre of computer games characterized by their very large branching factors. Our results show that as the branching factor grows, naive sampling outperforms the other sampling strategies.

The AIIDE 2015 Workshop Program

The workshop program at the Eleventh Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment was held November 14–15, 2015 at the University of California, Santa Cruz, USA. The program included 4 workshops (one of which was a joint workshop): Artificial Intelligence in Adversarial Real-Time Games, Experimental AI in Games, Intelligent Narrative Technologies and Social Believability in Games, and Player Modeling. This article contains the reports of three of the four workshops.

Reports of the Workshops Held at the Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment

The AIIDE-14 Workshop program was held Friday and Saturday, October 3–4, 2014 at North Carolina State University in Raleigh, North Carolina. The workshop program included five workshops covering a wide range of topics. The titles of the workshops held Friday were Games and Natural Language Processing, and Artificial Intelligence in Adversarial Real-Time Games. The titles of the workshops held Saturday were Diversity in Games Research, Experimental Artificial Intelligence in Games, and Musical Metacreation. This article presents short summaries of those events.

An Ensemble Architecture for Learning Complex Problem-Solving Techniques from Demonstration

We present a novel ensemble architecture for learning problem-solving techniques from a very small number of expert solutions and demonstrate its effectiveness in a complex real-world domain. The key feature of our “Generalized Integrated Learning Architecture” (GILA) is a set of heterogeneous independent learning and reasoning (ILR) components, coordinated by a central meta-reasoning executive (MRE). The ILRs are weakly coupled in the sense that all coordination during learning and performance happens through the MRE. Each ILR learns independently from a small number of expert demonstrations of a complex task. During performance, each ILR proposes partial solutions to subproblems posed by the MRE, which are then selected from and pieced together by the MRE to produce a complete solution. The heterogeneity of the learner-reasoners allows both learning and problem solving to be more effective because their abilities and biases are complementary and synergistic. We describe the application of this novel learning and problem solving architecture to the domain of airspace management, where multiple requests for the use of airspaces need to be deconflicted, reconciled, and managed automatically. Formal evaluations show that our system performs as well as or better than humans after learning from the same training data. Furthermore, GILA outperforms any individual ILR run in isolation, thus demonstrating the power of the ensemble architecture for learning and problem solving.