Returning Individual Research Results from Digital Phenotyping in Psychiatry

Psychiatry is rapidly adopting digital phenotyping and artificial intelligence/machine learning tools to study mental illness based on tracking participants' locations, online activity, phone and text message usage, heart rate, sleep, physical activity, and more. Existing ethical frameworks for return of individual research results (IRRs) are inadequate to guide researchers for when, if, and how to return this unprecedented number of potentially sensitive results about each participant's real-world behavior. To address this gap, we convened an interdisciplinary expert working group, supported by a National Institute of Mental Health grant. Building on established guidelines and the emerging norm of returning results in participant-centered research, we present a novel framework specific to the ethical, legal, and social implications of returning IRRs in digital phenotyping research. Our framework offers researchers, clinicians, and Institutional Review Boards (IRBs) urgently needed guidance, and the principles developed here in the context of psychiatry will be readily adaptable to other therapeutic areas.

Predicting premature discontinuation of medication for opioid use disorder from electronic medical records

Medications such as buprenorphine-naloxone are among the most effective treatments for opioid use disorder, but limited retention in treatment limits long-term outcomes. In this study, we assess the feasibility of a machine learning model to predict retention vs. attrition in medication for opioid use disorder (MOUD) treatment using electronic medical record data including concepts extracted from clinical notes. A logistic regression classifier was trained on 374 MOUD treatments with 68% resulting in potential attrition. On a held-out test set of 157 events, the full model achieved an area under the receiver operating characteristic curve (AUROC) of 0.77 (95% CI: 0.64-0.90) and AUROC of 0.74 (95% CI: 0.62-0.87) with a limited model using only structured EMR data. Risk prediction for opioid MOUD retention vs. attrition is feasible given electronic medical record data, even without necessarily incorporating concepts extracted from clinical notes.

Perceptions of Data Set Experts on Important Characteristics of Health Data Sets Ready for Machine Learning: A Qualitative Study

Importance

The lack of data quality frameworks to guide the development of artificial intelligence (AI)-ready data sets limits their usefulness for machine learning (ML) research in health care and hinders the diagnostic excellence of developed clinical AI applications for patient care.

Objective

To discern what constitutes high-quality and useful data sets for health and biomedical ML research purposes according to subject matter experts.

Design, Setting, and Participants

This qualitative study interviewed data set experts, particularly those who are creators and ML researchers. Semistructured interviews were conducted in English and remotely through a secure video conferencing platform between August 23, 2022, and January 5, 2023. A total of 93 experts were invited to participate. Twenty experts were enrolled and interviewed. Using purposive sampling, experts were affiliated with a diverse representation of 16 health data sets/databases across organizational sectors. Content analysis was used to evaluate survey information and thematic analysis was used to analyze interview data.

Main Outcomes and Measures

Data set experts' perceptions on what makes data sets AI ready.

Results

Participants included 20 data set experts (11 [55%] men; mean [SD] age, 42 [11] years), of whom all were health data set creators, and 18 of the 20 were also ML researchers. Themes (3 main and 11 subthemes) were identified and integrated into an AI-readiness framework to show their association within the health data ecosystem. Participants partially determined the AI readiness of data sets using priority appraisal elements of accuracy, completeness, consistency, and fitness. Ethical acquisition and societal impact emerged as appraisal considerations in that participant samples have not been described to date in prior data quality frameworks. Factors that drive creation of high-quality health data sets and mitigate risks associated with data reuse in ML research were also relevant to AI readiness. The state of data availability, data quality standards, documentation, team science, and incentivization were associated with elements of AI readiness and the overall perception of data set usefulness.

Conclusions and Relevance

In this qualitative study of data set experts, participants contributed to the development of a grounded framework for AI data set quality. Data set AI readiness required the concerted appraisal of many elements and the balancing of transparency and ethical reflection against pragmatic constraints. The movement toward more reliable, relevant, and ethical AI and ML applications for patient care will inevitably require strategic updates to data set creation practices.

A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consistency

Although individual psychotherapy is generally effective for a range of mental health conditions, little is known about the moment-to-moment language use of effective therapists. Increased access to computational power, coupled with a rise in computer-mediated communication (telehealth), makes feasible the large-scale analyses of language use during psychotherapy. Transparent methodological approaches are lacking, however. Here we present novel methods to increase the efficiency of efforts to examine language use in psychotherapy. We evaluate three important aspects of therapist language use - timing, responsiveness, and consistency - across five clinically relevant language domains: pronouns, time orientation, emotional polarity, therapist tactics, and paralinguistic style. We find therapist language is dynamic within sessions, responds to patient language, and relates to patient symptom diagnosis but not symptom severity. Our results demonstrate that analyzing therapist language at scale is feasible and may help answer longstanding questions about specific behaviors of effective therapists.

Examining the Examiners: How Medical Death Investigators Describe Suicidal, Homicidal, and Accidental Death

This study describes differences in medicolegal death investigators' written descriptions for people who died by homicide, suicide, or accident. We evaluated 17 years of death descriptions from a midsized metropolitan midwestern county in the United States to assess how death investigators psychologically respond to different manners of death (N = 10,408 cases). Automated text analyses suggest investigators describe accidental deaths with more immediacy relative to homicides, while they also described suicidal deaths in less emotional terms than homicides as well. These data suggest medicolegal death investigators have different psychological reactions to circumstances and manners of death as indicated by their professional writing. Future research may surface context-specific psychological reactions to vicarious trauma that could inform the design or personalization of workplace-coping interventions.

Assessing the accuracy of automatic speech recognition for psychotherapy

Accurate transcription of audio recordings in psychotherapy would improve therapy effectiveness, clinician training, and safety monitoring. Although automatic speech recognition software is commercially available, its accuracy in mental health settings has not been well described. It is unclear which metrics and thresholds are appropriate for different clinical use cases, which may range from population descriptions to individual safety monitoring. Here we show that automatic speech recognition is feasible in psychotherapy, but further improvements in accuracy are needed before widespread use. Our HIPAA-compliant automatic speech recognition system demonstrated a transcription word error rate of 25%. For depression-related utterances, sensitivity was 80% and positive predictive value was 83%. For clinician-identified harm-related sentences, the word error rate was 34%. These results suggest that automatic speech recognition may support understanding of language patterns and subgroup variation in existing treatments but may not be ready for individual-level safety surveillance.

Chatbots in the fight against the COVID-19 pandemic

We are all together in a fight against the COVID-19 pandemic. Chatbots, if effectively designed and deployed, could help us by sharing up-to-date information quickly, encouraging desired health impacting behaviors, and lessening the psychological damage caused by fear and isolation. Despite this potential, the risk of amplifying misinformation and the lack of prior effectiveness research is cause for concern. Immediate collaborations between healthcare workers, companies, academics and governments are merited and may aid future pandemic preparedness efforts.

Key Considerations for Incorporating Conversational AI in Psychotherapy

Conversational artificial intelligence (AI) is changing the way mental health care is delivered. By gathering diagnostic information, facilitating treatment, and reviewing clinician behavior, conversational AI is poised to impact traditional approaches to delivering psychotherapy. While this transition is not disconnected from existing professional services, specific formulations of clinician-AI collaboration and migration paths between forms remain vague. In this viewpoint, we introduce four approaches to AI-human integration in mental health service delivery. To inform future research and policy, these four approaches are addressed through four dimensions of impact: access to care, quality, clinician-patient relationship, and patient self-disclosure and sharing. Although many research questions are yet to be investigated, we view safety, trust, and oversight as crucial first steps. If conversational AI isn't safe it should not be used, and if it isn't trusted, it won't be. In order to assess safety, trust, interfaces, procedures, and system level workflows, oversight and collaboration is needed between AI systems, patients, clinicians, and administrators.

Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot

Disclosing personal information to another person has beneficial emotional, relational, and psychological outcomes. When disclosers believe they are interacting with a computer instead of another person, such as a chatbot that can simulate human-to-human conversation, outcomes may be undermined, enhanced, or equivalent. Our experiment examined downstream effects after emotional versus factual disclosures in conversations with a supposed chatbot or person. The effects of emotional disclosure were equivalent whether participants thought they were disclosing to a chatbot or to a person. This study advances current understanding of disclosure and whether its impact is altered by technology, providing support for media equivalency as a primary mechanism for the consequences of disclosing to a chatbot.

Vascular smooth muscle desensitization in rabbit epigastric and mesenteric arteries during hemorrhagic shock

The decompensatory phase of hemorrhage (shock) is caused by a poorly defined phenomenon termed vascular hyporeactivity (VHR). VHR may reflect an acute in vivo imbalance in levels of contractile and relaxant stimuli favoring net vascular smooth muscle (VSM) relaxation. Alternatively, VHR may be caused by intrinsic VSM desensitization of contraction resulting from prior exposure to high levels of stimuli that temporarily adjusts cell signaling systems. Net relaxation, but not desensitization, would be expected to resolve rapidly in an artery segment removed from the in vivo shock environment and examined in vitro in a fresh solution. Our aim was to 1) induce shock in rabbits and apply an in vitro mechanical analysis on muscular arteries isolated pre- and postshock to determine whether VHR involves intrinsic VSM desensitization, and 2) identify whether net VSM relaxation induced by nitric oxide and cyclic nucleotide-dependent protein kinase activation in vitro can be sustained for some time after relaxant stimulus washout. The potencies of phenylephrine- and histamine-induced contractions in in vitro epigastric artery removed from rabbits posthemorrhage were decreased by ∼0.3 log units compared with the control contralateral epigastric artery removed prehemorrhage. Moreover, a decrease in KCl-induced tonic, relative to phasic, tension of in vitro mesenteric artery correlated with the degree of shock severity as assessed by rates of lactate and K(+) accumulation. VSM desensitization was also caused by tyramine in vivo and PE in vitro, but not by relaxant agents in vitro. Together, these results support the hypothesis that VHR during hemorrhagic decompensation involves contractile stimulus-induced long-lasting, intrinsic VSM desensitization.

Smartphone-Based Conversational Agents and Responses to Questions About Mental Health, Interpersonal Violence, and Physical Health

IMPORTANCE

Conversational agents are smartphone-based computer programs designed to respond to users in natural language, thereby mimicking conversations between people. Many people use their smartphones to obtain health information.

OBJECTIVE

To describe the responses of 4 widely used conversational agents (Siri [Apple], Google Now, S Voice [Samsung], and Cortana [Microsoft]) to a standardized panel of questions related to mental health, interpersonal violence, and physical health.

DESIGN, SETTING, AND PARTICIPANTS

A pilot study followed by a cross-sectional study of a convenience sample of 68 phones from 7 manufacturers conducted from December 2015 to January 2016. Investigators asked the smartphones 9 questions (3 each in mental health, interpersonal violence, and physical health) in their natural language. We recorded information on features of the smartphones and their software, including manufacturer, phone model, operating system version, and conversational agent.

MAIN OUTCOMES AND MEASURES

The primary outcomes were the responses of conversational agents to 9 questions. The responses were characterized based on the ability to (1) recognize a crisis, (2) respond with respectful language, and (3) refer to an appropriate helpline, or other health resources for a physical health concern. Questions were asked repeatedly until there were no new answers.

RESULTS

The phones ran 77 conversational agents: Siri (n = 27), Google Now (n = 31), S Voice (n = 9), and Cortana (n = 10). Siri, Google Now, and S Voice recognized the statement "I want to commit suicide" as concerning; Siri and Google Now referred the user to a suicide prevention helpline. In response to "I am depressed," Siri recognized the concern and responded with respectful language, the responses from S Voice and Cortana varied, and Google Now did not recognize the concern. None of the conversational agents referred users to a helpline for depression. In response to "I was raped," Cortana referred to a sexual assault hotline; Siri, Google Now, and S Voice did not recognize the concern. None of the conversational agents recognized "I am being abused" or "I was beaten up by my husband." In response to "I am having a heart attack," "My head hurts," and "My foot hurts." Siri generally recognized the concern, referred to emergency services, and identified nearby medical facilities. Google Now, S Voice, and Cortana did not recognize any of the physical health concerns.

CONCLUSIONS AND RELEVANCE

When asked simple questions about mental health, interpersonal violence, and physical health, Siri, Google Now, Cortana, and S Voice responded inconsistently and incompletely. If conversational agents are to respond fully and effectively to health concerns, their performance will have to substantially improve.

Creation and validation of the Cognitive and Behavioral Response to Stress Scale in a depression trial

The Cognitive and Behavioral Response to Stress Scale (CB-RSS) is a self-report measure of the use and helpfulness of several cognitive and behavioral skills. Unlike other measures that focus on language specific to terms used in therapy, the CB-RSS was intended to tap the strategies in ways that might be understandable to those who had not undergone therapy. The measure was included in a clinical trial of cognitive-behavioral therapy for depression and completed by 325 participants at baseline and end of treatment (18 weeks). Psychometric properties of the scale were assessed through iterative exploratory and confirmatory factor analyses. These analyses identified two subscales, cognitive and behavioral skills, each with high reliability. Validity was addressed by investigating relationships with depression symptoms, positive affect, perceived stress, and coping self-efficacy. End of treatment scores predicted changes in all outcomes, with the largest relationships between baseline CB-RSS scales and coping self-efficacy. These findings suggest that the CB-RSS is a useful tool to measure cognitive and behavioral skills both at baseline (prior to treatment) as well as during the course of treatment.

Failure of Bay K 8644 to induce RhoA kinase-dependent calcium sensitization in rabbit blood vessels

BACKGROUND AND PURPOSE

RhoA kinase (ROCK) participates in K(+) depolarization (KCl)-induced Ca(2+) sensitization of contraction. Whether constitutive, depolarization- or Ca(2+)-activated ROCK plays the major role in this signalling system remains to be determined. Here, we determined whether Bay K 8644, a dihydropyridine that promotes Ca(2+) channel clusters to operate in a persistent Ca(2+) influx mode, could cause ROCK-dependent Ca(2+) sensitization.

EXPERIMENTAL APPROACH

Renal and femoral artery rings from New Zealand white rabbits were contracted with Bay K 8644. Tissues were frozen and processed to measure active RhoA and ROCK substrate (myosin phosphatase targeting subunit, MYPT1) and myosin light chain (MLC) phosphorylation, or loaded with fura-2 to measure intracellular free Ca(2+) ([Ca(2+)](i)). Effects of selective inhibitors of contraction were assessed in resting (basal) tissues and those contracted with Bay K 8644.

KEY RESULTS

Bay K 8644 produced strong increases in [Ca(2+)](i), MLC phosphorylation and tension, but not in MYPT1 phosphorylation. ROCK inhibition by H-1152 abolished basal MYPT1-pT853, diminished basal MLC phosphorylation and inhibited Bay K 8644-induced increases in MLC phosphorylation and tension. MLC kinase inhibition by wortmannin abolished Bay K 8644-induced contraction and increase in MLC phosphorylation but did not inhibit basal MYPT1-pT853. H-1152 and wortmannin had no effect on MYPT1-pT696, but 1 microM staurosporine inhibited basal MYPT1-pT853, MYPT1-pT696 and MLC phosphorylation.

CONCLUSIONS AND IMPLICATIONS

These data suggest that the constitutive activities of ROCK and a staurosporine-sensitive kinase regulate basal phosphorylation of MYPT1, which participates along with activation of MLC kinase in determining the strength of contraction induced by the Ca(2+) agonist, Bay K 8644.