Putting the Patient Back in Clinical Significance: Moderated Nonlinear Factor Analysis for Estimating Clinically Significant Change in Treatment for Posttraumatic Stress Disorder

Beyond Jacobson and Truax: Estimation of Clinical Significance Trajectories in the Coping Power Intervention Using Measurement Error-Corrected Multilevel Modeling

Coping Power (CP) is a preventive intervention that focuses on reducing child externalizing problems. Although it is typically delivered in a group format (GCP), individually delivered CP (ICP) has produced greater mean reductions in externalizing problems. However, standard analysis of randomized trials loses individual-level information regarding which youth improve, fail to improve, or get worse, whereas clinically significant change (CSC) metrics capture information on individual change. The present study is a secondary analysis of an ICP/GCP trial (N = 360) that examines differences in CSC-based individual-level inferences on externalizing. A novel method for assessing CSC under measurement error-corrected multilevel modeling was used, overcoming three limitations of traditional CSC methods: (a) restriction to two time points, (b) use of total scores, and (c) assumption of constant reliability across time and participants. Because of concerns about Type II errors with all CSC methods, an individual-level effect size metric for CSC was also developed. Based on individualized Cohen's d estimates, individual-level improvements in externalizing from 4th through 11th grades of d ≥ 0.5 were significantly greater in ICP (73%) versus GCP (45%). Further, GCP saw significantly higher percentages of youth with worsening of externalizing, underscoring concerns about diminished effects for GCP. Half of the sample had improvement that was not statistically significant but exceeded d ≥ 0.5, highlighting the susceptibility to Type II errors of CSC's results based on statistical significance. An examination of ICP/GCP differences under advanced CSC analysis gives more nuanced information than conventional randomized controlled trial analysis and greater precision in estimating individual-level outcomes than standard CSC methodologies.

Behavioral activation for veterans with co-occurring alcohol use disorder and posttraumatic stress disorder: Basis and methodology for a pilot randomized controlled trial

BACKGROUND

Nearly 2 million U.S. veterans live with co-occurring alcohol use disorder and posttraumatic stress disorder (AUD/PTSD). Extant AUD/PTSD treatments emphasize symptom reduction, sometimes overlooking psychosocial functioning improvements, and have dropout rates as high as 50 %. Additionally, current approaches to measuring psychosocial functioning are limited to self-report. This study protocol describes a 1:1 parallel, two-arm, pilot randomized controlled trial comparing Behavioral Activation (BA) psychotherapy to Relapse Prevention (RP) psychotherapy for veterans with AUD/PTSD.

METHODS

Forty-six veterans with AUD/PTSD will be block-randomized to eight weekly, virtual, hour-long individual sessions of BA or RP. Clinical interview, self-report, and geospatial assessments will be administered at pre- and post-treatment. Select outcome and exploratory measures will be administered during treatment. Analyses will focus on trial feasibility, BA acceptability, and preliminary efficacy. Geospatial analyses will explore whether pre- to post-treatment changes in geospatial movement can be used to objectively measure treatment response. The study site and an independent Data and Safety Monitoring Board will monitor trial progress, safety, and quality. De-identified data from consenting participants will be submitted to a sponsor-designated data repository.

CONCLUSION

If successful, this trial could help to provide veterans with AUD/PTSD with a more acceptable treatment option. Positive findings would also lay groundwork for testing BA in civilians with AUD/PTSD. Finally, by incorporating novel geospatial methods and technologies, this study could potentially yield a new approach to objectively measuring AUD/PTSD recovery that could be used in other clinical trials. This study was registered in ClinicalTrials.gov (NCT06249386).

Boundaries on Parent Involvement in Their Child's Anxiety Cognitive-Behavioral-Treatment Outcome: Parent Reinforcement and Relationship Behaviors Moderate Outcome

Extending a recent parent mediation efficacy trial, we identified parent reinforcement and relationship behaviors as setting boundary conditions, or moderators, of youth anxiety outcome in 254 youths and their parents, who were randomized to: (1) Cognitive Behavioral Treatment (CBT) with parent reinforcement behavior training (CBT + Reinf); (2) CBT with parent relationship behavior training (CBT + Relat); or (3) individual youth CBT - a comparator control arm. Findings revealed that parents with high baseline negative reinforcement levels and acceptance levels (i.e., above the mean) report their children as having lower anxiety at outcome, when assigned to CBT + Reinf, and CBT + Relat, respectively, versus CBT. No moderation effects were found for either parent positive reinforcement or parent psychological control. Implications for treating anxiety disorders and moving toward precision treatment approaches in youth are discussed, and the importance of research replication and extension.

Differential symptom weighting in estimating empirical thresholds for underlying PTSD severity: Toward a "platinum" standard for diagnosis?

OBJECTIVE

Symptom counts as the basis for Post-Traumatic Stress Disorder (PTSD) diagnoses in the DSM presume each symptom is equally reflective of underlying disorder severity. However, the "equal weight" assumption fails to fit PTSD symptom data when tested. The present study developed an enhanced PTSD diagnosis based on (a) a conventional PTSD diagnosis from a clinical interview and (b) an empirical classification of full PTSD that reflected the relative clinical weights of each symptom.

METHOD

Baseline structured interview data from Project Harmony (N = 2658) was used. An enhanced diagnosis for full PTSD was estimated using an empirical threshold from moderated nonlinear factor analysis (MNLFA) latent PTSD scale scores, in combination with a full conventional PTSD diagnosis based on interview data.

RESULTS

One in 4 patients in the sample had a PTSD diagnosis that was inconsistent with their empirical PTSD grouping, such that the enhanced diagnostic standard reduced the diagnostic discrepancy rate by 20%. Veterans, and in particular female Veterans, were at greatest odds for discrepancy between their underlying PTSD severity and DSM diagnosis.

CONCLUSION

Psychometric methodologies that differentially weight symptoms can complement DSM criteria and may serve as a platform for symptom prioritization for diagnoses in future editions of DSM.

Measurement Error-Corrected Estimation of Clinically Significant Change Trajectories for Interventions Targeting Comorbid PTSD and Substance Use Disorders in OEF/OIF Veterans

In randomized control trials (RCTs), a focus on average differences between treatment arms often limits our understanding of whether individuals show clinically significant improvement or deterioration. The present study examined differences in individual-level clinical significance trajectories between Concurrent Treatment of PTSD and Substance Use Disorders Using Prolonged Exposure (COPE) and Relapse Prevention (RP). Eighty-one treatment-seeking veterans with a comorbid PTSD/SUD diagnosis were randomized to COPE or RP; data from an additional n = 48 patients who did not meet criteria for both disorders was used to establish a normative threshold. A newly developed, modernized approach to the Jacobson and Truax (1991) clinically significant change framework, using (a) moderated nonlinear factor analysis (MNLFA) scale scoring and (b) measurement error-corrected multilevel modeling (MEC-MLM) was used; this approach was compared to other approaches using conventional total scores and/or assuming no measurement error. Using a conventional approach to estimating the Reliable Change Index (RCI) yielded no differences between COPE and RP in the percentage of patients achieving statistically significant improvement (SSI; 88.9% for both groups). However, under MNLFA/MEC-MLM, higher percentages of patients receiving COPE (75.0%) achieved SSI compared to RP (40.7%). Findings suggest that, even though COPE and RP appear to reduce the same number of PTSD symptoms, MNLFA scoring of outcome measures gives greater weight to interventions that target and reduce "hallmark" PTSD symptoms.

Estimating posttraumatic stress disorder severity in the presence of differential item functioning across populations, comorbidities, and interview measures: Introduction to Project Harmony

Multiple factor analytic and item response theory studies have shown that items/symptoms vary in their relative clinical weights in structured interview measures for posttraumatic stress disorder (PTSD). Despite these findings, the use of total scores, which treat symptoms as though they are equally weighted, predominates in practice, with the consequence of undermining the precision of clinical decision-making. We conducted an integrative data analysis (IDA) study to harmonize PTSD structured interview data (i.e., recoding of items to a common symptom metric) from 25 studies (total N = 2,568). We aimed to identify (a) measurement noninvariance/differential item functioning (MNI/DIF) across multiple populations, psychiatric comorbidities, and interview measures simultaneously and (b) differences in inferences regarding underlying PTSD severity between scale scores estimated using moderated nonlinear factor analysis (MNLFA) and a total score analog model (TSA). Several predictors of MNI/DIF impacted effect size differences in underlying severity across scale scoring methods. Notably, we observed MNI/DIF substantial enough to bias inferences on underlying PTSD severity for two groups: African Americans and incarcerated women. The findings highlight two issues raised elsewhere in the PTSD psychometrics literature: (a) bias in characterizing underlying PTSD severity and individual-level treatment outcomes when the psychometric model underlying total scores fails to fit the data and (b) higher latent severity scores, on average, when using DSM-5 (net of MNI/DIF) criteria, by which multiple factors (e.g., Criterion A discordance across DSM editions, changes to the number/type of symptom clusters, changes to the symptoms themselves) may have impacted severity scoring for some patients.

Adapting the multilevel model for estimation of the reliable change index (RCI) with multiple timepoints and multiple sources of error

OBJECTIVE

One of the primary tools in the assessment of individual-level patient outcomes is Jacobson and Truax, (1991's) Reliable Change Index (RCI). Recent efforts to optimize the RCI have revolved around three issues: (a) extending the RCI beyond two timepoints, (b) estimating the RCI using scale scores from item response theory or factor analysis and (c) estimation of person- and time-specific standard errors of measurement.

METHOD

We present an adaptation of a two-stage procedure, a measurement error-corrected multilevel model, as a tool for RCI estimation (with accompanying Statistical Analysis System syntax). Using DASS-21 data from a community-based mental health center (N = 379), we illustrate the potential for the model as unifying framework for simultaneously addressing all three limitations in modeling individual-level RCI estimates.

RESULTS

Compared to the optimal-fitting RCI model (moderated nonlinear factor analysis scoring with measurement error correction), an RCI model that uses DASS-21 total scores produced errors in RCI inferences in 50.8% of patients; this was largely driven by overestimation of the proportion of patients with statistically significant improvement.

CONCLUSION

Estimation of the RCI can now be enhanced by the use of latent variables, person- and time-specific measurement errors, and multiple timepoints.

Design and methodology for an integrative data analysis of coping power: Direct and indirect effects on adolescent suicidality

As suicide rates have risen in the last decade, there has been greater emphasis on targeting early risk conditions for suicidality among youth and adolescents as a form of suicide "inoculation". Two particular needs that have been raised in this nascent literature are a) the dearth of examination of early intervention effects on distal suicide risk that target externalizing behaviors and b) the need to harmonize multiple existing intervention datasets for greater precision in modeling intervention effects on low base rate outcomes such as suicidal behaviors. This project, entitled "Integrative Data Analysis of Coping Power (CP): Effects on Adolescent Suicidality", funded by the National Institute of Mental Health (NIMH), will harmonize and analyze data from 11 randomized controlled trials of CP (total individual-level N = 3183, total school-level N = 189). CP is an empirically-supported, child- and family-focused preventive intervention that focuses on reducing externalizing more broadly among youth who exhibit early aggression, which makes it ideally suited to targeting externalizing pathways to suicidality. The project utilizes three measurement and data analysis frameworks that have emerged across multiple independent disciplines: integrative data analysis (IDA), random treatment effects multilevel modeling (RTE-MLM), and propensity score weighting (PSW). If successful, the project will a) provide initial evidence that CP would have gender-specific indirect effects on suicidality through reductions in externalizing for boys and reductions in internalizing for girls and b) identify optimal conditions under which CP is delivered (e.g., groups, individuals, online) across participants on reductions in suicidality and other key intermediate endpoints.

Evaluating treatments for posttraumatic stress disorder, alcohol and other drug use disorders using meta-analysis of individual patient data: Design and methodology of a virtual clinical trial

This paper describes Project Harmony, a Virtual Clinical Trial (VCT) funded by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) to harmonize and analyze data from over 40 independent psychological, pharmacologic and/or combined pharmacological treatment studies for posttraumatic stress disorder and comorbid alcohol and other drug use disorders (PTSD/AOD). The study attends to three distinct analysis challenges: (1) variation in measurement of PTSD/AOD across studies, time, populations and reporters, (2) cross-study variation in treatment effect sizes and (3) non-randomized, cross-study variation in the classification of treatments (despite within-study randomization of treatment arms). To address these challenges, the study combines meta-analysis of individual patient data (MIPD), integrative data analysis (IDA) and propensity score weighting (PSW) to integrate raw data from these clinical trials. This protocol shows how this VCT analytic framework was used to (1) develop commensurate scale scores of PTSD and AOD severity when measures vary across studies, (2) compare the efficacy of evidence-based treatment models for PTSD/AOD, (3) test for potential mediators of treatment effects on AOD and PTSD across treatment models, and (4) explore individual- and study-level moderators to inform for whom each of the treatment models works best. The advantages of the general VCT approach are juxtaposed against the limitations of single randomized controlled trials and conventional meta-analysis.

Crossover between diagnostic and empirical categorizations of full and subthreshold PTSD

BACKGROUND

Most of the work on understanding subthreshold PTSD has focused on inconsistencies in defining subthreshold PTSD and how those inconsistencies impact prevalence rates. The present study distinguishes between full and subthreshold PTSD using empirical categorization and assesses the circumstances under which empirical categorization is discordant with full and subthreshold PTSD diagnoses.

METHODS

Using data from the NIDA CTN Women and Trauma Study (N = 353), we use a modernized adaptation of the Jacobson and Truax (1991) framework, assessing whether patients were above or below an empirical threshold on latent PTSD severity scores estimated under categorical confirmatory factor analysis; the empirical categorizations were then crossed with the diagnoses to form four diagnostic by empirical categorization groupings.

RESULTS

Compared to a reference group (full PTSD diagnosis and empirical categorization), patients who had a full PTSD diagnosis but a subthreshold empirical categorization had lower symptom endorsement rates on 15 PTSD symptoms, were more likely to be married, ethnic minorities with fewer lifetime traumas. Conversely, patients with a subthreshold PTSD diagnosis and a full PTSD empirical grouping looked similar to "Full/Fulls", only differing on avoidance symptoms.

LIMITATIONS

Alternative definitions of subthreshold PTSD and coding of symptom endorsement may impact results. The use of DSM-IV symptoms (though reconciled against overlapping symptoms from DSM-5) is also a key limitation.

CONCLUSIONS

Empirical categorization can be a useful supplement to diagnosis in distinguishing subthreshold PTSD from full PTSD, using a methodology that could provide a platform for melding dimensional and categorical nosology approaches in the DSM.