Augmenting visual analysis in single-case research with hierarchical linear modeling

Exploring the Effects of Yoga Therapy on Heart Rate Variability and Patient-Reported Outcomes After Cancer Treatment: A Study Protocol

BACKGROUND

Following cancer treatment, adults commonly report worsened patient-reported outcomes (PROs) such as anxiety, stress, depression, persistent and upsetting cognitive complaints, unrelenting fatigue, and reduced quality of life. Poorer PROs are associated with disrupted autonomic nervous system functioning as measured by heart rate variability (HRV), both of which have been associated with greater morbidity and mortality. Interventions to improve HRV and PROs among adults following cancer treatment are needed. Yoga therapy holds promise as an intervention to improve HRV and PROs. Therefore, we conducted a single-subject exploratory experimental study to investigate the effects of yoga therapy on HRV and specific PROs (ie, cancer-related fatigue, anxiety, cognitive function, depression, stress, quality of life) in adults treated for cancer. To reduce publication bias, improve reproducibility, and serve as a reference for forthcoming reporting of study results, we present the study protocol for this study herein.

METHODS

Participants were adults who completed cancer treatment that were recruited from the Ottawa Integrative Cancer Centre. Consenting and eligible participants received one 1:1 yoga therapy session (ie, 1 participant, 1 Yoga Therapist) and 6 weekly group-based yoga therapy sessions (ie, 2-3 participants, 1 Yoga Therapist). Participants completed assessments 7 times: 3 times prior to the program (ie, -6 weeks, -3 weeks, immediately prior to the 1:1 yoga therapy session), immediately following the 1:1 yoga therapy session, prior to the first group-based yoga therapy session, after the last group-based yoga therapy session, and at a 6-week follow-up. Hierarchical linear modeling will be used to test the average effects of the yoga therapy program across participants.

DISCUSSION

This study will explore several novel hypotheses, including whether yoga therapy can improve HRV and/or specific PROs among adults treated for cancer acutely (ie, during a 1:1 yoga therapy session) and/or through repeated exposure (ie, after completing 6 weeks of group-based yoga therapy). Although the findings will require confirmation or refutation in future trials, they may provide initial evidence that YT may benefit adults treated for cancer.

TRIAL REGISTRATION

ISRCTN registry, ISRCTN64763228. Registered on December 12, 2021. This trial was registered retrospectively. URL of trial registry record: https://www.isrctn.com/ISRCTN64763228.

Assessing consistency of effects when applying multilevel models to single-case data

In the context of single-case experimental designs, replication is crucial. On the one hand, the replication of the basic effect within a study is necessary for demonstrating experimental control. On the other hand, replication across studies is required for establishing the generality of the intervention effect. Moreover, the "replicability crisis" presents a more general context further emphasizing the need for assessing consistency in replications. In the current text, we focus on replication of effects within a study, and we specifically discuss the consistency of effects. Our proposal for assessing the consistency of effects refers to one of the promising data analytical techniques, multilevel models, also known as hierarchical linear models or mixed effects models. One option is to check, for each case in a multiple-baseline design, whether the confidence interval for the individual treatment effect excludes zero. This is relevant for assessing whether the effect is replicated as being non-null. However, we consider that it is more relevant and informative to assess, for each case, whether the confidence interval for the random effects includes zero (i.e., whether the fixed effect estimate is a plausible value for each individual effect). This is relevant for assessing whether the effect is consistent in size, with the additional requirement that the fixed effect itself is different from zero. The proposal for assessing consistency is illustrated with real data and is implemented in free user-friendly software.

Statistical analysis in Small-N Designs: using linear mixed-effects modeling for evaluating intervention effectiveness

BACKGROUND

Advances in statistical methods and computing power have led to a renewed interest in addressing the statistical analysis challenges posed by Small-N Designs (SND). Linear mixed-effects modeling (LMEM) is a multiple regression technique that is flexible and suitable for SND and can provide standardized effect sizes and measures of statistical significance.

AIMS

Our primary goals are to: 1) explain LMEM at the conceptual level, situating it in the context of treatment studies, and 2) provide practical guidance for implementing LMEM in repeated measures SND.

METHODS & PROCEDURES

We illustrate an LMEM analysis, presenting data from a longitudinal training study of five individuals with acquired dysgraphia, analyzing both binomial (accuracy) and continuous (reaction time) repeated measurements.

OUTCOMES & RESULTS

The LMEM analysis reveals that both spelling accuracy and reaction time improved and, for accuracy, improved significantly more quickly under a training schedule with distributed, compared to clustered, practice. We present guidance on obtaining and interpreting various effect sizes and measures of statistical significance from LMEM, and include a simulation study comparing two p-value methods for generalized LMEM.

CONCLUSION

We provide a strong case for the application of LMEM to the analysis of training studies as a preferable alternative to visual analysis or other statistical techniques. When applied to a treatment dataset, the evidence supports that the approach holds up under the extreme conditions of small numbers of individuals, with repeated measures training data for both continuous (reaction time) and binomially distributed (accuracy) dependent measures. The approach provides standardized measures of effect sizes that are obtained through readily available and well-supported statistical packages, and provides statistically rigorous estimates of the expected average effect size of training effects, taking into account variability across both items and individuals.

Improved analyses of single cases: Dynamic multilevel analysis

This project identifies some difficulties when analyzing single-case data and showcases a new method, dynamic multilevel analysis (DMA). We re-analyze a published, meta-analysis of single-case interventions for participants with autism. Analytic difficulties include missing data, nested data, baseline trends, time periods, recency effects, many hypotheses' false positives, interactions among explanatory variables, indirect effects (including false negatives), and sampling errors. Furthermore, non-overlapping analyses can yield contested results, overvalue data near overlap boundaries, lose statistical power, and lack estimates of explained variance or unexplained residuals. To address these difficulties, DMA integrates several methods, including multilevel and time-series analyses. DMA re-analysis not only showed robust intervention effects, but also time-, outcome-, and intervention component-specific effects. Moreover, DMA informs the suitability of time hypotheses or meta-analysis, and DMA's components can be used separately, notably its time-series analyses for small samples (e.g., one participant). Hence, DMA can help researchers analyze single-case data more accurately.

Computerized Sign Language-Based Literacy Training for Deaf and Hard-of-Hearing Children

Strengthening the connections between sign language and written language may improve reading skills in deaf and hard-of-hearing (DHH) signing children. The main aim of the present study was to investigate whether computerized sign language-based literacy training improves reading skills in DHH signing children who are learning to read. Further, longitudinal associations between sign language skills and developing reading skills were investigated. Participants were recruited from Swedish state special schools for DHH children, where pupils are taught in both sign language and spoken language. Reading skills were assessed at five occasions and the intervention was implemented in a cross-over design. Results indicated that reading skills improved over time and that development of word reading was predicted by the ability to imitate unfamiliar lexical signs, but there was only weak evidence that it was supported by the intervention. These results demonstrate for the first time a longitudinal link between sign-based abilities and word reading in DHH signing children who are learning to read. We suggest that the active construction of novel lexical forms may be a supramodal mechanism underlying word reading development.

Analyzing Therapeutic Change Using Modified Brinley Plots: History, Construction, and Interpretation

The paper reviews the history, construction, and interpretation of modified Brinley plots, a scatter plot used in therapy outcome research to compare each individual participant's scores on the same dependent variable at Time 1 (normally pretreatment baseline; x-axis), with scores at selected times during or after treatment (y-axis). Since 1965 eponymously named Brinley plots have occasionally been used in experimental psychology to display group mean data. Between 1979 and 1995 a number of clinical researchers modified Brinley plots to show individuals' data but these plots have received little subsequent use. When constructed with orthogonal axes having the same origin and scale values, little or no change over time is shown by individuals' data points lying on or closely about the diagonal (45⁰_o) while the magnitude and direction of any improvement (or deterioration), outliers, and the extent of replication across cases shows via dispersion of points away from 45⁰_o. Interpretation is aided by displaying reliable change boundaries, clinical cutoffs, means, variances, confidence intervals, and effect sizes directly on the graph. Modified Brinley plots are directly informative about individual change during therapy in the context of concurrent change in others in the same (or a different) condition, clearly show if outcomes are replicated and if they are clinically significant, and make nomothetic group information, notably effect sizes, directly available. They usefully complement other forms of analysis in therapy outcome research.

Recommendations for Choosing Single-Case Data Analytical Techniques

The current paper responds to the need to provide guidance to applied single-case researchers regarding the possibilities of data analysis. The amount of available single-case data analytical techniques has been growing during recent years and a general overview, comparing the possibilities of these techniques, is missing. Such an overview is provided that refers to techniques that yield results in terms of a raw or standardized difference and procedures related to regression analysis, as well as nonoverlap and percentage change indices. The comparison is provided in terms of the type of quantification provided, data features taken into account, conditions in which the techniques are appropriate, possibilities for meta-analysis, and evidence available on their performance. Moreover, we provide a set of recommendations for choosing appropriate analysis techniques, pointing at specific situations (aims, types of data, researchers' resources) and the data analytical techniques that are most appropriate in these situations. The recommendations are contextualized using a variety of published single-case data sets in order to illustrate a range of realistic situations that researchers have faced and may face in their investigations.

How Can Single-Case Data Be Analyzed? Software Resources, Tutorial, and Reflections on Analysis

The present article aims to present a series of software developments in the quantitative analysis of data obtained via single-case experimental designs (SCEDs), as well as the tutorial describing these developments. The tutorial focuses on software implementations based on freely available platforms such as R and aims to bring statistical advances closer to applied researchers and help them become autonomous agents in the data analysis stage of a study. The range of analyses dealt with in the tutorial is illustrated on a typical single-case dataset, relying heavily on graphical data representations. We illustrate how visual and quantitative analyses can be used jointly, giving complementary information and helping the researcher decide whether there is an intervention effect, how large it is, and whether it is practically significant. To help applied researchers in the use of the analyses, we have organized the data in the different ways required by the different analytical procedures and made these data available online. We also provide Internet links to all free software available, as well as all the main references to the analytical techniques. Finally, we suggest that appropriate and informative data analysis is likely to be a step forward in documenting and communicating results and also for increasing the scientific credibility of SCEDs.

Analyzing Two-Phase Single-Case Data with Non-overlap and Mean Difference Indices: Illustration, Software Tools, and Alternatives

Two-phase single-case designs, including baseline evaluation followed by an intervention, represent the most clinically straightforward option for combining professional practice and research. However, unless they are part of a multiple-baseline schedule, such designs do not allow demonstrating a causal relation between the intervention and the behavior. Although the statistical options reviewed here cannot help overcoming this methodological limitation, we aim to make practitioners and applied researchers aware of the available appropriate options for extracting maximum information from the data. In the current paper, we suggest that the evaluation of behavioral change should include visual and quantitative analyses, complementing the substantive criteria regarding the practical importance of the behavioral change. Specifically, we emphasize the need to use structured criteria for visual analysis, such as the ones summarized in the What Works Clearinghouse Standards, especially if such criteria are complemented by visual aids, as illustrated here. For quantitative analysis, we focus on the non-overlap of all pairs and the slope and level change procedure, as they offer straightforward information and have shown reasonable performance. An illustration is provided of the use of these three pieces of information: visual, quantitative, and substantive. To make the use of visual and quantitative analysis feasible, open source software is referred to and demonstrated. In order to provide practitioners and applied researchers with a more complete guide, several analytical alternatives are commented on pointing out the situations (aims, data patterns) for which these are potentially useful.

A brief primary care intervention to reduce fear of movement in chronic low back pain patients

Fear avoidance model of chronic pain-based interventions are effective, but have not been successfully implemented into primary care. It was hypothesized that speed walking times and key measures of the fear avoidance model would improve following the brief intervention delivered in primary care. A brief primary care-based intervention (PCB) that included a single educational session, speed walking (an in vivo desensitization exposure task), and visual performance feedback was designed to reduce fear avoidance beliefs and improve function in 4 patients with chronic low back pain. A multiple baseline across subjects with a changing criterion design indicated that speed walking times improved from baseline only after the PCB intervention was delivered. Six fear avoidance model outcome measures improved from baseline to end of study and five of six outcome measures improved from end of study to follow-up. This study provides evidence for the efficacy of a brief PCB fear avoidance intervention that was successfully implemented into a busy clinic for the treatment of chronic pain.

Assessing Functional Relations in Single-Case Designs

In the context of the evidence-based practices movement, the emphasis on computing effect sizes and combining them via meta-analysis does not preclude the demonstration of functional relations. For the latter aim, we propose to augment the visual analysis to add consistency to the decisions made on the existence of a functional relation without losing sight of the need for a methodological evaluation of what stimuli and reinforcement or punishment are used to control the behavior. Four options for quantification are reviewed, illustrated, and tested with simulated data. These quantifications include comparing the projected baseline with the actual treatment measurements, on the basis of either parametric or nonparametric statistics. The simulated data used to test the quantifications include nine data patterns in terms of the presence and type of effect and comprise ABAB and multiple-baseline designs. Although none of the techniques is completely flawless in terms of detecting a functional relation only when it is present but not when it is absent, an option based on projecting split-middle trend and considering data variability as in exploratory data analysis proves to be the best performer for most data patterns. We suggest that the information on whether a functional relation has been demonstrated should be included in meta-analyses. It is also possible to use as a weight the inverse of the data variability measure used in the quantification for assessing the functional relation. We offer an easy to use code for open-source software for implementing some of the quantifications.

The influence of the design matrix on treatment effect estimates in the quantitative analyses of single-subject experimental design research

The quantitative methods for analyzing single-subject experimental data have expanded during the last decade, including the use of regression models to statistically analyze the data, but still a lot of questions remain. One question is how to specify predictors in a regression model to account for the specifics of the design and estimate the effect size of interest. These quantitative effect sizes are used in retrospective analyses and allow synthesis of single-subject experimental study results which is informative for evidence-based decision making, research and theory building, and policy discussions. We discuss different design matrices that can be used for the most common single-subject experimental designs (SSEDs), namely, the multiple-baseline designs, reversal designs, and alternating treatment designs, and provide empirical illustrations. The purpose of this article is to guide single-subject experimental data analysts interested in analyzing and meta-analyzing SSED data.

Single-case experimental designs: reflections on conduct and analysis

In this editorial discussion we reflect on the issues addressed by, and arising from, the papers in this special issue on Single-Case Experimental Design (SCED) study methodology. We identify areas of consensus and disagreement regarding the conduct and analysis of SCED studies. Despite the long history of application of SCEDs in studies of interventions in clinical and educational settings, the field is still developing. There is an emerging consensus on methodological quality criteria for many aspects of SCEDs, but disagreement on what are the most appropriate methods of SCED data analysis. Our aim is to stimulate this ongoing debate and highlight issues requiring further attention from applied researchers and methodologists. In addition we offer tentative criteria to support decision-making in relation to the selection of analytical techniques in SCED studies. Finally, we stress that large-scale interdisciplinary collaborations, such as the current Special Issue, are necessary if SCEDs are going to play a significant role in the development of the evidence base for clinical practice.