Treatment Integrity Failures during Timeout from Play

Peter CC, Feng J, Long BP. Trends in Reporting Procedural Integrity: A Comparison

Procedural integrity refers to the extent to which an independent variable is implemented as described. Measuring procedural integrity is one important factor when considering internal and external validity of experiments. Experimental articles in behavior-analytic journals have rarely reported procedural-integrity data. The purpose of this study was to update previous reviews of whether articles published in the Journal of Applied Behavior Analysis reported procedural integrity, spanning a period from 1980 to 2020, and compare reporting in JABA to recent reviews of studies published in Behavior Analysis in Practice (2008-2019) and the Journal of Organizational Behavior Management (2000-2020). Procedural integrity continues to be underreported across all three journals, but an increasing trend in reporting procedural integrity is evident in the Journal of Applied Behavior Analysis and Behavior Analysis in Practice. In addition to our recommendations and implications for research and practice, we provide examples and resources to assist researchers and practitioners with recording and reporting integrity data.

Comparing Multiple Methods to Measure Procedural Fidelity of Discrete-trial Instruction

Procedural fidelity is the extent to which an intervention is implemented as designed and is an important component of research and practice. There are multiple ways to measure procedural fidelity, and few studies have explored how procedural fidelity varies based on the method of measurement. The current study compared adherence to discrete-trial instruction protocols by behavior technicians with a child with autism when observers used different procedural-fidelity measures. We collected individual-component and individual-trial fidelity with an occurrence-nonoccurrence data sheet and compared these scores to global fidelity and all-or-nothing, 3-point Likert scale, and 5-point Likert scale measurement methods. The all-or-nothing method required all instances of a component or trial be implemented without error to be scored correct. The Likert scales used a rating system to score components and trials. At the component level, we found that the global, 3-point Likert, and 5-point Likert methods were likely to overestimate fidelity and mask component errors, and the all-or-nothing method was unlikely to mask errors. At the trial level, we found that the global and 5-point Likert methods approximated individual-trial fidelity, the 3-point Likert method overestimated fidelity, and the all-or-nothing method underestimated fidelity. The occurrence-nonoccurrence method required the most time to complete, and all-or-nothing by trial required the least. We discuss the implications of measuring procedural fidelity with different methods of measurement, including false positives and false negatives, and provide suggestions for practice and research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43494-023-00094-w.

A parametric analysis of procedural fidelity errors following mastery of a task: A translational study

Procedural fidelity is defined as the extent to which the independent variable is implemented as prescribed. Research using computerized tasks has shown that fidelity errors involving consequences for behavior can hinder skill acquisition. However, studies examining the effects of these errors once skills have been mastered are lacking. Thus, this translational study investigated the effects of varying levels of fidelity following mastery of a computerized arbitrary matching-to-sample task. A group design (consisting of five groups) was used in which college students initially completed 250 trials during which no programmed errors (i.e., perfect fidelity) were arranged, followed by an additional 250 trials with consequences delivered across various levels of fidelity (i.e., 20, 40, 60, 80, and 100% of trials administered without errors). The results showed that participants assigned to higher fidelity conditions performed better (on average). These results extended the findings of previous studies by demonstrating how errors involving consequences affect behavior across various stages of learning.

Nominally acceptable integrity failures negatively affect interventions involving intermittent reinforcement

The finding that differential reinforcement of alternative behavior (DRA) is efficacious at 80% integrity when continuous reinforcement is programmed for alternative responding may have contributed to a perception that integrity at 80% or above is acceptable. However, research also suggests that other interventions (e.g., noncontingent reinforcement) may not remain effective at 80% integrity. The conditions under which 80% integrity is acceptable for common behavioral interventions remains unclear. Therefore, we conducted two human-operant studies to evaluate effects of 80% integrity for interventions with contingent or noncontingent intermittent reinforcement schedules. During Experiment 1, we compared noncontingent reinforcement (NCR) and DRA when implemented with 80% integrity. During Experiment 2, we compared 2 variations of DRA, which included either a ratio or interval schedule to reinforce alternative behavior. Results replicated previous research showing that DRA with a FR-1 schedule programmed for alternative responding resulted in consistent target response suppression, even when integrity was reduced to 80%. In contrast, neither NCR nor interval-based DRA were consistently effective when implemented at 80% integrity. These results demonstrate that 80% integrity is not a uniformly acceptable minimal level of integrity.

Strengthening the procedural fidelity research-to-practice loop in animal behavior

Procedural fidelity is the extent to which components of an intervention are implemented as designed. Procedural fidelity is measured as a dependent variable and manipulated as an independent variable. In research and practice, procedural-fidelity data should be collected, monitored, and reported. Procedural fidelity as an independent variable has been investigated in humans using parametric analyses, and the current article summarizes some of the research conducted on the effects of procedural-fidelity errors in behavior-reduction and skill-acquisition interventions. Connections were drawn to applied animal researchers and the work of animal behavior practitioners to produce implications for practice with human and animal clients and suggestions for future research. Further, there are multiple ways to measure procedural fidelity, and different conclusions can be drawn based on the measure and computation method. The current article describes procedural-fidelity measures that are most applicable to animal behavior researchers and professionals.