Association for Behavior Analysis International

Treatment integrity refers to the extent to which a procedure is implemented as it was designed. Recent research about treatment integrity has evaluated the extent to which variations of measurement procedures affect obtained integrity values, and the extent to which reduced integrity affects treatment outcomes. In this symposium, we will describe recent studies that have further advanced these two areas of enquiry. Our studies include both aspects of contemporary integrity research. Halbur et al. describe the extent to which variations in measurement systems may lead to different perceived levels of integrity in published studies. This line of research is furthered by Smothermon et al., who show that similar variations can over- or underestimate the performance of staff. Because integrity levels are often low without ongoing feedback, our final two studies evaluate the impact of reduced integrity on intervention outcomes. Brand et al. review the published literature in which treatment integrity was experimentally manipulated, and Mesches et al. describe a study in which integrity was manipulated during differential reinforcement of other behavior. Results across all studies continue to suggest that treatment integrity should be an important consideration for applied researchers and practitioners.

Collecting data on the integrity with which staff and caregivers implement prescribed treatments is a critical component of program evaluation. However, it can be challenging to collect data accurately on multiple procedural components in fast-paced instructional contexts. One possible approach is to evaluate performance across an entire session (e.g., whether the individual delivers prompts correctly during all trials of an observation) versus on a trial-by-trial basis (e.g., whether the individual delivers prompts correctly on each trial). In this study, we examined the sensitivity of data collected in this manner by comparing whole-session data to trial-by-trial data on the procedural integrity of 16 paraprofessionals who received training on how to implement discrete-trial teaching (DTT). We also compared the outcomes when data were collapsed across all procedural components versus individual components. Results suggested that whole-session data had adequate sensitivity but, in general, underestimated the performance of the individual implementing DTT. At the same time, trial-by-trial data were more likely to overestimate performance unless we examined the integrity of individual DTT components. Findings have important implications for assessing procedural integrity and selecting an appropriate mastery criterion during caregiver training.

Treatment integrity refers to the extent to which a treatment is accurately implemented according to the treatment plan (Gresham, 1989). There are multiple methods for collecting and calculating treatment integrity data. For example, experimenters may collect data on the trainees implementation of each step of intervention and calculate treatment integrity by dividing the number of correctly implemented treatment steps by a total number of steps per session. The present study describes variations in treatment integrity calculations and the implications of using each method of calculation. We will review trends in treatment integrity calculations used in staff/caregiver training studies and compare those methods to studies in which treatment integrity errors are evaluated. Raw data from staff/caregiver training studies published in the past seven years will be obtained and re-calculated using different criteria, and aggregated outcomes of these re-calculations will be presented and discussed. Clinical implications of the recalculations, suggestions for integrity calculations, and future research will be discussed.

Treatment integrity refers to the degree to which a treatment is implemented as it was designed. Treatment integrity failures are known negatively affect differential reinforcement of alternative behavior (DRA) and response cost, but little is known about effects of integrity failures on differential reinforcement of other behavior (DRO). We assessed the efficacy of a DRO procedure when implemented perfectly and when implemented with varying degrees of treatment integrity failures using a human-operant preparation with students from a university (Experiment 1) and with a child who engaged in socially significant challenging behavior (Experiment 2). The DRO was effective at all levels for 3 human operant participants, but lost efficacy when implemented with 60% or lower treatment integrity for the remaining 3 participants. We also obtained a loss of treatment effects at around 60% integrity with the clinical replication. These data suggest that DRO interventions may be effective with some level of failure in treatment integrity, but there may be a critical integrity level around 60% at which the intervention breaks down.