Intervention Integrity Part 1: Building Integrity into the Academic Intervention in the Design Phase

 Much of the effort to implement RTI involves the creation of a multi-tiered continuum of high-quality, scientifically valid academic interventions for at-risk students. A vexing problem for schools, though, is how to assure that academic interventions conducted in classrooms and other busy school settings are actually being done according to best ‘evidence-based’ practices.

 

 In RTI, the challenge to provide quality assurance of school-based interventions is often referred to as ‘intervention integrity’ (Gresham, 1989; Gresham, Gansle & Noell, 1993; Hawkins et al., 2008). Certainly, schools must be able to verify that an intervention has been carried out correctly and in full compliance with its design before they can assume that a student who failed to benefit from that intervention is a true ‘non-responder’.

There are two important steps to attaining a high level of intervention integrity in a school.
  1. The school first builds integrity into each academic intervention plan while developing it. To accomplish this step, the school can consult a ‘critical components’ checklist to verify –among other things--that the intervention plan being developed is of high quality, matched to the identified student needs, is formatted as a step-by-step script, and has the full understanding and support of the classroom teacher or other interventionist. The presence of these critical components raises the probability that the intervention selected is appropriate and will be put into practice correctly.
  2. The second step is for the school then to explicitly measure the integrity of the intervention plan on an ongoing basis while that plan is being implemented. Data collected about the integrity (quality of implementation) of the intervention provide objective information about how closely the actual intervention unfolding in the school setting conforms to the original intervention plan.
It is apparent that the steps of (1) designing integrity into an intervention with the use of a quality-control checklist and then (2) measuring the integrity of the intervention-in-progress are closely interlinked.  Just as the design of the intervention is strengthened by a consideration in the earliest stages of development of how to implement it with integrity, so too the measurement of intervention integrity is made easier when the teacher grasps its importance and is a full partner in collecting integrity data. The focus of this article is on how to build integrity into the intervention plan before it is implemented. (The question of how to measure the integrity of school-based interventions will be addressed in a future blog entry.)
Any critical-components checklist for academic interventions is likely to include the following categories:
  • Allocating sufficient contact time & assuring appropriate student-teacher ratio. The cumulative time set aside for an intervention and the amount of direct teacher contact are two factors that help to determine that intervention’s ‘strength’ (Yeaton & Sechrest, 1981). 
  • Matching the intervention to the student problem. Academic interventions are not selected at random. First, the student academic problem(s) is defined clearly and in detail. Then, the likely explanations for the academic problem(s) are identified to understand which intervention(s) are likely to help—and perhaps which should be avoided. The Instructional Hierarchy (Haring et al., 1978) is one useful tool for locating the best academic intervention for a student; it defines learners as being in the acquisition, fluency, generalization, or adaptation stage. Another important question to be answered when selecting an intervention is whether poor motivation plays a significant role in student underperformance.
  • Incorporating effective instructional elements into the intervention. The effective ‘building blocks’ of instruction are well-known and well-supported by the research. They include explicit instruction, appropriate level of challenge, active engagement, and timely performance feedback (Burns, VanDerHeyden & Boice, 2008; Skinner, Pappas & Davis, 2005). Schools should keep these effective instructional elements foremost in mind when selecting or creating any academic interventions.
  • Verifying teacher understanding & providing teacher support. The teacher is an active agent in the intervention, with primary responsibility for putting it into practice in a busy classroom. It is important, then, that the teacher fully understands how to do the intervention, believes that he or she can do it, and knows whom to seek out if there are problems with the intervention.
  • Documenting the intervention & collecting data. Interventions have meaning only if they are done within a larger data-based context. For example, interventions that lack baseline data, goal(s) for improvement, and a progress-monitoring plan are ‘fatally flawed’ (Witt, VanDerHeyden & Gilbertson, 2004).
Building integrity into an academic intervention plan during its design phase, then, is a proactive means to promote the integrity with which the intervention is eventually carried out. Schools can download the Academic Interventions ‘Critical Components’ Checklist (attachment at the bottom of this page) -- a guide that will help educators to construct academic interventions in a manner that increases the likelihood that they will be implemented with integrity--even in busy and under-resourced instructional settings.

References

  • Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools. Routledge: New York.
  • Gresham, F. M. (1989). Assessment of treatment integrity in school consultation & prereferral intervention. School Psychology Review, 18, 27-50.
  • Gresham, F. M., Gansle, K. A., & Noell, G. H. (1993). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26(2), 257-263.
  • Haring, N.G., Lovitt, T.C., Eaton, M.D., & Hansen, C.L. (1978). The fourth R: Research in the classroom. Columbus, OH: Charles E. Merrill Publishing Co.
  • Hawkins, R. O., Morrison, J. Q., Musti-Rao, S., & Hawkins, J. A. (2008). Treatment integrity for academic interventions in real- world settings. School Psychology Forum, 2(3), 1-15.
  • Skinner, C. H., Pappas, D. N., & Davis, K. A. (2005). Enhancing academic engagement: Providing opportunities for responding and influencing students to choose to respond. Psychology in the Schools, 42, 389-403.
  • Witt, J. C., VanDerHeyden, A. M., & Gilbertson, D. (2004). Troubleshooting behavioral interventions. A systematic process for finding and eliminating problems. School Psychology Review, 33, 363-383.