Technical review of published research on applied behaviour analysis interventions for people with Autism Spectrum Disorder
New Zealand Ministries of Education and of Health requested a technical review of the evidence base on the effectiveness of Applied Behaviour Analysis (ABA) for people with Autism Spectrum Disorders (ASD).
Released on Education Counts: April 2010
Author(s): Oliver Mudford, Neville Blampied, Katrina Phillips, Dave Harper, Mary Foster, John Church, Maree Hunt, Jane Prochnow, Dennis Rose, Angela Arnold-Saritepe, Heather Peters, Celia Lie, Katrina Jeffrey, Eric Messick, Catherine Sumpter, James McEwan and Susan Wilczynski (2009), Auckland UniServices Limited.
Date Published: 15 January 2009 - Revised 16 January 2009
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Results 8. Comprehensive Programmes: Early Intensive Behavioural Intervention
Early Intensive Behavioural Intervention is defined as comprehensive treatment programs that involve a combination of applied behavior analytic procedures (e.g., discrete trial, incidental teaching, etc.) which is delivered to young children (generally under the age of 8). These treatments may be delivered in a variety of settings (e.g., home, self-contained classroom, inclusive classroom, community), involve a low student-to-teacher ratio (e.g., 1:1). All of the studies falling into this category met the strict criteria of: (a) targeting the defining symptoms of ASD, (b) have treatment manuals, (c) providing treatment with a high degree of intensity, and (d) measuring the overall effectiveness of the program [i.e., studies that measure subcomponents of the program are listed elsewhere in this report]. These treatment programs may also be referred to as ABA programs or behavioral inclusive classrooms.” (NSP definition, Wilczynski, Personal Communication)
Studies in this category are comprehensive programmes that target behaviours that span the Ministry of Education and NSP categories of behaviours for intervention. Some programmes target general symptoms, others target many of the defining symptoms of ASD.
Evidence from NSP review
The scientific evidence reviewed by the NSP provides strong evidence for the effectiveness of these programmes. Across all years, 128 items received SMRS scores of ≥ 2.0, and 42 of these items scored 3. The NSP review found strong evidence for increases in skills in the outcomes it categorized as communication, higher cognitive functions, interpersonal, personal responsibility, and more regular school placement, and that there was emerging evidence for increases in learning readiness. The review reports that there was no established scientific evidence of effects from EIBI programmes for the play, self-regulation and academic categories, but that there was emerging evidence for decreases in behaviours classified as behaviour problems and general symptoms of ASD. The NSP review found strong scientific evidence for the effects of EIBIs for children with diagnoses of Autistic Disorder (AD) and Pervasive Developmental disorder (PDD). There was also strong scientific evidence found for children in the 3-5 year age range, and emerging evidence for ages 0-3 and 6-9 years.
In summary, the NSP review found that there was strong scientific evidence or emerging scientific evidence for the use of EIBI to increase some skills and to decrease some problem behaviours with children with diagnoses of AD or PDD in the 0-9 year old range, with the strongest evidence for increasing higher cognitive, personal responsibility, and interpersonal skills and for improving school placement for those between 3 to 5 years old. All items were classified as demonstrating either beneficial or unknown effects.
Additional evidence from New Zealand reviewers
There was no additional evidence on the effectives of early intensive behavioural interventions from New Zealand reviewers.
Evidence from studies published from 1998-2007
Among all items reviewed by NSP, 76% were published during this time. Of the 112 items examined (from twelve studies), 33 (from three studies) received SMRS scores of 3 for all targeted categories. Nineteen of these items showed beneficial effects. All other items received scientific merit rating scores of 2 for all targeted categories, including the four items from the one single-subject design study. Beneficial treatment effects were reported for 36 of the SMRS = 2 items in the NSP database and the rest were classified as having unknown treatment effects. No items were classified as showing ineffective treatments or adverse treatment effects. The majority of the studies resulted in some beneficial effects. Ten of the twelve studies had 50% or more of their effects classified as beneficial, the remaining two had all 36 of their items classified as having unknown effects (4 from one and 32 from the other study). The 1998-2207 findings can be interpreted as being similar to the 24% larger set that contributed data to the “Evidence from NSP review” summarised above.
Summary of EIBI
The effectiveness of comprehensive programmes such as EIBI is difficult to research. It is not easy to use the randomised control studies seen as desirable for such evaluations particularly not when participation is required over a number of years. It is hard to keep observers or those providing the intervention ‘blind’ as to which participant is getting which treatment. In addition the design of placebo treatments is difficult and, in fact, it may not be ethical or desirable to use a placebo once there is some evidence as to the positive effects of some aspects of the package. To carry out such studies is a very time consuming and expensive exercise. Given this, it is not surprising that there are not many studies in this area, and that none of those found received SMRS score of 4 or 5.
The research reviewed provides some strong scientific evidence for the use of EIBI in the development of some cognitive (thinking) skills, in social development and relating to others, in the development of some independent organisational skills and in increasing engagement and flexibility in some developmentally appropriate tasks and play and later engagement in some vocational activities. Strong evidence for the benefits of EIBI on the school placement variable was found. The research reviewed also provides emerging scientific evidence for increasing other skill areas. The studies show strong evidence of effectiveness with children with diagnoses of AD or PDD who are in the 0-9 year old range. Given this is early intervention, it is not surprising that the research focuses on the 0-9 year old range. The strongest evidence found of beneficial effects was for those between 3 to 5 years old in the development of cognitive (thinking) skills and in social development and relating to others. It is clear more research is needed, particularly with 0-3 and 6-9 year olds and with both these and other skills.
There appears to be enough evidence of beneficial effects of EIBI that it could be time for a study comparing EIBI with other early interventions that also have some strong evidence of effectiveness.
Sources of information for evaluating generalisation and maintenance effects in Sections 9 and 10
The degree to which generalisation (i.e., the transfer of treatment effects from target to non-target behaviours or to non-treatment settings) and maintenance (i.e., the durability of treatment effects over time, post-treatment) effects were reported by NSP reviewers was unavailable. The reason for this was that NSP did not compute generalisation and maintenance effects unless the type of behavioural intervention (Table 2) reached the “strongest evidence” criterion for multiple studies showing beneficial effects with SMRS scores of 4 or 5 (see Appendix A Table 3, left hand column). As can be determined from previous sections (1 to 8), there were no “Strongest evidence” ratings. Hence, NSP did not report on effects, except the main effects. Nevertheless, we reviewed the NSP database to determine the quality of experimental design and method for determining whether generalisation and/or maintenance had been assessed. That was achieved by examination of SMRS ratings for “Generalization of Tx (Treatment) Effect(s)” (see right hand column in Appendix A Table 1: SMRS).
Among the items in the NSP database (and original NZ-unique database), 12% (21%) objectively measured for both generalisation and maintenance, 31% (25%) objectively measured either generalisation or maintenance, 3% (6%) reported subjective generalisation and/or maintenance effects, and 53% (48%) did not have a sufficient experimental method to report on either generalisation or maintenance. A possible reason for the NZ database showing higher percentages of studies with designs enabling objective generalisation and maintenance effects to be reported was that studies published after 1997 were more likely to have assessed these effects than studies published before then.
We can report on the obtained generalisation and maintenance effects from the original NZ-unique database of 169 items. New Zealand reviewers had originally coded effects as “strong support” for generalisation or maintenance if ≥ 67% of the research item showed positive effects in other settings, or with other people, or across materials (generalisation) or at follow up (maintenance). “Limited support” was noted if effects were noted for between 50% and 66% of items.The summary of generalization and maintenance effects must be interpreted with caution since interobserver agreement for these categories of review was 68% and 64% respectively. Thus, these results should be considered indicative rather than accurate.
Where to find out more
For more information on this publication or Special Education services, please email: Special Education Mailbox