An evaluation of some programmes for children with Autistic Spectrum Disorder in Auckland: Opportunities, contingencies, and illusions Publications
This research evaluates three educational programmes in the Auckland area for children with Autistic Spectrum Disorder (ASD). The programmes evaluated were Autism Action Precision Teaching Centre (AAPTC) programme, intensive Applied Behaviour Analysis (ABA) home-based programmes, and mainstream pre-school and school education. The report describes each service, their cost and how effective the programmes were in providing education for children with ASD ending with recommendations and conclusions regarding educational programmes for children with ASD.
Author(s): Rebecca Godfrey, Dennis Moore, Claire Fletcher-Flinn and Angelika Anderson, Auckland UniServices Limited, University of Auckland.
Date Published: July 2002
The research presented in this report evaluates three educational programmes in the Auckland area for children with autistic spectrum disorders (ASD). The research was commissioned by the Ministry of Education and carried out by the Research Centre for Interventions in Teaching and Learning at the University of Auckland.
The evaluation was designed and implemented by Dr Rebecca Godfrey, Associate Professor Dennis Moore and Dr Claire Fletcher-Flinn in consultation with the Ministry of Education, the Specialist Education Services1, the Autistic Association and the Autism Action Precision Teaching Centre (AAPTC). Dr Angelika Anderson and Christina Birkin also contributed to sections of the research. The report represents the period between June 1, 2000 and September 30, 2001.
The three groups of programmes evaluated were the Autism Action Precision Teaching Centre (AAPTC) programme, intensive applied behaviour analysis (ABA) home-based programmes, and mainstream pre-school and school education.
In all, four studies were conducted. The first study (Study 1) aimed to provide the Ministry with information regarding descriptions on the types of services provided, their costs and how effective the programmes were in providing education for children with ASD. This study, which represents the original scope of the evaluation, is addressed in Part 1 of this report.
Because of limitations in the scope and method of the initial study, three further analyses of the data were undertaken: an alternative analysis of classroom learning and behaviour (Study 2); a social skills analysis of social interactions and play behaviour (Study 3); and an investigation of the stereotypic behaviour of three children (Study 4). These studies are discussed in Part 2 of this report. Part 3 presents the researchers' conclusions and recommendations regarding educational programmes for ASD children.
Autism is a disorder associated with impairments or deficits in three broad interacting zones of behaviour; namely, impairments in social interaction, impairments in communication, and restricted repetitive and stereotyped patterns of behaviour, interests and activities. These areas form the basis of the diagnostic criteria for autism, as specified in the American Psychiatric Association's Diagnostic and Statistical Manual (DSM-IV). A degree of deficit in each area must be present for a diagnosis of autism to be specified. More details on the diagnostic criteria for ASD are presented in Appendix 1.
A literature survey identified elements of best practice which were used as a framework for evaluating the effectiveness of the programmes. The best practice features identified included: early intervention; teaching language and academic skills; teaching social skills; using behavioural techniques; generalisation; dosage and intensity of treatment; assessment; decreasing maladaptive behaviour; involvement of family; integration; and an individualised programme.
Initially, the evaluation was to look at the AAPTC and a mainstream school-based centre. A group of children in mainstream settings who received either early intervention or inclusive services support from SES was also included. The mainstream school-based centre withdrew its involvement in the evaluation and, following consultation with the Ministry of Education, it was decided to include a group of children who were receiving home therapy. Thus the three groups of programmes evaluated were AAPTC, home-based programmes and mainstream settings.
At the time of the evaluation, the AAPTC programme was in its first year and was still in the process of development. The main goal of AAPTC was to implement Intensive Model (Autism) of Generative Instruction Excellence (IMAGINE) as an early intervention educational programme so that children with autism may be mainstreamed.The IMAGINE programme involved teaching fine, gross, and oral motor imitation; receptive and expressive communication; pre-reading and reading skills; pre-writing and writing; mathematics; and play skills. These skills were taught across several learning channels (i.e. see, do, say, hear, write, and touch).
School routines were practiced from the moment the children arrived at AAPTC. Parents sending their child to AAPTC also underwent training.
Special needs policies were developed by the three mainstream settings (one pre-school and two schools) involved in the evaluation. The rationale for all was that children with special needs were entitled to equal educational opportunities in the areas of physical, emotional, social, and intellectual development. While the guidelines differed somewhat for each school, they shared a common purpose which was to ensure that children with special needs had their needs met and that their teachers were supported. Furthermore, they recognised that children with special education needs required extra assistance, adapted programmes or learning environments, and specialised materials or equipment to support them in regular mainstreamed education settings.
There were three home-programmes involved in this evaluation. Each was different in its content and approach to therapy and reflected the age and level of functioning of the child. The parents determined the specific aims and objectives for each child, with therapists having varying roles in the programming details. There was no common therapist, overall supervisor, or supervision from an outside organization across the programmes. Furthermore, none of these home-programmes employed one particular programme or scripted curriculum. All the programmes aimed to increase the language, academic, social, daily living, and physical and functional play skills of the child. At least one parent or therapist from each family regularly attended a home-therapy group meeting where resources and ideas were shared.
The children who participated
A total of 11 children took part in the evaluation.Criteria for inclusion in the evaluation were: (1) a previous diagnosis of autism had been made; (2) the parents and all staff involved in the teaching of the child gave their consent; (3) the child spoke English as a first language2; (4) the child was 8 years old or younger; and, (5) for children attending the Autism Action Precision Teaching Centre or mainstream school that they were not involved in any other intensive, home-based treatment programmes.Five children (three pre-school and two school-aged) attended the AAPTC for part of their school day and a mainstream setting for the remainder. Three children (two pre-school and one school-aged) received home-based therapy, with two of them also attending a mainstream setting. The third group of three children (one pre-school and two school-aged) attended a mainstream setting only.
The children ranged from having mild-moderate to severe symptoms of autism, and their language ability varied from well-below chronological age to age appropriate. The number of hours of teacher-aide or education support worker (ESW) support varied from 50 - 100 % of the time they attended the mainstream setting.
The pre- and post-test measures conducted for each child included the Childhood Autism Rating Scale (CARS), Peabody Picture Vocabulary Test III (PPVT-III) and, when appropriate, the Burt Reading Test - New Zealand Revision and the Neale Analysis of Reading Ability Test 3rd edition.
The Interview Edition of the Vineland Adaptive Behaviour Scales (VABS) was administered to the child's primary caregiver and the Teacher Edition of VABS was given to the child's main teacher in the target setting.
Observational data was gathered by video-tape over two observation periods, on average 121 days apart, in each child's main educational setting, with typically five visits made for each child. For the children attending AAPTC or a home programme who also attended a mainstream school or pre-school, additional data was gathered from their mainstream setting over one half-day.
Using the observational data, the study analysed the time spent on various activities such as on-task behaviour; instruction from teachers, teacher-aides or education support workers (ESW); praise or feedback; child-chosen activity; and stereotypical behaviour. These are traditional methods of assessing teaching and learning which provide quantifiable but indirect measures of teaching and learning.Home visits and observations were also made in order to establish if tasks and skills mastered in one setting had generalised to other settings and people.
The study also conducted interviews with parents and teachers regarding programme costs, effectiveness, benefits and concerns.
Results of Study 1
The results indicated that children at AAPTC or in home-based programmes received significantly higher rates of instruction, praise and feedback; spent greater amounts of time on-task; and emitted no stereotypical behaviours.
In comparison, children in the mainstream received little by way of instruction, praise or feedback and spent less time on-task but more time emitting stereotypical behaviours.
The observational system used, however, imposed artificial ceilings on instances of instruction. Furthermore, the definition of on-task included those times when students were awaiting the next instruction, but not actually engaged in academic activity. Levels of `on-task' behaviour reported in Study 1 might therefore be inflated. In addition, as `on-task' was only a time-based measure of dosage, it did not adequately reflect programme intensity. Furthermore, the measures used did not capture a crucial aspect of effective instruction; that is the relationship between the instruction, the student behaviour, and the feedback given. Consequently a second, more fine-grained analysis of the data was carried out as reported in Study 2.
Other findings in Study 1 included information on the cost of the various programmes. The maximum amount of costs to parents for a child to attend AAPTC in 2000 was just over $19,000 (annual instruction and joining fees) plus an additional fee of AUS$5,000.00. Funding for support workers and teacher aides for children in mainstream settings was at variable rates and came from various sources including SES, ORRS funding and schools' special education grants. For home-based programmes, the cost of 15 hours of therapy a week for 40 weeks was between $6,000 and $15,000 a year, with parents receiving some funding support from various sources.
Study 2 used the same classroom observations as Study 1 but analysed them using the concept of the `learn unit', which provides a more direct and educationally valid measure of teaching and learning than the categories such as hours of instruction or time on-task used in the first study.
A learn unit refers to an interaction between the student and teacher, and can be defined as an A-B-C sequence of antecedent, behaviour and consequence; for example, an instruction from the teacher, the behaviour of the student in response to the instruction, and the praise or feedback resulting from that response.
A learn unit may be complete (comprise all three components in the A-B-C sequence) or incomplete (comprise just one or two components).Learn units are considered fundamental to the process of learning, and the literature demonstrates a positive relationship between the provision of complete learn units and student learning and progress. The basic premise is that learn units are present in effective instruction and not in ineffective instruction.
The concept of learn units is discussed in more detail in Section 5.0.
Results of Study 2
The results of this study confirmed the broad findings of Study 1; namely that there were many more learning opportunities provided at AAPTC and in home-programmes than in the mainstream settings.
While both AAPTC and home programmes delivered high rates of learn units, the children at AAPTC received more complete learn units whereas the children in home programmes received more two-term components (instruction and behaviour, but no feedback).
The mainstream settings were characterised by low levels of each of the components of the learn unit: instruction, behaviour and feedback. Furthermore, complete learn units were extremely rare.
The children at AAPTC or in home programmes had a clear advantage over children in mainstream settings. The rate of instruction and opportunity to respond in the mainstream settings was disappointing given the level of teacher aide and ESW support these children had.
Mainstream settings have the potential advantage of providing ASD children the opportunity to practice social skills with typically developing peers in a mainstream environment. The extent to which this occurred was explored in the third study, which focused on the social interactions and play behaviour of the target children in free-play situations in mainstream settings.
The study used video-footage of periods of free play and classroom recordings used for Study 1.
Play was categorised by using an adapted version of the Parten Scale, which defines six categories of play (ranging from unoccupied behaviour through to co-operative or organised supplement play). Social interaction was assessed using an adaptation of Ballard's (1981) system for coding social interactions which captures reciprocal interactions and sharing behaviour, distinguished between interactions with adults and interactions with peers, and codes the target child's responses to others' initiations as positive or negative.
Results of Study 3
Study 3 showed clear differences in the play behaviour and pattern of social interactions between the target children and other children in the same settings. All target children, whether pre-school or school-aged, engaged in less mature play behaviour; they had fewer social interactions and a high proportion of those interactions were with adults.
There were no discernable differences between the target groups when target children played in a mainstream setting. The academic benefits associated with home programmes and AAPTC did not appear to translate into more adaptive social behaviour.
Teacher aides and education support workers varied in the extent to which they interacted with the children in free play situations and in terms of how they interacted, and appeared unclear as to whether their role was a teaching or caretaking one. Some adults tried to facilitate target children's interaction with their peers while others compensated for the children's deficits and assisted them with tasks. The facilitative interactions were associated with increased levels of peer interaction while the compensatory interactions acted as a barrier.
Study 4 aimed to assess the reasons for the stereotypical behaviour of three children in mainstream settings.
Stereotypic behaviours consist of repetitive movements which do not appear to have any function other than providing sensory input for the individual concerned. The behaviours typically include rocking, hand flapping and the spinning of objects, but may also involve auditory feedback such as tapping or shaking of items.
The research literature suggests that stereotypic behaviour may interfere with learning, relationships and neurological development, and its suppression has resulted in spontaneous increases in appropriate play and academic behaviour.
This study undertook a functional analysis of the stereotypic behaviour of the three children, using the video-taped observations from Study 1. Functional analysis is a behavioural assessment which focuses on observable target behaviour and the observable antecedents to and consequences of the behaviour. It is a valuable tool for planning behaviour modification programmes because it provides objective data which can be used to select an effective intervention for a specific individual.
The data for each child was compared to see if there were any similarities in the situations associated with their stereotypical behaviours.
Results of Study 4
As predicted, the analysis showed differing patterns of antecedents across the three subjects. This indicates that the stereotypic behaviours serve functions specific to the individual, and that it is neither useful nor accurate to assume that these functions can be generalised to others exhibiting similar behaviours.
This supports the need for individual programming for children with ASD, as is suggested by the literature.
Study 4 concluded by suggesting that though stereotypical behaviour was not necessarily or inherently detrimental in itself, it needed to be controlled in educational settings so as to maximise learning opportunities. It highlighted the need for more and better training for people working with children with ASD, in order that they can devise effective programmes for the management of stereotypic behaviour.
Evaluating Programme Effectiveness
Because not all data sought was available and because the majority of the participants in this evaluation were involved in more than one programme, it was not possible to make direct judgements about programme effectiveness by outcome measures such as academic achievement or progress made.
In terms of assessing programme effectiveness in relation to known parameters of effective teaching practice generally and specifically for children with ASD, all programmes had some strengths and weaknesses. However, the research shows children in the mainstream were receiving a relatively ineffective education in comparison with children at the AAPTC or in home-based programmes.
All programmes specified learning objectives on some level and assessed learning in some way. Both AAPTC and home programmes provided an educational plan that attempted to meet the needs of children with ASD by incorporating many of the characteristics outlined as best practice in the literature. The educational programmes provided in mainstream settings on the other hand, contained few of these features.AAPTC and home-programmes offered more than mainstream settings did in terms of assessing the child's level of functioning at entry, providing more highly structured learning activities at the child's level, and providing corrective feedback and reinforcement for child performance.
Although the home-programmes targeted social behaviour and play, there was no discernable difference on these dimensions between the children who attended the home programmes and the others. It could be that it was not adequately taught, or, more likely that generalisation of acquired skills was not facilitated in the mainstream settings.
Similarly, all programmes took some measures to control undesirable behaviour, but in the mainstream settings there was a lot more evidence of undesirable behaviours occurring than in the other settings.
Even though all programmes provided a high teacher-student ratio, the mainstream programmes provided little instruction and feedback, and children here spent little time actively engaged in academic responding. There was a lot of waiting time, which might contribute to the high occurrence of stereotypic behaviour in mainstream settings.
In sum, no one programme provided the perfect plan. However, the research suggests the mainstream settings do not adequately meet the needs of children with ASD and are not in line with approaches shown to be effective for these children.
Issues for the education of children with ASD in New Zealand
The research literature reviewed at the beginning of this report show the most effective treatment for children with ASD appears to be a programme which incorporates 1-to-1, direct instruction using behavioural techniques, begins rapidly upon diagnosis, and teaches a comprehensive curriculum including academic, daily living, language, play, and social skills. Children involved in such programmes have been reported to make substantial, lasting progress with the best gains recorded by children who start intervention programmes at 2 or 3 years of age. The likely outcome for most children with ASD who do not receive early, intensive, behavioural intervention is a need for lifelong specialised services. Therefore, a front-end investment has high potential for beneficial outcomes long-term, both in terms of financial savings to the government and in terms of human costs.
Currently in New Zealand there appears to be a lack in clear directives and promotion of good practice regarding the education of children with ASD. Some effective services are available but to access them parents need information about them, and they need to be able to meet their cost. The large financial costs associated with both AAPTC and home-programmes are draining for parents who choose these options and appear exclusive to those who cannot afford them. The support provided currently by the government, though often substantial in financial terms, is largely ineffective.
A crucial aspect of best-practice for children with ASD is that the treatment programme be delivered by individuals with extensive training in the methods of applied behaviour analysis. Provision of intervention programmes in the mainstream by ineffective teachers using ineffective methods is a waste of resources and prevents the child accessing more effective programmes.Current in-service training provided by SES does not appear to be equipping education staff with effective tools for teaching children with ASD. An obvious first step to incorporating best practices into the mainstream environment would be to provide professional development for teachers, teacher-aides and education support workers.
The effectiveness of programmes could be improved by increasing collaboration and coordination between the different programmes any particular child is involved in. Improved communications across settings would allow skills mastered by a child in one setting to be the target for generalisation programming in another. Improved communications would also enable the opportunities provided by mainstream settings to be used for developing adaptive social functioning and interactions with typically developing peers.
This evaluation does not identify one or other of the three programmes as providing the `solution' to the educational needs of children with ASD. The data presented in this report go beyond this - they identify a number of key issues regarding both how to measure the effectiveness of the programmes and how to incorporate features deemed `best-practice' into educational programmes for children with ASD.
The international literature suggests that much can be done to help children with ASD develop and learn skills that will enable them to live independent, and in some cases normal lives. To achieve best outcomes, interventions need to begin early, be of high intensity for a number of years and involve the child's family. The kinds of interventions that work are behavioural programmes characterised by a high density of direct instruction, and a high degree of structure, consistency, and predictability. Maintenance and generalisation need to be programmed and facilitated.
Currently in New Zealand there is no one coherent programme that offers an optimal service for children with ASD. There are some good programmes available at a high cost to families. This cost makes obtaining a sufficient level of effective services almost impossible for most families.
The effectiveness of these programmes is further compromised by inadequate provision for generalisation of skills. Effective generalisation programming requires a high degree of communication and collaboration across settings.
The state-funded provision for these children in the mainstream (teacher-aides and education support workers) appears inconsistent and lacks clear focus. The level of provision (number of hours provided) did not always reflect the level of functioning of the child as much as reflecting the parent's ability to secure funding. It is unclear if the purpose of the support is to facilitate learning and the development of independent functioning of children with ASD in mainstream settings, or if it is to help children with ASD, compensating for their deficits so that the running of the classroom is not disrupted by their presence.
To improve the current situation requires a number of changes. These include:
- A conceptual shift that moves from a compensatory system to a facilitative one.
- A funding structure which reflects the conceptual shift, so that effective provision is not punished by an immediate reduction in funds.
- Further research into an optimal programme of intensive early intervention for children with ASD.
- Training programmes for teacher aides, support workers and teachers in the mainstream.
- Clear information and guidelines making the right kind of services available to all families.
In sum, despite the small number of children involved in this evaluation and the relatively short time-frame of the project, some interesting observations were made from which a number of important messages emerged. There is much that can be done for children with ASD and their families. Investment in the development and implementation of effective early intervention programmes promises long-term benefits to individuals with ASD, and is also likely to impact positively on general education in New Zealand.
- This report refers to Specialist Education Services throughout. However, since the initiation of the evaluation, SES has been disbanded and is now part of the Ministry of Education and referred to as Group Special Education.
- Note that, one child, Fred, was in an environment where two languages were spoken to him, but he was the only mainstream pre-school child in the evaluation so he was not excluded.
Education Data Requests
If you have any questions about education data please contact us:
Email: Requests EDK
Phone: +64 4 463 8065