Executive Summary

Background to this report

This report provides the findings of an evaluation of a nationwide introductory professional development programme centred on the role of teacher aides/kaiāwhina working with students with special educational needs, which was funded by the Ministry of Education. The evaluation was undertaken from mid 2001, to the end of 2002. The evaluation in 2001 was formative, designed to contribute to the development of the programme and its companion resources. Our findings were reported to the Ministry of Education to inform decisions on the programme before it began. These findings are summarised in the introduction to this report.

The programme was undertaken with around a third of New Zealand schools, two-thirds of kura kaupapa Māori, and around a third of the teacher aides/kaiāwhina in 2002. It started with whole-school workshops of several hours, using a video developed by a collaboration between the 4 provider institutions as a spur for discussion and identification of school strengths and areas for future development. Around a quarter of the schools requested additional follow-on sessions and support from the providers. Separate workshops for teacher aides/kaiāwhina were held over 2 sessions. The programme ended with a facilitation session, attended by school SENCOs and principals, going through the print and video resources, and suggesting uses and activities to accompany that use. The providers also distributed the resources to all schools, including those that did not take part in the professional development programme, with some using the opportunity to have useful discussions about schools' processes and approaches to meeting the needs of students with special needs.

In 2002, the NZCER evaluation focused on the impact of the programme for people in the schools which participated, using "before and after" surveys of schools and teacher aides/kaiāwhina, case studies of 12 English-medium schools, and discussions with staff at 4 kura kaupapa Māori, and with the Māori facilitators who worked on the programme.

The overall focus of the NZCER evaluation was to assess how well this professional development programme met its aims to:

1. increase the knowledge and skills of teacher aides/kaiāwhina so that they can optimise the support and assistance they provide to teachers in their work with students who have special learning needs; and
2. increase school knowledge and strategies for supporting teacher aides/kaiāwhina as team members responsible for assisting teachers in the planning, delivery, and evaluation of the learning of students with special needs.

Through the evaluation, the Ministry of Education wished to get a clearer picture of:

• how well/effectively the teacher aides/kaiāwhina introductory professional development programme was delivered (including an assessment of barriers and keys to success for delivering and receiving the programme);
• whether the teacher aides/kaiāwhina introductory professional development programme engaged its target audience, and what aspects of the programme and the presentation of the programme schools and teacher aides/kaiāwhina found most effective;
• what impact the professional development programme had on school understanding of the roles of teacher aides/kaiāwhina;
• what impact the professional development and continuing access to Kia Tūtangata Ai/Supporting Learning (the resource) has on school practices, and the role and practice of teacher aides/kaiāwhina;
• any differences in engagement and impact, in relation to the way the programme was used in professional development, its accessibility to schools, any updating of the programme over the period of research, school type, and school organisation, including the extent of inclusion of students with special educational needs; and
• ways to improve this kind of professional development provided in the future.

We summarise the findings related to each research question below.

Was there an increase in the knowledge and skills of teacher aides/kaiāwhina?

The teacher aides/kaiāwhina workshops did result in an increase in their knowledge and skills, particularly in relation to an understanding of their role. Around 40-50 percent of the teacher aides/kaiāwhina who participated in the workshops said they had gained a lot on at least a third of the topics covered in their workshops. The topics with the most participants recording a lot of gain from their workshops were: roles and responsibilities, maintaining confidentiality, individual education plans, strategies to support learning, fostering friendships, effective communication, and behaviour.

Gains were highest in terms of understanding a topic (average of 40 percent for all topics), with similar averages for confidence and strategies learnt (35 and 33 percent respectively). Gains in knowledge and skills were most likely for those who had identified a need for a given topic - but they also occurred for those who had not seen a topic as necessary or relevant to them at the start of the workshop.

Most of the examples of new knowledge which came through the case studies were related to ways of supporting students' positive behaviour and engagement in learning, including stepping-back from doing things for the students in ways which made it harder for the students to learn or experience independence.

Some also gained the knowledge that the Ministry of Education expected teachers rather than teacher aides/kaiāwhina to take responsibility for the programmes of students with special needs, and to work collaboratively with teacher aides/kaiāwhina rather than leaving them to work alone with individual students. Where schools had also used the professional development to improve their systems and practice, this knowledge could be used by the teacher aides/kaiāwhina. But if teachers they worked with did not change their practice, and school systems did not change, the teacher aides/kaiāwhina found it difficult to use the knowledge they had gained.

Was there an increase in school knowledge and strategies for supporting teacher aides/kaiāwhina as team members?

The school surveys show some increase in school knowledge and strategies for supporting teacher aides/kaiāwhina as team members, with between 10-17 percent of schools indicating some progress in terms of teachers and teacher aides/kaiāwhina meeting regularly and working together, and 22 percent indicating improvements in their employment conditions, and appraisal to give them feedback on their work. About a third of the school respondents said their school did not change their practice because staff felt things were already working well.

Some increase in school knowledge and strategies to support teacher aides/kaiāwhina as team members was evident in the case study schools that were open to improving their practice, and prepared to put some resources into ensuring that staff could work together.

How well was this professional development delivered?

The case study schools' experience of the professional development was variable, and very dependent on the quality of the facilitator, their knowledge of their subject and of how schools work, and their confidence and willingness to respond usefully to the particular issues raised by school staff in both the whole-school workshops and the teacher aides/kaiāwhina workshops.

Providers were also aware of the importance of the knowledge and quality of the facilitators delivering the professional development, and because of the difficulties in finding such people nationwide at similar times, were aware of variable quality, even though they provided training and support.

Did this professional development programme engage its target audience?

Thirty-one percent of New Zealand schools took part in the introductory whole-school staff workshops. This was a lower proportion than expected. Likely reasons include: existing school professional development commitments, given that this programme was announced when many schools had already decided on their professional development programme for 2002; some schools were already engaged in professional development about the role of teacher aides/kaiāwhina with other providers; some schools thought the professional development would be offered in subsequent years; and the continuing reluctance of schools or school management to give special education priority in their professional development.

Two thousand, five hundred and sixty five teacher aides/kaiāwhina took part in the workshops - probably around a third of all teacher aides/kaiāwhina.

There was a lower than expected take-up rate for the follow-up in-school support which was offered, perhaps because many of the priority areas identified for further work in the initial whole-school workshops were areas which the schools could activate themselves. However, respondents from schools which had taken part in the teacher aides/kaiāwhina workshops and the in-school support sessions were more likely to report that the professional development had had a large, positive impact in their school.

Evidence from the case studies and providers indicates that the whole-school workshops did not always engage all school staff, particularly in secondary schools, where some of those who most needed to be there (principals and teachers working with teacher aides/kaiāwhina) were not present, indicating a school lack of priority to the subject of this professional development. However, those who did attend found value in the workshops, particularly in bringing people together who did not have time in their daily rounds to share their experiences and ideas for how their work together for students with special needs could be improved.

The providers reported greater interest in the print and video resources where they delivered them personally to schools and kura kaupapa Māori; they noted that the print resource was, however, not reader or copier-friendly. Some schools reported intentions to use these for the induction of new teacher aides/kaiāwhina.

Were there any differences in engagement and impact?

There was a higher-take up rate of the introductory staff workshops in secondary, special, small town, low decile, and large schools. This is consistent with the ACNielsen survey data from 2001, which showed that more teacher aides/kaiāwhina were employed in these schools. From the providers' reflections and the case study material, it would seem that schools which did not give priority to meeting the needs of students with special needs were less likely to engage in the professional development, as were schools where material about the professional development did not reach staff who were most keen to have the professional development.

School interest in meeting the needs of students with special needs was also a large factor in the impact of the professional development; the clarity of school organisation and systems (itself a focus of the professional development), was another. There were no school features (e.g. size or type) that were related to the perceived impact of the professional development.

Teacher aides/kaiāwhina who were highly experienced were less likely to get as much out of the teacher aides/kaiāwhina workshops as others, but even here there were some descriptions of changes in approach and understanding. It was difficult for teacher aides/kaiāwhina who did gain a deeper understanding of their role in working with teachers, and of teachers having prime responsibility for student programmes, when they returned to schools where they were expected to continue to take that responsibility themselves, with what they could see was insufficient support.

School representatives whose schools had taken part in the teacher aides/kaiāwhina workshops and follow-up work were more likely to report that the professional development had had a large impact on their school practice.

Are there ways to improve this kind of professional development?

Each of the components of this professional development had some value; schools that participated in all of them probably gained most. The whole-school workshops, while not always engaging all who needed to be engaged if schools were to develop or change their practice, did allow the identification of needs, and an affirmation that the teacher aides/kaiāwhina were part of a team, and that a team approach and systems were needed. We cannot tell from this evaluation whether this participation did sow seeds which were later able to be taken up by school staff, particularly SENCOs and principals, and RTLBs.

One theme which was identified in the different parts of this evaluation was that local expertise and knowledge could have been used and built on. The programme has left many teacher aides/kaiāwhina with a desire for more professional development, and this could be offered locally if there was some co-ordination.

Another is that it is important to customise a national programme where possible, so that it engages school staff, and leads them to see how they can make changes in their own particular contexts. There was flexibility in this programme to do so. However, either a longer lead-in time to develop resources, or the use of specialist expertise in the making of videos is necessary in offering a national programme, to avoid the very late notification of this programme, which led to a lower take-up rate than planned or desirable.

The quality of the professional development was generally high, but school staff were disappointed in some facilitators. The providers did experience some issues in recruiting and retaining their teams. The experience of the professional development programme has raised issues about the development and retention of expertise and skills, particularly in relation to work with kura kaupapa Māori. There are some ongoing questions about the sustainability of expertise in special education professional development, if it is dependent on periodic contracts.

This professional development succeeded in showing the roles and responsibilities of teachers, to the extent that school staff queried why they had not also been targeted by the programme.

The weight of schools' existing beliefs about how best to work with students with special needs, and their willingness to give this work priority, was a strong factor in both engagement and impact. This suggests that it is important to include working with students with special needs in all curriculum areas, e.g. literacy, numeracy, science, the arts, so that this knowledge can reach school managers and teachers who would not otherwise take part. It would also make sense to include teacher aides/kaiāwhina in curriculum-related professional development. This would reinforce their role, and give them further knowledge and skills.