2015 Annual Evaluation Report for the Teach First NZ programme pilot
Delivered in partnership with the University of Auckland
This is the third annual evaluation report of the Teach First NZ programme pilot, delivered in partnership with the University of Auckland. It confirms that the Teach First NZ programme continues to be effectively and efficiently implemented. Teach First NZ and the University of Auckland’s Faculty of Education continue to find ways to strengthen the programme and to ensure it is well known and well supported. Participants are strong ambassadors for the programme, including the mission of reducing educational inequalities. Almost all participants have made a valued contribution in their school, have supported their students to engage and progress, and intend to stay in teaching, at least in the short-term.
Author(s): Jo MacDonald, Jenny Whatman and Liesje Stevens, New Zealand Council for Educational Research. Report prepared for the Ministry of Education.
Date Published: August 2016
The Teach First NZ pilot programme is an alternative field-based Initial Teacher Education (ITE) programme.1 The pilot programme operates between 2013 and 2016 with three annual intakes of up to 20 participants. The third and final intake included in this evaluation is Cohort 15 (beginning at the start of the 2015 school year).
The New Zealand Council for Educational Research (NZCER) is undertaking a 4-year evaluation of the Teach First NZ pilot programme. The first report focused on the programme's first year, Cohort 13.2 The second report focused on Year 2 for Cohort 13 and Year 1 for Cohort 14.3 This third report focuses on Year 2 for Cohort 14 and Year 1 for Cohort 15, and begins to look at the programme's alumni pathways. It also makes some comparisons across all 3 years of the evaluation. The key evaluation questions are:
- How well (effectively and efficiently) has the programme been implemented?
- To what extent has the programme achieved its overall outcomes and objectives?
The evaluation this year has confirmed that key success elements of the Teach First NZ programme are:
- the robust selection process resulting in high-calibre participants
- the responsiveness of the programme, in part made possible because of its small size, but also as a consequence of the robust partnership between Teach First NZ and the University of Auckland's Faculty of Education (hereafter referred to as the Teach First NZ partnership)
- effective support and mentoring for participants from schools and Teach First NZ partnership personnel
- immersion in the classroom, coupled with opportunities for participants' critical reflection on themselves and their teaching.
The successful implementation of all of these elements is critical for an employment-based ITE programme. While there is variability in the way these elements play out, particularly in relation to mentoring and in-school support, and participants' match with the school, in combination they provide a powerful platform for this model of an alternative pathway into secondary teaching.
In 2015 we collected data from four main sources:
- programme and participant documentation
- interviews with participants, key personnel in Cohort 15 schools and with the Teach First NZ partnership
- online surveys for Cohort 14 participants, their mentors and co-ordinators
- online survey (Me and My Class, NZCER) for students.
How well (effectively and efficiently) has the programme been implemented?
We focused our attention on evaluative criteria established at the start of the evaluation (see Appendix 1), to assess the extent to which the programme has been effectively implemented in its third year. These criteria pertained to the participants themselves (their recruitment and selection, and their intentions after the pilot programme); programme factors managed by the Teach First NZ partnership (such as the Summer Initial Intensive, termly clinics and workshops, Faculty papers, and support provided by the Teach First NZ partnership); and school factors (support for participants provided by schools, in particular by mentors). We also evaluated the viability4 of the pilot programme.
In this third annual evaluation report we continue to report that the programme has been effectively and efficiently implemented. Points we have noted in the past continue to be important. These include the selection of high-calibre—and resilient—participants; strong mentor support with time allowance; and a responsive and relevant programme with clear communication between the Teach First NZ partnership personnel and those who provide school-based support.
This year it was notable that more participants had a mentor from a department different from their own. More schools appear to be selecting mentors who can build relationships and provide strong pedagogical support, and where those mentors teach a different subject from the participant, participants are getting curriculum and subject-specific support in other ways (for example, from heads of department (HoD), visiting teaching specialists (VTS) and learning area specialists (LAS)). This year we conclude that, overall, mentoring support is more consistently strong than in previous years. The co-ordinator role also appears to have been strengthened with co-ordinators and mentors working well together.A well-functioning department continues to be an important contributor to participant success. There still appears to be most variability for participants teaching te reo Māori, some of whom have very limited departmental support and may even be expected to take on responsibility for the department, and others who are well-supported within a strong department. This year, the importance of appropriate school placement was raised more often by interviewees. Some were concerned about schools that were isolated, and found it challenging to provide the level of support required, or to expose participants to good teaching. With this variability in school and departmental support there appears to be greater recognition from the Teach First NZ partnership that participants need different types and levels of support.
Teach First NZ partnership personnel, including VTS and LAS, have stepped in to provide additional support when required while acknowledging that participants themselves need to find ways to manage challenging situations.
One aspect of quality we judged was the extent to which the pilot programme attracted high-quality applicants, some of whom may not otherwise have undertaken teaching at this time (particularly in schools serving lower socioeconomic communities).
As in 2013 and 2014, the programme was successful in attracting high-quality applicants. Twenty participants (8.5 percent of those who filled in the online application) were selected for the 2015 cohort. Almost a third (six participants) of the 2015 intake are Māori or Pasifika and eight (40 percent) are male. Eight majored in English,5 four in te reo Māori, three in mathematics, and five in chemistry or physics. Ten have postgraduate qualifications, including Masters or Doctorate degrees. The evaluation team talked to different groups of people and, as a result, perceive Cohort 15 participants to be of high calibre. Under half of Cohort 15 had considered teaching prior to hearing about Teach First NZ.
Completion and retention rates
Completion rates for the 2-year programme for Cohort 13 and Cohort 14 have exceeded the expectation of 90 percent. The majority of participants are retained in teaching in New Zealand secondary schools after they complete the programme.
Cohort 13: In 2015, 13 (of the 15 participants) have been employed in New Zealand secondary schools. In 2016 all 15 will be employed in teaching, 14 of them in New Zealand. Five are teaching in decile 1–3 schools, and nine in decile 4–9 schools.
Cohort 14: At the end of the second year of the programme, 19 of the 20 participants remain, although two have been on parental leave. As at March 2016,6 15 have been appointed to permanent positions in New Zealand secondary schools. Nine of these are with their original host school (deciles 1–4), four in other decile 1–4 schools, and two at decile 7 schools. At the end of the programme one is going into full-time study, one will be overseas, and another will be doing relief teaching.
Cohort 15: At the end of the first year of the programme, 17 of the 20 participants remain. The 17 remaining expect to complete the programme and stay in teaching, at least to gain full certification. Many have plans to stay in teaching much longer than that.
Participant connections with the Teach First NZ community
This year we were particularly interested in the connections at different layers of the Teach First community. Interviews and surveys confirmed that participants' connections are strongest with their own cohort, followed by other cohorts, the wider Teach First NZ partnership community, and finally the global network of Teach For All. All participants from Cohort 14 and Cohort 15 were connected with their cohort, about half had strong connections with other cohorts and the partnership, and only a few are strongly connected with the global network of Teach For All. These data were collected before the Teach For All Global Conference was held in Auckland in October 2015.
The programme: Teach First NZ and faculty components
The taught programme
Participants continue to feel well prepared by the SII. Areas where participants would like more preparation are each raised by just a few people. One area we highlight is more opportunities to learn about Pasifika education in preparation for teaching in schools with large numbers of Pasifika students.
As in previous years, participants from both cohorts commented on the relevance of their university papers and assignments as a strength of the programme. All Cohort 14 participants found the coursework useful, and most found it appropriately challenging. Overall, Cohort 14 participants were more positive than Cohort 13 participants about the usefulness of different components of the programme.
All Cohort 15 participants commented positively on aspects of the taught programme, and many commented on multiple aspects: usefulness of clinics, relevance of assignments, and time with curriculum specialists. Participants highlighted the value of ongoing course work and assignments as a way of connecting theoretical understanding with the practicalities of working in the classroom. Improvements suggested by Cohort 15 were seen by them as minor adjustments, and reflected individual participant preferences.
As in earlier years, the vast majority of participants from both cohorts reported that they found visits and feedback from both their VTS and LAS useful. The provision of resources, curriculum knowledge and subject-specific strategies were particularly important for participants whose school mentors taught in another curriculum area.
The Teach First NZ partnership has actively sought feedback on how the programme could be improved and continues to reflect on and make changes to the programme.
Due to changes in the timing and implementation of the 'away practicum', we have not given affiliate schools a strong focus in the evaluation.Most Cohort 13 participants completed their 3-week 'away practicum' in the affiliate schools in Term 4 2014, after senior students left (four completed it earlier than that). Teach First NZ described these practicums as "positive and productive", but also reported that the required 3-week block (an Education Council requirement) was very challenging for host schools because of the length of time participants must be released for. Most Cohort 14 participants undertook their 'away practicum' in Term 4 2015.
The programme: school components
Support for participants from host schools
Nearly all Cohort 15 participants reported positively on the way their school community had responded to them as a Teach First NZ participant. All but one Cohort 14 participant told us that they felt fully accepted as a staff member at their school.
This year, the importance of appropriate school placement was raised more often by interviewees discussing Cohort 15. Some were concerned about schools that found it challenging to provide the level of support required, were isolated, and could not expose participants to good teaching. Timetabling and assignment of participants to classes that were likely to respond well to them appeared to be less of an issue in 2015.
Mentor support to participants
Mentoring is a critical component of the programme and ineffective mentoring can impact negatively on participants' learning and, therefore, on the overall quality of the programme. Through mentor and participant interviews and surveys, we reviewed how mentors were selected; perspectives on their role; and the extent to which they provided regular high-quality observation, mentoring and feedback to participants, and helped participants to become part of the wider school community. This year we report that, overall, mentor support is strong, and appears less variable than in previous years.
More Cohort 14 participants have had the same mentor for years 1 and 2 of the programme than we reported for Cohort 13. There are a number of possible reasons for this, but one may be that Cohort 14 schools have been able to make good matches between mentors and participants.
Schools continue to weigh up a number of factors before assigning a mentor to a participant. For Cohort 15 it is notable that more participants were in a department different from their mentor. On the whole this was working well, with mentors providing core support supplemented by others providing more subject-specific support.
As in previous years, the frequency of classroom observations (and mentors' individual approaches to them) varied, as did the extent and nature of the feedback given. Participants valued mentors' pedagogical knowledge, help with pastoral support, feedback from observations, and help with student behaviour management. Many of the mentors commented on their own learning from the experience.
Preparation and support for host schools' roles
Most mentors and co-ordinators considered they were well supported by the Teach First NZ partnership in terms of initial preparation and ongoing support. Clarity and timing of communications continued to be an issue for a few mentors and co-ordinators.
All mentors and co-ordinators felt valued by programme participants, and nearly all felt valued by others in their schools.
Viability of the programme
We considered that the programme would be viable if the Teach First NZ model can successfully adjust to larger numbers of participants within agreed funding arrangements.
In 2013 there were nine schools in the programme, in 2014 there were 16, and in 2015 there are 20. Seven schools joined the programme for the first time in 2015, six decile 1 and one decile 2. We judge that the programme has successfully adjusted to the larger number of schools and participants, although some schools found it harder in 2015 to provide the level of support participants needed.
While most of the schools in the programme are positive about ongoing involvement, some are likely to limit this involvement to one cohort at a time. This could have implications for the proposed expansion of the programme. Reasons for this include the level of support required (the demand for and on mentors), falling rolls, and hopes to be able to retain participants at the end of the 2-year programme, having to do so within staffing entitlements.
There does not appear to have been an issue for 2015 Auckland schools in funding Cohort 14 and Cohort 15 (who, unlike Cohort 13, are not supernumerary), although funding continues to be raised by a few principals as a potential barrier to participation. The Ministry funded the four Northland participants in Cohort 15 as supernumerary, which allowed those schools to accept participants.
We will report more on alumni pathways in the final evaluation report in 2016. This year, we report that nine Cohort 14 participants have been employed in their host school as PCTs in 2016,7 with a further four in other decile 1–4 schools. Two have been employed in decile 7 schools. Cohort 13 participants are now entering their PCT2 year. Nearly all (14 out of 15) will be teaching in New Zealand secondary schools in 2016, five in decile 1–3 and nine in decile 4–9 schools. (Information accurate at March 2016.)
To what extent has the programme achieved its overall outcomes and objectives?
To answer the second evaluation question, we focused on: the effectiveness of participants' teaching, their level of support for the pastoral life of the school, the leadership development strand of the programme, the ongoing involvement and/or retention of participants (see earlier in this summary), and programme impact on quality of teaching and learning in participating schools and the status of teaching. Based on the data from the Me and My Class survey we considered that most students in participants' classes were engaged in their learning and that they were not disadvantaged by being in participants' classes, in the first or second year of participants' teaching.
Effectiveness of participants' teaching
Informed professional judgement was one of the key measures we used to ascertain the effectiveness of participants' teaching. We asked co-ordinators and mentors, and triangulated their considerable experience and expertise with the judgements made by the participants themselves. Judgements included evidence of student engagement and academic progress and achievement.Participants are valued by their schools for their confidence, ability to take responsibility for leading change and a "sense of maturity" about their practice. Almost all co-ordinators and principals considered that students responded
'very positively' or 'positively' to the participants with many commenting on the impact participants were having with students, both academically and in a pastoral role. Some mentors considered a few participants lacked pedagogical and subject knowledge, and, for Cohort 15, behaviour management strategies. Cohort 15 participants rated lesson planning, behaviour management and assessment knowledge as their most frequent challenges.
Everyone surveyed about Cohort 14 considered that participants had made considerable progress and were more confident, had stronger relationships with students, and were playing a stronger role in their department and often in the school than they had in their first year of teaching. Cohort 15 participants were also felt to be making progress, with almost all of these at least making 'expected' progress.
Support by participants for pastoral life of school
Participants reported being involved in a variety of aspects of school life allowing them to strengthen relationships with students from their own classes and throughout the school. Cohort 15 participants tended to support rather than lead these types of activities. One interesting difference from previous years was the high number of participants from both cohorts who were involved in running or assisting with dance and drama activities.
Leadership development strand
All but three of the Cohort 14 participants took on significant leadership roles in 2015. Participants attributed their leadership roles to their Teach First NZ leadership project, the school giving them responsibility, encouragement from their mentor or HoD, and their own knowledge and initiative. Unlike Cohort 13, most participants' leadership projects related to curriculum and departmental leadership where participants' in-depth knowledge and skills in e-learning, te reo Māori, or aspects of mathematics or English were highly valued.
Programme impact on quality of teaching and learning in participating schools and the status of teaching
All Cohort 14 co-ordinators and principals considered that participants had had either a "high positive impact" or "some positive impact" on teaching and learning in the school. Almost all school personnel in Cohort 15 schools noted the contribution participants made to different activities in the school, their department and in staff meetings, and their willingness to share new ideas. All co-ordinators thought that Teach First NZ has had a 'very positive' or 'positive' impact on the perceived status of teaching as a "competitive profession attracting top graduates and talented individuals".8 Mentors were more muted in their responses with a small number identifying no impact or a negative impact on the status of teaching as a competitive profession. All but one participant thought Teach First NZ has had a 'very positive' or 'positive' impact on the perceived status of teaching as a competitive profession.
- A full description of the Teach First NZ pilot programme can be found in previous evaluation reports.
- View the report: 2013 Annual Evaluation Report for the Teach First NZ Pilot Programme Delivered in Partnership with the University of Auckland. on Education Counts website.
- View the report: 2014 Annual Evaluation Report for the Teach First NZ Pilot Programme Delivered in Partnership with the University of Auckland. on Education Counts website.
- We considered that the programme would be viable if the Teach First NZ model successfully adjusted to larger numbers of participants within agreed funding arrangements. Note that Cohort 15 participants in Northland schools were funded by the Ministry as supernumeracy. This is additional funding added to the Ministry contribution for Cohorts 13 and 14.
- One English-major participant left the programme in the first term; one te reo Māori-major participant left the programme in Term 3; one mathematics-major participant left the programme in Term 3, after the evaluation team had completed fieldwork.
- Previous annual evaluation reports have presented destination data as at the end of the school year, usually November. This year we have updated the information so it is accurate at March 2016.
- For Cohort 14, this includes two alumni teachign in a decile 4 school.
- This is a Teach First NZ's term.
Where to find out more
Education Data Requests
If you have any questions about education data then please contact us at:
Email: Requests EDK
Phone: +64 4 463 8065