Executive Summary

The main research question for this evaluation is How well and in what ways does Te Kotahitanga work towards the goal of improving Māori student achievement? To address this question, the evaluation was designed specifically to answer the following sub-questions:

• What is the quality of the overall design, content and implementation of Te Kotahitanga?
• How valuable are the outcomes for the teachers who participate—what new knowledge, understandings and skills do they develop, and how valuable are these learnings?
• How valuable are the outcomes for Māori students, and what is the impact on other classmates/peers?
• How valuable are the outcomes for whānau?
• How beneficial (or detrimental) are the effects of Te Kotahitanga on school culture (covering any changes in formal systems and policies; informal practices, or "the way we do things around here"; and underlying beliefs, values, assumptions and attitudes)?
• What are the enablers and barriers for getting Te Kotahitanga to work most effectively?
• To what extent is Te Kotahitanga likely to work effectively in other settings and contexts? How sustainable is the initiative likely to be when ministry investment of resources is scaled back?
• What are the most critical factors in improving teacher efficacy?

To address these questions, our evaluation team of Māori and non-Māori researchers carried out school visits at 22 Phase 3 and Phase 4 schools participating in Te Kotahitanga. We observed classrooms and professional development (PD) components, and we interviewed school leaders, teachers, project facilitators, whānau, students and board of trustees (BOT) chairs. Hundreds of interviews with participants and classroom observations were analysed using quantitative and qualitative approaches. We also reviewed school and Ministry of Education reports and other documents, and we reviewed student achievement data and information on other student outcomes from a variety of sources including official records in the Ministry of Education and New Zealand Qualifications Authority (NZQA).

This executive summary first summarises the evidence addressing each of the evaluation sub-questions to answer the main research question of How well and in what ways does Te Kotahitanga work towards the goal of improving Māori student achievement? Next, key findings are highlighted with respect to Te Kotahitanga as a model for teacher professional development and its impact on teachers' classroom practice, students and schools, reported in more detail in the body of the report. Key findings are also presented on the issue of sustainability of Te Kotahitanga as a school-based teacher professional development programme designed to enhance Māori student outcomes including achievement in mainstream secondary schools. Finally, we include a set of recommendations emerging from the evaluation that have the potential to enhance the effectiveness and sustainability of Te Kotahitanga towards improving Māori student achievement outcomes.

Summary of Evidence for the Evaluation Questions

To answer the main research question for this evaluation How well and in what ways does Te Kotahitanga work towards the goal of improving Māori student achievement?, the evidence with respect to each of the eight sub-questions is summarised:

What is the quality of the overall design, content and implementation of Te Kotahitanga?

Across the board and with very few exceptions, teachers, principals, Boards of Trustees chairs, and facilitators were most enthusiastic about the Te Kotahitanga professional development model, viewing it as a sound and effective process for improving classroom teaching for Māori students. Teachers valued the interconnected parts of the model, voicing most enthusiasm for the classroom observations with feedback which they saw as not only improving their teaching but also improving their ability to reflect on their teaching. Components of the model working well were facilitator observations with feedback to teachers and co-construction meetings; shadow coaching did not appear to be well understood or used according to the model. Teachers, principals and other school leaders affirmed that their own expectations had been raised for and relationships improved with Māori students, and they attributed this shift to Te Kotahitanga. Teachers were less clear about development of Māori culturally grounded identity as an educational outcome for students.

The Te Kotahitanga professional development model is associated with improved classroom teaching. In comparison with classrooms where Te Kotahitanga was not yet implemented, classroom observation results indicate that the majority of teachers (approximately 75%) evidenced either moderate or high implementation according to criteria based on the Effective Teaching Profile (ETP). More than one in five teachers demonstrated a high level of implementation of the ETP in Year 9-10 classrooms across subjects and schools. At the same time, observational data indicate variability across subjects and schools in the quality of implementation. On average, one in four teachers had not mastered sufficiently key dimensions of the ETP; in these classrooms, PD needs extend beyond those Te Kotahitanga was designed to address.

How valuable are the outcomes for the teachers who participate—what new knowledge, understandings and skills do they develop, and how valuable are these learnings?

Interview analysis confirmed that teachers valued relationship-based pedagogies. The majority of teachers affirmed that Te Kotahitanga professional development had an impact on classroom instruction leading to enhanced outcomes for Māori students as well as for all students. Most teachers were able to highlight particular teaching strategies and methods in their subjects introduced by Te Kotahitanga that had a relational/interaction focus towards improving practice and outcomes for Māori students. Teachers spoke of change in their classrooms in terms of: (a) change in teacher beliefs, expectations and understandings; (b) change in teacher agency, and (c) increased teacher job satisfaction, motivation and empowerment. They described increased experimentation and risk-taking in the classroom; increased understanding and awareness of Māori students needs such as valuing, respecting and including Māori students language and/or cultural knowledge; teacher repositioning, co-construction, power-sharing and student-focused classrooms; group work and cooperative learning approaches; teacher monitoring and related assessment activities; and an increase in teacher satisfaction, motivation and empowerment.

How valuable are the outcomes for Māori students, and what is the impact on other classmates/peers?

Students reported enhanced valuing of their identity as Māori learners and increases in culturally responsive practices at most schools; perceptions of school personnel and whānau provide additional support for growing appreciation for Māori cultural identity in schools. Māori students were proud of Māori culture and identity and felt that, on the whole and in most schools, they were able to "be Māori" in school rather than having to leave that identity outside the school entrance in order to succeed academically. When students discussed ways in which the school as a whole either did or did not demonstrate valuing of Māori culture and language, they gave examples such as use of powhiri, kapahaka and waiata, and they were able to define places and people—such as the Te Kotahitanga room, the marae, and Māori teachers—that helped them to 'feel Māori' at school in a positive way. While appreciated by the students, these examples appeared to be episodic rather than reflecting systemic changes to the overall culture of the school.

Students were able to articulate how teachers showed they valued them as learners and as Māori, and they were able to discuss how teachers had changed in establishing positive relationships with them as learners. They emphasised the importance of teachers' caring about them as persons to support their learning. They commented on how difficult it was for them to care about how well they did and do the work in classes if teachers made it clear they did not. However, there were still perceptions among Māori students in a few schools that a 'double standard' continued to exist whereby Māori students were singled out and disciplined for behaviour ignored for students from other cultural groups.

Teachers, facilitators, principals and other school leaders reported improvements in student attendance, participation, motivation, and engagement in school and classroom learning activities. There is numerical evidence of enhanced student retention and increases in Māori student enrolment in the senior school and National Certificate of Educational Achievement (NCEA) credit attainment at Year 11 for Phase 3 schools in comparison to 12 matched comparison schools.

Systematic comparisons of Year 11 student performance between the 12 Te Kotahitanga Phase 3 and 12 matched schools reveal higher increases (gain) across 2004-2008 in the percentage of Year 9 entrants attaining NCEA Level 1 in Year 11; Te Kotahitanga schools also evidenced twice the increase in this percentage gain than the average gain nationally. These comparisons also reveal lower achievement outcomes for literacy and similar achievement outcomes for numeracy at NCEA level 1 in 2008.

Systematic comparisons of Māori student NCEA achievement at Te Kotahitanga Phase 3 schools compared to matched schools reveal statistically significant differences favouring the Te Kotahitanga schools in mathematics, physics and science, and no differences across the two groups of schools in English and history. NCEA findings, however, are early days for the Phase 3 schools, where the first student cohort to experience full implementation of Te Kotahitanga did not reach Year 11 until 2008. It would be inappropriate to make NCEA comparisons at the Phase 4 schools where no student cohort had yet experienced full implementation and thus reached Year 11 by 2008-2009.

Te Kotahitanga schools were associated with a higher mean percentage of the total school population at Year 13 who gained University Entrance in comparison with the 12 matched comparison schools.

How valuable are the outcomes for whānau?

For whānau whom we interviewed, Te Kotahitanga was associated with major changes in the way their children approached school and their motivation to do well. Most stressed that while they themselves had never enjoyed coming to school, their children were enthusiastic about school and did not have to be persuaded or forced to attend. They valued high achievement for their children, and many emphasised whānau expectations that their young people would do better in school than the previous generation of Māori. They also valued that their children were 'able to be Māori' while learning, unlike how they themselves recalled feeling about being in school. At a few schools, whānau were critical of the extent to which Māori culture and te reo were supported, and they felt that their children had to struggle to be both Māori and high achievers at school.

The commitment of Māori whānau and the school community to Te Kotahitanga and to Māori student achievement in the mainstream requires ongoing communication and information sharing. Present communication strategies with communities are tenuous, nor are effective strategies for engaging with Māori whānau evident. Enhanced communication links would further support sustainability, particularly in periods of change in school leadership. These links were not key elements in Te Kotahitanga as implemented during Phases 3-4, but future development of such linkages would enhance the model and school capacities to support achievement and other positive outcomes for Māori students. To some extent, the small sample of whānau whom we interviewed can be viewed as an indicator of less than optimal levels of involvement and communication with Māori families.

How beneficial (or detrimental) are the effects of Te Kotahitanga on school culture (covering any changes in formal systems and policies; informal practices, or "the way we do things around here"; and underlying beliefs, values, assumptions and attitudes)?

School leaders, teachers and students noted a focus and a change in the relationships within the school as a result of the Te Kotahitanga programme.

Principals were able to articulate achievement targets and achievements for students, but these were not always shared with the school community including teachers, facilitators, whānau and the Boards of Trustees. Chairs of Boards of Trustees and particularly whānau expressed the desire to know more about Te Kotahitanga and an interest in closer connections between the school and its community. There is, for example, potential for improving the use of the Marae and facilities in enhancing these relationships.

Te Kotahitanga has created new professional leadership opportunities in schools, including facilitation, mentoring, and leadership skills for teachers through the creation of new roles. There is less evidence of leadership distributed across the school with respect to responsibility for the GPILSEO framework; the support of Deans, Heads of Departments and DPs is more philosophical than structural. In a few schools there is evidence that this power has been shared with leadership opportunities extended for Māori students with the creation of mentoring roles, prefect and head boy/girl positions.

Schools struggle over the dilemma of voluntary participation or full inclusion of staff. Shared problem-solving and decision-making by co-construction teacher groups work best when all members of the group are participating in Te Kotahitanga and can be prohibitive when some are not. The implementation of the programme can initially cause division amongst staff as some are resistant to change, although there is evidence that this dissipates after time. There continue to be concerns at some schools that targeting of Māori student achievement may be misconstrued as deficit theorising in attributing less than satisfactory outcomes to the students and their families rather than schools and teachers assuming agency for student results.

Principals generally indicated that Te Kotahitanga had not had significant impact on other school practices and/or school policy. Their discussions of the programme emphasised the teacher change in developing the Effective Teaching Profile rather than the GPILSEO framework and how it had impact on their school overall. Thus, they did not generally see Te Kotahitanga as a school reform initiative, but rather as focused on teacher professional development.

What are the enablers and barriers for getting Te Kotahitanga to work most effectively?

The Te Kotahitanga professional development model works best when: it has active support from the school's leadership team, particularly the principal and the other senior managers; the leadership team views it as an essential vehicle to improve academic achievement of Māori students; and there are effective communications between the school's senior management team and the lead facilitator.

Trained facilitators are critical to the success of this professional development model. Facilitators as well as teachers affirmed that the facilitator role required expertise in Māori culture and its relationship to culturally responsive classroom pedagogy; subject matter expertise that they can connect with culturally responsive pedagogy; and the process of working with teachers and other adult learners. This is not a role that can be shifted to other personnel in the school who have not developed this expertise. The challenges for facilitators in providing effective professional development support for teachers included uneven availability of: curriculum expertise for ETP exemplars across different subject areas; timely student outcome data for feedback to teachers; and differentiated PD activities to accommodate teachers at different stages of implementation, expertise and cultural knowledge. Enabling teachers in the different subject areas who have demonstrated high levels of implementation of the ETP to play a greater role in mentoring other teachers could provide a way forward as well as recognise teacher leadership.

The Te Kotahitanga professional development model appears to function best when there is stability in the facilitation team; when most facilitators are based within the school thus connected to its school community; and either full-time or, if part-time, have a sufficiently flexible schedule for project responsibilities; and all facilitators have sufficient training around issues of culture, pedagogy and subject knowledge.

Planning for the implementation of Te Kotahitanga is crucial to the success of the programme as new staff needed to be employed, provision to timetables needed to be made and physical space appropriated. The interview data demonstrate that the physical space given to the programme signals the importance and permanence of the programme to staff and students. Some schools indicated that initial implementation is challenging for schools given the necessity of making changes to systems and structures to accommodate Te Kotahitanga. Principals felt that networking and/or mentoring relationships with colleagues more experienced with the model could have assisted in this process and expressed interest in playing this role for schools new to Te Kotahitanga.

To what extent is Te Kotahitanga likely to work effectively in other settings and contexts? How sustainable is the initiative likely to be when ministry investment of resources is scaled back?

Principals emphasised sustainability of Te Kotahitanga was dependent on continued resources and expertise associated with the facilitator role, although some principals explored ideas for embedding the culture of Te Kotahitanga in school relationships and related school processes including staff appraisal and peer support networks.

The BOT chairs who were interviewed also emphasised that sustainability of Te Kotahitanga depended upon both people and financial resources, and they expressed concern about funding being reduced or withdrawn. They generally saw the lead facilitator's role as key to sustainability. They emphasised existing budget limitations and wondered aloud whether their Boards would support re-directing funds from other initiatives in order to continue funding Te Kotahitanga should targeted Ministry of Education funding end.

Sustainability of Te Kotahitanga is currently dependent upon delivery of professional workshops and hui from the Waikato research team to develop school leader and facilitator skills and expertise in support of the initiative. Without the availability of ongoing training and mentoring opportunities, there is risk of losing expertise needed to sustain teacher professional development programme towards enhancing Māori student outcomes in mainstream schools.

Without better access to student outcome data on a regular basis, teacher participation in the professional development activities may wane once teachers themselves feel they have mastered the critical components or no longer have interest in doing so. Sustainability will require more efficient and relevant data on student outcomes at the school level for teacher use throughout the year. With some exceptions, the present data collection and reporting systems are not achieving this.

What are the most critical factors in improving teacher efficacy?

There is agreement across school personnel at all levels that the role of lead facilitator is central to Te Kotahitanga with its focus on teacher professional development towards the Effective Teaching Profile towards enhancing student outcomes for Māori in the mainstream. There are also concerns that integrating the role within the school with additional professional development coordination duties could have a negative impact on programme effectiveness if responding to multiple initiatives shifts the focus away from Māori student achievement. There was strong support for a permanent senior teacher leadership role held by a person with the necessary cultural and instructional expertise so that this work would continue.

Across schools and across subjects, Te Kotahitanga has communicated effectively to teachers that relationships in the classroom are important. The success to which these relationships were transferred into successful learning situations was variable in some subjects and some schools. A key challenge was how to overcome a lack of change in some classrooms, particularly for teachers shown and perceived to be low implementers. Some teachers may require additional work and exemplars to assist them in constructing lessons that enable learning relationships to develop in the classroom.

Factors associated with low implementers were the absence of stated learning outcomes and achievement criteria; low expectations for students; and classroom management challenges. These classrooms did not evidence culturally responsive pedagogies of relations, and students did not appear to be engaged as active participants in building understandings and bringing their own knowledge, cultural identity and experiences to new learning opportunities. Difficulties being experienced by some teachers indicated professional needs beyond those that Te Kotahitanga is designed to address. Some may be performance appraisal issues, but these teachers require more support and advice than Te Kotahitanga is designed to give.

The percentage of high implementers was high, at approximately 2 in 5 at the Phase 4 and 1 in 5 at the Phase 3 schools. The fact that the percentage of high implementers was highest at Phase 4 schools could be due to a number of factors. One possibility is that refinements to the Te Kotahitanga programme model might have resulted in enhanced effectiveness at Phase 4 compared to Phase 3 schools. Another could be that the benefits for teaching practice reach their peak within two-three years, and teachers may lose momentum for demonstrating high implementation when the cycle becomes repetitive. Facilitators indicated that it seemed unnecessary to continue to carry out the same observations and feedback sessions for teachers who had long since demonstrated their skills on the ETP. There could be further differentiation of the model for the involvement of high implementers. This differentiation could be done by moving onto senior secondary subjects once a certain level of master is reached in years 9-10. Alternatively, high implementers along with Heads of Departments could become more active in mentoring others or even serving on the facilitation team. Such initiatives might also enable the ETP to be better integrated into the different subject areas and across the senior secondary school.

There also needs to be better access to student outcome data on a regular basis to inform the co-construction planning meetings. Without this, teacher participation in the professional development activities may wane once teachers feel they have mastered the critical components or no longer have interest in doing so. Most importantly, the co-construction teams' problem-solving and planning processes require these data if they are to focus change based on evidence rather than collective impressions regarding impact on students. Sustainability will require more efficient and relevant data on student outcomes at the school level for teacher use throughout the year. With some exceptions, the present data collection and reporting systems are not achieving this.

Appendix 1 lists reference to specific sections and page numbers of the report where evidence relating to each sub-question can be found.

Key Findings for PD, Classrooms, Students and Schools

In addition to reporting our findings for each of the evaluation sub-questions, this section reports the evidence with respect to four "categories" that have more functional utility for educational policy and practice. The first set of findings is related specifically to Te Kotahitanga as a professional development model, and we then report findings with respect to impact on teachers' classroom practice, students, and the school as a whole. This alternative organisation for our findings does not introduce new evidence but instead strives to present that evidence for particular constituent or stakeholder groups including those engaged in professional development design and implementation; teachers and curriculum specialists; educators and whānau/families; and school leaders and the school community. These include evidence around the issue of sustainability for Te Kotahitanga in schools.

Findings for Te Kotahitanga as a professional development model

• Across the board and with very few exceptions, teachers, principals, Boards of Trustees chairs, and facilitators were most enthusiastic about the Te Kotahitanga professional development model, viewing it as a sound and effective process for improving classroom teaching for Māori students.
• Teachers valued the interconnected parts of the model, voicing most enthusiasm for the classroom observations with feedback which they saw as not only improving their teaching but also improving their ability to reflect on their teaching. Co-construction meetings appeared most effective and useful when all teachers in the group were trained in Te Kotahitanga and problem-solving was based on student evidence provided to the group on a regular basis. The implementation of shadow coaching across Phase 3 and Phase 4 schools appeared variable. Data analysis indicated that some lead facilitators and facilitators were less knowledgeable and/or confident about the implementation of shadow-coaching in their school. Some teachers also indicated that they were unclear about the process of shadow-coaching, how it differed from other components of the PD model and/or its contribution to their ongoing professional development. More attention needs to be placed on facilitators' knowledge and use of shadow-coaching and the quality of its implementation within participating schools.
• Trained facilitators are critical to the success of this professional development model. Facilitators as well as teachers affirmed that the facilitator role required expertise in Māori culture and its relationship to culturally responsive classroom pedagogy; subject matter expertise that they can connect with culturally responsive pedagogy; and the process of working with teachers and other adult learners. This is not a role that can be shifted to other personnel in the school who have not developed this expertise.
• The Te Kotahitanga professional development model works best when: it has active support from the school's leadership team, particularly the principal and the other senior managers; the leadership team views it as an essential vehicle to improve academic achievement of Māori students; and there are effective communications between the school's senior management team and the lead facilitator. It also appears to function best when there is stability in the facilitation team; when most facilitators are based within the school thus connected to its school community; and either full-time or, if part-time, have a sufficiently flexible schedule for project responsibilities.
• The challenges for facilitators in providing effective professional development support for teachers included uneven availability of: curriculum expertise for ETP exemplars across different subject areas; timely student outcome data for feedback to teachers; and differentiated PD activities to accommodate teachers at different stages of implementation, expertise and cultural knowledge.

Findings for impact on teachers' classroom practice

• Classroom observation results indicate that the majority of teachers (approximately 75%) evidenced either moderate or high implementation according to objective assessment using criteria based on the Effective Teaching Profile.
• More than one in five teachers demonstrated a high level of implementation of the Effective Teaching Profile in Year 9-10 classrooms across subjects and schools.
• Major changes reported by teachers who were interviewed which they attributed to Te Kotahitanga professional development were: (a) change in teacher beliefs, expectations and understandings; (b) change in teacher agency, and (c) increased teacher job satisfaction, motivation and empowerment.
• Sub-themes associated with these changes included increased teacher experimentation and risk-taking in the classroom; increased understanding and awareness of Māori students needs such as valuing, respecting and including Māori students language and/or cultural knowledge; teacher repositioning, co-construction, power-sharing and student-focused classrooms; group work and cooperative learning approaches; teacher monitoring and related assessment activities; and an increase in teacher satisfaction, motivation and empowerment.
• Interview analysis confirmed that teachers valued relationship-based pedagogies, and the majority of teachers affirmed that Te Kotahitanga professional development had an impact on classroom instruction leading to enhanced outcomes for Māori students as well as for all students. Teachers held varied beliefs about the extent to which the ETP differed from good teaching generally.
• Most teachers were able to highlight particular teaching strategies and methods in their subjects introduced by Te Kotahitanga that had a relational/interaction focus towards improving practice and outcomes for Māori students.
• Observational data indicate variability across subjects and schools in the quality of implementation. On average, one in four teachers was not observed to be implementing key features of the Effective Teaching Profile. In addition to an absence of mastery of culturally responsive pedagogies of relations, students in these classrooms experienced an absence of explicit learning outcomes, criteria for success and high expectations, along with high levels of off-task and disruptive behaviour likely to interfere with learning. The PD needs associated with low implementation of the ETP go beyond factors that are the responsibility of Te Kotahitanga and would seem to indicate the need for good teaching support generally.
• Interview analysis identified ongoing challenges in the attempt to improve practice and outcomes for Māori students. A key challenge was how to overcome a lack of change in some classrooms, particularly for teachers shown and perceived to be low implementers.

Findings for impact on students

• Students reported enhanced valuing of their identity as Māori learners and increases in culturally responsive practices at most schools, and the perceptions of school personnel and whānau provide additional support for growing appreciation for Māori cultural identity in schools. Māori students were proud of Māori culture and identity and felt that, on the whole and in most schools, they were able to "be Māori" in school rather than having to leave that identity outside the school entrance in order to succeed academically.
• Students were able to articulate how teachers showed they valued them as learners and as Māori, and they were able to discuss how teachers had changed in establishing positive relationships with them as learners. They emphasised the importance of teachers' caring about them as persons to support their learning. They commented on how difficult it was for them to care about how well they did and do the work in classes where teachers made it clear they did not. However, there were still perceptions among Māori students in a few schools that a 'double standard' continued to exist whereby Māori students were singled out and disciplined for behavior ignored for students from other cultural groups.
• Students discussed ways in which the school as a whole either did or did not demonstrate valuing of Māori culture and language. They gave examples such as use of powhiri, kapahaka and waiata, and they were able to define places and people—such as the Te Kotahitanga room, the marae, and Māori teachers—who helped them to 'feel Māori' at school in a positive way.
• Interviews with whānau affirmed that their children view school positively, loved coming to school, and had greatly improved their attendance and participation, which many contrasted with their own more negative memories of schools and schooling.
• Teachers, principals and other school leaders affirmed that their own expectations had been raised for and relationships improved with Māori students, and they attributed this shift to Te Kotahitanga rather than to the ETP specifically. Teachers generally did not use the language of the ETP when they discussed Māori learners, expectations, relationships and pedagogy in the classroom. While they noted the relevance of culture to teaching and learning, few discussed specifically that the development of a culturally-grounded identity was an educational outcome for students. The work of facilitators with teachers may require more explicit focus on culturally responsive pedagogies and how to support learning grounded in students' Māori identity towards adding to and enhancing existing conceptions of good teaching. This will also require more dialogue with the Māori community.
• Teachers, facilitators, principals and other school leaders reported improvements in student attendance, participation, motivation, and engagement in school and classroom learning activities which they attributed to Te Kotahitanga.
• There is numerical evidence of enhanced student retention and increases in Māori student enrolment in the senior school and NCEA credit attainment at Year 11 for Phase 3 schools in comparison to 12 matched comparison schools.
• Systematic comparisons of Year 11 student performance between the 12 Te Kotahitanga Phase 3 and 12 matched schools reveal higher increases (gain) across 2004-2008 in the percentage of Year 9 entrants attaining NCEA Level 1 in Year 11; Te Kotahitanga schools also evidenced twice the increase in this percentage gain than the average gain nationally. Comparisons also reveal lower achievement outcomes for literacy and similar achievement outcomes for numeracy at NCEA level 1 in 2008.
• Systematic comparisons of Māori student NCEA achievement at Te Kotahitanga Phase 3 schools compared to matched schools reveal statistically significant differences favouring the Te Kotahitanga schools in mathematics, physics and science, and no differences across the two groups of schools in English and history.
• Te Kotahitanga schools were associated with a higher mean percentage of the total school population at Year 13 who gained University Entrance in comparison with the 12 matched comparison schools.

Findings for impact on schools

• School principals were able to articulate achievement targets and achievements for students, but these were not always shared with the school community including teachers, facilitators, whānau and the Boards of Trustees.
• Across schools and across subjects, Te Kotahitanga has communicated effectively to teachers that relationships in the classroom are important. The success to which these relationships were transferred into successful learning situations was variable in some subjects and some schools.
• While evident within the schools, there is less evidence that this focus on relationships has been extended beyond the school to relationships between the school and its Māori community and whānau. Chairs of Boards of Trustees and particularly whānau expressed the desire to know more about Te Kotahitanga and an interest in closer connections between the school and its community. There is, for example, potential for improving the use of the marae and facilities in enhancing these relationships.
• Planning for the implementation of Te Kotahitanga is crucial to the success of the programme as new staff needed to be employed, provision to timetables needed to be made and physical space appropriated. The interview data demonstrate that the physical space given to the programme signals the importance and permanence of the programme to staff and students.
• Some schools indicated that initial implementation is challenging for schools given the necessity of making changes to systems and structures to accommodate Te Kotahitanga. Principals felt that networking and/or mentoring relationships with colleagues more experienced with the model could have assisted in this process and expressed interest in playing this role for schools new to Te Kotahitanga.
• Te Kotahitanga has created new professional leadership opportunities in schools, including facilitation, mentoring, and leadership skills for teachers through the creation of new roles. There is less evidence of leadership distributed across the school with respect to responsibility for the GPILSEO framework; the support of Deans, Heads of Departments and DPs is philosophical rather than structural. In some schools there is evidence that this power has been shared with leadership opportunities extended for Māori students with the creation of mentoring roles, prefect and head boy/girl positions.
• The implementation of the programme can initially cause division amongst staff whose different perspectives on enhancing student achievement may result in resistance. There is evidence that resistance to the programme dissipates over time, but schools still struggle over the dilemma of voluntary participation or full inclusion of staff. Shared problem-solving and decision-making by co-construction teacher groups works best when all members of the group are participating in Te Kotahitanga and can be prohibitive when some are not.
• Schools leaders, teacher and students noted a focus and a change in the relationships within the school as a result of the Te Kotahitanga programme. The classroom observations indicate that further emphasis on fostering learning relationships between students within some classrooms is needed.
• There continue to be concerns at some schools that targeting of Māori student achievement may be misconstrued as deficit theorising in attributing less than satisfactory outcomes to the students and their families rather than schools and teachers assuming agency for student results.
• Whānau at a few schools were critical of the extent to which Māori culture and te reo were supported, and they felt that their children had to struggle to be both Māori and high achievers at school. There is evidence from our school visits that a few staff at a few schools engaged in deficit theorising and racist attitudes, seen by students, teachers and whānau as continuing to impede progress for Māori students at those schools.
• Principals generally indicated that Te Kotahitanga had not had significant impact on other school practices and/or school policy. Their discussions of the programme emphasised teacher change in developing the Effective Teaching Profile rather than the GPILSEO framework and how it had impact on their school overall. They did not generally regard Te Kotahitanga as a school reform initiative, but rather as focused on teacher professional development.

Findings for sustainability of Te Kotahitanga in schools

• School leaders emphasised that sustainability of Te Kotahitanga at their schools was dependent on three factors: (a) the lead facilitator role, performed by a professional with the necessary cultural knowledge, secondary curricular and pedagogical expertise, and credibility and skill in providing technical advice and support to teachers; (b) individualized expert advice to teachers and support for co-construction team activities; and (c) the availability of ongoing expert training and consultation as had been provided by the University of Waikato research team.
• There is agreement across school personnel at all levels that the role of lead facilitator is central to Te Kotahitanga with its focus on teacher professional development towards the Effective Teaching Profile towards enhancing student outcomes for Māori in the mainstream. There are also concerns that integrating the role within the school with additional professional development coordination duties could have a negative impact on programme effectiveness if responding to multiple initiatives shifts the focus away from Māori student achievement. There was strong support for a permanent senior teacher leadership role held by a person with the necessary cultural and instructional expertise so that this work would continue.
• Principals emphasised that sustainability of Te Kotahitanga was dependent on continued resources and expertise associated with the facilitation team, although some principals explored ideas for embedding the culture of Te Kotahitanga in school relationships and related school processes including staff appraisal and peer support networks.
• BOT chairs also emphasised sustainability of Te Kotahitanga depended upon both people and financial resources, and they expressed concern about funding being reduced or withdrawn. They generally saw the lead facilitator's role as key to sustainability. They emphasised existing budget limitations and wondered aloud whether their Boards would support re-directing funds from other initiatives in order to continue funding Te Kotahitanga should targeted Ministry of Education funding end.
• While also important, the role of additional facilitators in schools has presented various challenges to schools regardless of targeted Te Kotahitanga funding. Reasons for this include staff time, expertise and credibility around issues of culture, pedagogy and subject knowledge. Further, the evidence on uneven implementation of the Effective Teaching Profile, despite length of participation, suggests that teacher participation could be better differentiated. Enabling teachers in the different subject areas who have demonstrated high levels of implementation of the ETP to play a greater role in mentoring other teachers could provide a way forward as well as recognise teacher leadership.
• Sustainability of Te Kotahitanga within the Phase 3 and Phase 4 model is dependent upon delivery of professional workshops and hui from the Waikato research team to develop school leader and facilitator skills and expertise in support of the initiative. Without the availability of ongoing training and mentoring opportunities, there is risk of losing expertise needed to sustain teacher professional development programme towards enhancing Māori student outcomes in mainstream schools.
• Without better access to student outcome data on a regular basis, teacher participation in the professional development activities may wane once teachers themselves feel they have mastered the critical components or no longer have interest in doing so. Sustainability will require more efficient and relevant data on student outcomes at the school level for teacher use throughout the year. With some exceptions, the present data collection and reporting systems are not achieving this.
• The commitment of Māori whānau and the school community to Te Kotahitanga and to Māori student achievement in the mainstream requires ongoing communication and information sharing. Present communication strategies with communities do not appear to be effective, nor are effective strategies for engaging with Māori whānau evident. Enhanced communication links would further support sustainability, particularly in periods of change in school leadership.

Recommendations based on Evaluation Findings

Based on the findings from this evaluation, the following recommendations are made:

For teacher professional development

1. There should be continued support for the Te Kotahitanga professional development model as a viable process for improving classroom teaching, and particularly teachers' ability to teach Māori students. Schools require further advice and technical assistance to determine how they can institutionalise the model without relying on an ongoing stream of additional funding.
2. Given the key role of the lead facilitator and the importance of trained facilitators who are critical to the Te Kotahitanga professional development model, there should be encouragement for development of alternative training formats such as university-based programmes along the lines of the RTLB professional, postgraduate programme leading to a formal qualification.
3. Since schools have at least some teachers who are implementing the effective teaching profile at a high level, consideration should be given to facilitation teams that include more opportunities for high implementer teachers to serve part-time as facilitators for limited time periods, especially to provide subject-specific advice and support. This offers rich possibilities for developing teacher leadership while addressing concerns that facilitation teams lack expertise in specific subject matter areas.
4. Teachers indicated the need for more subject-focused advice on culturally responsive pedagogies, suggesting possible alignment whereby Heads of Departments and experienced Te Kotahitanga teachers might become more involved in supporting colleagues in the different curriculum areas where they have expertise.

For classroom instruction

1. Te Kotahitanga professional development should be recognised for contributing to positive change in teachers' classroom practice. Analysis of interview data indicated changes in teacher beliefs, expectations and understandings and teacher agency as well as teacher satisfaction, motivation and empowerment.
2. Teachers also should be recognised for their role in creating more responsive classrooms for Māori students, particularly in the area of relationship based and interactive pedagogies. Interview and observational analyses also confirmed the value of relationship-based pedagogies.
3. At present the professional development programme does not differentiate between teachers who are working as 'High' 'Moderate' or 'Low' implementers. Teachers have different strengths and challenges, whereas the professional development programme now represents a 'one size fits all' approach. There is a need for more specific and targeted professional development to better cater for teachers' needs with more targeted PD goals for improvement.
4. Teachers working as High Implementers could be used more effectively in coaching/mentoring activities, specifically working in classrooms with those teachers who are currently operating at a Low Implementation level. This recommendation also has implications for school leadership, emphasising the involvement of Heads of Departments working alongside other, new teacher leaders both within and across schools. Teacher leaders should be recognised and their contributions towards ongoing improvement valued.

For students

1. Whenever initiatives are intended to have an impact on student outcomes, systematic measure of those outcomes should be reported on an annual basis by the school to the Ministry of Education and to the school's Board of Trustees, students and families. For Te Kotahitanga, these data should minimally include an agreed measure of student achievement as well as achievement-related motivation and/or engagement. There are measures now available normed in New Zealand to measure each of these and already in use in some schools, some of which may require disaggregated norms for Māori students as well.
2. Schools receiving special initiative funding should be provided a template and technical support for reporting required student outcome data annually to the funding agency in a format that is consistent across schools. These data should also be shared annually with school personnel, parents and their Boards of Trustees. Minimally, these data should include the overall data and data disaggregated by ethnicity covering: average percentage daily attendance; retention as a percentage of students returning to school in the year following their 16th birthday; the total number of suspensions, stand-downs and expulsions; and numbers and levels of streamed groups including percentage composition by ethnicity.
3. The Ministry of Education should also provide technical support to schools as needed in order for them to establish reliable data management systems and designated school personnel who will be responsible for these data.
4. If achievement results are expected to be demonstrated beyond the year groups affected by a project or initiative, there should be systematic planning for extending an appropriate level of project activities. For the Te Kotahitanga project, for example, this could comprise selected facilitator observations and feedback sessions focused on NCEA subjects in the senior secondary school for teachers once they have demonstrated mastery in the junior school, rather than assuming transfer of processes and new skills without scaffolding.

For schools

1. All schools and particularly mainstream schools with significant Māori populations should establish staffing patterns, policy and procedures, and cultural advice to teachers across disciplines to support student achievement in culturally responsive ways. Accountability for these systemic processes and changes should be with the principal and the senior management team.
2. There should be focus on high achievement for all Māori students alongside evidence of high expectations represented by access to enhanced learning opportunities, gifted and talented programmes, and appropriate educational supports including lesson differentiation for students with special educational needs
3. Data should be collected, analysed and summarised in a manner appropriate for use by school personnel to improve instruction and programmes. Summaries of student outcome data should also be shared on a regular basis with teachers, whānau and the students themselves. Whānau and students should be included in the feedback loop and given opportunity to participate in the visioning and goal setting for the school. This includes creating opportunities for whānau to participate in reform; some schools may need to address relationships with whānau and local iwi given evidence that historical disputes and/or past schooling experiences for whānau at the school were seen as barriers to participation.
4. There needs to be an expectation of overall school change associated with Te Kotahitanga driven by school principals and other school leaders accountable for that change. This should include consideration of the implications for specific school and personnel systems such as supports for provisionally registered teachers, professional mentoring, performance management, professional development activities, curricular reform, and relationships with the community and families.