Summary

From 2004 to 2007, Te Kotahitanga was introduced in 33 New Zealand secondary schools. Its aim: to develop culturally responsive pedagogies designed to enhance Māori student achievement based on the Effective Teaching Profile concept.

This independent evaluation of Te Kotahitanga addresses the research question: How well and in what ways does Te Kotahitanga work towards the goal of improving Māori student achievement?

Background

This Summary Report focuses on major findings from the evaluation of Te Kotahitanga as implemented at 12 Phase 3 and 21 Phase 4 schools. During these phases, the Te Kotahitanga model was focused on teacher professional development to enhance teacher practice as described in Bishop, Berryman, Cavanagh, and Teddy (2009).

The evaluation team of Māori and non-Māori researchers visited 22 of these schools in 2008 to observe classrooms and interview school leaders, teachers, project facilitators, whānau, students, and boards of trustees chairs. Project reports were reviewed, and student achievement data analysed.

In 2010, Phase 5 of the programme was introduced to 17 new secondary schools. Informed by the evaluation, the Phase 5 model is focused not only on teacher professional development, but also on school leadership to achieve whole-school change and the use of evidence of student outcomes to improve and inform practice.

NB. For details of Te Kotahitanga please see the full evaluation report.

Key Findings

This section summarises key findings for each of the sub-questions that were designed to assess the overarching question:

1. What is the quality of the overall design, content and implementation of Te Kotahitanga?

With few exceptions, teachers, principals, boards of trustees chairs, and facilitators were overwhelmingly positive about the Te Kotahitanga professional development model as a sound and effective process for improving classroom teaching and learning for Māori students. Teachers were enthusiastic about facilitator classroom observations and the feedback they received towards improving their teaching and reflective practice. Co-construction meetings across the team were working well, while use of shadow coaching was limited. Teachers, principals and other school leaders affirmed enhanced relationships with and expectations for Māori students, and attributed these to Te Kotahitanga. Teachers knew less about Māori cultural identity as an educational outcome for students.

The Te Kotahitanga professional development model is associated with improved classroom teaching. The majority of teachers (approximately 75%) in Te Kotahitanga schools evidenced moderate or high implementation of the Effective Teaching Profile (ETP) in Year 9-10 classrooms. More than one in five demonstrated high levels although there was variability across subjects and schools. On average, mastery of the Effective Teaching Profile was not evident in 25% of classrooms, where professional development needs appeared to extend beyond those which Te Kotahitanga was designed to address.

2. How valuable are the outcomes for the teachers who participate – what new knowledge, understandings and skills do they develop, and how valuable are these learnings?

Teachers reported valuing relationship-based pedagogies, affirming that Te Kotahitanga professional development had an impact on classroom instruction, which led in turn to enhanced outcomes for Māori students as well as for all students. Across schools and across subjects, Te Kotahitanga has communicated effectively to teachers that relationships in the classroom are important.

Most teachers were able to highlight subject strategies introduced by Te Kotahitanga with a relational focus that improved practice and outcomes for Māori students. Teachers spoke of changes in factors including their beliefs, expectations and understandings; improved teacher agency; and increased job satisfaction, motivation and empowerment. They reported increased valuing of Māori students' language and cultural knowledge, a shift to student-focused classrooms, improved assessment practices, and more use of group work and cooperative learning.

3. How valuable are the outcomes for Māori students, and what is the impact on other classmates/peers?

Māori students were proud of Māori culture and identity. On the whole and in most schools, they felt that they were able to 'be Māori' as learners rather than leaving their culture outside school, in order to succeed academically. Students reported enhanced valuing of their identity as Māori learners and increases in culturally responsive practices. Students gave examples of how schools either did or did not demonstrate valuing of Māori culture and language. They were able to define places and people – the Te Kotahitanga room, the marae, and Māori teachers – that helped them to 'feel Māori' at school in a positive way.

Students articulated how teachers showed they valued them as learners and as Māori, and they discussed how teachers had established positive relationships with them as learners, which they saw as essential for their learning. They commented on how difficult it was for them to be motivated and work hard in class if teachers did not care and had low expectations for them. In a few schools, there were still perceptions among Māori students that a 'double standard' existed whereby Māori students were singled out and disciplined for behaviour that was typically ignored for students from other cultural groups.

Whānau reported that their children felt appreciated as Māori in school and were more positive about school than they themselves had been. Parents, teachers, facilitators, principals and other school leaders reported improvements in student attendance, participation, motivation, and engagement in school and classroom learning activities. There is evidence of enhanced student retention leading to increases in Māori student enrolment in the senior school. In terms of real numbers, there is an average increase in the Y11 Māori student enrolment of approximately 250% from 2005 to 2008 at Te Kotahitanga schools.

Given the Te Kotahitanga implementation timeframes, these are early days for analysing its impact on NCEA outcomes. At the Phase 3 schools, 2008 Year 11 NCEA results were analysed for the first student cohort exposed to full implementation of Te Kotahitanga across Years 9 and 10. Year 11 NCEA results, compared with those at demographically similar schools from 2004-2008, revealed enhanced performance for Māori students at Te Kotahitanga schools on several achievement indicators. Te Kotahitanga schools also had a higher mean percentage of the total school population gaining University Entrance in Year 13.

No comparisons of NCEA results were done at Phase 4 schools as the first student cohort exposed to Te Kotahitanga in Years 9 and 10 will not reach Year 11 at these schools until 2010 – beyond the timeframe for the evaluation.

4. How valuable are the outcomes for whānau?

Whānau associated Te Kotahitanga with major changes in how their children viewed school. Most stressed that their children were enthusiastic about attending and motivated to achieve. Whānau valued achievement and expected young people to do better in school than they had. They perceived that their children were 'able to be Māori' while learning, unlike the previous generation of Māori. At a few schools, whānau expressed that Māori culture and te reo were not adequately supported, and they felt that in these schools, their children still struggled to be both Māori and high achievers.

Findings suggest less than optimal levels of involvement and communication between many schools and the Māori community. Ongoing communication and partnership work were generally not happening to support commitment of Māori whānau and the school community to Māori student achievement in these schools. Information sharing with communities and effective strategies for engaging with Māori whānau were limited.

5. How beneficial (or detrimental) are the effects of Te Kotahitanga on school culture?

(covering any changes in formal systems and policies; informal practices, or 'the way we do things around here'; and underlying beliefs, values, assumptions and attitudes)

Principals generally articulated student achievement targets and outcomes, but these were not widely known to others in the school community. Chairs of boards of trustees and whānau wanted more information about Te Kotahitanga and closer connections between the school and its community. There is potential for improving the use of the marae in enhancing these relationships.

Schools leaders, teachers and students attributed positive change in relationships within the school to Te Kotahitanga. New professional leadership opportunities in schools included facilitation, mentoring, and leadership development for teachers with the creation of new roles. There was less evidence of leadership distributed across the school with respect to responsibility for Te Kotahitanga's GPILSEO framework. The support of deans, heads of departments and deputy principals tended to be philosophical rather than structural. In a few schools, leadership opportunities have been extended for Māori students with the creation of mentoring roles, prefect and head boy/girl positions.

Schools struggle over the dilemma of voluntary or required participation by staff. Shared problem-solving and decision-making by co-construction teacher groups worked best when all members of the group were participating in Te Kotahitanga. Such groups were challenged when some team members were not involved. Programme implementation was sometimes associated with division amongst staff, although this appeared to dissipate after time. At some schools, there was a risk that targeting Māori student achievement was being misconstrued as deficit theorising about students and families, by attributing to students predetermined outcomes based on socio-economic and family influences, rather than emphasising the focus of Te Kotahitanga on actions by school leaders and teachers as agents of change to enhance student outcomes.

Principals generally indicated that Te Kotahitanga had not had significant impact on other school practices and/or school policy. They emphasised teacher change towards development of the Effective Teaching Profile rather than whole-school change, and use of the GPILSEO framework (see Chapter 6 of this report) was limited. School leaders did not generally see Te Kotahitanga as school reform, but rather as focused on teacher professional development. It is important to emphasise that at this time, Te Kotahitanga was focused on teacher professional development.

6. What are the enablers and barriers for getting Te Kotahitanga to work most effectively?

The Te Kotahitanga professional development model works best with active support from school leaders, particularly the principal and other senior managers, who see it as essential for improving academic achievement of Māori students. Communications between the school leadership and the lead facilitator varied and were largely dependent on personal factors rather than a systematic feedback loop to the senior management.

Trained facilitators were seen as critical to the success of the model. Facilitators and teachers affirmed that the facilitation role required:

• expertise in Māori culture and culturally responsive classroom pedagogy
• subject matter expertise related to culturally responsive pedagogy
• effective strategies for working with teachers and colleagues.

There were some substantial challenges for facilitators in providing effective professional development. These included:

• uneven availability of curriculum expertise across different subject areas related to the effective teaching profile
• timely access to student outcome data for use by teachers
• lack of differentiated professional development activities to accommodate teachers at different stages of implementation, expertise and cultural knowledge.

It is suggested that teachers demonstrating high levels of implementation of the Effective Teaching Profile in different subject areas could play a greater role in mentoring other teachers.

The Te Kotahitanga professional development model appears to function best with:

• a stable facilitation team
• facilitators who are part of the school community and who have a sufficiently flexible schedule for project responsibilities
• facilitators who have expertise related to culture, pedagogy and subject knowledge.

Physical space for the project and the employment status of the facilitators appeared to signal the importance and permanence of the programme to staff and students.

Successful implementation requires willingness from leaders to change systems and structures. Principals felt that networking and/or mentoring relationships with colleagues who had prior experience with the model would assist in this process and they expressed an interest in playing this role for schools new to Te Kotahitanga.

7. How sustainable is the initiative likely to be when investment of resources is scaled back?

Principals emphasised that the effectiveness and sustainability of Te Kotahitanga was dependent on the resources and expertise associated with the facilitator role. The model appeared to be dependent on provision of the expertise associated with a facilitator position. This position supported the professional development of teachers across the school. Some principals explored ideas for embedding the culture of Te Kotahitanga in school relationships and related school processes, including staff appraisal and peer support networks. The boards of trustees chairs who were interviewed also emphasised that Te Kotahitanga depended upon both people and financial resources. They expressed concern about funding being reduced or withdrawn and how they would maintain facilitator staffing. Boards of trustees chairs also emphasised budget limitations and wondered whether there would be support for re-directing funds from other initiatives in order to continue funding Te Kotahitanga when additional funding ended.

In addition to the role of facilitators, sustainability of Te Kotahitanga has also been dependent upon delivery of professional workshops and hui from the Waikato Te Kotahitanga team to develop school leader and facilitator skills. Additional opportunities for professional development of cultural expertise, such as development of a qualification to provide ongoing availability of specialist facilitators, would further enhance sustainability.

Student achievement-related outcome data were not generally available on a regular basis to teacher teams. Sustainability of Te Kotahitanga will require more efficient and relevant data on student outcomes at the school level for teacher use throughout the year.

8. What are the most critical factors in improving teacher efficacy?

School personnel agreed that the role of lead facilitator is central to Te Kotahitanga as teacher professional development towards enhancing Māori student outcomes. There were concerns that integrating the role with other duties could have a negative impact on programme effectiveness if the emphasis shifted from Māori student achievement. There was strong support for a permanent senior teacher leadership role held by someone with the necessary cultural and instructional expertise.

The percentage of high implementers was high. Approximately two in five at the Phase 4 schools and one in five at the Phase 3 schools were evidencing high implementation of the Effective Teacher Profile (for more information refer to page 10). The percentage of high implementers was highest at Phase 4 schools, and this could reflect improvements to the Te Kotahitanga programme model towards enhanced effectiveness at Phase 4 compared to Phase 3 schools. Alternatively, benefits for teaching practice may reach their peak within two-three years, after which momentum for demonstrating high implementation declines without additional activities. There were some teachers and facilitators who felt the cycle became repetitive once mastery of the Effective Teaching Profile was evidenced. The model could be better differentiated for high implementers, perhaps through moving onto senior secondary subjects. Or, high implementers, along with heads of departments could become more active in mentoring others or as facilitators. Such approaches could also enable the Effective Teaching Profile to be better integrated into the different subject areas and across the senior secondary school.

Better access to student outcome data on a regular basis is needed to inform co-construction planning meetings. Without this, teacher participation in the professional development activities may wane once teachers feel they have mastered the critical components of the Effective Teaching Profile.

A remaining challenge is the lack of change in some classrooms where there is low implementation of the Effective Teaching Profile. Difficulties evidenced by some teachers indicated professional needs beyond those that Te Kotahitanga is designed to address. Some may be performance appraisal issues, and a minority of teachers do require more support and advice.

In relation to Te Kotahitanga, some teachers may require additional work and subject-specific exemplars to assist them in constructing lessons that use culturally responsive pedagogies. Shadow-coaching was either missing or not operating effectively within the professional development model in many schools. More emphasis on shadow-coaching could assist the professional development process.

Factors associated with low implementers were the absence of stated learning outcomes and achievement criteria, low expectations for students, and classroom management challenges. These classrooms did not evidence culturally responsive pedagogies of relations, and students did not appear to be active participants by bringing their own knowledge, cultural identity and experiences to learning.