Executive Summary

The purpose of this report is to document the outcomes of the implementation of Te Kōtahitanga in Phase 3 and 4 secondary schools from 2007 to 2010. Most quantitative data pertain to the years 2007 to 2009, whereas the qualitative data were collected during the period 2009 to 2010. During these four years, the Phase 3 schools were in their fourth to seventh year of implementing the project in their schools. Phase 4 schools were in their first to fourth years of the programme.

These questions were investigated using a mixed-methods research approach using both:

• quantitative methods, including student achievement data, observations of teacher and student behaviour in previously defined categories, and self-reports of intentions and experiences collated through questionnaires and rating scales
• qualitative methods, including semi-structured and open-ended interviews and documentary analysis.

Data collection was carried out in accordance with the University of Waikato’s ethical requirements and the American Education Research Association (AERA) guidelines for reporting on empirical social science research in educational settings (AERA, 2006).

Findings are presented in Chapters 3 to 6, against each of the four key questions.

Chapter 3 presents the changes that were maintained in the Phase 3 schools as evidenced through improvements in Māori student outcomes, changes in teachers’ discursive positioning and classroom practices, and changes in school management and structures to support project implementation (key question 1).

Chapter 4 discusses the degree to which patterns of Māori student outcomes, changes in teachers’ discursive positioning and classroom practices, and changes in school management and structures in Phase 4 were replications of those established earlier in Phase 3 schools (key question 2).

Chapter 5 considers the elements of effective professional development that sets out to refine and develop new ways of working to support the improvement of Māori students’ learning and achievement (key question 3).
Chapter 6 identifies the degree to which Te Kōtahitanga is being embedded in the Phase 3 schools as indicators of sustainability (key question 4).

Chapter 7 draws some conclusions and points to future directions.

1. Maintaining changes for teachers and gains for Māori students in Phase 3 schools

• In every comparison of asTTle gain scores in reading and mathematics in Years 9 and 10, Māori students improved between pre- and post-tests.
• Where in 2007 and 2008, Māori and non-Māori were achieving the same asTTle gain scores in reading and mathematics in Years 9 and 10, or where non-Māori in Phase 3 schools were achieving better, by 2009 Māori students were achieving at least as well, if not better, than non-Māori in most comparisons.
• Year 10 Māori students achieved the same as or higher asTTle gain scores than the national norm for all students in the reading and mathematics assessments, in 2008 and 2009.
• Māori students’ achievements in NCEA outcomes improved between 2007 and 2009 in Years 11 and 12. In Year 13 there was a non-significant decrease in achievement, also reflected at national level.
• Between 2005 and 2006, Year 11 Māori students in Phase 3 schools doubled the percentage points gain of the national cohort of Māori students¹.  In subsequent years they improved on this achievement level.
• Between 2007 and 2009, the percentage points gain made by Year 12 Māori students in terms of NCEA Level 2 and above was double that of the national cohort of Māori students.

Data from the Observation Tool indicated that:

• for all cohorts, a high level of relationships was maintained from the first year of teachers’ participation
• there was a general increase in the use of discursive practices by term 4, 2009, relative to term 1 of the first year of participation
• the percentage of time teachers spent at the front of the classroom decreased in their first year of participation relative to the baseline
• there was an increase in the cognitive level of the lessons relative to the baseline, that was maintained through the first three years of participation
• the percentage of student engagement increased significantly in the first year relative to the baseline and was maintained in the second and third year
• the level of work completion by Māori students increased and was maintained.

Embedding the project in Phase 3 schools

Overall, in the surveys that were undertaken, teachers and leaders reported positively on the processes of embedding GPILSEO elements in their schools.

Overall, leaders were also very positive in their self-evaluations as leaders in Te Kōtahitanga schools and also in their evaluations of the systems and organisation supporting the project in their schools, but in this latter case slightly less so.

In terms of staff retention in the project, the greatest number of withdrawals took place in the first two years of teachers in cohort 1. Subsequently very few teachers chose to withdraw.

2. Replicating changes for teachers and gains for Māori students in Phase 4 schools

Māori student achievement in Phase 4 schools showed that:

• in every comparison of asTTle gain scores in reading and mathematics in Years 9 and 10, Māori students’ achievement improved between pre- and post-tests
• in 2007, 2009 and 2010, in half of the comparisons of mathematics achievement within Phase 4 schools, non-Māori students improved more than Māori students and in half they improved equally. However, Year 10 Māori students achieved higher gain scores than the national norm for all students in asTTle mathematics assessments in both 2008 and 2009
• in asTTle reading by 2009 Māori students outperformed non-Māori in terms of learning gain within Phase 4 schools in three out of four results
• in terms of national comparisons, Year 10 Māori students achieved a 50% increase in gain scores in asTTle reading assessments between 2008–2009, and almost achieved the same level of gain scores as the national norm for all students in 2009.
• NCEA Level 1 and above in Year 11, and Level 2 and above in Year 12, showed a marked improvement:
  • Year 11 students made twice the gain of the national cohort of Māori students at Year 11 in NCEA Level 1 and above, in terms of percentage points
  • Year 12 Māori students also made a greater percentage points gain at NCEA Level 2 and above (9% higher) than the national cohort of Year 12 students

Changes in Te Kōtahitanga teachers’ classroom practices in Phase 4 schools, as evidenced by data from the Observation Tool, included:

• for all cohorts, a high level of relationships, maintained from the first year of participation
• a general increase in the use of discursive practice by term 4, 2009, relative to term 1 of the first year of participation
• a decrease in the percentage of time teachers spent at the front of the classroom in teachers’ first year of participation relative to the baseline
• an increase in cognitive level of the class, relative to the baseline, that was maintained across time
• an increase in the percentage of student engagement that was maintained across time
• maintenance of the high level of work completion by Māori students already established at baseline.

Replicating the project in Phase 4 schools

Overall, in the surveys that were undertaken, teachers and leaders reported positively on the processes of beginning to understand the GPILSEO elements in their schools.

Leaders were very positive in their self-evaluations as leaders in Phase 4 schools and also in their evaluations of the systems and organisation supporting the project in their schools.

In terms of staff retention in the project, very few teachers withdrew from the project. Those few left mostly in the first year of membership.

3. The process of effective professional development

Analysis of evidence from Te Kōtahitanga professional development indicates that:

• the tools developed to support the professional development associated with Te Kōtahitanga were trialled, evaluated and proven to be fit for purpose
• providing inconsistent direction and less hands-on support to Phase 3 schools in 2007 and 2008 resulted in variable outcomes in terms of sustainability
• the central project team should provide support for school leadership in implementing and maintaining the project
• the integrity of the professional development cycle in schools should be maintained
• Phase 4 schools have shown more adherence to the integrity of the Te Kōtahitanga model; however
  • there is still wide variance from one school to another
  • one-third of schools still require more intensive support
  • there is a wide range of knowledge and prior experiences across the group of lead facilitators
  • school leadership provide differing levels of support for implementation of Te Kōtahitanga
• the Review of Practice and Development (RP&D) cycle developed by the central project team is effective in quality assuring facilitation teams’ responses to their teachers and teachers’ responses to their Māori students, whilst also maintaining the integrity and purpose of Te Kōtahitanga
• distance support is most effective when it is blended with face-to-face relationships and support.

4. The degree to which Te Kōtahitanga is embedded in Phase 3 schools

It is clear from the analysis of the overall patterns from Phase 3 schools that Māori student achievement patterns continued to improve in association with the maintenance of changes in teacher’s practices. However, the individual case studies analysis that was undertaken in 2009 and 2010 of Phase 3 schools in their sixth and/or seventh year of the project, and which used the GPILSEO model as an analytical tool to investigate the degree to which schools were supporting the pedagogic intervention, showed that there were marked differences in the degree to which the schools had actually implemented the model and how they were maintaining the implementation of the project, with consequent implications for sustainability.

Phase 3 schools can be seen as falling within one of four categories:

• high implementers and high maintainers of the project (four schools)
• previously high implementers but currently low maintainers (three schools)
• previously partial implementers, but currently poised to implement fully (four schools)
• low implementers and low maintainers (one school).

In three of the four years between 2006–2009, Year 11 Māori students’ achievements at NCEA Level 1 were significantly better in high/previously high implementers (the first two categories above) than in the previously partial/low implementers (the lower two categories above).

A comparison of Māori students’ achievements in two schools²,  both of the same decile rating, indicates that, in the high implementing, high maintaining school:

the proportion of Māori to non-Māori students’ suspension and stand-down rates was consistently lower than the lowest implementing and maintaining school

• Māori students’ achievements in literacy and numeracy requirements at NCEA Level 1 and in NCEA Level 1 science overall were higher than in the lowest implementing and maintaining school
• in each school and at every level of responsibility there are shining examples of colleagues who implement the Effective Teaching Profile to a very high degree, and who are supporting Māori students to enjoy success in education as Māori very effectively.

The seven schools in the first two categories are, or have been, very effective implementers of the Effective Teaching Profile in the majority of their teachers’ classrooms through use of the project’s central institutions (Induction hui, observations, feedback, co-construction meetings and shadow coaching). They have also reported steady gains in Māori student attendance, retention, engagement and achievement (AREA) in their schools. In terms of the GPILSEO model, there are a number of additional features these schools have in common. In these schools:

• the senior leadership teams are agentic leaders and present a united front in their determination to support the implementation of the school’s goal to improve the achievement of Māori students and to reduce educational disparities and these visions are of a long-term nature. There are also clear specific measurable goals in regards to Māori student achievement in these schools
• there has been marked changes made in the institutional and structural arrangements in the school in a manner that is clearly responsive to the needs of the pedagogic intervention, including policy development and implementation
• there has been a concerted effort to effectively distribute leadership throughout the school
• most or all of their staff are included in the project
• there is evidence of their making steady progress towards improving positive supportive learning relations with their Māori parents and community
• there has been a concerted effort to ensure improvements in evidence gathering, analysis and use
• ownership of the project, its goals and means of implementation is fundamental to their thinking and practice. This latter development means that there has been a reprioritising of funds available in the school so as to support the establishment of an ongoing professional development function (facilitators) in these schools.

Footnotes

1 The national cohort was weighted for decile levels.

2. One school was high implementing, high maintaining, and the other low implementing and low maintaining.