Evaluation of Te Kotahitanga: 2004-2008
In 2007, Victoria University was contracted by the Ministry to produce an external evaluation of the effectiveness of Te Kotahitanga. It is the first external evaluation of Te Kotahitanga.
This is the full technical report of the evaluation of Te Kotahitanga in 22 schools from phase three and four of the programme, from 2004-2008. Substantive findings from the evaluation report concluded that Te Kotahitanga is a sound and effective process for improving classroom teaching and learning for Māori students.
Also available on Education Counts is the Te Kotahitanga Summary Report, which outlines the key findings.
Author(s): Luanna Meyer, Wally Penetito, Anne Hynds, Catherine Savage, Rawiri Hindle, and Christine Sleeter. Report for the Ministry of Education.
Date Published: August 2010
This report is available as a download (please refer to the 'Downloads' inset box). To view the individual chapters please refer to the 'Sections' inset box. For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Many people contributed to our evaluation of Te Kotahitanga and to this report, which synthesises findings reported in the two preliminary interim reports across all 12 Phase 3 and 10 of the Phase 4 schools participating in the project. The project proposal funded by the Ministry of Education was originally developed by Professor Janice Wearmouth of Victoria University (VUW) in collaboration with Professor Wally Penetito (VUW), with input from an international team of researchers including Drs Christine Sleeter, Bruce Wilson and Celia Haig-Brown. Following Dr Wearmouth's departure to the U.K., Professor Luanna Meyer took on leadership responsibility for the project at the start of 2008, and Rawiri Hindle, Dr Anne Hynds, and Dr Catherine Savage (VUW) joined the project along with Professors Meyer, Penetito, and Sleeter who continued with the project as a key member of the research team throughout. The evaluation project has benefited from the additional expert input provided through the participation of the international consultants, key personnel in the Ministry of Education, and our national advisory group.
A number of staff from the Ministry of Education have played key roles in supporting the evaluation from its inception. These include our initial project manager Lynette Bradnam, followed by Martin Henry and Colin Brown. Kate Shaw as senior project officer has also worked with us throughout the reporting and final revision phases. Sarah Hopkins assumed major responsibility for the project as Senior Advisor Māori in English medium for the Ministry of Education in the final revision phase. Others in the Ministry of Education who provided crucial input include Cheree Shortland-Nuku, Prue Kyle, Fred Bishop and Heleen Visser. We were also fortunate in being able to consult with Dr Michael Johnston at New Zealand Qualifications Authority (NZQA) regarding the student outcome databases, and he also served on our national advisory group. We thank also the members of the evaluation project's national advisory group which met on three occasions to review the research plan, critique preliminary findings, and review the draft final report. While not all participated throughout the project, those with whom we consulted at various stages include Dr Johnston, Cheree Shortland-Nuku, Garrick Cooper, Lynette O'Brien, Para Meha, Arthur Graves, Angela Roberts, Keri Milne-Ihimaera, and Arihia Stirling.
At VUW, Rochelle Finlay and Sian Wright provided Research Office support during this phase of project activities. It has been important as well to liaise with the University of Waikato Te Kotahitanga Research Team to ensure that our evaluation had access to the necessary information and data sources, and we express our continuing appreciation to Professor Russell Bishop (Project Director) and his staff. Professor Bishop has been consistently open and collaborative in providing advice on data collection approaches, addressing queries from our team and discussing issues arising from the data.
We were fortunate to have the help of many additional key personnel at the Jessie Hetherington Centre (JHC) for Educational Research who assisted in data collection, coding, data analysis, and report preparation. JHC's Research & Development Officers Lynanne McKenzie and then Dr Susan Davidson oversaw data entry by project coders, guided coding of qualitative data using NVIVO, and provided technical support in the analysis of classroom observations and interview data including preparing summaries of the qualitative analyses using NVIVO. Executive Assistants Pam Ritchie and Virginia Neal took on the mammoth tasks of organising school visits; communications with school personnel and others; and performing the myriad of support functions needed in a project of this scope. Virginia Neal also joined the Phase 4 school site visit research team, carrying out interviews and observations along with our core research team. The JHC's Research Administrator Susan Kaiser provided technical assistance in the preparation of data displays and production of the report. Expert statistical advice was provided at various stages by Professor Richard Harker (Massey University), Dr Michael Johnston (NZQA), and Dr Flaviu Hodis (VUW), and Larissa Kus (PhD student in Psychology at Victoria) worked on the data analyses of student National Certificate of Educational Achievement (NCEA) achievement outcomes and also provided support to the analyses of qualitative data. Thanks are also extended to Joe McClure, Melissa Romanov, Sylvia Kwang, Brenda Watson, and Anna Thompson who prepared Word transcripts from audio-recorded interviews and/or from handwritten focus group notes.
Gathering data on-site at the 12 Phase 3 and 10 Phase 4 schools covered by this report involved over several hundred classroom observations and interviews with hundreds of participants over a period of many weeks in March-April and October 2008. This required a large team comprising both Māori and non-Māori researchers with relevant expertise and training for this task. We are particularly grateful to senior Māori researchers Dr Mere Kēpa, Dr Tepora Emery, and Dr Georgina Stewart; their participation along with the VUW Māori research team members was key to our commitment that expertise in Māori research methodologies and Māori Kaupapa was respected and represented throughout. For the Phase 3 school data collection, the team comprised VUW team members Hindle, Hynds, Meyer, Penetito and Savage; our international consultants Sleeter and Wilson; and independent Māori researchers Kēpa, Emery and Stewart. For the Phase 4 school data collections, the team comprised VUW team members Hindle, Hynds, Meyer, Neal, Penetito, and Savage; our international consultant Sleeter; and independent Māori researcher Emery.
Finally, of course, this report and the work that underpins it would not have been possible without the cooperation and participation of the many adults and students associated with the twelve Phase 3 and ten Phase 4 Te Kotahitanga schools. We also wish to acknowledge the contribution made by ten Phase 5 schools that allowed our team to carry out classroom observations prior to implementation of Te Kotahitanga, both as baseline for their future development but also to provide a sample for comparison with the Phase 3-4 schools. The principals and lead facilitators were particularly helpful and accommodating in arranging and coordinating observations of lessons, meetings, and interviews involving teachers, students, whānau, and various other school personnel who met and talked with us about Te Kotahitanga and their experiences. These participants have opened their schools to us and trusted us with reporting our findings with integrity and clarity. Our response to their extraordinary good will must be by ensuring that the information and ideas they have shared with us will not just be recorded in the pages of evaluation reports but will be put to good use. They and we share the same goal—to utilise evidence on processes, outcomes and people to inform educational practice.