Evaluation of Partnership Schools | Kura Hourua Policy Publications
Publication Details
The Ministry of Education has a clear focus on improving student achievement, and employs a range of approaches to support the sector’s efforts, including provision of strategic leadership, resources, and targeted interventions. Partnership Schools | Kura Hourua (PSKH) is a new policy that provides an innovative addition to this mix.
Author(s): MartinJenkins. Report prepared for the Ministry of Education.
Date Published: May 2017
Executive Summary
Partnership School│Kura Hourua policy
The PSKH policy has been implemented in two rounds to date: five schools/kura opened in February 2014 and a further four schools/kura opened in February 2015. The agreement with one of the Round 1 kura was terminated by the Minister of Education from 7 March 2016.
Evaluation overview
MartinJenkins is conducting a multi-year evaluation of the PSKH policy for the Ministry. The overall purpose of the multi-year evaluation is to assess the extent to which the PSKH policy has delivered what it intended to deliver with regard to flexibility, innovation and student outcomes.1
This second year of evaluation focused on two things:
- the approaches taken by PSKH, to understand better what the policy is delivering in practice
- an in-depth analysis of schools' assessment practices and use of assessment data to inform decision-making in the classroom and across the school, as an important condition for success.
Year 2 of the evaluation used a mixed-method approach; drawing on document review, interviews and an online survey to develop a comprehensive understanding of school-level provision, supplemented by quantitative analysis of outcomes data, which was provided to the MartinJenkins evaluation team by the Ministry of Education. The research was undertaken in two parts.
Part 1 explored at a broad level the approaches schools/kura are taking to meet the needs of priority students. The research questions for Part 1 are:
- Who are the priority students and what are their needs?
- What are schools/kura doing to meet the needs of priority students?
- What is the rationale for approaches taken?
- What outcomes do schools/kura attribute to the approaches taken?
- What has enabled or prevented schools/kura from implementing desired approaches?
Part 2 explored in depth the assessment practices within schools/kura. The research questions for Part 2 are:
- What is the value of good assessment practice and what does best practice look like?
- What does assessment practice and information use look like in the schools/kura?
- What is the quality of schools'/kura's assessment practice and information use?
- What is the relationship between assessment practice and student achievement outcomes?
Findings
Part 1: Broad approaches taken by schools/kura to meet the needs of priority students
Question 1: Who are priority students and what are their needs?
Schools/kura are reaching priority students, both in terms of students who meet the policy's categorical definition and those who are individually at greater risk of not achieving.
In our view, all of the schools/kura have good understandings of the needs of the categories of students they seek to serve and intimate understandings of the needs of the individual students on their rolls. Schools/kura report that while students do bring a range of positive attributes to their education experience, large proportions have high and sometimes complex needs related to:
- Low academic baselines and lack of core skills for learning
- Histories of disengagement from education
- Complex socio-economic and health needs that create barriers to education
- Lack of education role models to support and encourage education success.
Question 2: What approaches are schools/kura taking to meet the needs of priority students?
Schools/kura take a range of approaches to meet student needs, specifically approaches related to four identified needs:
- NEED 1:
- Low academic baselines and lack of core skills for learning
- Ensuring teaching and learning is of top quality
- Ensuring early identification of students falling behind
- Providing targeted support
- NEED 2:
- Histories of disengagement from education
- Clear attendance and behaviour expectations with consequences for breaches
- Systematic and rapid follow-up when students are late/absent
- Creating an environment that students want to be in - Designing the school timetable to maximise student engagement
- NEED 3:
- Complex socio-economic and health needs that create barriers to education
- Reducing (or eliminating) the costs of education - Providing or brokering access to social and health services
- NEED 4:
- Lack of education role models to support and encourage education success
- Instilling high aspirations for every student and broadening student horizons
- Involving parents/family/whānau in their student's education journey.
For the most part, their approaches reflect good practice and sometimes innovative practice. We have not found any evidence of poor practice, but we have found some signs of practices that indicate schools/kura are still refining their approaches and tailoring their approaches to meet the needs of their students and communities, as one would expect of newly established schools.
The extent to which schools have an embedded and established 'approach' overall is reflective of the length of time they have been operating, and/or the depth of experience their Sponsor organisation has in delivering services within the education sector. Those with greater experience are further along this pathway; those who are newer are still consolidating their practice and tailoring it to meet the specific needs of their local communities.
Question 3: What is the rationale for approaches taken?
Sponsors have applied to deliver a PSKH because they believe they have something valuable to offer and have a deep belief in the value of their approach. Their underlying worldview, developed from experiences prior to operating a PSKH, influenced their initial school/kura design and determines how they respond to emerging requirements.
With Sponsor worldview as the bedrock, schools/kura also draw on leaders' experience, research evidence and emerging best practice, and ongoing evaluative and reflective practice to develop and refine their approaches.
Last, but by no means least, schools/kura are very mindful of their contracted targets and the wide public interest in whether they succeed or fail.
Question 4: What outcomes do schools/kura attribute to the approaches taken?
Schools/kura report positive outcomes for students across a range of areas, many of which are not captured through reporting of contracted targets.
With regard to contracted targets, we see mixed results:2
- Student achievement: 5 out of 8 schools/kura almost met or exceeded their targets
- Student attendance: 7 out of 8 schools/kura met or exceeded their targets
- Student engagement: 6 out of 8 schools/kura met most or all of their targets.
Other positive outcomes reported by schools/kura that are not captured in contracted performance targets expand on the areas above (e.g. achievement in subjects other than reading, writing and mathematics for primary age students) and go beyond (e.g. improved self-esteem and self-worth, development of high aspirations, adopting school/kura values and greater security of identity, culture and language).
For the most part, schools/kura attribute their outcomes to the package of approaches that they use, rather than to any single approach that can be taken in isolation.
Question 5: What has enabled or inhibited schools/kura from taking their desired approaches?
Schools/kura do not report being prevented from implementing any of the approaches they would like to take; rather some approaches may not yet have been implemented as schools/kura have prioritised what they do due to limited resources and the demands of rapid establishment.
The flexibilities enabled by the policy, and the special status of being a new type of school, are key enablers, but not without challenge. Most schools/kura perceive reporting requirements to be burdensome, and some report unresolved contract issues and/or a complex relationship with their key partner, the Ministry. These issues have at times diverted attention and resources away from delivery.
Part 2: Assessment practice and information use
Question 6: What is the value of good assessment practice and what does best practice look like?
Good assessment practice and information use are key components of quality teaching, and can play a foundational role in raising achievement and improving student outcomes.
The New Zealand Curriculum identifies six features of effective assessment practice: benefits students; involves students; supports teaching and learning goals; is planned and communicated; is suited to the purpose; and is valid and fair.
These features are reinforced and expanded in the targeted literature that we reviewed (and at times, are cast in a slightly different way). In particular, the literature emphasises four themes that characterise effective assessment: places students at the centre; is delivered by assessment capable systems, school leaders, teachers, and whānau; involves effective use of information to support inquiry; and collects data/information in a planned way and that data is used.
The six NZC features of effective assessment practice and the four themes identified in wider literature together form ten touchstones, in relation to which we describe and assess the quality of practice in partnership schools/kura.
Question 7: What does assessment practice look like in the schools/kura?
All of the schools/kura report that assessments that are administered by them are the primary source of information they collect to develop a comprehensive baseline understanding of students' position in relation to National Standards, Ngā Whanaketanga Rumaki Māori, or National Certificate of Educational Achievement (NCEA) expectations. They report using a range of standardised and non-standardised existing assessments, and also develop their own formal assessment tools or tailor existing tools to meet their student needs.
Across all of the schools/kura, staff at all levels have some level of responsibility for tracking and monitoring student progression.
All of the schools/kura report using baseline assessment data to develop individualised learning plans for their students and class-based learning plans tailored to students collective needs/capabilities (i.e. by adapting curriculum, pedagogy and learning resources). They also report drawing on assessment data to inform school-wide and management/board-level decisions (e.g. to target investment and resources).
Schools/kura report using a range of age-appropriate approaches to involve students in assessment, with daily feedback and regular written reports being common. Efforts to engage parents/family/whānau in assessment are not limited to reporting student progress: most schools/kura also make efforts to improve parent/family/whānau understanding of how progress is measured and what expected performance looks like, and to enlist parent/family/whānau support to support students to achieve.
Question 8: What is the quality of schools'/kura's assessment practice and information use?
We assess the quality of assessment practice in partnership schools/kura in relation to ten touchstones drawn from the features of effective assessment practice identified in the New Zealand Curriculum and themes identified in wider literature. Our assessment of the quality of school's/kura's practice in relation to these touchstones is based on document review, in-depth interviews with academic leaders (during which they were asked to evidence their assessment practice by describing tangible examples), notes from previous evaluation visits (which included interviews and observations) and aggregated feedback received through the online teacher survey (which asked respondents to rate their own ability, practice and confidence in a range of assessment areas; and to report on practices across the school/kura overall)
Based on the evidence we reviewed, in our view:
- Overall, we find that assessment practice across the schools/kura is 'good'. All of the schools/kura have very good understandings of assessment practice at leadership level and appropriate systems and tools in place to support it, although the extent to which these are fully embedded varies.
- We found examples of schools/kura that are demonstrating 'good' or 'very good' assessment practice overall. These schools/kura are 'assessing for learning' and have assessment practices that are woven throughout the fabric of school/kura management, classroom management and individual student management. This is not to say that these schools/kura have no room for improvement. Rather, they are aware of the relative strengths and weaknesses of their practices, are continuing to assess and review their practices and, where relevant, have measures in place to support improvement.
- We found examples of schools/kura that are demonstrating 'adequate' assessment practice overall. These schools/kura face challenges that are particular to their circumstances, or they are somewhat earlier in their journey to embed consistently good assessment practice. These schools/kura are aware of these challenges and are introducing measures to support improvement.
- We did not find any examples of schools/kura demonstrating assessment practice that is 'poor' or 'inadequate' overall.
- We are confident that all of the schools/kura are either already delivering, or are on a path towards delivering, assessment practice that is 'good' or 'very good' overall.
Question 9: What is the relationship between assessment practice and student achievement outcomes?
Leaders in all of the schools/kura have a good understanding of the relationship between assessment practice and student achievement outcomes. They see good assessment practice as a core component of quality teaching and learning, both for groups of students and for individuals.
The quality of assessment practice does not correlate directly with student outcomes within the small sample of partnership schools/kura. It would therefore be overly simplistic to say that assessment practice is the cause of good or bad outcomes for individual students or for groups of students in these schools/kura. Rather, assessment practice contributes to good outcomes in these schools/kura; in particular by supporting effective implementation of targeted support, and continued improvement of assessment practice that is implemented alongside other initiatives is likely to contribute to further improvements in student outcomes over time.
Navigation
Where to find out more
Contact Us
Education Data Requests
If you have any questions about education data please contact us:
Email: Requests Data and Insights
Phone: +64 4 463 8065