Introduction

The evaluation

The evaluation was designed to support the implementation and roll-out of the Positive Behaviour for Learning (PB4L) Restorative Practice programme within 27 schools, and to identify lessons for other schools implementing the programme. The evaluation began in 2014 and this is the final evaluation report.

• An initial Establishment Phase in 2014 tested key evaluation tools and methods with a group of 20 'pilot' schools^[1]. Survey tools and reporting were further refined in 2015.
• The evaluation focused on early implementation from 2016 through to early 2017.
  • Interim Report, First Findings, June 2016 - early feedback on implementation.
  • Restorative Practice Progress Dashboard, April 2017 - snapshot of progress in implementing a Restorative Practice environment and emerging themes.

• From mid-2017 the evaluation continued to track implementation, but added a focus on early outcomes.
  • Progress Report, June 2017 - detailed feedback on progress in implementing a Restorative Practice environment, and early impact of Restorative Practice.
  • The current report presents further implementation and outcomes data, and makes final evaluative conclusions about Restorative Practice. This is the final major evaluation report (the evaluation formally concludes in June 2018, when a small amount of administrative data will be updated).

The aim of the evaluation is to determine how well Restorative Practice is being implemented and what the outcomes are within in-scope schools. The key evaluation questions are:

• How well has the Restorative Practice programme been implemented in schools?
• What outcomes has the programme had:
  • On schools' restorative environments?
  • For students within the schools?
• What lessons can be learnt from the implementation of the Restorative Practice programme?

The evaluation used a mixed-methodology, involving surveys of schools, interviews (with school staff and students), and analysis of administrative data.

Scope of the evaluation 

The evaluation focuses on 27 schools: 24 secondary schools and three composite schools who began participating in the Restorative Practice programme in 2015. Other schools that have since joined the Restorative Practice Programme are not within scope of this evaluation.^[2]

This report refers to the 27 in-scope schools as Uptake schools.

• All 27 Uptake schools participate in the PB4L School-Wide programme.
• 10 of the Uptake schools are low decile (1-3), 16 are medium decile (4-7), and one is high decile (8-10).

The report also refers to Focus schools: a group of 13 of the Uptake schools that we have a robust data set for (this is further explained in Caveats and Methodology, below). 

Appendix 1 outlines the decile, size and location of the 27 Uptake schools and the 13 Focus schools. Appendix 1 also contains details of the full group of 174 schools currently implementing the Restorative Practice programme.

Caveats

As outlined above, the evaluation focuses on 27 schools who began participating in the Restorative Practice programme in 2015. The findings cannot be generalised to all 174 schools currently participating in Restorative Practice, due to the sample size.

However, we are confident that the evaluation has surfaced key issues and findings likely to be indicative of what would be found across the wider group of Restorative Practice schools because:

• The Uptake schools' are broadly similar to the full group of Restorative Practice schools (when compared by region, decile, student ethnicity and size – see Appendix 1).
• Consistent feedback was received from the schools involved in the evaluation about implementation issues, enablers and barriers (indicating that most key issues had surfaced).
• Feedback on outcomes was consistent across the schools who provided feedback (with schools reporting small but steady improvements).

It is important to note that outcomes data was not consistently provided by all 27 Uptake schools for each of the three years:

• 13 schools provided sufficient data to allow a robust analysis across the three years (referred to as the Focus schools – see Methodology, below).
• Quantitative findings in the Outcomes section are for the 13 Focus schools only (ie the most robust data set), however, the Focus schools are broadly similar to the full group of Uptake schools.
  • The Focus school data showed a similar pattern over time to the full data set for the Uptake schools – a small, steady improvement.
  • Comparison of the Focus and the Uptake school demographics shows that the two groups of schools are broadly similar (see Appendix 1).

About PB4L Restorative Practice 

The PB4L Restorative Practice programme aims to build respectful, positive relationships between students, their teachers, and the wider school community. The programme is one of a suite of seven PB4L initiatives currently being rolled out to New Zealand schools. PB4L Restorative Practice aims to:

• improve student wellbeing
• improve student engagement
• address problem behaviour
• develop professional relationships
• increase student achievement.

Specific outcomes sought by PB4L Restorative Practice include:

• a calmer school environment, with less classroom disruption and more time for teaching
• an increase in the engagement and learning of students in the classroom
• growth in relational and problem-solving skills, both for adults and students across the school community
• improvements in attitudes and relationships across the whole school community
• a consistent best-practice approach across the whole school community that aligns with the school's shared values.

Training and support for implementing PB4L Restorative Practice is being delivered through the University of Waikato's Institute of Professional Learning (IPL). IPL has five Regional Coordinators and two National Trainers, who facilitate training sessions and workshops with staff across the country, and are available to answer staff questions about Restorative Practice. The Ministry also provides support and resources for the programme (through production of the Restorative Practice Kete and regional Learning Support staff, who provide support to PB4L schools).

The Kete includes a Restorative Practice matrix for schools to assess their environment and practices against. This matrix was used for the evaluation, to track outcomes in the Environment survey.

Purpose of this report

The purpose of the report is to provide final feedback to the Ministry on the implementation of PB4L Restorative Practice, and the impact that it is having. The lessons identified in the report are intended to support decision making in the Ministry about Restorative Practice and similar programmes.

It is also expected that the report will be useful for IPL and the Uptake schools themselves, to support ongoing programme implementation and improvement.

Report content

This final evaluation report it summarises findings that have been previously reported as well as the new research to make evaluative judgements.

The new research included in this report includes (details are given in Methodology, below):

• the third (and final) iteration of the Environment survey
• interviews with six Uptake schools, involving school staff and students.

This report covers:

• implementation of Restorative Practice: training, resources, and support received by schools
• the outcomes achieved by the Uptake schools, including progress towards a restorative environment and specific changes taking place within schools
• lessons learned from the Uptake schools, relating to the design and improvement of the PB4L Restorative Practice programme.

Methodology

The evaluation used mixed-methods design to ensure that a wide range of perspectives were included.

Initial piloting of research instruments and evaluation design

Research instruments were developed and tested with 20 'pilot' schools, and further refined with the Ministry's steering group and operational policy owners. The initial piloting with schools involved in-depth interviews, focus groups with school staff, and testing of surveys.

The design, analysis and reporting of the evaluation has been informed by the Ministry's feedback throughout the evaluation period.

Ministry staff and IPL staff have provided input to analysis and report content through ongoing conversations and telephone interviews throughout the evaluation period.

Surveys

Surveys provided key data for the evaluation, and were designed to capture changes and progress in the Uptake schools over time. Note that the findings of the 2018 Environment survey are reported for the first time in this report.

Footnotes

1. These 20 schools were all PB4L School-Wide schools that were beginning to implement Restorative Practice; they were not involved in the evaluation after this initial work.
2. A total of 174 schools are currently implementing PB4L Restorative Practice.