Executive Summary

Introduction

There are three programmes; one programme for literacy: Accelerating Literacy Learning (ALL) and two programmes for mathematics: Accelerating Learning in Mathematics (ALiM) and Mathematics Support Teacher (MST). The ALL is aimed at students identified as below and well-below the National Standards in reading or writing. The ALiM programme is aimed at students below the National Standards in mathematics and the MST programme is aimed at students identified as well-below the National Standards in mathematics.

The programmes are designed to be delivered by an effective mathematics or literacy teacher within a school. The teacher delivers the programme to small groups of students in addition to regular classroom teaching. The teachers receive support from a mentor throughout the programme. ALiM and ALL are 10-15 week programmes. The MST programme runs across the whole-school year with students receiving support for around two-terms (20 weeks). A requirement of the MST programme is that teachers complete a post-graduate paper.

Participating Schools

Schools were selected to participate in the programmes by regional Ministry of Education offices with input from mathematics facilitators, literacy advisors and the PfS National Leaders. School selection criteria included effective school leadership, having students identified as below or well-below national standards and effective classroom practice in literacy (ALL) and/or mathematics (ALiM/MST).

In 2012, 307 schools took part in ALL programmes, 337 schools took part in ALiM programmes and 76 schools took part in MST programmes.

Evaluation Methodology

Three evaluative studies were undertaken to see how the programmes were developing and help inform the PfS programme leaders to make adjustments to programme operation and delivery. The studies were designed to look at student progress in ALL and ALiM; the first six months of the MST programme; and factors contributing to successful and sustainable programmes.

The research questions that related to each study were:

1. What progress did students who participated in ALiM and ALL programmes in 2012 make? Was this progress accelerated?
2. How did the MST programme operate in schools during the first six months of implementation? What progress did students involved in the MST programme make?
3. What factors contributed to the success and sustainability of the programmes in schools?

The evaluation collected data from a number of sources: online surveys, analysis of student achievement data, interviews and school reports. Student achievement data were sought from all schools involved in the programmes in 2012. The student achievement data collected pre- and post- programmes were:

Visits were made to 12 MST schools in August and September 2012. Online surveys were emailed to all MSTs and MST school principals. Visits were made to an additional 15 ALiM and ALL schools to understand success and sustainability factors of the programmes.

Findings from the Evaluative Studies

Student Progress in ALiM and ALL in 2012

The pre and post programme student achievement analyses shows the 2012 programmes were successful in accelerating the progress of students in mathematics, writing and reading. In this study, progress was looked at in relation to norm expectations and whether the progress made by students was to expected levels for their year group by the end of the programme.

Overall, the majority of students participating in the ALiM and ALL programmes in 2012 made more than expected progress over the timeframe of the programmes (10-15 weeks). Students in ALiM, on average, made more than two terms progress on PAT assessments and students in some Years 4, 5 and 7 ended the programme above the expected levels for their year. Students, on average, made more than one year's progress as measured by GLoSS (Global Strategy Stage). These patterns were observed across gender and ethnicity.

E-asTTle writing student achievement data showed progress in writing was in excess of one years' progress. For some year groups this meant the students ended the programmes close to the expected levels. In junior literacy programmes most students made progress on all aspects of the Observation Survey and on average reached the expected stanine for their age.

MST Programme

The findings from this initial exploratory research show the MST programme was leading to positive changes for students, teachers and schools. The MST programme had only been operating in schools for six months when the fieldwork was undertaken. The post-graduate paper had been well received and was assisting teachers to change their practice. The paper was seen as valuable, interesting and relevant but had led to additional work for the MST teacher.

The MST programme student achievement analyses shows the programmes implemented in schools were effective in accelerating progress for most students. The GLoSS assessment data showed 70 percent of students increased one or more stages over the intervention. A year's progress is around one stage. For PAT the mean scale score progress made by students was above that expected for two terms. The length of programme students received was not provided in the data. However, students were expected to receive programme support for around 20 weeks or two terms, if this was the case it suggests accelerated progress was achieved by many students.

Factors Contributing to Successful Programmes

A 2011 evaluation³ by Wilf Malcolm Institute of Educational Research (WMIER) identified key elements of programme organisation that contributed to programme success. These were small groups, targeted teaching, sessions of 30 or more minutes at least four times a week and a school focus on using student achievement data.

The Ministry refined the PfS model for 2012 based on the evaluation findings and schools reports from 2011. Messages at the planning days and the accompanying documentation encouraged schools to accelerate progress for their target students but also encouraged schools to plan for how they would maintain or sustain any positive changes from the programme.

The 2012 evaluative studies sought to identify factors that contributed to success and sustainability of the programmes by purposefully focusing on more successful schools based on their student achievement data. The 2012 evaluation visited 27 schools that had run successful programmes in terms of improvements in student achievement. Through these visits it was evident there were common important elements of the programmes run by these schools. These elements were:

• Leadership and strategic planning
• Programme organisation
• Inquiry teaching
• Capability building through professional learning communities, support – mentors/advisors/coach
• Strong relationships between students, teacher, parents/whānau and communities
• A focus on student's need/outcomes.

Leadership and Planning

Principal and leadership support are key to on-going success and sustainability. School leaders were responsible for managing the organisation elements (release time, relief teacher) and set the tone and expectations for the programme within the school. With effective support the programmes were able to reach a wider number of students and influence additional teaching staff, not just those directly involved in the programme. Without this support programmes were likely to only impact on student achievement for those in the programme.

Leadership enabled a whole-school focus on literacy or mathematics by giving priority and visibility to the ALL, ALiM or MST programme which, in some schools, included a restructure of resources and timetables. Through leadership support, teachers also felt supported to make pedagogical changes.

Programme Organisation

Previous participation in PfS was helpful to schools as it made the programme organisation easier to manage, set expectations of success and enabled schools to focus on capability building and sustainability.

Use of multiple teachers, mentors and inquiry teams was common in the successful schools. These techniques helped to transfer the knowledge gained by the programme teachers to other teachers to improve teaching practice and the wider culture of learning.

Inquiry Teaching

Involving multiple teachers in PfS and providing time for teachers to share ideas and suggestions about students through inquiry teaching was powerful in building trust, respect and inter-classroom connections. Successful schools offered teachers opportunities to share and collaborate in a safe environment using inquiry teaching. Involving more than one teacher requires good organisation at the beginning of the programme but the outcomes for schools that did this suggest this effort is rewarded. The inclusion of multiple teachers also allows for capability building, succession planning and sustainability.

Working in inquiry teams meant teachers felt supported to take queries and difficulties to their team meetings and share ideas. Some schools also organised collaborative planning sessions and moderation of students' work. Teachers in these schools commented on the value of the team approach as this shared the responsibility and helped all team members to adapt their teaching through learning conversations and reflections.

Capability Building

Teachers in the successful schools found the programmes provided them with learning and development opportunities. This is particularly true for the MST programme due to the post-graduate paper but many ALiM and ALL teachers saw the programmes as a learning and development experience.

Schools benefited from building knowledge gained from running the programmes within their own school and using this to further develop and refine their programmes into the future.

Relationships

Knowing their students and having strong student-teacher relationships were important to accelerate student progress. Building students' trust and confidence in the subject were also important steps for the programme. Working in small groups in safe environments with student focused tasks helped to develop those relationships.

Relationships between programme and classroom teachers were enhanced through use of inquiry teams and led to higher expectations for students across the schools. Schools noted strong relationships with whānau and families were also important to support student learning. Schools used different approaches to encourage whānau and families to come to school, for example some schools had success with breakfasts, open evenings and invitations to visit sessions. This was an area where most schools visited felt they needed to continue to focus.

Sustainability

Many of the schools visited had seen positive changes through their participation in PfS and acknowledged the value of continuing with a form of the programme after the funding ended. In order to be able to maintain the PfS programme some schools had restructured roles within schools.

The factors that enabled schools to run successful programmes were the same factors that led to sustainable programmes. The degree to which the programmes were being sustained after the Ministry funding finished and reaching more students than the initial group was dependent on the schools ability to disperse the programme throughout the school. If factors such as leadership, inquiry teaching, student focus, strong relationships and capability building were present the programmes were likely to be successful as well as sustainable over time.

Conclusion

Looking across the three studies the evidence suggests that the 2012 ALiM, ALL and MST programmes were effective in accelerating the progress of students who were not achieving the National Standards. The majority of ALiM and ALL writing students made progress that exceeded expected progress during the programmes but not all reached the level expected for their year by the end of the programmes. The findings were mixed for those students who participated in ALL junior literacy and MST. This is partly due to data measurement issues and partly due to the amount of progress needed to reach expected levels as many students began the programmes well-below those levels.

PfS are not generic interventions, rather they offer an opportunity for a school to use release time for their teachers in the most suitable way to raise student achievement. Principals and teachers said the programmes provided an opportunity to make on-going improvements for their under-performing students and also for the teaching and learning at their school.

Footnotes

1. A brief summary of each tool is contained in the Appendix.
2. Re-calibration of the e-asTTle writing tool occurred in April 2013. As both pre and post programme data was collected in 2012 the results are not affected by the re-calibration.
3. Cowie, B. et al (2012). Evaluation of Literacy and Mathematics Additional Learning Programmes for Students 2011. Wellington: Ministry of Education.