Summary 

This is the first comprehensive study by the Ministry on the use of the e-asTTle tool for research purposes. e-asTTle is used by a large number of schools to assess a large number of students from Years 1 to 10 for mathematics, reading and writing.  It provides a robust and rich source of achievement data that can be analysed to add to the current evidence base regarding achievement and progress in mathematics, reading and writing. 

Since the main purpose of the tool is to support teachers in their teaching, it is important to understand the strengths and limitations of the tool for research purposes. Therefore, the first objective of this project was to assess the potential of the e-asTTle dataset as a source of progress and achievement data for research purposes.

The second objective was to report on initial findings from the data as a way of illustrating its potential. The analysis was mainly descriptive and was informed by the following overarching research questions:

• How well do students perform in mathematics, reading and writing, and how does this compare to other findings?
• What is the pattern of student achievement/progress over time? How does this pattern relate to the expectations outlined in the New Zealand Curriculum?
• To what extent does student performance differ when analysed by student and school characteristics?

To fully understand how the different characteristics of students or their schools are related to student achievement, techniques that are able to control for the interaction between the different characteristics need to be used.  This research does not attempt to do that; it does not study causality or try to establish which student or school characteristics explain high or low performance.

Key Findings

The e-asTTle dataset and its use for research

• Data generated by e-asTTle as part of the assessment process in schools, provides a rich dataset that can be used for research into the achievement and progress in mathematics, reading and writing for Years 1 to 10.
• The assessment tool is not used evenly across different types of schools and it is used for different purposes throughout the year.  This means that subsets of the full data need to be selected purposefully and adjustments for bias in the data may need to be considered..
• The end-of-year sample allows for computing achievement measures that can be better compared with the expectations described in the New Zealand Curriculum and to therefore understand year on year progress.
• Data for years 9 and 10 students has been weighted by school decile to adjust for an over representation of students from lower decile schools observed in the dataset.

Achievement and progress in mathematics, reading and writing from 2011 to 2016

• The majority of students at the end of Year 4 are achieving within level 2 of the curriculum or above for mathematics, reading and writing, which reflects the expectation of the New Zealand curriculum.
• At least a third of students at the end of their primary education (end of Year 8) are achieving scores in writing and mathematics within curriculum levels that are lower than the level expected by the New Zealand Curriculum.  
• Results relating to achievement against curriculum expectations are broadly consistent with results from the National Monitoring Study of Student Achievement for all three learning areas.
• As expected, students' levels of achievement in all the three learning areas improve as students move through the years of schooling. The rate of achievement, or progress, declines on average as students move up the years of schooling.
• The transition from Year 8 to Year 9 is associated with the lowest mean annual progress for all three learning areas. 
• There is a wide variation in the yearly progress made by students in the same year level for the three learning areas.  
• Overall achievement scores in mathematics, reading and writing differ widely for students at the same year level. 
• Difference in average achievement can also be seen when the data is analysed by school decile, ethnicity and gender. These observed differences are consistent with evidence from international studies and other national achievement data.
• Unlike achievement, there is no clear relationship between annual progress in the three different learning areas and school decile, gender or ethnicity. Therefore, the differences in achievement by school decile, gender and ethnicity observed at higher levels of primary school reflect different starting points rather than differing progress throughout Years 4 to 8.