Achievement over time for New Zealand students

Year 5 students had higher maths achievement in 2015 than in 1994.

TIMSS was first conducted in 1994, and PISA began in 2000. TIMSS is conducted every four years, while PISA is conducted every three years. The trend in maths achievement of Year 5 and Year 9 students has been measured since 1994, while PISA provides data on the maths achievement of 15-year-old students since 2003. Although maths was assessed in 2000, 2003 was the first time maths was the focus area of the PISA assessment; all four content areas of the current PISA mathematics framework were covered in 2003, but not in 2000¹.

Exhibit 1 shows the trends in maths achievement for students across the cycles of both studies.

• Year 5 maths achievement was higher, on average, in 2014 than it was for the students 20 years ago, but was not significantly² different from the last three cycles.
• For Year 9 students, there has been no significant change in average³ maths achievement since 1994.
• The average maths score of New Zealand 15-year-old students declined between 2003 and 2015, but most of the decline occurred between 2009 and 2012.

Exhibit 1: Trends in maths achievement for TIMSS and PISA

Note: A score of 500 is not the same across the year levels. However, within each year level, a score of 500 is the same across assessment years.

Range of maths achievement

The range of achievement, that is, the gap between low and high achievers, is an important measure to use to examine equity in education. Rather than using the very highest and lowest scores, the range is commonly measured as the difference between the score at which only 5% of students achieved a lower score, and the score at which only 5% of students achieved a higher score. However, even when different measures of range are used, in PISA, the range for 15-year-old students in maths has been wider than for many other countries, while in TIMSS, the range has varied over time. In recent cycles of TIMSS, the range of maths achievement has been fairly moderate for both Year 5 and Year 9 students, compared to other countries. However, for both Year 5 and Year 9 students, the range increased (and was therefore wider) in 2014/15 than in 2010/11, but not as wide as the first two cycles.

Low to advanced performance in maths

22% of 15-year-old students completed only basic maths tasks.

Both TIMSS and PISA classify students by their performance, and link this performance to descriptions of the types of maths tasks students could do.⁴ In TIMSS these classifications are called benchmarks and there are four: the advanced at 625 scale score points; the high at 550; the intermediate at 475; and the low at 400 scale score points.

In PISA these are called proficiency levels, and are numbered 1 to 6. Students at Level 6 are capable of advanced maths thinking and reasoning, whereas those at Level 1 can only complete relatively basic maths tasks. Level 2 is considered to be a baseline level at which students begin to demonstrate the competencies that will enable them to participate actively in maths-related life situations.

• Six percent of Year 5 and Year 9 students reached the advanced benchmark in TIMSS 2014/15 (see Exhibit 2). Twelve percent of 15-year-olds reached the Level 5 proficiency level or above in PISA 2015 (3% were at Level 6).
• Around half of the 15-year-old students who were top performers in PISA maths were also top performers in reading and science.
• At the other end of the spectrum, 16 percent of Year 5, and 15 percent of Year 9 students did not reach the low benchmark in TIMSS, and 22 percent of 15-year-olds performed below Level 2 proficiency in maths in PISA. These students did not perform the simple mathematics tasks designed for students at their grade and age grouping.

Exhibit 2: Proportion of high and low achievers in maths in TIMSS and PISA

Note: Benchmarks and proficiency levels are not directly comparable across year levels, but are indicative of the range of achievement within New Zealand.

Changes over time

There have been some changes at each year level in the proportions of students at each benchmark - these changes tend to reflect the movement in average maths achievement.

• In TIMSS, the proportions of Year 5 students reaching the high and advanced benchmarks have increased significantly between 2010/11 and 2014/15.
• The proportions of Year 9 students reaching each benchmark in 2014/15 were not significantly different from the previous cycle. However, there has been an increase in the proportion of Year 9 students who did not reach the low benchmark since 1994/95 (15% in 2014/15, compared with 11% in 1994/95).
• In PISA, the proportion of 15-year-old students who attained Level 5 or above declined between 2003 (21%) and 2015 (11%). The proportion of students who attained the most advanced proficiency level (Level 6) was also lower in 2015 (3%) than in 2003 (7%). In line with this decline in the more advanced proficiency levels, the proportion of 15-year old students below Level 2 increased from 15 percent in 2003, to 22 percent in 2015.

How does New Zealand compare with other countries?

Compared to other countries New Zealand maths achievement was better at secondary school than primary school.

Compared with other countries, New Zealand students' average maths achievement ranked relatively low for primary students, but was relatively higher at the secondary level (see Exhibit 3). Rankings for New Zealand change as other countries enter or leave studies or if they improve or see a decline in their scores. However, rankings help form part of the picture of maths achievement.

• The average maths achievement of New Zealand Year 5 students was significantly higher than 13 countries (out of 49 countries), but lower than 33 countries, including all the other predominantly English- speaking countries who participated at the middle primary level.⁵
• Out of the 39 countries participating in TIMSS at the lower secondary level, New Zealand Year 9 students' average maths achievement was significantly higher than 18 countries, but lower than 17 countries, including all the other predominantly English speaking countries who participated.
• In PISA, New Zealand 15-year-old students performed above the OECD average and 41 countries, but significantly below 19 countries. New Zealand had a similar average score to 9 other countries, including Australia and the United Kingdom.

High and low performers

Compared with other countries, New Zealand was around the middle when considering the proportion of top performers in maths, for both PISA and TIMSS. At the middle primary level, New Zealand had a relatively high proportion of lower performers, but by the secondary level, the proportion was similar to the middle (the international median in TIMSS, or the OECD average in PISA).

Exhibit 3: Comparisons of average maths scores for TIMSS and PISA countries

Notes:

1. The shading on these lists shows those countries significantly above or below New Zealand in a darker shade, while those that are not significantly different are shaded the same as New Zealand.
2. B-S-J-G (China) refers to the four PISA participating China provinces: Beijing; Shanghai; Jiangsu; and Guangdong.
3. FYR Macedonia refers to the Former Yugoslav Republic of Macedonia.
4. Standard errors are presented in parentheses.

Strength and weaknesses of New Zealand students within maths

New Zealand students perform better in statistics questions than geometry, algebra or number.

Both TIMSS and PISA show that statistics is an area of relative strength for New Zealand students, compared to other areas of maths. PISA 2012, the last cycle where maths was the focus, showed that questions on geometry and measurement was a relative weakness for New Zealand students. TIMSS showed that for Year 9 students, both geometry and algebra were relative weaknesses. At the Year 5 level, both number questions and geometry and measurement questions were a relative weakness, compared with statistics. At the middle primary level 'number' included the simple algebraic concepts, including pattern recognition, so this is consistent with the Year 9 information.

Both TIMSS and PISA also examine the thinking skills students are most likely to employ when answering a question or working on a problem. Both studies show that questions requiring higher-order thinking skills were a relative strength for New Zealand students, compared with relatively simple tasks such as recalling knowledge and using procedures.

These strengths and weaknesses have been fairly consistent over time within New Zealand. Note that there is no consistent pattern of strengths across high-performing countries, with each country having different combinations of strengths and weaknesses.

Maths achievement of New Zealand boys and girls

Within all age groups, in the most recent TIMSS and PISA assessments, boys and girls had similar average maths achievement; however, boys had a wider range than girls (see Exhibit 4).

In most TIMSS cycles, there has been no significant gender difference in maths achievement, but in 2010/11 there was a difference for the first time, with Year 9 boys having higher average maths scores than Year 9 girls. This difference was due to a significant decrease for the girls between 2002/03 and 2010/11. However, the decrease was reversed again so that by 2014/15, girls' average maths achievement was the same as in 2002/03. In PISA, 15-year-old boys have had higher average achievement than 15-year-old girls since 2003; the difference in 2009, however, was not statistically significant.

Exhibit 4: Average maths achievement of boys and girls in TIMSS and PISA

Notes:

1. The darker colour bar indicates the mean score for that gender is significantly higher than the other gender.
2. Standard errors are presented in parentheses.

Maths achievement of students in different ethnic groupings

Both TIMSS and PISA asked students which ethnic groupings they belonged to, with the option of selecting more than one grouping. Nearly one-quarter of middle primary students, and around one-fifth of secondary students, identified with more than one grouping; the groupings were Māori, Pasifika, Pākehā/European, Asian, and Other. There was a wide variation in maths achievement within ethnic groupings, with advanced achievers and very low achievers in all ethnic groupings.

Across both TIMSS and PISA, Māori had lower average maths achievement than non- Māori. Similarly, Pasifika students had lower average maths achievement than non-Pasifika. Analyses show that this achievement gap narrows when socio-economic factors are taken into account.

Changes over time

At the Year 5 level, the average maths achievement within ethnic groupings had not changed significantly since 2010/11. However, for secondary students, there were some significant changes.

• At Year 9, Pākehā/European girls' average maths score increased, while the average of those students classified as Other decreased. The average maths achievement for Māori and Pasifika students, as well as Pākehā/European boys, and Asian students, did not change significantly.
• For 15-year-old students, the average maths score for Māori, Asian, and Pākehā/ European students decreased significantly between 2003 and 2015. For Māori and Pākehā/European students the decrease mainly occurred between 2009 and 2012. Over this period, average scores for Pasifika students did not change significantly.

Maths achievement and socio-economic status

In PISA, compared to earlier cycles, a student's socio-economic background is not such a strong predictor of their achievement.

Regardless of the measure of socio-economic status, TIMSS and PISA both show that students in homes with many resources for learning had higher maths achievement, on average, than those whose homes had fewer resources. Similarly, at a school level, maths achievement was higher, on average, for students in schools with more affluent students than those in schools with more economically disadvantaged students. In both studies, the difference in maths achievement between these two groupings within New Zealand was higher than in most other countries. However, in PISA, compared to earlier cycles, a student's socio-economic background is not such a strong predictor of how well they have achieved.

Footnotes

1. The PISA 2000 mathematics assessment only covered space and shape, and change and relationships.
2. The word 'significant' in this report is used to describe statistical significance. Statistical significance means we are 95% certain that we would get the same results if the study was repeated with a different sample.
3. The 'average' refers to the mean unless otherwise stated.
4. Full descriptions of benchmarks are provided in Mullis, Martin, Foy, & Hooper (2016), and descriptions of proficiency levels are provided in PISA 2015 Assessment and Analytical Framework (OECD, 2016a).
5. Year 5 is referred to as middle primary, and Year 9 is referred to as lower secondary where international comparisons are drawn.

References

• Caygill, R., Singh, S., & Hanlar, V. (2016). Mathematics Year 5: Trends over 20 years in TIMSS. Wellington: Ministry of Education.
• Caygill, R., Hanlar, V., & Singh, S. (2016). Mathematics Year 9: Trends over 20 years in TIMSS. Wellington: Ministry of Education.
• May, S., with Flockton, J., & Kirkham, S. (2016). PISA 2015 New Zealand summary report. Wellington: Ministry of Education.
• Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in mathematics.
• OECD. (2016a). PISA 2015 assessment and analytical framework: Science reading, mathematic and financial literacy. Paris: Author.
• OECD. (2016b). PISA 2015 results (volume 1): Excellence and equity in education. Paris: Author.