Executive Summary

The Trends in International Mathematics and Science Study, 2002-2003 (TIMSS-02/03), conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA), was the third in a cycle of international assessments designed to measure trends in mathematics and science education. This report presents an overview of the mathematics and science achievement results for Year 5 students for New Zealand’s participation in TIMSS in 2002. Data collected in this cycle are examined in comparison with that collected from the previous two cycles. Contextual information, such as students’ demographic background, as well as attitudes to mathematics and science, is also provided and the relationship with achievement discussed.

Background

The first cycle, TIMSS-94/95, was an extensive study involving students from Years 4 and 5, Years 8 and 9, and Years 12 and 13¹, with the administration conducted during 1994 and 1995. The release of the results from this study in mid-1997 highlighted areas of concern in mathematics and science education in New Zealand. In particular, the study showed that Years 4 and 5 students were generally achieving well below the international mean in mathematics and were very weak in some aspects of science. Concerns with New Zealand students’ achievement combined with teachers’ reported difficulties implementing the new curricula during this period, led to the establishment of the Mathematics and Science Taskforce in August 1997. The taskforce recommended changes including new and revised teacher resource materials. This has been an ongoing development along with increased initiatives for in-service teacher education.

In 1998 the second cycle of the study, TIMSS-98/99, was administered in New Zealand. This second cycle was not as extensive as TIMSS-94/95 as the focus internationally was on lower secondary or Grade 8; in New Zealand this was Year 9. However, New Zealand also chose to do a replication of the middle primary 1994-1995 assessment for Year 5 students at the same time in 1998.

Assessment Content

TIMSS assesses two learning areas: mathematics and science. The design of the study is such that a range of cognitive skills are tested in the context of these two learning areas using a variety of test question formats. At the middle primary level, the mathematics questions require students to use their knowledge, reasoning, and problem-solving skills in five domains: Number, Algebra (Patterns and Relationships), Measurement, Geometry, and Data. The science questions assess students’ knowledge, conceptual understanding, and reasoning and analytical skills to solve problems from three domains: Life Science, Physical Science, and Earth Science.

Final assessment match with the New Zealand intended curricula

In order to aid interpretation of the TIMSS assessment scores, participating countries undertook the Test Curriculum Matching Analysis (TCMA). The purpose of this analysis was to determine which of the assessment questions, in terms of the mathematics and science content, were in the intended curriculum. In New Zealand, using the content of Levels 1 through to 3 from the respective curricula documents, just over 80 percent of the science questions were judged to be in the intended curricula for Year 5 students. The mathematics questions ranged by content area from 70 percent of measurement questions to 96 percent of algebra questions judged to be in the intended curriculum for Year 5 students.

Based on the estimations of their teachers, the majority of Year 5 students who participated in TIMSS in 2002 were working at Level 3 for each strand within both the mathematics and science curricula.

Achievement in mathematics of Year 5 students

• New Zealand Year 5 students, on average, achieved at about the international mean in mathematics for the 25 countries participating in TIMSS-02/03 at the middle primary level.
• New Zealand was one of six countries that recorded a significant improvement in mean mathematics achievement between TIMSS-94/95 and TIMSS-02/03.
• There were significantly higher proportions of New Zealand students reaching the High, Intermediate and Low benchmarks in 2002/03 relative to 1994/95; there was virtually no change in the proportion reaching the Advanced benchmark.
• There was no difference between the mean mathematics achievement of boys and girls in 2002. Both boys and girls showed a significant improvement in the eight-year period between 1994 and 2002.
• There was no significant difference observed between the mathematics achievement of Pākehā/European and Asian students; similarly there was no difference between Māori and Pasifika students. However, Pākehā and Asian students achieved significantly higher mean mathematics scores than Māori and Pasifika students.
• The improved performance of New Zealand Year 5 students since 1994 appears to be due primarily to the higher mean scores obtained by students who were identified by their school as Māori or Pasifika. Three measures show this improvement: significantly higher mean scores in 2002; significantly greater proportions achieving at or above international benchmarks; and reduced effect sizes for these students when compared with Pākehā/European and Asian students.
• Relative to other countries, as well as Year 5 students overall performance, New Zealand students were stronger in Geometry and Data and relatively weak in Number. Girls performed significantly better than boys in Geometry.

Achievement in science of Year 5 students

• New Zealand Year 5 students, on average, achieved significantly above the international mean in science for the 25 countries participating in TIMSS-02/03 at the middle primary level.
• New Zealand was one of nine countries reporting a significant increase in mean science achievement between TIMSS-94/95 and TIMSS-02/03.
• There were significantly higher proportions of New Zealand students reaching the Low and Intermediate benchmarks in 2002/03 relative to 1994/95; there was no significant change in the proportions reaching the High and Advanced benchmarks.
• There was no difference between the mean science achievement of boys and girls in 2002. Both boys and girls showed a significant improvement in the eight-year period between 1994 and 2002.
• There was no significant difference observed between the science achievement of Pākehā/European and Asian students; similarly there was no difference between Māori and Pasifika students. However, Pākehā and Asian students achieved significantly higher mean science scores than Māori and Pasifika students.
• The improved performance of New Zealand Year 5 students since 1994 appears to be due primarily to the higher mean scores obtained by students who were identified by their school as Māori or Pasifika. Three measures show this improvement: significantly higher mean scores in 2002; significantly greater proportions achieving at or above international benchmarks; and reduced effect sizes for these students when compared with Pākehā/European and Asian students.
• New Zealand students overall showed quite even levels of competency across Life, Physical and Earth Sciences. Girls performed significantly better than boys in both Life and Physical Science.

Attitudes to mathematics and science of Year 5 students

• New Zealand middle primary students generally enjoyed learning mathematics and science. There was a significant increase between TIMSS-94/95 and TIMSS-02/03 in the proportions of New Zealand students who reported enjoying these subjects a lot.
• About the same proportions of girls and boys reported that they enjoyed learning mathematics and science. The relationship between enjoyment and achievement for each subject was linear for boys but less clear-cut for girls.
• The majority of students in each ethnic grouping in the New Zealand TIMSS-02/03 sample reported that they enjoyed learning mathematics and science a lot. The reported levels of enjoyment learning mathematics or science were generally not related to achievement for Pākehā/European, Pasifika or Asian students. For Māori students, while the enjoyment of learning mathematics had little relationship with achievement, those who reported enjoying learning science a lot had higher mean achievement than those who did not.
• About half of the New Zealand Year 5 students in TIMSS-02/03 expressed a high level of self-confidence in mathematics and science.
• Students who had a high self-concept in learning mathematics or science usually, on average, achieved higher scores in that subject than those students who reported having a low self-concept.
• Proportionally more boys than girls in New Zealand expressed a high level of self-confidence in both mathematics and science. The relationship between confidence and achievement observed for all students was also evident within each gender group.
• Proportionally more Pākehā/European students, when compared with students in other ethnic groupings, expressed a high level of self-confidence in both mathematics and science. Nevertheless, students who were more confident did better in both the mathematics and science assessments.

Year 5 students’ background

• The proportion of students in each ethnic grouping has changed since 1994, with greater proportions of students being classified in the Asian, Pasifika, and Other ethnic groupings. Consistent with this trend, a greater proportion of students reported they were not born in New Zealand in 2002 compared with 1994.
• Students who always spoke the test language at home achieved, on average, higher in mathematics and science than those who sometimes spoke the test language. The difference between the mean achievements of students in the two language categories has decreased since 1994.
• The household composition with regards to the people living with the student made little difference to achievement.
• Students with more books in the home and those with a greater number of possessions in the home (indicating higher economic status) had higher mean achievement in mathematics and science.
• Consistent with previous studies, the proportion of students experiencing unsafe behaviours at school was relatively high compared with their international counterparts; however, most students liked being at school.

Final comment

This report does not cover all the data collected during TIMSS 2002; for New Zealand Year 9 achievement results see Chamberlain, M. C. (2007). In addition a number of reports published both nationally and internationally are listed at the end of this report. Data collected from school principals and mathematics and science teachers is only touched upon in this report and further analysis of the data could provide insight into Year 5 student achievement in mathematics and science in 2002.

TIMSS 2002 provides a second international data point on which comparisons can be made with other countries for Year 5 students. It is encouraging to note the improvement in achievement since the 1994-1995 TIMSS study which sparked the further development of teacher resources. However at the point in time of the 2002 assessment, many of the more recent initiatives, for example the Numeracy project, would have had little time to have bedded down and for the potential improvements to have been fully realised in the Year 5 student population. The fourth cycle of TIMSS, TIMSS-06/07 conducted in New Zealand in 2006 at the Year 5 level only, will provide a chance to examine the Year 5 student achievement at a point when the initiatives should be starting to impact more widely amongst primary school students. Data from this fourth cycle of TIMSS will be published in December 2008.

Footnote

1. In 1994 and 1995 these class levels used the ‘standard’ and ‘form’ nomenclature, therefore in previous reports these students are said to be in Standard 2 and 3, Form 2 and 3, and Form 6 and 7.