New Zealand mathematics achievement in 2006 in an international context

As shown in Figure 2, the mean mathematics score for New Zealand Year 5 students in TIMSS 2006/07 was 492 scale score points. New Zealand’s mean score was similar to the Czech Republic (486), Scotland (494), the Slovak Republic (496) and Armenia (500), and significantly higher than 12 other countries. In contrast, 19 countries had higher mean mathematics achievement, including Singapore (599), England (541), the United States (529) and Australia (516).

The range of achievement (from the 5th to 95th percentile) in New Zealand was 284 score points from 341 (the 5th percentile) to 626 (the 95th percentile)¹. This was relatively wider than the ranges of many of the higher achieving countries but much the same as that of England (284). Another measure of spread, the inter-quartile range (from the 25th to 75th percentile) can also be examined. For New Zealand (117) this was relatively wider than the higher performing countries.

Given the number of countries now participating in TIMSS, it is more meaningful to compare New Zealand to a selection of countries (such as English-speaking or high-performing). Compared to the countries that tested in English (Singapore, England, the United States, Australia, and Scotland), New Zealand had significantly lower mathematics achievement, on average, than all of them except for Scotland (numerically New Zealand was lower but the difference is not significant).

Alongside Figure 2, Table 6 presents some information to help put mathematics achievement in context. Countries are presented in the same order as in Figure 2. It contains information on the number of years of primary schooling students will have undertaken by the time of the assessment, along with students’ average age at the time of testing. Also given in the table is the average number of hours of time spent in mathematics instruction during the assessment year according to teacher reports. Three bits of information are presented about the economic circumstances, on average, across each country, the Human Development Index, and two versions of the Gross National Income per Capita (described later).

New Zealand spends less time at the middle primary level teaching mathematics, on average, according to teacher reports, than any of the other English-speaking countries. However, New Zealand did spend more time on mathematics instruction than the top-performing non-English speaking countries with the exception of Hong Kong SAR. Teachers in Hong Kong SAR reported spending a similar number of mathematics instructional hours as New Zealand.

Table 6 also presents the Human Development Index provided by the United Nations Development Programme (UNDP – for details see 'Human Development Report 2007/2008', p. 229-232). This index was included by Mullis, Martin, and Foy (2008) in the international reporting to provide some context around the economic and educational development of TIMSS participating countries. The index ranges from a minimum value of 0 to a maximum value of 1, with high values indicating that people in a country generally enjoy long life expectancy, high levels of school enrolment and adult literacy, and a good standard of living as measured by per capita GDP. New Zealand was relatively high on this scale with a value of 0.943, similar to that of Italy (0.941), and England and Scotland (0.946 – this value is actually for the United Kingdom as no disaggregated data is available for England and Scotland) and lower than that of Australia (0.962) and the United States (0.951).

Perhaps easier to relate to than the HDI, two versions of the Gross National Income (GNI) per Capita are also presented in Table 6. The first of the two columns gives the GNI per Capita in United States dollars while the second is an adjusted value that takes account of comparative purchasing power between each country and the United States. Compared to the countries that assessed in English, New Zealand has the lowest income regardless of which of these values is used.

Figure 2: Distribution of middle primary mathematics achievement in TIMSS 2006/07

Note:
* Met guidelines for sample participation rates only after replacement schools were included.
** Nearly satisfied guidelines for sample participation rates only after replacement schools were included.
1 National Target Population does not include all of the International Target Population defined by TIMSS.
2 National Defined Population covers 90% to 95% of National Target Population.
► Kuwait and Dubai, UAE tested the same cohort of students as other countries, but later in 2007, at the beginning of the next school year.
Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.

Source: Adapted from Exhibits 1.1 and D.1 Mullis, Martin, and Foy, 2008.
 

Table 6: Selected contextual factors for TIMSS 2006/07 countries

Country	Years of formal schooling*	Average age at time of testing	Human Development    Index**	Gross National Income per Capita (in US dollars)	GNI per Capita (Purchasing Power Parity)	Average hours of instructional time in mathematics (teacher reports)
Hong Kong SAR	4	10.2	0.937	29040	39200	150 (3.4)
Singapore	4	10.4	0.922	28730	43300	201 (0.8)
Chinese Taipei	4	10.2	0.932	17294	-	112 (2.6)
Japan	4	10.5	0.953	38630	32840	136 (1.2)
Kazakhstan	4	10.6	0.794	3870	8700	133 (1.8)
Russian Federation	4	10.8	0.813	5770	12740	110 (1.3)
England	5	10.2	0.946	40560	33650	183 (2.1)
Latvia	4	11.0	0.855	8100	14840	121 (3.1)
Netherlands	4	10.2	0.953	43050	37940	179 (4.6)
Lithuania	4	10.8	0.862	7930	14550	118 (1.7)
United States	4	10.3	0.951	44710	44070	171 (3.7)
Germany	4	10.4	0.935	36810	32680	145 (1.5)
Denmark	4	11.0	0.949	52110	36190	125 (1.2)
Australia	4	9.9	0.962	35860	33940	174 (5.4)
Hungary	4	10.7	0.874	10870	16970	110 (1.3)
Italy	4	9.8	0.941	31990	28970	201 (2.8)
Austria	4	10.3	0.948	39750	36040	126 (1.1)
Sweden	4	10.8	0.956	43530	34310	104 (2.3)
Slovenia	4	9.8	0.917	18660	23970	141 (1.0)
Armenia	4	10.6	0.775	1920	4950	133 (3.4)
Slovak Republic	4	10.4	0.863	9610	17060	143 (0.6)
Scotland	5	9.8	0.946	40560	33650	181 (2.7)
New Zealand	4.5 - 5.5	10.0	0.943	26750	25750	148 (1.8)
Czech Republic	4	10.3	0.891	12790	20920	144 (1.1)
Norway	4	9.8	0.968	68440	50070	115 (2.5)
Ukraine	4	10.3	0.788	1940	6110	104 (1.4)
Georgia	4	10.1	0.754	1580	3880	130 (1.5)
Iran, Islamic Rep. of	4	10.2	0.759	2930	9800	105 (2.6)
Algeria	4	10.2	0.733	3030	5940	177 (4.7)
Colombia	4	10.4	0.791	3120	6130	175 (4.7)
Morocco	4	10.6	0.646	2160	3860	162 (2.5)
El Salvador	4	11.0	0.735	2680	5610	147 (2.6)
Tunisia	4	10.2	0.766	2970	6490	166 (1.6)
Kuwait	4	10.2	0.891	30630	48310	x x
Qatar	4	9.7	0.875	-	-	x x
Yemen	4	11.2	0.508	760	2090	134 (7.2)

Note:
* Represents years of schooling counting from the first year of primary schooling.
** Taken from United Nations Development Programme’s Human Development Report 2007/2008. See Mullis, Martin, and Foy for details.
*** Data on GNI taken from the World Bank’s 2008 World Development Indicators. Purchasing Power Parity adjusts the GNI to take account of comparative purchasing power between the country and the United States.
Standard errors are presented in parentheses

Source: Adapted from Exhibits 3, 1.1, and 5.2, Mullis, Martin, and Foy, 2008.

International trends in mathematics achievement at the middle primary level

There are several ways that trends since 1994 can be examined for the countries participating in TIMSS. The analyses presented here will include only those countries that have participated in all three international cycles, 1994/95, 2002/03, and 2006/07. Table 7 shows the change in mean mathematics scores since 1994/95, ordered so that those countries that have had the biggest positive change since the first cycle are at the top and those with the biggest negative change are at the bottom.

Table 7: Trends in middle primary school mean mathematics achievement in three cycles of TIMSS

Country	1994/95 to 2006/07 difference	2002/03 to 2006/07 difference
England	57 (4.4) ▲	10 (4.7) ▲
Hong Kong SAR	50 (5.4) ▲	32 (4.8) ▲
Slovenia	40 (3.6) ▲	23 (3.2) ▲
Latvia	38 (5.1) ▲	4 (3.8)
New Zealand	23 (5.0) ▲	-3  (3.2)
Australia	22 (4.9) ▲	17 (5.3) ▲
Iran , Islamic Rep. of	15 (6.6) ▲	13 (5.7) ▲
United States	11 (3.8) ▲	11 (3.4) ▲
Singapore	9 (5.9)	5 (6.7)
Japan	1 (2.8)	4 (2.6)
Scotland	1 (4.7)	4 (3.9)
Norway	-3 (4.1)	22 (3.5) ▲
Hungary	-12 (5.1) ▼	-19 (4.8) ▲
Netherlands	-14 (3.7) ▼	-5 (3.0)

Note:
▲ 2006/07 score significantly higher.
▼ 2006/07 score significantly lower.
Standard errors are presented in parentheses.

Source Adapted from Exhibit 1.3 Mullis, Martin, and Foy, 2008.

 

England is the country with the largest change over time in mean mathematics score. Hong Kong SAR, Slovenia, Latvia, New Zealand and Australia have also all had significant increases in mean mathematics achievement since 1994. Both England and Hong Kong have undergone curriculum reforms since the first implementation of TIMSS. In addition Latvia and Slovenia have both made significant changes in their education systems since 1994; summaries of these along with summaries of the changes in England and Hong Kong are presented in the following paragraphs.

England

The English National Curriculum was revised in 1999/2000 and the non-statutory National Numeracy Strategy was introduced in 1998 then formally implemented at primary school level in 1999, influencing the teaching of mathematics particularly at primary level. The English curriculum is structured in three age related key stages: key stage 1 is years 1 and 2 of primary school, ages 5 to 7; key stage 2 is years 3 to 6 of primary school, ages 7 to 11; and key stage 3 is years 7 to 9 of secondary school, ages 11 to 14. Students are assessed in English and mathematics at the end of each of these key stages; for the students at ages 11 and 14, their tests are externally scored and the school-by-school results published nationally (Ruddock, 2008).

Hong Kong

In Hong Kong, a new curriculum framework (including new syllabuses) was introduced in primary schools in 2002, with a move away from traditional rote learning to a more student centred learning system. The Basic Competency Assessment (BCA), comprising a student assessment and a system assessment, was also introduced from 2003 to help monitor learning at key stages for Chinese, English and Mathematics (Grades 3, 6, and 9). The student assessment consists of an online resource bank that teachers can use to design appropriate assessment tasks for their students. The system assessment is administered at the territory level by the government and provides feedback to schools, which can then feed into schools’ planning around effectiveness in learning and teaching (Tse & Loh, 2007 and Lam, 2002).

The Hong Kong government has also increased its focus on teacher education and qualifications. Mathematics specialists can be found teaching students at primary school level. From the 2004-2005 school year, all graduates of pre-service primary and secondary teacher education programmes have been degree holders. The percentage of primary school teachers who are degree holders has increased to 80.4 percent in the 2006-2007 school year, compared to 49.6 percent in the 2001-2002 school year (Leung & Leung, 2008).

Latvia

Since 1998, Latvia has had a basic education standard for students in grades 1 to 9 (aged 7 to 16).² Subject standards, which are part of the basic education standard, determine the main aims and tasks of the subject, the mandatory content of the subject, and the forms and order of the evaluation of achievement. The number of lessons per week is set nationally and mandatory. In grades 1 to 4 students have one teacher for all subjects; from grade 5, students have specialist subject teachers. Latvian students have tests in all grades, but the first national assessments occur at the end of grade 3 (students aged 9 and 10 – see Geske, Grinfelds, & Ozola, 2008).

Slovenia

Slovenia has been undergoing some significant changes in its schooling system, the most obvious of which is the lowering of the school starting age from 7 to 6 and revised national curricular documents for all levels of pre-university education. The goal of the reforms, implementation of which began in 1999, are:

“a higher level of interconnectedness of disciplinary knowledge, and increased active role of students, internationally comparable standards and levels of knowledge, improvement in functional literacy, and an increase in the quality and longevity of acquired knowledge.” (Japelj Pavešić, 2008, p. 537).

The Slovenian syllabus specifies the exact number of yearly and weekly lessons for individual subjects. In grades 1 to 3, nearly all subjects are taught by general class teachers. During grades 4 to 6, specialist teachers become more and more involved in the teaching process.

Relative rankings

In many summaries of the international data, relative rankings of mean scores are used to describe change. This is not a particularly desirable practice as any mean scores derived from a sample and ascribed to a population have some level of uncertainty around them and rankings ignore this uncertainty. In addition, some presentations of rankings fail to mention the number of countries included in the ranking.

Table 8 presents relative ranking changes between 1994/95 and 2006/07. This should be read with caution, because, although a country may be ranked higher, the mean scores may not be significantly different when the uncertainties are taken into account. For example, the mean mathematics achievement for Hong Kong SAR and that of Singapore in 2006/07 are not significantly different.

Table 8 shows that for New Zealand, the mean mathematics scores have improved quite considerably between 1994 and 2006, and there has been very little movement in our position in the ‘league tables’ when only those countries in both assessments are included. Despite New Zealand’s Year 5 mean mathematics score rising over time, it has remained lower than the mean of the 14 countries as this mean has also risen over time.

Table 8: Middle primary mean mathematics scores for countries participating in three cycles of TIMSS from 1994/95 to 2006/07

1994 /95 mean mathematics score		2002/03 mean mathematics score		2006/07 mean mathematics score
Singapore	590 (4.5) ▲	Singapore	594 (5.6) ▲	Hong Kong SAR	607 (3.6) ▲
Japan	567 (1.9) ▲	Hong Kong SAR	575 (3.2) ▲	Singapore	599 (3.7) ▲
Hong Kong SAR	557 (4.0) ▲	Japan	565 (1.6) ▲	Japan	568 (2.1) ▲
Netherlands	549 (3.0) ▲	Netherlands	540 (2.1) ▲	England	541 (2.9) ▲
Hungary	521 (3.6) ▲	Latvia	533 (3.1) ▲	Latvia	537 (2.3) ▲
United States	518 (2.9) ▲	England	531 (3.7) ▲	Netherlands	535 (2.1) ▲
Latvia	499 (4.6)	Hungary	529 (3.1) ▲	United States	529 (2.4) ▲
Australia	495 (3.4) ▼	United States	518 (2.4)	Australia	516 (3.5)
Scotland	493 (4.2) ▼	Australia	499 (3.9) ▼	Hungary	510 (3.5) ▼
England	484 (3.3) ▼	New Zealand	496 (2.1) ▼	Slovenia	502 (1.8) ▼
Norway	476 (3.0) ▼	Scotland	490 (3.3) ▼	Scotland	494 (2.2) ▼
New Zealand	469 (4.4) ▼	Slovenia	479 (2.6) ▼	New Zealand	492 (2.3) ▼
Slovenia	462 (3.1) ▼	Norway	451 (2.3) ▼	Norway	473 (2.5) ▼
Iran, Islamic Rep. of	387 (5.0) ▼	Iran, Islamic Rep. of	389 (4.2) ▼	Iran, Islamic Rep. of	402 (4.1) ▼
Mean for all 14*	505 (1.1) ▼	Mean for all 14*	514 (1.0) ▼	Mean for all 14*	522 (0.9) ▼

Note:
* This mean has been calculated for the 14 countries common to all cycles. It is calculated by pooling all student results for the 14 countries and weighting so that each country contributes equally to the mean.
▲ Country mean is significantly higher than the mean for the 14 countries
▼ Country mean is significantly lower than the mean for the 14 countries
Standard errors are presented in parentheses

 

International trends in mathematics benchmarks

As shown in Table 9, five percent of New Zealand Year 5 students reached the advanced benchmark, the point at where students were deemed capable of applying their understanding and knowledge in a variety of relatively complex situations and explain their reasoning. This was a similar proportion to countries including Italy (6%), Germany (6%), the Slovak Republic (5%), and Scotland (4%) and higher than Slovenia (3%), Austria (3%), and Sweden (3%). However, Singapore was the country with the greatest proportion of students at the advanced benchmark, more than eight times the proportion of New Zealand students, at 41 percent.

Examining the low benchmark, 15 percent of New Zealand students did not reach this benchmark and therefore, in terms of the benchmark definition, did demonstrate some basic mathematical knowledge. Most countries had some students in this group, with Chinese Taipei (1%), Singapore (2%), Japan (2%) and the Netherlands (2%) having the fewest students unable to reach the low benchmark. All students in Hong Kong SAR were at least able to reach this low benchmark. Countries with similar proportions to New Zealand at the advanced benchmark generally had fewer students unable to reach the low benchmark when compared to New Zealand.

Included in the table is the international median percentage of students at each benchmark. The same proportion of New Zealand Year 5 students reached the advanced and high benchmarks as the international median, so New Zealand was around the middle of the countries for these benchmarks. For the intermediate and low benchmarks, proportionally fewer New Zealand Year 5 students reached these benchmarks compared to the international median.

Table 9: Proportion of middle primary students at each international benchmark

		Percentage of Year 5 students reaching each benchmark
	Country	Advanced	High	Intermediate	Low
	Singapore	41 (2.1)	74 (1.7)	92 (0.9)	98 (0.3)
	Hong Kong SAR	40 (2.2)	81 (1.6)	97 (0.5)	100 (0.1)
	Chinese Taipei	24 (1.2)	66 (1.2)	92 (0.5)	99 (0.2)
	Japan	23 (1.2)	61 (1.2)	89 (0.8)	98 (0.4)
1	Kazakhstan	19 (2.1)	52 (3.5)	81 (2.9)	95 (1.5)
	England	16 (1.2)	48 (1.4)	79 (1.2)	94 (0.7)
	Russian Federation	16 (1.8)	48 (2.3)	81 (1.7)	95 (0.7)
1	Latvia	11 (0.8)	44 (1.5)	81 (1.2)	97 (0.5)
2 *	United States	10 (0.8)	40 (1.3)	77 (1.2)	95 (0.5)
1	Lithuania	10 (0.7)	42 (1.4)	77 (1.4)	94 (0.7)
	Hungary	9 (0.8)	35 (1.4)	67 (1.7)	88 (1.2)
	Australia	9 (0.8)	35 (1.9)	71 (1.7)	91 (1.0)
	Armenia	8 (1.5)	28 (1.8)	60 (1.8)	87 (1.2)
*	Denmark	7 (0.7)	36 (1.5)	76 (1.2)	95 (0.8)
**	Netherlands	7 (0.7)	42 (1.6)	84 (1.3)	98 (0.4)
	Germany	6 (0.5)	37 (1.3)	78 (1.2)	96 (0.5)
	Italy	6 (0.7)	29 (1.6)	67 (1.6)	91 (1.0)
	New Zealand	5 (0.5)	26 (1.0)	61 (1.1)	85 (1.0)
	Slovak Republic	5 (0.7)	26 (1.4)	63 (1.8)	88 (1.5)
*	Scotland	4 (0.5)	25 (1.1)	62 (1.4)	88 (0.9)
	Slovenia	3 (0.4)	25 (1.1)	67 (0.9)	92 (0.6)
	Austria	3 (0.3)	26 (1.0)	69 (1.4)	93 (0.8)
	Sweden	3 (0.3)	24 (1.4)	68 (1.4)	93 (0.7)
	Ukraine	2 (0.5)	17 (1.1)	50 (1.5)	79 (1.2)
	Czech Republic	2 (0.4)	19 (1.4)	59 (1.6)	88 (1.1)
	Norway	2 (0.3)	15 (1.0)	52 (1.6)	83 (1.1)
1	Georgia	1 (0.4)	10 (1.0)	35 (1.8)	67 (2.0)
	Colombia	0 (0.1)	2 (0.4)	9 (1.1)	31 (2.0)
	Morocco	0 (0.2)	2 (0.8)	9 (1.1)	26 (2.0)
	Iran , Islamic Rep. of	0 (0.1)	3 (0.5)	20 (1.5)	53 (2.0)
	Algeria	0 (0.1)	2 (0.4)	14 (1.4)	41 (2.2)
	Tunisia	0 (0.1)	1 (0.2)	9 (0.7)	28 (1.6)
	El Salvador	0 (0.0)	1 (0.2)	6 (0.5)	22 (1.6)
►	Kuwait	0 (0.0)	0 (0.1)	5 (0.6)	21 (1.2)
	Qatar	0 (0.0)	0 (0.1)	2 (0.2)	13 (0.4)
	Yemen	0 (0.0)	0 (0.1)	1 (0.4)	6 (0.8)
	International Median	5	26	67	90
Benchmarking Participants
2	Massachusetts, US	22 (1.8)	63 (2.1)	92 (1.1)	99 (0.3)
2 *	Minnesota, US	18 (2.1)	55 (3.2)	85 (2.2)	97 (1.2)
2	Quebec, C	5 (0.7)	34 (2.2)	74 (1.6)	96 (0.6)
2	British Columbia, C	4 (0.5)	27 (1.3)	67 (1.7)	93 (0.9)
2	Ontario, C	4 (0.6)	29 (1.8)	71 (1.8)	94 (1.1)
2	Alberta, C	3 (0.6)	25 (1.8)	69 (1.9)	94 (1.0)
►**	Dubai, UAE	2 (0.3)	12 (0.7)	37 (1.2)	69 (1.3)

Note:
* Met guidelines for sample participation rates only after replacement schools were included.
** Nearly satisfied guidelines for sample participation rates only after replacement schools were included.
1 National Target Population does not include all of the International Target Population defined by TIMSS.
2 National Defined Population covers 90% to 95% of National Target Population.
►Kuwait and Dubai, UAE tested the same cohort of students as other countries, but later in 2007, at the beginning of the next school year.
Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.

 

Figures 3 to 6 present examples of questions that Year 5 students achieving at or above the advanced, high, intermediate, and low benchmarks were likely to have answered correctly. An example of a correct answer and a summary of the scoring guide are presented. In addition, proportions of students successfully completing the question for a selection of countries, including the best performing country on that question, are shown. The international average is also presented as an indication of how students in all 37 countries performed on this question.

Figure 3: Question students reaching the advanced benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source Adapted from Exhibit 2.6, Mullis, Martin, and Foy, 2008.

Figure 4: Question students reaching the high benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.8, Mullis, Martin, and Foy, 2008.
 

Figure 5: Question students reaching the intermediate benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.12, Mullis, Martin, and Foy, 2008.

Figure 6: Question students reaching the low benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.15, Mullis, Martin, and Foy, 2008.

 

International trends in mathematics content and cognitive domains

As mentioned earlier, New Zealand Year 5 students demonstrated a relative strength in data display questions and a relative weakness in number questions. Relatively higher data display mean scores and relatively lower number mean scores were also observed for Japan and Scotland. In contrast, the higher achieving countries, Singapore, Chinese Taipei, and Hong Kong SAR all showed a relative strength in the number domain.

In the cognitive domains, New Zealand Year 5 students demonstrated a relative strength in questions that required them to use their reasoning and were relatively worse at questions that required demonstrating their knowledge. None of the other English-speaking or high-performing countries had a pattern quite like this. Scotland’s and Australia’s students demonstrated a relative weakness in the knowing domain and a relative strength in the applying domain. In contrast, Chinese Taipei, Hong Kong SAR, Singapore, England and the United States all showed a relative strength in the knowing domain.

 

Footnotes

1. Because results are rounded to the nearest whole number, this difference may appear inconsistent.
2. Pre-primary education is compulsory for students aged 5 and 6 in Latvia.