New Zealand science achievement in 2006 and in an international context

As shown in Figure 2, the mean science score for New Zealand Year 5 students in TIMSS 2006/07 was 504 scale score points. New Zealand’s mean score was similar to Scotland (500) and significantly higher than 13 other countries. In contrast, 21 countries had higher mean science achievement, including Singapore (587), England (542), the United States (539) and Australia (527).

The range of achievement (from the 5th to 95th percentile) in New Zealand was 299 score points from 344 (the 5th percentile) to 643 (the 95th percentile). This was relatively wider than many of the higher achieving countries, but narrower than that of Singapore (309). Another measure of spread, the inter-quartile range (from the 25th to 75th percentile) can also be examined. For New Zealand (121) this was wider than many of the higher performing countries, but the same as Singapore (also 121).

Given the number of countries now participating in TIMSS, it is more meaningful to compare New Zealand to a selection of countries (such as English-speaking or high-performing). Compared to the other countries that tested in English (Singapore, England, the United States, Australia, and Scotland), New Zealand had significantly lower science achievement, on average, than all of them except for Scotland.

Alongside Figure 2, Table 6 presents some information to help put science achievement in context. Countries are presented in the same order as in Figure 2. It contains information on the number of years of primary schooling students will have undertaken by the time of the assessment, along with students’ average age at the time of testing. Also given in the table is the average number of hours of time spent in science instruction during the assessment year according to teacher reports. Three bits of information are presented about the economic circumstances, on average, across each country, the Human Development Index, and two versions of the Gross National Income per Capita (described later).

New Zealand spends less time at the middle primary level teaching science, according to teacher reports, than most of the English-speaking countries and many of the high-performing countries. Note that teachers were asked first whether they taught science as a separate subject and if not, to estimate the amount of time it was taught as part of an integrated programme. Teachers who reported a similar amount of time on average as New Zealand teachers included Australian and Latvian, while teachers in the Russian Federation reported relatively less time in science instruction. Of particular note when examining the context in which to interpret science achievement results, New Zealand teachers are reporting far fewer hours spent in science instruction for the year (21 fewer hours on average) compared with 2002.

Table 6 also presents the Human Development Index (HDI) provided by the United Nations Development Programme (UNDP – for details see Human Development Report 2007/2008, p. 229-232). This index was included by Martin, Mullis and Foy (2008) in the international reporting to provide some context around the economic and educational development of TIMSS participating countries. The index ranges from a minimum value of 0 to a maximum value of 1, with high values indicating that people in a country generally enjoy long life expectancy, high levels of school enrolment and adult literacy, and a good standard of living as measured by per capita GDP. New Zealand was relatively high on this scale with a value of 0.943, similar to that of Italy (0.941), and England and Scotland (0.946 – this value is actually for the United Kingdom as no disaggregated data is available for England and Scotland) and lower than that of Australia (0.962) and the United States (0.951).

Perhaps easier to relate to than the HDI, two versions of the Gross National Income (GNI) per Capita are also presented in Table 6. The first of the two columns gives the GNI per Capita in United States dollars while the second is an adjusted value that takes account of comparative purchasing power between each country and the United States. Compared to the countries that assessed in English, New Zealand has the lowest income regardless of which of these values is used.

Figure 2: Distribution of middle primary science achievement in TIMSS 2006/07

Note:
* Met guidelines for sample participation rates only after replacement schools were included.
** Nearly satisfied guidelines for sample participation rates only after replacement schools were included.
1 National Target Population does not include all of the International Target Population defined by TIMSS.
2 National Defined Population covers 90% to 95% of National Target Population.
Kuwait and Dubai, UAE tested the same cohort of students as other countries, but later in 2007, at the beginning of the next school year.
Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.

Source: Adapted from Exhibits 1.1 and D.1 Martin, Mullis and Foy, 2008.

Table 6: Selected contextual factors for TIMSS 2006/07 countries

Country	Years of formal schooling*	Average age at time of testing	Human Development Index**	Gross National Income per capita (in US dollars)***	GNI per capita (purchasing -power parity)***	Average hours of instructional time in science (teacher reports)
Singapore	4	10.4	0.922	28730	43300	82 (0.9)
Chinese Taipei	4	10.2	0.932	17294	-	79 (1.6)
Hong Kong SAR	4	10.2	0.937	29040	39200	72 (5.2)
Japan	4	10.5	0.953	38630	32840	82 (1.2)
Russian Federation	4	10.8	0.813	5770	12740	40 (1.1)
Latvia	4	11.0	0.855	8100	14840	48 (1.2)
England	5	10.2	0.946	40560	33650	70 (1.7)
United States	4	10.3	0.951	44710	44070	89 (2.5)
Hungary	4	10.7	0.874	10870	16970	54 (1.5)
Italy	4	9.8	0.941	31990	28970	68 (1.4)
Kazakhstan	4	10.6	0.794	3870	8700	52 (1.3)
Germany	4	10.4	0.935	36810	32680	106 (2.1)
Australia	4	9.9	0.962	35860	33940	46 (2.2)
Slovak Republic	4	10.4	0.863	9610	17060	59 (0.7)
Austria	4	10.3	0.948	39750	36040	92 (1.1)
Sweden	4	10.8	0.956	43530	34310	56 (2.5)
Netherlands	4	10.2	0.953	43050	37940	33 (1.5)
Slovenia	4	9.8	0.917	18660	23970	84 (0.8)
Denmark	4	11.0	0.949	52110	36190	59 (0.9)
Czech Republic	4	10.3	0.891	12790	20920	41 (1.3)
Lithuania	4	10.8	0.862	7930	14550	51 (0.6)
New Zealand	4.5 - 5.5	10.0	0.943	26750	25750	45 (2.5)
Scotland	5	9.8	0.946	40560	33650	51 (3.1)
Armenia	4	10.6	0.775	1920	4950	81 (4.0)
Norway	4	9.8	0.968	68440	50070	44 (1.9)
Ukraine	4	10.3	0.788	1940	6110	33 (1.1)
Iran , Islamic Rep. of	4	10.2	0.759	2930	9800	83 (2.4)
Georgia	4	10.1	0.754	1580	3880	35 (2.8)
Colombia	4	10.4	0.791	3120	6130	139 (3.9)
El Salvador	4	11.0	0.735	2680	5610	135 (3.5)
Algeria	4	10.2	0.733	3030	5940	67 (4.7)
Kuwait	4	10.2	0.891	30630	48310	x x
Tunisia	4	10.2	0.766	2970	6490	71 (2.7)
Morocco	4	10.6	0.646	2160	3860	54 (4.2)
Qatar	4	9.7	0.875	-	-	x x
Yemen	4	11.2	0.508	760	2090	83 (5.7)

Note:
* Represents years of schooling counting from the first year of primary schooling.
** Taken from United Nations Development Programme’s Human Development Report 2007/2008. See Martin, Mullis, and Foy for details.
*** Data on GNI taken from the World Bank’s 2008 World Development Indicators. Purchasing Power Parity adjusts the GNI to take account of comparative purchasing power between the country and the United States.
Standard errors are presented in parentheses.

Source: Adapted from Exhibits 3, 1.1, and 5.3, Martin, Mullis, and Foy, 2008.

International trends in science achievement at the middle primary level

There are several ways that trends since 1994 can be examined for the countries participating in TIMSS. The analyses presented here will include only those countries that have participated in all three international cycles, 1994/95, 2002/03, and 2006/07. Table 7 shows the change in mean science scores since 1994/95, ordered so that those countries that have had the biggest positive change since the first cycle are at the top and those with the biggest negative change are at the bottom.

Table 7: Trends in middle primary school mean science achievement over three cycles of TIMSS

Country	1994/95 to 2006/07 difference	2002/03 to 2006/07 difference
Singapore	63 (6.4) ▲	22 (6.8) ▲
Latvia	56 (5.4) ▲	12 (3.5) ▲
Iran , Islamic Rep. of	55 (6.3) ▲	22 (5.9) ▲
Slovenia	54 (3.6) ▲	28 (3.2) ▲
Hong Kong SAR	46 (4.8) ▲	12 (4.6) ▲
Hungary	28 (4.8) ▲	6 (4.5)
England	14 (4.2) ▲	1 (4.4)
Australia	6 (4.9)	7 (5.3)
New Zealand	-1 (5.9)	-18 (3.5) ▼
United States	-3 (4.3)	3 (3.5)
Japan	-5 (2.6) ▼	4 (2.5)
Netherlands	-7 (4.0)	-2 (3.1)
Scotland	-14 (5.0) ▼	-2 (3.6)
Norway	-27 (5.2) ▼	10 (3.5) ▲

Note:
▲ 2006/07 score significantly higher.
▼ 2006/07 score significantly lower.
Standard errors are presented in parentheses

Source: Adapted from Exhibit 1.3 Martin, Mullis, and Foy, 2008.

Singapore, Latvia, the Islamic Republic of Iran, Slovenia, Hong Kong SAR, Hungary, and England have all had significant increases in mean science achievement since 1994. Of these countries, Latvia and Slovenia have both made significant changes in their education systems since 1994; summaries of these are presented in the following paragraphs. However first we look at Singapore.

Singapore

Singapore is the country with the largest change over time in mean science score. Singapore has had some changes in the education system over this time with a new science curriculum introduced in 2001, along with a change in the philosophy of the education system since 2004. Singapore has moved toward

“more quality in terms of classroom interaction, opportunities for expression, and the learning of life long skills through innovative and effective teaching approaches, and away from quantity in terms of rote learning, repetitive tests, and following prescribed answers and set formulae. It also reaffirms the learner at the center [sic] of all that is being done and better recognizes and caters to the various needs and interests of different learners.” (Quek et al., 2008, p. 537).

Singapore places a great emphasis on the teaching and learning of science and mathematics. Primary schools have science rooms and sometimes mathematics rooms that serve as a focal point for science and mathematics (respectively) activities and innovations. From upper primary levels onward, students have specialist teachers in science and mathematics. Guidelines are given nationally on the number of hours per week mathematics and science should be taught at each grade level.

Latvia

Since 1998, Latvia has had a basic education standard for students in grades 1 to 9 (aged 7 to 16).¹ Subject standards, which are part of the basic education standard, determine the main aims and tasks of the subject, the mandatory content of the subject, and the forms and order of the evaluation of achievement. The number of lessons per week is set nationally and mandatory. In grades 1 to 4 students have one teacher for all subjects; from grade 5, students have specialist subject teachers. Latvian students have tests in all grades, but the first national assessments occur at the end of grade 3 (students aged 9 and 10 – see Geske, Grinfelds, & Ozola, 2008).

Slovenia

Slovenia has been undergoing some significant changes in its schooling system, the most obvious of which is the lowering of the school starting age from 7 to 6 and revised national curricular documents for all levels of pre-university education. The goal of the reforms, implementation of which began in 1999, are:

“a higher level of interconnectedness of disciplinary knowledge, and increased active role of students, internationally comparable standards and levels of knowledge, improvement in functional literacy, and an increase in the quality and longevity of acquired knowledge.” (Japelj Pavešić & Svetlik, 2008, p. 537).

The Slovenian syllabus specifies the exact number of yearly and weekly lessons for individual subjects. In grades 1 to 3, nearly all subjects are taught by general class teachers. During grades 4 to 6, specialist teachers become more and more involved in the teaching process.

Relative rankings

In many summaries of the international data, relative rankings of mean scores are used to describe change. This is not a particularly desirable practice as any mean scores derived from a sample and ascribed to a population have some level of uncertainty around them and rankings ignore this uncertainty. In addition, some presentations of rankings fail to mention the number of countries included in the ranking.
Table 8 presents relative ranking changes between 1994/95 and 2006/07. This should be read with caution, because, although a country may be ranked higher, the mean scores may not be significantly different when the uncertainties are taken into account. For example, the mean science achievement for Hong Kong SAR and that of Japan in 2006/07 are not significantly different.

Table 8 shows that for New Zealand, not only are the mean science scores essentially the same for 1994 and 2006, but that there is very little movement in our position in the ‘league tables’ when only those countries in both assessments are included. Despite New Zealand’s Year 5 mean science score being the same in 1994/95 as it was in 2006/07, it has moved from being around the mean of the 14 countries to being below the mean over this time period.

Table 8: Middle primary mean science scores for countries participating in three cycles of TIMSS from 1994/95 to 2006/07

1994/95 mean science score		2002/03 mean science score		2006/07 mean science score
Japan	553 (1.8)▲	Singapore	565 (5.5)▲	Singapore	587 (4.1)▲
United States	542 (3.3)▲	Japan	543 (1.5)▲	Hong Kong SAR	554 (3.5)▲
Netherlands	530 (3.2)▲	Hong Kong SAR	542 (3.1)▲	Japan	548 (2.1)▲
England	528 (3.1)▲	England	540 (3.6)▲	Latvia	542 (2.3)▲
Singapore	523 (4.8)▲	United States	536 (2.5)▲	England	542 (2.9)▲
Australia	521 (3.8)▲	Latvia	530 (2.8)▲	United States	539 (2.7)▲
Scotland	514 (4.5)▲	Hungary	530 (3.0)▲	Hungary	536 (3.3)▲
Hong Kong SAR	508 (3.3)	Netherlands	525 (2.0)▲	Australia	527 (3.3)
Hungary	508 (3.4)	New Zealand	523 (2.3)▲	Netherlands	523 (2.6)
New Zealand	505 (5.3)	Australia	521 (4.2)	Slovenia	518 (1.9)▼
Norway	504 (3.7)	Scotland	502 (2.9)▼	New Zealand	504 (2.6)▼
Latvia	486 (4.9)▼	Slovenia	490 (2.5)▼	Scotland	500 (2.3)▼
Slovenia	464 (3.1)▼	Norway	466 (2.6)▼	Norway	477 (3.5)▼
Iran , Islamic Rep. of	380 (4.6)▼	Iran , Islamic Rep. of	414 (4.1)▼	Iran , Islamic Rep. of	436 (4.3)▼
Mean for all 14*	505 (1.1)	Mean for all 14*	516 (1.0)	Mean for all 14*	524 (1.1)

Note:
▲Country mean is significantly higher than the mean for the 14 countries
▼Country mean is significantly lower than the mean for the 14 countries
* This mean has been calculated for the 14 countries common to all cycles. It is calculated by pooling all student results for the 14 countries and weighting so that each country contributes equally to the mean.
Standard errors are presented in parentheses

 

International trends in science benchmarks

As shown in Table 9, eight percent of New Zealand Year 5 students reached the advanced benchmark, the point where students were deemed capable of applying knowledge and understanding of scientific processes and relationships in beginning scientific inquiry. This was a similar proportion to countries including Austria (9%), Sweden (8%), and the Czech Republic (7%), and higher than Slovenia (6%), Scotland (4%), and the Netherlands (4%). However, Singapore was the country with the greatest proportion of students at the advanced benchmark, more than four times the proportion of New Zealand students, at 36 percent.

Examining the low benchmark, 13 percent of New Zealand students did not reach this benchmark and therefore, in terms of the benchmark definition, did not demonstrate some elementary knowledge of life science and physical science. Most countries had some students in this group, with Hong Kong SAR (2%) and Latvia (2%) having the fewest students unable to reach the low benchmark. Countries with similar proportions at the advanced benchmark generally had fewer students unable to reach the low benchmark when compared to New Zealand.

Included in the table is the international median percentage of students at each benchmark. Approximately the same proportion of New Zealand Year 5 students reached the advanced benchmark as the international median, so New Zealand was around the middle of the countries for the advanced benchmark. For the high, intermediate, and low benchmarks, proportionally fewer New Zealand Year 5 students reached these benchmarks compared to the international median.

Table 9: Proportion of middle primary students at each international benchmark

		Percentage of students reaching each benchmark
	Country	Advanced	High	Intermediate	Low
	Singapore	36 (1.9)	68 (1.9)	88 (1.1)	96 (0.5)
	Chinese Taipei	19 (1.0)	55 (1.2)	86 (0.7)	97 (0.4)
	Russian Federation	16 (1.9)	49 (2.3)	82 (1.7)	96 (0.9)
2 *	United States	15 (0.9)	47 (1.4)	78 (1.1)	94 (0.6)
	England	14 (1.2)	48 (1.6)	81 (1.1)	95 (0.6)
	Hong Kong SAR	14 (1.4)	55 (2.2)	88 (1.2)	98 (0.4)
	Hungary	13 (1.0)	47 (1.8)	78 (1.6)	93 (0.8)
	Italy	13 (1.0)	44 (1.6)	78 (1.3)	94 (0.7)
	Japan	12 (1.0)	51 (1.1)	86 (1.0)	97 (0.4)
	Armenia	12 (1.8)	27 (1.8)	52 (1.8)	77 (1.6)
	Slovak Republic	11 (0.8)	42 (2.0)	75 (1.8)	92 (1.3)
	Australia	10 (0.7)	41 (2.2)	76 (1.6)	93 (0.8)
1	Latvia	10 (1.1)	47 (1.7)	84 (1.3)	98 (0.4)
	Germany	10 (0.7)	41 (1.3)	76 (1.2)	94 (0.6)
1	Kazakhstan	10 (1.3)	44 (3.1)	79 (2.6)	95 (1.0)
	Austria	9 (0.7)	39 (1.3)	76 (1.3)	93 (0.6)
	Sweden	8 (0.6)	37 (1.6)	76 (1.5)	95 (0.6)
	New Zealand	8 (0.5)	32 (1.0)	65 (1.2)	87 (1.0)
	Czech Republic	7 (0.7)	33 (1.9)	72 (1.4)	93 (0.8)
*	Denmark	7 (0.8)	35 (1.5)	72 (1.5)	93 (0.8)
	Slovenia	6 (0.6)	36 (1.3)	74 (1.0)	93 (0.6)
*	Scotland	4 (0.6)	26 (1.2)	65 (1.3)	90 (0.8)
**	Netherlands	4 (0.8)	34 (1.8)	79 (1.4)	97 (0.5)
1	Lithuania	3 (0.4)	30 (1.4)	74 (1.4)	95 (0.6)
	Ukraine	2 (0.3)	17 (1.1)	52 (1.5)	82 (1.3)
	Iran, Islamic Rep. of	2 (0.3)	12 (1.0)	36 (1.7)	65 (1.9)
	Norway	1 (0.4)	17 (1.4)	54 (2.0)	84 (1.4)
	Colombia	1 (0.2)	6 (0.8)	22 (1.7)	51 (2.4)
1	Georgia	1 (0.2)	5 (0.8)	26 (2.0)	59 (2.1)
	El Salvador	0 (0.1)	4 (0.5)	18 (1.2)	47 (1.5)
►	Kuwait	0 (0.2)	4 (0.6)	16 (1.3)	37 (1.3)
	Morocco	0 (0.2)	2 (0.5)	9 (1.4)	21 (1.9)
	Algeria	0 (0.1)	2 (0.5)	11 (1.3)	33 (2.1)
	Tunisia	0 (0.1)	3 (0.6)	14 (1.1)	32 (1.7)
	Qatar	0 (0.0)	2 (0.2)	8 (0.3)	23 (0.7)
	Yemen	0 (0.0)	0 (0.1)	2 (0.5)	8 (1.1)
	International Median	7	34	74	93
Benchmarking participants
2	Massachusetts, US	22 (2.2)	64 (2.4)	92 (1.1)	99 (0.4)
	Minnesota, US	17 (1.9)	54 (3.2)	84 (2.1)	96 (1.5)
2	Alberta, C	12 (1.3)	48 (2.0)	82 (1.5)	96 (0.7)
2	Ontario, C	12 (1.2)	45 (2.2)	79 (1.7)	95 (1.0)
2	British Columbia, C	11 (0.8)	44 (1.7)	81 (1.5)	96 (0.6)
2	Quebec, C	5 (0.6)	32 (1.9)	74 (1.9)	96 (0.6)
►**	Dubai, UAE	4 (0.5)	21 (1.0)	48 (1.3)	72 (1.4)

Note:  
* Met guidelines for sample participation rates only after replacement schools were included.
** Nearly satisfied guidelines for sample participation rates only after replacement schools were included.
1 National Target Population does not include all of the International Target Population defined by TIMSS.
2 National Defined Population covers 90% to 95% of National Target Population.
►Kuwait and Dubai, UAE tested the same cohort of students as other countries, but later in 2007, at the beginning of the next school year.
Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent.

Source: Adapted from Exhibit 2.2 Martin, Mullis and Foy, 2008. 

 

Figures 3 to 6 present examples of questions that Year 5 students achieving at or above the advanced, high, intermediate, and low benchmarks were likely to have answered correctly. An example of a correct answer and a summary of the scoring guide are presented. In addition, proportions of students successfully completing the question for a selection of countries, including the best performing country on that question, are shown. The international average is also presented as an indication of how students in all 37 countries performed on this question.

Figure 3: Question students reaching the advanced benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.6, Martin, Mullis, and Foy, 2008.

 

Figure 4: Question students reaching the high benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.9, Martin, Mullis, and Foy, 2008.

 

Figure 5: Question students reaching the intermediate benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.12, Martin, Mullis, and Foy, 2008.

 

Figure 6: Question students reaching the low benchmark are likely to have answered correctly

Note:
Standard errors are presented in parentheses.

Source: Adapted from Exhibit 2.15, Martin, Mullis, and Foy, 2008.

 

International trends in science content and cognitive domains

As mentioned earlier, New Zealand Year 5 students demonstrated a relative strength in earth science questions and a relative weakness in physical science questions. Relatively higher earth science mean scores and relatively lower physical science mean scores were also observed for Scotland and Australia. In contrast, the higher achieving countries, Singapore, Chinese Taipei, and Japan all showed a relative strength in the physical science domain.

In the cognitive domains, New Zealand Year 5 students demonstrated a relative strength in questions that required them to demonstrate their knowledge and a relative weakness in questions that required them to apply their knowledge. This pattern was the same as Scotland and the United States. In contrast, Chinese Taipei, Hong Kong SAR, and Japan all showed a relative strength in the reasoning domain. Interestingly, Singaporean students were relatively stronger at questions in the knowing domain and relatively weaker at the questions in the reasoning domain.

 

Footnote

1. Pre-primary education is compulsory for students aged 5 and 6 in Latvia.