Science - Trends in Year 5 science achievement 1994 to 2006

Introduction

This report examines the science results for New Zealand Year 5 students from TIMSS 2006/07.¹ Along with the report on New Zealand’s results for mathematics (Caygill & Kirkham, 2008), this report forms the beginning of a series of reports around New Zealand’s participation in TIMSS 2006/07. International findings for science for TIMSS 2006/07 have been published by the IEA (Martin, Mullis & Foy, 2008). A separate international report on mathematics was also published at this time (Mullis, Martin & Foy, 2008).

This report begins by examining trends in New Zealand science achievement at the Year 5 level from 1994 to 2006. It then looks at New Zealand’s science achievement in relation to other countries that participated in the study. An examination of the TIMSS assessment questions in relation to New Zealand’s science curriculum is presented along with analyses of achievement by sub-groupings (such as gender and ethnicity) and background factors. Lastly, a statistical model that attempts to explain variations among students, classes, and schools, using the background information discussed in this report is also described.

Assessment of science in TIMSS

The TIMSS assessment has two main dimensions: a dimension that describes the content or subject matter to be assessed; and a dimension that describes the cognitive processes used to answer the questions. The three content dimensions for science are: life science, physical science, and earth science. The detail about the topic areas covered in these domains at each grade or year level assessed and a set of assessment objectives for each topic area are presented in the 'TIMSS 2007 assessment frameworks' (Mullis, Martin, Ruddock, O’Sullivan, Arora, & Erberber, 2005). Briefly, each of the content areas is described in the frameworks as follows.

“Life science includes understandings of the characteristics and life processes of living things, the relationships between them, and their interaction with the environment.” (p. 43).

“Physical science includes concepts related to matter and energy, and covers topics in the areas of both chemistry and physics. Since students in fourth grade have only a beginning knowledge of chemistry, the framework places more emphasis on physics concepts.” (p. 47).

“Earth science is concerned with the study of Earth and its place in the solar system.” (p. 50).

In order to answer questions in the TIMSS test correctly, as well as being familiar with the science content, students need to draw on a range of cognitive skills. Also, in their lives outside and beyond school, students will need to do more than just accurately recall a range of science facts. This is acknowledged in the framework with three aspects to the cognitive dimension entitled knowing, applying, and reasoning. Briefly, each cognitive dimension is described in the framework as follows.

“The first domain, Knowing, covers facts, procedures, and concepts students need to know, while the second domain, Applying, focuses on the ability of the student to apply knowledge and conceptual understanding in a problem situation. The third domain, Reasoning, goes beyond the solution of routine problems to encompass unfamiliar situations, complex contexts, and multi-step problems.” (p. 68).

Data collection

Each student was assessed in two timed sessions of 36 minutes, and answered a combination of mathematics and science questions. The assessment was a pencil-and-paper test containing both multiple-choice and constructed-response questions. Following this, students were given a questionnaire containing questions about themselves, their opinions about science and mathematics, their computer use and time spent on homework. Principals and teachers were also given questionnaires in order to gain further information about the context in which the science teaching and learning take place. In New Zealand, the assessments and questionnaires were conducted in English.²

International participants in TIMSS

The number of participants in TIMSS at the Year 5 or grade 4 level has steadily increased since 1994, when 26 countries took part. In 2002, 25 countries and 3 benchmarking participants took part. Benchmarking participants are usually states or parts of countries and are not included in international averages. In 2006, the number of education systems participating at the middle primary level had risen to 37 countries and 7 benchmarking participants.

Technical information

A lot of information is gathered during the TIMSS administration and a number of techniques are applied when collecting and analysing the data. The 'TIMSS 2007 technical report' (Olson, Martin, & Mullis, (Eds.), 2008) contains a detailed account of the procedures for scoring, translation of materials, sampling, survey operations, quality assurance, sampling weights, item analysis, scaling, and reporting. In addition, the 'TIMSS 2007 user guide for the international database' (to be published in early 2009) contains information on how to analyse the data. Brief details of the technical information are given in the definitions and technical notes at the end of this report.

TIMSS encyclopaedia

In order to provide a context in which the TIMSS results can be examined, TIMSS also publishes the 'TIMSS 2007 encyclopedia: a guide to mathematics and science education around the world' (Mullis, Martin, Olson, Berger, Milne, & Stanco (Eds.), 2008). This encyclopaedia contains short reports from each country describing mathematics and science education policies and practices in that country.