Executive Summary

Nature and Work of IEA

The International Association for the Evaluation of Educational Achievement (IEA) is an international, non-profit-making cooperative of research institutions. A permanent secretariat is located in Amsterdam, in the Netherlands . The principal purposes of IEA are to:

• undertake comparative educational research on an international scale;
• promote research aimed at examining educational problems in order to provide factual information which can help in the ultimate improvement of education systems;
• provide means whereby research centres in the various member countries of IEA can undertake cooperative projects.

Within each country, a leading research organisation represents their country on IEA. The organisation is responsible for the conduct of any IEA research project in which it and national authorities agree participation is desirable. In New Zealand, the Ministry of Education is the member institution of IEA.

New Zealand participated in its first IEA study — the Six Subject Survey — in the early 1970s, followed by the Second International Mathematics Study (SIMS) in 1981. Other studies in which New Zealand has taken part include the Written Composition Study (1984), the Computers in Education Study (1989), and the Reading Literacy Study (1990). New Zealand did not take part in the Second International Science Study (SISS, 1983).

New Zealand 's participation in the current study — The Third International Mathematics and Science Study (TIMSS) — is the subject of a series of reports, of which this is the third.

Background to the Study

Overview

One of the most important features of TIMSS is that it enabled the collection of information on the nature of teaching and learning at both international and national levels. The result will be a current description of student achievement in mathematics and in science. The study will be able to place these achievements into context as a result of investigation of the curricula and teaching and classroom practices in participating countries. It will also provide a benchmark for educational systems to evaluate the current status of their mathematics and science education, and determine their needs in terms of assessment practices and resources for the twenty-first century. For New Zealand this is particularly pertinent, as data for the study were collected at a time when new mathematics and science curricula were being developed and introduced.

Coordination of the Study

In New Zealand, TIMSS is being administered by the IEA Unit within the Research and International Section of the Ministry of Education, Wellington. International administration of the study and subsequent analyses of the data and production of the international reports is the ongoing responsibility of the TIMSS Study Centre, Boston College , Boston, Massachusetts, USA . The Study Centre is supported by the International Coordinating Centre (ICC) located at the University of British Columbia, Vancouver, Canada .

Participating Countries

New Zealand is one of more than 40 countries or educational systems taking part in the study. The 26 participating countries at the middle primary level are:

New Zealand Report

The main aim of this report is to describe the achievement of New Zealand students in mathematics and science at the standards 2 and 3 levels. While the focus of the report is on the New Zealand results, where possible international data have been included to put the achievement of New Zealand students into an international context. Unless otherwise stated, TIMSS population 1 data for other participating countries is unpublished at the time of writing and, while it is very unlikely to change, should be regarded as preliminary. Where items have been used as examples, for some only a description of the content is given due to an international policy on the release of achievement items.

Other New Zealand reports

Two reports have already been published:

• Mathematics Performance of New Zealand Form 2 and Form 3 Students (Garden, 1996a); and
• Science Performance of New Zealand Form 2 and Form 3 Students (Garden, 1996b).

Topics for later reports will be:

• Mathematics and science literacy of New Zealand 's school leavers;
• The achievement of New Zealand students in mathematics and science in an international setting;
• The New Zealand technical report.

The two earlier reports included only New Zealand data. This report, and subsequent reports, will include international data.

International Reports

A series of major reports dealing with the mathematics and science achievement for each population is being published by the Study Centre in Boston . Two of these reports — published in November 1996 — present the international mathematics and science results for population 2 (forms 2 and 3 in New Zealand). For further details see Beaton et al, 1996a and 1996b. Results of the curriculum analysis component are being published from Michigan State University , where the team responsible for TIMSS curriculum analysis is located; the first volume on mathematics was released in March, 1997, with the companion volume on science to follow. See Schmidt et al, 1997a and 1997b. In addition, a series of monographs is to be published.

The first three have been published:

• Curriculum Frameworks for Mathematics and Science (Robitaille et al, 1993);
• Research Questions and Study Design (Robitaille & Garden, 1996);
• Mathematics Textbooks: a comparative study of grade 8 texts (Howson, 1995).

The fourth monograph deals with mathematics and science literacy.

Rationale for the Study

Conceptual Framework of the Study

The conceptual framework for TIMSS is based on the concepts of the intended, implemented, and attained curricula and was derived and adapted from the model used in the IEA Second International Mathematics Study (SIMS), as shown in Figure 1.

Figure 1: An IEA research model

Note: Source: Robitaille & Maxwell, 1996 (adapted from Travers & Westbury, 1989).

This model has subsequently been modified and updated during the development of a framework for TIMSS. This framework is shown in Figure 2.

Figure 2: Conceptual framework for TIMSS

Note: Source: Robitaille & Maxwell, 1996.

For the purpose of this study, the intended curriculum refers to the aims, content, and methods for teaching and learning mathematics and science as defined by education authorities at the national, regional, or local level. It is described in published documents such as curricula guides, prescriptions, syllabuses, policy statements, and other official statements produced to guide schools and teachers. Textbooks, resources, and examinations also reflect the essence of the intended curriculum. The mathematics and science content students are expected to learn may be described in the form of concepts, processes, skills, and attitudes. As well as being set within a specific educational context which includes institutional arrangements for the system, the intended curriculum is also set within the context of a society. Societal factors influencing the context may include the goals and expectations the society holds for schooling, participation rates, the role of independent schools, the professional preparation of teachers, the status accorded to teachers, the resources society has, and the proportion of those resources allocated to education.

Teachers in turn interpret, translate, and implement the intentions of the mathematics and science curricula according to their own experiences and beliefs. The educational milieu in which the implemented curriculum is placed embodies institutional arrangements made at the school and class level but is largely influenced by system level arrangements. Teacher interpretation of content and teaching practice in terms of teaching strategies used in their lessons, and time allocation, also contribute to the nature of the implemented curriculum. The local community, while often reflecting society-at-large, provides the context for the setting of the implemented curriculum. Social, cultural, and economic characteristics of the community, parent involvement in the community, expectations held for schooling, and participation rates of students are just some of the contextual features believed to influence achievement.

The attained curriculum consists of the concepts, processes, skills, and attitudes towards mathematics and science that students have acquired during their schooling. Student learning will be affected by what was intended and by the quality and types of opportunities made available to them. Institutional arrangements such as allocation of staff and time made available for instruction also provide a context for the attained curriculum. The attained curriculum can also be placed in the broader context of students' backgrounds. Student backgrounds are likely to be influenced by their community and by society-at-large. Attitudes to education, students' perceptions of their own abilities, and the economic well-being of their families are likely to have an influence on students and are therefore important to take into account when examining the attained curriculum (Robitaille et al, 1993; Robitaille & Maxwell, 1996).

The framework depicted in Figure 2 thus provides the rationale and context for the development of the research questions.

Research questions

The research questions were formulated at two levels. The first level is of broad generality. At the second level, a large number of specific questions, amenable to the provision of specific answers using the data collected, were also framed.

The four general questions under which all specific questions were grouped are:

1. How do countries vary in the intended learning goals for mathematics and science; and what characteristics of educational systems, schools, and students influence the development of those goals?
2. What opportunities are provided for students to learn mathematics and science; how do instructional practices in mathematics and science vary among nations; and what factors influence these variations?
3. What mathematics and science concepts, processes, and attitudes have students learned; and what factors are linked to students' opportunity to learn?
4. How are the intended, the implemented, and the attained curricula related with respect to the contexts of education, the arrangements for teaching and learning, and the outcomes of the educational process?

Note: Source: Robitaille & Maxwell, 1996.

Curriculum Frameworks

To enable systematic analyses of curricula as well as valid inter-country comparisons of mathematics and science education in participating countries, common curriculum frameworks for each of mathematics and science were developed. They provide structures from which the intended curricula of countries can be compared, as well as providing a basis for developing appropriate achievement tests and questionnaires.

In previous IEA studies, particularly those pertaining to mathematics, "content-by-cognitive-behaviour" grids were used for comparing curricula and constructing achievement measures. They were usually represented by a two-dimensional grid with the horizontal dimension representing student cognitive behaviours in hierarchical levels (eg simple recall, comprehension, applications, analysis). Content areas were specified in the vertical dimension. Achievement test items or curricular material were assigned to cells within this grid. There were limitations associated with such grids as they failed to take account of the inter-relationship of content areas or of cognitive behaviours. Consequently, for TIMSS there are three dimensions to the frameworks: content, performance expectations, and perspectives.

Content was defined as the mathematics or science subject matter being considered. Performance expectations is a re-conceptualisation of the cognitive behaviour dimension and is intended to be non-hierarchical in nature. The aim is to enable a description of many kinds of performance expected of students who experience the intended curriculum. The intention of the perspectives aspect is to describe curricular goals that promote positive attitudes, interest, and motivation in mathematics and in science. In addition, learning situations which promote the participation of under-represented groups, as well as those which encourage students to pursue careers in mathematics, science, or technology, can be described using these perspectives (Robitaille et al, 1993).

Figure 3: The mathematics framework

Figure 4: The science framework

Source: Minor adaptations from Robitaille et al, 1993.

Summary

New Zealand 's commitment to one of the biggest ever international studies on mathematics and science educational achievement began in 1991. In New Zealand , this culminated in the data collection at the standards 2 and 3 and forms 2 and 3 levels in 1994, and forms 6 and 7 levels in 1995. This chapter has provided an outline of the framework used by IEA in the design of this very complex study, as well as an outline of the methodology used in New Zealand . The purpose of this report is to examine achievement in mathematics and science of standards 2 and 3 students and the factors that influence achievement, making comparisons with other countries where appropriate.