Executive Summary

Nature and Work of IEA

The International Association for the Evaluation of Educational Achievement (IEA) is an international, non-profit-making cooperative of research institutions. A permanent secretariat is located in Amsterdam , in the Netherlands . The principal purposes of IEA are to:

• undertake comparative educational research on an international scale;
• promote research aimed at examining educational problems in order to provide factual information which can help in the ultimate improvement of education systems;
• provide means whereby research centres in the various member countries of IEA can undertake cooperative projects.

Within each country a leading research organisation represents their country on IEA. The organisation is responsible for the conduct of any IEA research project in which it and national authorities agree participation is desirable. In New Zealand , the Ministry of Education is the member institution of IEA.

New Zealand participated in its first IEA study - the Six Subject Survey - in the early 1970s, followed by the Second International Mathematics Study (SIMS) in 1981. Other studies in which New Zealand has taken part include the Written Composition Study (1984), the Computers in Education Study (1989), and the Reading Literacy Study (1990). New Zealand did not take part in the Second International Science Study (SISS, 1983).

New Zealand 's participation in the current study - The Third International Mathematics and Science Study (TIMSS) - is the subject of a series of reports, of which this is the second.

Background to the Study

Overview

One of the most important features of TIMSS is that it enabled the collection of information on the nature of teaching and learning at both international and national levels. The result will be a current description of student achievement in mathematics and in science. The study will be able to place these achievements into context as a result of investigation of the curricula and teaching and classroom practices in participating countries. It will also provide a benchmark for educational systems to evaluate the current status of their mathematics and science education, and determine their needs in terms of assessment practices and resources for the twenty-first century. For New Zealand this is particularly pertinent, as data for the study was collected at a time when new mathematics and science curricula were being developed and introduced.

Coordination of the Study

In New Zealand , TIMSS is being administered by the IEA Unit within the Research and International Section of the Ministry of Education, Wellington . International administration of the study and subsequent analyses of the data and production of the international reports is the ongoing responsibility of the TIMSS Study Centre, Boston College , Boston , Massachusetts , USA . The Study Centre is supported by the International Coordinating Centre (ICC) located at the University of British Columbia , Vancouver , Canada .

Participating Countries

New Zealand is one of 46 countries or educational systems taking part in the study. The participating countries are:

New Zealand Report

The main aim of this report is to describe the achievement of New Zealand students in mathematics at the forms 2 and 3 levels. International data was not available at the time of writing this report, therefore the focus of the report is the New Zealand results only. Where items have been used as examples, only a description of the content is given for some due to an international policy on the release of achievement items. In addition, certain information which puts this achievement into context will be included.

Other New Zealand reports

The topics of these reports include:

• Science achievement of New Zealand students in form 2 and form 3;
• Mathematics and science achievement of New Zealand students in standard 2 and standard 3;
• Mathematics and science literacy of New Zealand 's school leavers;
• The achievement of New Zealand students in mathematics and science in an international setting;
• The New Zealand technical report.

International Reports

A series of major reports dealing with mathematics and science achievement for each population will be published by the Study Centre in Boston . The first of these, which will consist of population 2 results, is due for publication in November 1996. Results of the curriculum analysis component will be published from Michigan State University , where the team responsible for TIMSS curriculum analysis is located. In addition, a series of monographs is to be published. The first three have been released. The topics these deal with are:

• Curriculum frameworks for mathematics and science;
• Research questions and study design;
• A comparative study of grade 8 mathematics textbooks.

Rationale for the Study

Conceptual Framework of the Study

The conceptual framework for TIMSS is based on the concepts of the intended, implemented, and attained curricula and was derived and adapted from the model used in the IEA Second International Mathematics Study (SIMS), as shown in Figure 1. This model has subsequently been modified and updated to enable the development of the framework for TIMSS.

The concepts embodied in this framework were represented as shown in Figure 2.

Figure 1: An IEA research model

Note: Source: Robitaille & Maxwell, 1996 (adapted from Travers & Westbury, 1989).

For the purpose of this study, the intended curriculum refers to the aims, content, and methods for teaching and learning mathematics as defined by education authorities at the national, regional, or local level. It is described in published documents such as curriculum guides, prescriptions, syllabuses, policy statements, and other official statements produced to guide schools and teachers. Textbooks, resources, and examinations also reflect the essence of the intended curriculum. The mathematics content students are expected to learn may be described in the form of concepts, processes, skills, and attitudes. As well as being set within a specific educational context which includes institutional arrangements for the system, the intended curriculum is also set within the context of a society. Societal factors influencing the context may include the goals and expectations the society holds for schooling, participation rates, the role of independent schools, the professional preparation of teachers, the status accorded to teachers, the resources society has, and the proportion of those resources allocated to education.

Figure 2: Conceptual framework for TIMSS

Note: Source: Robitaille & Maxwell, 1996.

Teachers in turn interpret, translate, and implement the intentions of the mathematics curriculum according to their own experiences and beliefs. The educational milieu in which the implemented curriculum is placed embodies institutional arrangements made at the school and class level but is largely influenced by system level arrangements. Teacher interpretation of mathematical content and teaching practice in terms of teaching strategies used in their lessons, and time allocation, also contribute to the nature of the implemented curriculum. The local community, while often reflecting society-at-large, provides the context for the setting of the implemented curriculum. Social, cultural, and economic characteristics of the community, parent involvement in the community, expectations held for schooling, and participation rates of students are just some of the contextual features believed to influence achievement.

The attained curriculum consists of the concepts, processes, skills, and attitudes towards mathematics students have acquired during their schooling. Student learning will be affected by what was intended and by the quality and types of opportunities made available to them. Institutional arrangements such as allocation of staff and time made available for instruction also provide a context for the attained curriculum. The attained curriculum can also be placed in the broader context of students' backgrounds. Student backgrounds are likely to be influenced by their community and by society-at-large. Attitudes to education, students' perceptions of their own abilities, and the economic well-being of their families are likely to have an influence on students and are therefore important to take into account when examining the attained curriculum (Robitaille et al, 1993; Robitaille & Maxwell, 1996).

The framework depicted in Figure 2 thus provides the rationale and context for the development of the research questions.

Research questions

The research questions were formulated at two levels. The first level is of broad generality. At the second level, a large number of specific questions, amenable to provision of specific answers using the data collected, were also framed.

Four general questions under which all specific questions were grouped are:

1. How do countries vary in the intended learning goals for mathematics and science; and what characteristics of educational systems, schools, and students influence the development of those goals?
2. What opportunities are provided for students to learn mathematics and science; how do instructional practices in mathematics and science vary among nations; and what factors influence these variations?
3. What mathematics and science concepts, processes, and attitudes have students learned; and what factors are linked to students' opportunity to learn?
4. How are the intended, the implemented, and the attained curricula related with respect to the contexts of education, the arrangements for teaching and learning, and the outcomes of the educational process?

This report is concerned only with New Zealand results. Comparisons with results from other countries will be made in a later report.

Curriculum Frameworks

To enable systematic analyses of curricula as well as valid inter-country comparisons of mathematics and science education in participating countries, common curriculum frameworks for each of mathematics and science were developed. They provide structures from which the intended curricula of countries can be compared, as well as providing a basis for developing appropriate achievement tests and questionnaires.

In previous IEA studies, particularly those pertaining to mathematics, "content-by-cognitive-behaviour" grids were used for comparing curricula and constructing achievement measures. They were usually represented by a two-dimensional grid with the horizontal dimension representing student cognitive behaviours in hierarchical levels (eg simple recall, comprehension, applications, analysis). Content areas were specified in the vertical dimension. Achievement test items or curricular material were assigned to cells within this grid. There were limitations associated with such grids as they failed to take account of the inter-relationship of content areas or of cognitive behaviours. Consequently, for TIMSS there are three dimensions to the frameworks: content, performance expectations, and perspectives.

Content was defined as the mathematics or science subject matter being considered. Performance expectations is a reconceptualisation of the cognitive behaviour dimension and is intended to be non-hierarchical in nature. The aim is to enable a description of many kinds of performance expected of students who experience the intended curriculum. The intention of the perspectives aspect is to describe curricular goals that promote positive attitudes, interest, and motivation in mathematics and in science. In addition, learning situations which promote the participation of under-represented groups, as well as those which encourage students to pursue careers in mathematics, science, or technology, can be described using these perspectives (Robitaille et al, 1993).

Figure 3: The mathematics framework

Note: Source: Robitaille et al, 1993.

The science framework can be found in the publications on science achievement of form 2 and 3 students and in mathematics and science achievement of standard 2 and 3 students. A more detailed description of the categories identified in the figure is found in Appendix 1.

Summary

New Zealand 's commitment to one of the biggest ever international studies on mathematics and science educational achievement began in 1991. In New Zealand , this culminated in the data collection at the standard 2 and 3 and form 2 and 3 levels in 1994, and forms 6 and 7 level in 1995. This chapter has provided an outline of the framework used by IEA in the design of this very complex study, as well as an outline of the methodology used in New Zealand . While outlining the design for all populations, the focus of this chapter and subsequent chapters is the performance of New Zealand 's form 2 and form 3 students in mathematics, and factors likely to influence their achievement.