TIMSS 2010/11 & PIRLS 2010/11 Key Findings: New Zealand’s Participation in PIRLS and TIMSS

Publication Details

This summary report presents the key findings from New Zealand's participation in PIRLS (Progress in International Reading Literacy Study) 2010/11 and TIMSS (Trends in International Mathematics and Science Study) 2010/11.

Author(s): Megan Chamberlain and Robyn Caygill

Date Published: December 2012

What are PIRLS and TIMSS?

Both PIRLS and TIMSS  are international research studies that are designed to measure trends in  student achievement. They are coordinated by the International Association for  the Evaluation of Educational Achievement (IEA). PIRLS looks at reading literacy  achievement of middle primary school students. New Zealand's Year 5 students  take part in PIRLS. It is administered every five years, with the first cycle in  2001, the second in 2005/06, and then the third in 2010/11.

TIMSS looks at the  mathematics and science achievement of both middle primary and lower secondary  school students. New Zealand's Year 5 and Year 9 students take part. TIMSS is  administered on a regular four-year cycle with the first cycle in 1994/95, and  then again in 1998/99, 2002/03, 2006/07, and then the fifth cycle in 2010/11.

TIMSS also provides  information on the relative progress of the middle primary school cohort four  years later when they are in lower secondary school. For example and although  not exactly the same students, the Year 5 student cohort assessed in TIMSS in  2006/07 formed the Year 9 TIMSS cohort in 2010/11.

As well as  comprehensive assessment information in three essential learning areas—reading,  science, and mathematics—a rich array of contextual background information is  collected from students, teachers, parents/caregivers, and school principals.  National educational policy information is also provided by each country to aid  the interpretation of results.

PIRLS and TIMSS in 2010/11

Both PIRLS and TIMSS  were administered in 2010/11. This provided a unique opportunity for many  countries participating in both PIRLS and TIMSS at the middle primary level as  it had the advantage of one comprehensive assessment in all three learning  areas: reading, mathematics, and science. Many countries that took part chose  to assess the same middle primary school students in all three areas. New Zealand  chose to assess two different groups of students: one group that took part in  PIRLS and one group that took part in TIMSS.

What  countries took part?

Approximately 60  countries including 28 OECD countries took part in either PIRLS or TIMSS or  both during 2010 and 2011. There were countries from Europe, northern and  southern Africa, the Americas, the Middle East, the Caribbean, and Asia-Pacific.  Southern Hemisphere countries that took part administered PIRLS and TIMSS in  late 2010 and Northern Hemisphere countries administered them in early 2011.

Who participated in New Zealand?

PIRLS and TIMSS used a  three-tiered approach to sampling in order to be able to describe the  achievement of Year 5 and Year 9 students. First, representative samples of  schools based on characteristics such as size, decile, location, and at the  secondary level, authority and type (single-sex and co-educational), were selected  for each study and educational level. Then one or two or in some cases all  classes or groups with just Year 5 or Year 9 students were randomly selected  from each school. All students in the selected classes or groups took part.1 Approximately 5,600 Year 5 students from 192 schools took part in  PIRLS and 5,300 Year 9 students from 158 schools took part in TIMSS, in  November 2010. TIMSS was administered to 5,600 Year 5 students in 180 schools  in early October 2011.

What is the focus of PIRLS?

PIRLS looks at the two  main reasons why students at the middle primary level read. They are reading  for literary experience and to acquire and use information. As well as looking  at the reasons for reading, the study looks at the processes and skills of  reading comprehension. For example, when students read they often need to  interpret and integrate ideas in order to understand the underlying message of  a story; in other situations they are required to locate a specific piece of  information from part of a text to answer a question that they bring to a reading task, or to be able to check their  understanding of some aspect of the text's meaning.

What did the students have to do in  PIRLS?

Each student was given  a booklet that contained either two literary (story) texts, two information  texts, or one of each. There were five different literary texts and five  different information texts so that students did not all have the same  material. Each passage (story or information text) was followed by a series of  questions that were designed to assess the student's reading comprehension.  Some questions were closed (i.e., students selected an answer from those  provided) and some questions were open (i.e., students had to write their own  response to the question with some questions requiring one or two sentences  using examples from the texts to explain their answers).

What is the focus of TIMSS?

TIMSS is organised  around two aspects: content or subject matter within mathematics and science;  and the cognitive or thinking processes involved when answering questions. The mathematics  content dimensions are:

  • number; geometric shapes and measures; and data  display at middle primary level
  • number; algebra; geometry (including measurement);  and data and chance at the lower secondary level.

The science content  dimensions are:

  • life science; physical science (aspects of  chemistry and physics); and Earth science at the middle primary level
  • biology, chemistry, physics, and Earth science  at the lower secondary level.

Three cognitive areas were  assessed in both mathematics and science and were defined under three broad  descriptions, with each encompassing skills and behaviours specific to  mathematics or science areas: knowing, applying, and reasoning.

What did the students have to do in  TIMSS?

Each student answered  sets of mathematics and science questions in a booklet. There were 14 different  booklets at each education level. The different booklets meant that more  subject-matter could be covered without making the test longer for an  individual student. Some questions were closed (i.e., students selected an  answer from those provided) and some questions were open (short answers or  extended answers with explanations).

In which language were students assessed?

All countries that  took part in PIRLS and TIMSS  assessed students according to the language in which they received their  instruction. Many countries tested in more than one language because more than  one language was used for instruction in their country. In TIMSS, New Zealand  students were assessed in English and in PIRLS, schools were given the option  of testing students in either English or te reo Māori.2 When reporting at the national/country level,  countries combine the results for the different languages and so does New  Zealand. The reading literacy achievement results described relate to all Year  5 New Zealand students irrespective of whether their language of instruction is  te reo Māori or English or both.

What is the quality of the PIRLS and TIMSS information?

The assessment  frameworks, assessments, and contextual questionnaires were developed  cooperatively with representatives from all participating countries with input  from subject-matter experts. The procedures for developing, implementing, and  reporting are designed to ensure the reliability, validity, and comparability  of the data through standardised procedures, and attention to quality control  throughout. Procedures, such as field-testing of questions, detailed manuals  covering procedures, rigorous training for all involved, and quality assurance monitoring  during the implementation, ensure good quality information.

Footnotes

  1. There were some exceptions to this. For  example, children with an intellectual disability, less than 1 or 2 years in  the language of instruction, a physical disability which would prevent them  taking the assessment, or parents who did not wish their child to participate.
  2. Where the curriculum was delivered in  classes or schools for 81–100% of the time.

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065