TIMSS 2010/11 & PIRLS 2010/11 Key findings: New Zealand’s participation in PIRLS and TIMSS Publications
This summary report presents the key findings from New Zealand's participation in PIRLS (Progress in International Reading Literacy Study) 2010/11 and TIMSS (Trends in International Mathematics and Science Study) 2010/11.
Author(s): Megan Chamberlain and Robyn Caygill
Date Published: December 2012
What are PIRLS and TIMSS?
Both PIRLS and TIMSS are international research studies that are designed to measure trends in student achievement. They are coordinated by the International Association for the Evaluation of Educational Achievement (IEA). PIRLS looks at reading literacy achievement of middle primary school students. New Zealand's Year 5 students take part in PIRLS. It is administered every five years, with the first cycle in 2001, the second in 2005/06, and then the third in 2010/11.
TIMSS looks at the mathematics and science achievement of both middle primary and lower secondary school students. New Zealand's Year 5 and Year 9 students take part. TIMSS is administered on a regular four-year cycle with the first cycle in 1994/95, and then again in 1998/99, 2002/03, 2006/07, and then the fifth cycle in 2010/11.
TIMSS also provides information on the relative progress of the middle primary school cohort four years later when they are in lower secondary school. For example and although not exactly the same students, the Year 5 student cohort assessed in TIMSS in 2006/07 formed the Year 9 TIMSS cohort in 2010/11.
As well as comprehensive assessment information in three essential learning areas—reading, science, and mathematics—a rich array of contextual background information is collected from students, teachers, parents/caregivers, and school principals. National educational policy information is also provided by each country to aid the interpretation of results.
PIRLS and TIMSS in 2010/11
Both PIRLS and TIMSS were administered in 2010/11. This provided a unique opportunity for many countries participating in both PIRLS and TIMSS at the middle primary level as it had the advantage of one comprehensive assessment in all three learning areas: reading, mathematics, and science. Many countries that took part chose to assess the same middle primary school students in all three areas. New Zealand chose to assess two different groups of students: one group that took part in PIRLS and one group that took part in TIMSS.
What countries took part?
Approximately 60 countries including 28 OECD countries took part in either PIRLS or TIMSS or both during 2010 and 2011. There were countries from Europe, northern and southern Africa, the Americas, the Middle East, the Caribbean, and Asia-Pacific. Southern Hemisphere countries that took part administered PIRLS and TIMSS in late 2010 and Northern Hemisphere countries administered them in early 2011.
Who participated in New Zealand?
PIRLS and TIMSS used a three-tiered approach to sampling in order to be able to describe the achievement of Year 5 and Year 9 students. First, representative samples of schools based on characteristics such as size, decile, location, and at the secondary level, authority and type (single-sex and co-educational), were selected for each study and educational level. Then one or two or in some cases all classes or groups with just Year 5 or Year 9 students were randomly selected from each school. All students in the selected classes or groups took part.1 Approximately 5,600 Year 5 students from 192 schools took part in PIRLS and 5,300 Year 9 students from 158 schools took part in TIMSS, in November 2010. TIMSS was administered to 5,600 Year 5 students in 180 schools in early October 2011.
What is the focus of PIRLS?
PIRLS looks at the two main reasons why students at the middle primary level read. They are reading for literary experience and to acquire and use information. As well as looking at the reasons for reading, the study looks at the processes and skills of reading comprehension. For example, when students read they often need to interpret and integrate ideas in order to understand the underlying message of a story; in other situations they are required to locate a specific piece of information from part of a text to answer a question that they bring to a reading task, or to be able to check their understanding of some aspect of the text's meaning.
What did the students have to do in PIRLS?
Each student was given a booklet that contained either two literary (story) texts, two information texts, or one of each. There were five different literary texts and five different information texts so that students did not all have the same material. Each passage (story or information text) was followed by a series of questions that were designed to assess the student's reading comprehension. Some questions were closed (i.e., students selected an answer from those provided) and some questions were open (i.e., students had to write their own response to the question with some questions requiring one or two sentences using examples from the texts to explain their answers).
What is the focus of TIMSS?
TIMSS is organised around two aspects: content or subject matter within mathematics and science; and the cognitive or thinking processes involved when answering questions. The mathematics content dimensions are:
- number; geometric shapes and measures; and data display at middle primary level
- number; algebra; geometry (including measurement); and data and chance at the lower secondary level.
The science content dimensions are:
- life science; physical science (aspects of chemistry and physics); and Earth science at the middle primary level
- biology, chemistry, physics, and Earth science at the lower secondary level.
Three cognitive areas were assessed in both mathematics and science and were defined under three broad descriptions, with each encompassing skills and behaviours specific to mathematics or science areas: knowing, applying, and reasoning.
What did the students have to do in TIMSS?
Each student answered sets of mathematics and science questions in a booklet. There were 14 different booklets at each education level. The different booklets meant that more subject-matter could be covered without making the test longer for an individual student. Some questions were closed (i.e., students selected an answer from those provided) and some questions were open (short answers or extended answers with explanations).
In which language were students assessed?
All countries that took part in PIRLS and TIMSS assessed students according to the language in which they received their instruction. Many countries tested in more than one language because more than one language was used for instruction in their country. In TIMSS, New Zealand students were assessed in English and in PIRLS, schools were given the option of testing students in either English or te reo Māori.2 When reporting at the national/country level, countries combine the results for the different languages and so does New Zealand. The reading literacy achievement results described relate to all Year 5 New Zealand students irrespective of whether their language of instruction is te reo Māori or English or both.
What is the quality of the PIRLS and TIMSS information?
The assessment frameworks, assessments, and contextual questionnaires were developed cooperatively with representatives from all participating countries with input from subject-matter experts. The procedures for developing, implementing, and reporting are designed to ensure the reliability, validity, and comparability of the data through standardised procedures, and attention to quality control throughout. Procedures, such as field-testing of questions, detailed manuals covering procedures, rigorous training for all involved, and quality assurance monitoring during the implementation, ensure good quality information.
- There were some exceptions to this. For example, children with an intellectual disability, less than 1 or 2 years in the language of instruction, a physical disability which would prevent them taking the assessment, or parents who did not wish their child to participate.
- Where the curriculum was delivered in classes or schools for 81–100% of the time.
Where to find out more
For queries about the Progress in International Reading Literacy Study (PIRLS) please email the: PIRLS Mailbox