PIRLS 2010/11 in New Zealand: An overview of findings from the third cycle of the Progress in International Reading Literacy Study (PIRLS)
PIRLS is an international research study designed to measure trends in reading literacy achievement of middle-primary school students every five years; in New Zealand the study involves Year 5 students. This was the third cycle and was administered in New Zealand in late 2010 and in Northern Hemisphere countries in early 2011. The first study was conducted in 2001 and the second, during 2005 and 2006. This mainly descriptive report presents a snapshot of the findings for 2010/11 in both an international and national context. Where possible comparisons are made with information from the two previous cycles.
Author(s): Megan Chamberlain, Research Division, Ministry of Education.
Date Published: July 2013
PIRLS-2010/11 involved approximately 5,600 New Zealand Year 5 students from 192 schools, their teachers, and parents/caregivers. An overview of the key results pertaining to New Zealand was presented in a summary report that was released to coincide with the announcement of the international results by the IEA and Boston College in December 2012.3
The focus of the following report is to describe Year 5 student achievement in both a national and international context and to describe any changes that may have occurred since 2001.
Year 5 reading achievement in an international context
- The mean reading score for New Zealand Year 5 students (531) was significantly higher than the international PIRLS Scale Centrepoint (500);4 32 countries, including New Zealand, were significantly higher than the PIRLS Scale Centrepoint.5
- The New Zealand mean was statistically similar to the mean scores of seven countries, including Australia and four other OECD countries-Slovak Republic, Slovenia, Austria, and Poland.
- It was significantly lower than the means for 20 countries or education systems, including 14 OECD countries. Five of these jurisdictions with achievement higher than New Zealand assessed in English: Northern Ireland, United States, Ireland, England, and Canada.
- The New Zealand mean was significantly higher than the means for 17 countries, including France, Norway, and Spain, and higher than two countries that tested in English-Trinidad and Tobago and Malta.
- There was no significant change in the mean achievement of New Zealand Year 5 students from 2001 to 2010/11.
- Compared to many other countries, among New Zealand Year 5 students there was a relatively large group who demonstrated that they were good readers. This was highlighted in two ways:
- the value of the 75th percentile (592) this being the point where 25 percent of Year 5 students achieved a higher score; and
- Year 5 students who achieved a score at or above the PIRLS High International Benchmark (45% of students including 14% who reached the PIRLS Advanced International Benchmark).
- Relative to higher-performing countries there was however a notable-sized group of New Zealand Year 5 students who showed that they were somewhat weaker readers. This was highlighted in two ways:
- the value of the 25th percentile (474) this being the point where 25 percent of Year 5 students achieved a lower score; and
- Year 5 students who did not reach the PIRLS Intermediate International Benchmark (25% of students including 8% who did not reach the PIRLS Low International Benchmark).
- New Zealand’s Year 5 girls (541) and Year 5 boys (521) achieved relatively well internationally.
- There was no significant change in either Year 5 girls’ or Year 5 boys’ mean reading achievement from 2001 to 2010/11.
- As observed in previous cycles of PIRLS, New Zealand had one of the largest differences between girls’ and boys’ mean reading scores, favouring girls, internationally.
Year 5 reading achievement in a national context
- Pākehā/European (558) and Asian (542) students scored, on average, at a significantly higher level than Māori (488) and Pasifika (473) students.
- There was no significant change in the mean reading achievement of Year 5 students in any of the main four ethnic groupings from 2001 to 2010/2011.
- The group of lower-achieving students-defined as the group of students who did not reach the PIRLS Intermediate International Benchmark-was over-represented by Pasifika students, Māori students (boys more so than girls), and to a lesser extent, Year 5 boys.
- The likelihood of Māori boys being in the lower-achievers group decreased a little from 2005/06 to 2010/11, but increased for both Pasifika boys and girls.
- The magnitude (size) of the difference between Year 5 girls and boys decreased slightly from 2001 to 2010/11, largely due to small (albeit non significant) increases in the mean achievement of both Māori and Pākehā/European boys.
- New Zealand Year 5 students were found to have a significant strength in literary reading compared to their overall reading performance; reading informational texts was neither a weakness nor strength.
- There were no significant changes in New Zealand Year 5 students’ achievement in either reading purpose from 2001 to 2010/11; the changes that were observed reflect a shift from 2005/06.
- Pākehā/European students’ mean score in literary reading increased significantly from 2005/06 to 2010/11, returning to about the same as in 2001.
- Asian students recorded a significant decrease in their informational reading from 2005/06 to 2010/11.
- Informational reading was a significant weakness for Year 5 girls compared to their overall reading achievement. Furthermore, Year 5 girls’ mean score in informational reading decreased significantly from 2005/06 to 2010/11.
- As was the case in 2001 and 2005/06, New Zealand Year 5 students showed significantly stronger performance when required to use their reasoning skills (interpreting, integrating, and evaluating) but significantly weaker performance using their text-based skills (retrieving information and making straightforward inferences).
Year 5 students’ home context
- Children’s early childhood experiences, including the number of years they attended an early childhood facility, were positively related to their reading literacy achievement when in Year 5. The relationship was stronger for Year 5 boys than it was for Year 5 girls.
- New Zealand’s Year 5 students who regularly spoke the language of the PIRLS assessment generally had higher reading literacy achievement (543) than Year 5 students who sometimes or rarely did (499).
- The difference between the means for Year 5 students (43) in the two home language categories was larger in New Zealand than their counterparts in other countries, such as Australia (18) and Ireland (25), but similar to the United States (40).
- The difference for New Zealand increased markedly from 2005/06 (24) to 2010/11 (43).
- Looking specifically at Year 5 students assessed in English (the vast majority of students in PIRLS), the increase was more marked for Pasifika students than students in other ethnic groupings.
- The odds of a Year 5 student who did not speak the test language at home being in the lower-achievers group in 2010/11 increased from 2005/06 to 2010/11.
Schools and school climate for learning
- New Zealand’s Year 5 students’ reading achievement tended to be lower in schools where proportionately few of the student body had the pre-requisite skills at school entry than in schools where more of the student body had these skills.
- The student intake of lower decile schools, particularly deciles 1 and 2 schools, was less likely to have early literacy skills when beginning primary than higher decile schools, particularly deciles 9 and 10 schools.
- The achievement difference in reading for students in New Zealand primary schools according to their economic composition6-the mix of students from economically affluent backgrounds and those from economically disadvantaged backgrounds-was very high compared to many other countries.
- New Zealand’s deciles 1 and 2 schools and to a lesser extent deciles 3 and 4 schools had the greatest concentration of students from economically disadvantaged backgrounds, while decile 9 and 10 schools had the greatest concentration of students from economically affluent backgrounds.
- Although there were high-performing and low-performing students across all deciles, students in deciles 1 and 2 and to a lesser extent decile 3 and 4 schools tended to have the weakest performance.
- About 50 percent of Year 5 students in deciles 1 and 2 schools reached the PIRLS Intermediate International Benchmark compared with 90 percent of their counterparts in deciles 9 and 10 schools.
- Ten percent of New Zealand’s Year 5 students were lower-achieving students in deciles 1 and 2 schools; six percent were attending deciles 3 and 4 schools.
- When students’ reading achievement was examined according to school decile, there was no significant change from 2001 to 2010/11.
- Compared with other countries, New Zealand Year 5 students were more likely to be attending schools where both their principals and teachers endorsed aspects of ‘academic optimism’; they shared a common view of academic success through their understanding of the school’s curricular goals, implementation of the school’s curriculum, and expectations for student achievement.
- New Zealand principals of lower decile schools tended to be less positive about the climate for learning in their schools and more likely to have concerns about negative student behaviours in their schools than their counterparts in mid-range and higher decile schools.
- Consistent with reporting in 2001 and 2005/06, New Zealand’s Year 5 students were more likely to have experienced bullying behaviours than many of their international counterparts.
- Students in lower decile schools were more likely to report they experienced the negative behaviours than students in higher decile schools.
- Despite experiencing negative behaviours, New Zealand Year 5 students reported positively that they liked school.
- Students in deciles 1 and 2 schools tended to be the most positive and those in deciles 7 and 8 schools the least positive.
- The 2010/11 Year 5 student cohort tended to be more positive than either their 2001 or 2005/06 counterparts; this was largely due to more positive endorsements by Year 5 boys, particular Pākehā/European boys.
The classroom context for learning
- New Zealand spent the fourth highest number of hours, on average (actual and as a proportion of total instructional hours), teaching reading, both formally and informally, during the school year.
- This amount of time is consistent with the situation in New Zealand schools in 2001 and 2005/06.
- While teachers tend to use a variety of approaches for organising their reading instruction, in New Zealand the single organisational approach reported to be used ‘almost always’ by teachers is to arrange Year 5 students into same-ability groups.
- Teaching reading as a whole-class activity was an approach used often in many countries but infrequently by New Zealand teachers.
- Consistent with findings in both 2001 and 2005/06, the majority of New Zealand teachers reported using a reading series as a basis for their reading programmes, often as a dual approach with children’s books.
- New Zealand Year 5 students tended to like reading more than students in many other countries.
- New Zealand’s Asian students generally liked reading more than students in other groups, with Māori students liking it less. Māori boys were, however, less likely to find reading a boring activity in 2010/11 than in 2001.
- New Zealand’s Year 5 students’ self confidence in their reading ability was relatively low internationally.
- New Zealand’s Asian students tended to be more confident; Pasifika students were less confident.
- New Zealand’s Year 5 students’ motivation to read in was comparable to the international average.
- In New Zealand, Asian and Pasifika students were generally more motivated than other student groups, with Pākehā/European students somewhat less motivated.
- New Zealand teachers tended to use a series of practices to engage Year 5 students during reading a little less often than teachers in many other countries; their students were found to be mostly somewhat engaged during their reading.
- International Association for the Evaluation of Educational Achievement (IEA).
- Internationally, this cycle is referred to as PIRLS 2011. In this report it is referred to as PIRLS-2010/11 to acknowledge the timing of the study which was administered in Southern Hemisphere countries in late 2010 and in Northern Hemisphere countries in early 2011.
- Key findings from New Zealand’s participation in the Progress in International Reading Literacy Study (PIRLS) and Trends in International Mathematics and Science Study (TIMSS) in 2010/11 (Chamberlain & Caygill, 2012).
- The PIRLS achievement scale uses the same point of reference (500) from assessment to assessment (refer to page 23 or the Technical Notes for details).
- The use of ‘significant’ hereafter is to be understood in terms of statistical significance at 5% level. See Technical Notes at the end of this report.
- A comparison of students in schools with more than 25% of the student body from economically affluent homes and 25% or fewer from disadvantaged homes with schools with more than 25% of students coming from economically disadvantaged homes and 25% or fewer coming from economically affluent homes and "schools with neither more affluent nor more disadvantaged students".