Reading literacy at Year 5: New Zealand’s participation in PIRLS 2021 Publications
Publication Details
New Zealand took part in the fifth cycle of the Progress in International Reading Literacy Study, ‘PIRLS 2021’ in Term 4, 2020. PIRLS provides information on Year 5 students’ reading literacy/comprehension every five years.
Author(s): Megan Chamberlain and Jessica Forkert [Educational Measurement and Assessment, Ministry of Education]
Date Published: May 2023
Executive Summary
This report provides an overview of findings from the IEA’s1 Progress in International Reading Literacy Study, PIRLS 2021, which was implemented in New Zealand in Term 4, 2020, the first year of the COVID-19 pandemic. Fifty-seven countries and eight jurisdictions took part in PIRLS during this extraordinary period. It is in this context that we provide information on Year 5 students’ reading literacy in both a national and international context. A comprehensive description of the reading policy context and overview of the impact of COVID-19 on each country is published in the PIRLS 2021 Encyclopedia: Education Policy and Curriculum in Reading (see https://pirls2021.org/encyclopedia/ ).
Background
PIRLS 2021 is the fifth in the series of assessments that provide trend achievement measures in reading literacy. It is 20 years since the first cycle was implemented in 2001. New Zealand has taken part in all five PIRLS cycles. Grade 4, or Year 5 in New Zealand, is an important point in students’ development as most have finished ‘learning to read’ and are ‘reading to learn’. Most of the students who took part in PIRLS 2021 had started primary school in 2015. Five years on, the students were typically ten years old.
Implementing PIRLS 2021 during the COVID-19 pandemic
PIRLS 2021 was implemented during the COVID-19 pandemic. Despite the disruptions to schooling around the world, the PIRLS 2021 data collection was successful. There were three waves to the data collection, with the testing window starting in late 2020 and finishing in 2022, instead of finishing in 2021, as planned. It is worth noting that PIRLS 2021 cannot provide data on how students would have performed without COVID-19, as every country and its citizens were impacted.
Transitioning to digital assessment
PIRLS 2021 is the first cycle in which countries, including New Zealand, had the opportunity to implement the assessment digitally, via an online portal. To ensure trends in achievement could still be measured, and to make any adjustments to account for differences in the mode students completed the assessment with, a separate ‘bridge’ sample of students completed the assessment on paper.
Who took part in New Zealand?
Year 5 students from 249 schools (184 ‘digital’ and 65 ‘bridging’ schools), their teachers, principals and parents/caregivers took part in the study in late 2020. The results in this report are for those students who completed the assessment digitally – 5,557 students from 184 schools, their principals and teachers.
New Zealand offered PIRLS assessments in English and te reo Māori.
Implementing a group adaptive design
A group adaptive design was introduced to allow better measurement between the assessment difficulty and students’ levels of reading achievement across countries. It meant that students in countries with traditionally high average achievement were all located a greater proportion of more difficult assessment ‘booklets’ (two texts), while students in countries with low average achievement were allocated a greater proportion of easy booklets. New Zealand students average scores are in the middle group of countries taking part in PIRLS, so were allocated an even mix of easy, moderate and difficult booklets.
PIRLS 2021 reporting – assigning countries into two groups
When reporting the achievement results internationally and nationally, countries are grouped according to the time of the school year in which they assessed. There are two groups of countries:
- Group 1 (43 countries and five benchmarking systems) assessed their students towards the end of the school year. New Zealand was in this group. Thirty-two Group 1 countries took part in PIRLS 2016.
- Group 2 (14 countries and three benchmarking systems) delayed implementation until the beginning of a new school year (September), when the students had moved to the next grade or year level. Their students’ average age is about six months older than cohorts in previous PIRLS cycles. It also appears that being older was an advantage as these countries exhibited relatively large increases in achievement between PIRLS 2016 and PIRLS 2021. Thirteen Group 2 countries took part in PIRLS 2016.
Key Findings
What did the international results show for PIRLS 2021?
- Singapore had the highest average reading achievement in PIRLS 2021 (587). Its students significantly2 outperformed students in 56 other countries. Singaporean students in PIRLS 2021 also demonstrated stronger performance than their Singaporean peers in previous PIRLS cycles.
- It is likely that disruptions to students' learning because of the COVID-19 pandemic impacted the results for many countries, with 21 out of 32 Group 1 countries showing generally weaker performance in PIRLS 2021 than in PIRLS 2016.
- Internationally, girls generally had higher reading achievement than boys. New Zealand was no exception and continued to have one of the largest differences between girls’ and boys’ average reading achievement.
Reading achievement and attitudes of Year 5 students
- The mean reading score for New Zealand Year 5 students (521) was significantly higher than the PIRLS Scale Centrepoint (500)3, this being the mean from the first cycle of PIRLS and the reference point for measuring trends over time.
- There were no significant changes in the mean, the distribution of scores, or the proportions of Year 5 students reaching each of the PIRLS reading benchmarks from PIRLS 2016 to PIRLS 2021.
- New Zealand’s relative standing internationally appears to have ‘improved’ when compared with 24 other countries common to three PIRLS cycles, but this shift is largely due to the declines in the average performance of many of these countries.
- Year 5 students’ attitudes to reading and their reading confidence both appear to have somewhat waned over the five-year period since PIRLS 2016 was implemented in New Zealand in 2015.
- The relationship between Year 5 students’ reading achievement and their confidence as readers was particularly strong, with those who were very confident readers scoring about 117 score points higher on average than children who were not confident.
- Year 5 students were more positive about their digital prowess than they were about their reading.
Gender and Year 5 students' reading achievement
- New Zealand girls and boys on average achieved above the international means for girls and boys.
- Year 5 girls and boys achieved about the same on average as their PIRLS 2016 cohort peers, but significantly lower than their respective cohorts in PIRLS 2011.
- At 19 score points, the average difference between Year 5 girls’ and boys’ reading achievement is the lowest it has been in the 20 years of PIRLS. The shift over time is largely due to a slight weakening of achievement for girls.
Ethnic identity and Year 5 student' reading achievement
- There were no significant changes in the average achievement of students in any of the four main ethnic groupings.
- The average reading achievement gap between Māori girls and boys widened since from a non-significant 13 score-point difference in 2015 to a significant 21 score-point difference in 2020.
- At 31, the average difference between Pacific girls’ and boys’ reading achievement scores is the largest of any ethnic grouping.
Home background and experiences of Year 5 students
- PIRLS 2021 illustrates the increasing diversity of the middle primary student population, with proportionally fewer Year 5 students in the Pākehā/European grouping (47%) and proportionally more in the Asian grouping (15%) than 20 years ago.
- Eighteen percent of Year 5 students reported that they had been born overseas, with about half of this group moving to New Zealand when they were under five years of age (and the other half five or older).
- Students’ home languages also illustrate an increasingly diverse population. Apart from English, the languages most often spoken at home were te reo Māori, or one of the Chinese (such as Mandarin) or Indian (such as Hindi) languages.
- Year 5 students who frequently spoke the test language at home (English or te reo Māori) generally had higher reading achievement than those who rarely or never spoke it. The average achievement gap between students in the two language groups – those who almost/almost always and those who sometimes/never spoke the test language – continued to track down and in 2020 was at its lowest in 20 years.
- About 40 percent of Year 5 students had attended two or more primary schools before to PIRLS 2021 was implemented in 2020.
- Unsurprisingly, being absent from school was more frequent than usual as PIRLS took place in New Zealand in the first year of the COVID-19 pandemic. PIRLS 2021 showed that there was a decrease in the proportion of students responding they were never or almost never absent in 2020 from 2015 when PIRLS 2016 was implemented. The decrease was accompanied by an increase in reports of being absent once a week.
- The relationship between attendance and reading achievement was strong, with the difference between students who were never or almost never absent and those Year 5 students absent weekly averaging 96 score points.
School and classroom context for Year 5 students' learning
- The average difference in reading achievement between New Zealand schools where most students were from economically affluent backgrounds and those where most were from economically disadvantaged backgrounds remained high compared to many other countries.
- Students’ sense of school belonging appeared to have waned from PIRLS 2016 to PIRLS 2021 while their experiences of negative behaviours continue to be at the same level. Proportionally fewer students arrived at school feeling happy every day/almost every (75% in 2020, down from 80% in 2015).
- In 2020, teachers appeared to be less reliant on using same ability grouping to organise Year 5 students for reading instruction, varying their approaches by creating mixed ability groups and teaching reading as a whole class activity more frequently than previously.
- Nearly all (98%) of Year 5 students had access to digital devices to use during their reading lessons.
Footnotes
- International Association for the Evaluation of Educational Achievement (IEA).
- The use of ‘significant’ in the report is to be understood as statistical significance at the 5 percent level. For details, see Glossary of Terms and Technical Notes at the end of the report.
- The PIRLS reading achievement scale was established in the first PIRLS cycle (2001). The mean of 500 (and standard deviation of 100) was set and remains constant from assessment to assessment (referred to as the PIRLS Scale Centrepoint). The reference point of 500 has the same implication over time and trends can be measured with confidence.
Navigation
Contact Us
PIRLS
For queries about the Progress in International Reading Literacy Study (PIRLS) please email the: EMA Mailbox