PISA 2009: Our 21st century learners at age 15 Publications
Publication Details
This report provides a high level picture of New Zealand’s 15-year-old performance in reading literacy (main focus), mathematical literacy and scientific literacy. It compares New Zealand’s results with other top- and high-performing countries. In July and August 2009 4,643 New Zealand 15 year-old students from 163 New Zealand schools took part in PISA 2009.
Author(s): Maree Telford with Steve May [Ministry of Education]
Date Published: December 2010
Reading literacy (main focus)
What aspects of reading literacy does PISA measure and report on?
Reading was the main focus of PISA when the study first began in 2000. Four administrations of PISA have now been completed, and PISA 2009 marks the beginning of a new cycle with a return to a focus on reading. This allows for reading to be measured in more detail.8 The reading tasks in the assessment covered three dimensions or characteristics, knowledge domain (the form of reading materials): continuous texts, non-continuous texts, mixed texts and multiple texts; competencies or aspects (types of reading task or process): access and retrieve, integrate and interpret, reflect and evaluate, and complex9; and situations or context (the use for which the text is constructed): personal, educational and occupational and public and scientific (see Figure 1). |
These are the same as in PISA 2000, except for the integrating and interpreting scale and accessing and retrieving scale; in PISA 2000 students were only assessed on how they interpreted what they read, whereas PISA 2009 also looked at how well they integrated it. In PISA 2000 students were only assessed on how well they retrieved information, in PISA 2009 they were also assessed on how well they accessed it.
Students’ reading results are reported by mean scores and proficiency levels on the two knowledge domain scales (continuous and non-continuous texts10) and on three of the four competency or aspect scales (access and retrieve, integrate and interpret, and reflect and evaluate).11 Reading is also reported, as for the previous administrations of PISA, in terms of the overall reading literacy scale, previously referred to as the combined reading literacy scale.
This section compares the reading results of New Zealand’s 15-year-olds with the average for the 34 OECD countries, as well as the countries with a mean reading performance that was either better than, or not statistically different to, that of New Zealand’s students.
An electronic reading assessment that measured how well students can read digital texts was also administered in this round of PISA. Twenty countries, including New Zealand, elected to take part in this option. The results of this assessment will be released by the OECD in 2011.
Figure 1: The PISA 2009 reading literacy framework
Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science, Vol 1. OECD: Paris.
Figure 2: What the reading literacy proficiency measures
Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science, Vol 1. OECD: Paris.
Student performance in reading
Reading literacy scale
The reading literacy scale summarises student performance on the tasks that cover the three dimensions of the framework. This allows for an ongoing, high-level picture of reading, whether it is a minor or a major focus of a PISA administration.
The mean overall reading literacy performance of the 65 countries or economies and the percentage of students on each of the seven proficiency levels are shown in Figure 3.
Mean scores by country comparison on the reading literacy scale
New Zealand’s 15-year-old students performed very strongly in reading literacy, with a mean score of 521 points. This was statistically better than the average score for the 34 OECD countries (493).
- Only two OECD countries, Korea (539) and Finland (536), achieved a better reading result than New Zealand’s 15-year-olds. *Shanghai-China12 (556) and *Hong Kong-China (533)13, two non-OECD partner economies, also gained significantly higher mean reading scores.
- New Zealand’s mean performance was similar to that of *Singapore14 (526), Canada (524), Japan (520) and Australia (515).
- New Zealand’s 15-year-olds on average outperformed their peers in 56 of the participating countries or economies, including the United States (500), the United Kingdom (494) and 26 of the other 33 OECD member countries.
Because these results are derived from samples of students and schools, there is a margin of error associated with the mean scores reported.15 As a result, it is not possible to determine a precise ranking of a country’s performance. However, it is possible to determine, with a 95 percent likelihood, the range of ranks in which a country’s mean performance lies. This is expressed in terms of an upper rank and a lower rank for each country.
Among the 34 OECD countries, New Zealand’s highest upper ranking in mean reading performance is third and its lowest ranking is fifth. When compared with all 65 participating countries or economies, New Zealand’s upper rank is sixth and its lower rank is ninth.16
Proficiency levels (seven levels) by all students on the reading literacy scale
In this administration of PISA the reading proficiency levels have been extended from five to seven levels with the introduction of a higher level, Level 6, and a lower level, Level 1b. In PISA 2000 the most advanced reading level was Level 5 or higher (scores of 625 points or higher). At the other end of the spectrum, the lowest proficiency level was referred to as Level 1, and students at this level were those with scores lower than 334 points.
Figure 3: Reading literacy proficiency levels
Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science, Vol 1. OECD: Paris.
This new approach allows for the identification of students with either very low (Level 1b, scores from 262 to 334 points) or very high (Level 6, scores above 708 points) reading proficiency, and provides better and more detailed descriptions of their skills and knowledge.
Level 2 has been established as the baseline level at which students begin to demonstrate competencies that will enable them to participate actively in life situations that are related to the three key subject areas assessed.
For a full description of the types of tasks that students can typically perform at each reading proficiency level, see Figure 2. It should be noted that students achieving a particular proficiency level are also considered to be proficient at the lower levels. For example, students proficient at Level 5 are considered to be proficient in reading at the four lower levels.
Level 6 (scores above 708 points) and Level 5 (scores 626 to 708 points)
Proficiency Level 6 tasks required students to demonstrate very advanced reading skills. For example, students at this level showed they were able to make multiple inferences, comparisons and contrasts from texts that were both detailed and precise.
Students achieving Level 5 are also described as top performers, as they were required to demonstrate they were capable of dealing with concepts that are not straightforward.
- Three percent of New Zealand’s and *Singapore’s 15-year-olds showed advanced reading skills; that is, they achieved Level 6. This proportion was similar to that found in *Shanghai-China, Australia and Japan − all with two percent.
- New Zealand (16%) and Finland (15%), along with the two new PISA participants *Shanghai-China (19%) and *Singapore (16%), had at least 15 percent of their students showing proficiency in reading at Level 5 or higher (a combination of Levels 5 and 6). In Japan (13%), Korea (13%), Australia (13%), Canada (13%) and *Hong Kong-China (12%), more than 10 percent of students were at these levels.
- In the other two large English-speaking countries, the United States (10%) and the United Kingdom (8%), the proportion reaching Level 5 or higher was smaller. This was also the case for the average across the 34 OECD member countries (8%).
Level 4 of higher (scores above 552 points)
Readers at this level were required to demonstrate an accurate understanding of long or complex texts whose content or form may be unfamiliar.
- More than half of *Shanghai-China’s 15-year-olds (54%) were proficient at Level 4 or higher, and at least 40 percent of students from Korea (46%), Finland (45%), *Hong Kong-China (44%), *Singapore (41%), New Zealand (41%), Japan (40%) and Canada (40%) demonstrated proficiency at these levels. Statistically smaller proportions of students from Australia (37%), the United States (30%) and the United Kingdom (28%) were proficient at one of these three upper levels relative to New Zealand. The average for the OECD was 28 percent.
Level 1a (scores from 335 to 407 points) and Level 1b (scores from 262 to 334 points) or below (scores below 262 points)
Students with Level 1a proficiency were able to locate one or more pieces of information explicitly stated and identify the main theme of a text. The reader was explicitly directed to consider relevant factors in the task and in the text. At the new lower level, Level 1b, there was minimal competing information requiring interpretation. The reader may have needed to make simple connections between adjacent pieces of information as the text typically provided support to the reader.
- Fourteen percent of New Zealand students did not achieve Level 2 reading literacy. This is the same proportion as Australia (14%) and Japan (14%). When compared to New Zealand, smaller proportions were at these levels in the six other top- or high-performing countries or economies: *Shanghai-China (4%), Korea (6%), Finland (8%), *Hong Kong-China (8%), Canada (10%) and *Singapore (12%).
- The proportion of students not reaching Level 2 in the two other large English-speaking countries − the United Kingdom (18%) and the United States (18%) − was larger than in New Zealand, Australia or Canada.
- At the lowest end of the proficiency scale, PISA 2009 Level 1b or below, Japan (5%), New Zealand (4%) Australia (4%) and *Singapore (3%) all showed slightly larger proportions than the five other top-or high-performing countries or economies. In the four top-performing countries or economies, two percent or less were at these levels. The proportion of students for the United States (5%) and the United Kingdom (5%) at the PISA 2009 lowest level was similar to that in New Zealand.
- Across the 34 OECD countries, an average of 19 percent of students were not proficient in reading at Level 2, and six percent of these students did not reach Level 1a.
Overall 15-year-olds from *Shanghai-China, a new non-OECD PISA participant, clearly outperformed their peers in each of the 64 other participating countries or economies with a score that was 17 points higher than any other participating country or economy.
At the top reading proficiency levels, 16 percent of New Zealand’s students achieved a score of at least 626 points. Among these students three percent achieved the new advanced PISA 2009 reading proficiency level, with a reading score of more than 708 points. While the percentage of New Zealand student at Level 5 or higher was three percentage points lower than in 2000 (19%), only *Shanghai-China (19%) exceeded the proportion of New Zealand’s students at the top level of proficiency in 2009.
The proportion of New Zealand’s weaker readers was larger than in the majority of the top-or high-performing countries, but similar to that in Australia and Japan.
Mean scores by gender on the reading literacy scale
In each of the 65 participating countries or economies, girls showed a significantly stronger reading performance than boys.
- New Zealand’s 15-year-old girls achieved an average of 544 score points, 46 points greater than their male counterparts (499).17
- The New Zealand girls’ average reading performance was stronger than that of their Australian counterparts (533), but similar to that in *Hong Kong-China (550), *Singapore (542), Canada
- (542) and Japan (540). However, girls in *Shanghai-China (576), Korea (558) and Finland (563) achieved a significantly stronger result.
- The average performance of boys in Japan (501) and Australia (496) was similar to that of New Zealand’s boys. Boys in the remaining top- or high-performing countries or economies − *Shanghai-China (536), Korea (523), *Hong Kong-China (518), *Singapore (511), Finland (508) and Canada (507) − showed a significantly stronger result.
More than 20 points separated boys’ and girls’ mean reading performance in every participating country or economy, except for *Colombia (a difference of 9 points). This disparity is reflected in the OECD average gender difference of 39 points (girls 513, boys 474). When comparing the nine top-or high-performing countries, Finland (55 score point difference) and New Zealand (46) showed the largest difference in favour of girls.
Proficiency levels by gender on the reading literacy scale
Level 6 (scores above 708 points) and Level 5 (scores 626 to 708 points)
- Not surprisingly, a larger proportion of New Zealand’s girls (4%) than boys (2%) achieved the highest level of reading proficiency, Level 6. It should be noted that no other participating country achieved a larger proportion of either boys or girls at this level.
- *Hong Kong-China (boys 1%, girls 2%) and Korea (boys 1%, girls 1%) had proportionally fewer boys and girls than New Zealand at Level 6 and Finland (1%) slightly fewer boys. The remaining five top- or high-performing countries achieved similar proportions of both boys and girls to New Zealand.
- At least one in five 15-year-old girls in New Zealand (20%), *Shanghai-China (26%) and Finland (21%) were top-performing readers; that is, they were proficient in reading at one of the two highest levels, a proportion that was twice the OECD average.18
- In contrast, the proportion of boys succeeding at Level 5 or higher was substantially smaller; *Shanghai-China (13%), New Zealand (12%), *Singapore (12%) and Japan (10%) were the only participating countries or economies that had more than 10 percent of boys at these levels. Nevertheless, as found for girls, the proportion of New Zealand boys at these levels was over twice the OECD average (5%).
Level 4 or higher (scores above 552 points)
- Around half (49%) of New Zealand’s girls were at Level 4 or higher, while a third (32%) of their male counterparts were at these levels.
- Japan (34%), Canada (33%), Finland (32%) and Australia (30%) also had around a third of their 15-year-old boys achieve Level 4 or higher, but the average proportion of boys across OECD countries was substantially smaller (22%). In the remaining four top- or high-performing countries or economies, more than 35 percent of boys were proficient at these levels: *Shanghai-China (44%), Korea (38%), *Hong Kong-China (37%) and *Singapore (36%). In the United States (27%) and the United Kingdom (24%), the average was around a quarter.
- Well over half of the girls in *Shanghai-China (64%), Finland (58%), Korea (55%) and *Hong Kong-China (53%) were at Level 4 or higher. Japan (47%), *Singapore (47%), Canada (46%) and Australia (43%) also had substantially more girls at these levels than the OECD average (35%), the United States (34%) and the United Kingdom (31%).
Level 1a (scores from 335 to 407 points) and Level 1b (scores from 262 to 334 points) or below (scores below 262 points)
- Although fewer than 10 percent of New Zealand’s girls (8%) did not reach Level 2, one in every five boys was at these lower levels (21%). A similar pattern to New Zealand was observed in Australia (boys 20%, girls 9%) and Japan (boys 19%, girls 8%).
- *Singapore (9%) had a similar proportion of girls to New Zealand, while the other top-or high-performing countries all achieved smaller proportions of boys (including *Singapore for boys) and girls at these levels (ie, not scoring above 407 points).
- Seven percent of New Zealand’s boys scored below 334 score points (the former PISA 2000 - 2006 ‘below Level 1’ reading proficiency), while only one percent of their female counterparts were at this level. This was also the case in Japan (boys 7%, girls 2%), Australia (boys 6%, girls 2%), *Singapore (boys 5%, girls 1%) and the average for all OECD countries (boys 8%, girls 3%). Statistically smaller proportions of boys were observed at Level 1b or below in the other top- or high-performing countries or economies.
In short, in all the participating countries or economies, girls achieved significantly better than boys, whether we look at mean scores or proficiency levels. The difference in the mean reading performance for girls and boys in New Zealand was the second largest among the top-or high-performing countries or economies. It was exceeded only by Finland, which had proportionally fewer boys with high levels of proficiency.
Mean scores by ethnic group/s on the reading literacy scale
Students taking part in PISA were asked to provide information about their ethnic group/s. Students were asked to identify the ethnic groups they belonged to. Those who identified that they belonged to more than one group were counted in each of those groups.19
- Students identifying as Pākehā/European, who comprised 71 percent of all students, achieved an average reading score of 541 score points in reading literacy. Students identifying as Asian (14%) scored 522 points, which was still significantly above the OECD mean (493).
- Those students identifying as Māori (19%) and Pasifika (10%) scored 478 score points and 448 score points,
- respectively. This was below the OECD mean.
Proficiency levels by ethnic group/s on the reading literacy scale
Level 6 (scores above 708 points) and Level 5 (scores 626 to 708 points)
- One in five (19%) of the students identifying as Pākehā/European demonstrated a reading ability that was at Level 5 or higher. Four percent of these students had exceptional skills, achieving Level 6. The proportion of Asian students at Level 5 or higher was 16 percent. Smaller proportions of students who identified as Māori (7%) and Pasifika (4%) achieved a score that was greater than 626 (Level 5 or higher).20
Level 4 or higher (scores above 552 points)
- Almost half of students identifying as Pākehā/ European (48%) were proficient readers at Level 4, and a slightly smaller proportion of Asian (41%) students scored at this level. A quarter of Māori (23%) students achieved a score that was greater than 552 (ie, Level 4 or higher). Proportionally fewer (17%) Pasifika students scored at this level.
Level 1a (scores from 335 to 407 points) and Level 1b (scores from 262 to 334 points) or below (scores below 262 points)
- Around a third of Pasifika (35%) and a quarter of Māori (24%) students did not show reading proficiency above Level 1a. A substantially smaller proportion of Asian (15%) and Pākehā/European (9%) students scored at this level.
- A similar pattern was seen for the proportions of students at Level 1b and below, where there were proportionally fewer Pākehā/European (2%) and Asian (3%) than Māori (6%) and Pasifika (13%) students.
The performance of students in each ethnic group was diverse. In each group there were students whose performance was poor and students who had advanced knowledge and skills.
However, the reading mean score for Māori and Pasifika students was lower than for Pākehā/European and Asian students. A small proportion of Māori and Pasifika students achieved at the highest levels of proficiency, but they were over-represented at the lower levels relative to the students from the other two ethnic groups.
In numerical terms, Pākehā/European was the largest ethnic group at the low proficiency levels as well as the high proficiency levels. This was also the case for mathematics and science.
Distribution of students on the reading literacy scale
Examining the distribution of student reading scores provides a high-level picture of the extent of the diversity in students’ reading ability within a country (see Figure 4). Countries with a wide spread of student achievement will have a larger disparity between high and low achievers within their country.
Figure 4: Reading literacy distribution
Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science, Vol 1. OECD: Paris.
Highest- and lowest-performing students: 95th and 5th percentile
- Among the eight top- or high-performing countries or economies, New Zealand had the widest range of scores (335) between the bottom five percent (5th percentile) and top five percent (95th percentile) of students, followed by Japan (328) and Australia (325). This is mostly accounted for by the fact that the top five percent of New Zealand’s students (678) outperformed their counterparts in Australia (668) and Japan (667)21, while the 5th percentile in Australia (343) and Japan (339) was about the same as in New Zealand (344).
- New Zealand (678), *Shanghai-China (679) and *Singapore (676) were the only countries or economies where five percent or more of students achieved more than 670 score points. However, the performance of New Zealand’s lowest five percent with scores of 344 or less, was lower than all but three, (Japan, Australia – as noted above – and *Singapore) of the top-or high-performing countries or economies.
Middle-range performing students:25th−75th percentile
- The range of scores (143 points) between New Zealand’s bottom quarter (25th percentile) and top quarter (75th percentile) of students was also greater than in any other top-or high-performing country or economy. In contrast, the range for the four top-performing countries was less than 120 score points (Korea 105, *Hong Kong-China 110, *Shanghai-China 109 and Finland 116). The other four top-or high-performing countries ranged between 124 (Canada) and 137 (*Singapore) score points.
- New Zealand was one of six OECD countries that exhibited more than 140 score points between the top and bottom quarter students. The others were Israel (153), Belgium (147), Austria (146), France (143) and Luxembourg (143).
- As found for New Zealand’s bottom five percent of students, the 25th percentile score of 452 was about the same as in Japan (459) and Australia (450), but lower than the other top- or high-performing countries or economies.
- In contrast, New Zealand’s top quarter of students achieved scores of 595 or higher. This was higher than in Australia (584) and Canada (588) and, with the exception of *Shanghai-China (613), much the same as the rest of the top- or high-performing countries or economies.
Although New Zealand continues to show a wide range of scores in reading, not all this difference can be attributed to low performing students. The success of the highest performing students also increased the size of the spread.
Knowledge domains: text formats
Continuous and non-continuous texts
Student performance on the continuous and non-continuous texts format scales is reported here because they describe the two ways in which texts are commonly structured in everyday life: either in sentences and paragraphs (continuous), or in other formats such as lists, diagrams, graphs and tables (non-continuous). Just under two-thirds of the questions related to continuous texts and around a third to non-continuous texts. Mixed (a mixture of both continuous and noncontinuous) and multiple (from more than one source) text formats are also incorporated into the reading assessment so that PISA’s definition of reading literacy is adequately covered.
Mean scores by all students on continuous and non-continuous texts
- New Zealand (532) performed strongly on the tasks that measured students’ ability to read noncontinuous texts relative to those that measured students’ ability to read continuous texts (518). Australia, the United Kingdom and *Singapore, as well as *Liechtenstein and Estonia, were the only other PISA participating countries or economies to show a score of more than 10 points higher on the non-continuous texts than the continuous texts.
- Students from three of the East-Asian countries or economies − Korea (542), *Shanghai-China (539) and *Singapore (539) − were the only participants to achieve a better result than New Zealand’s students on the non-continuous texts scale. Finland’s (535) and Canada’s (527) mean performance was about the same as New Zealand’s, while that of the three other top- or high-performing countries was lower: Australia (524), *Hong Kong-China (522) and Japan (518). This was also the case for the United Kingdom (506), the United States (503) and the OECD average (493).
- The mean performance of students on the continuous texts scale from three of the top-or high-performing OECD countries − Korea (538), Finland (535) and Canada (524) − and two of the partner economies – *Shanghai-China (564) and *Hong Kong-China (538) – was better than New Zealand’s. Although New Zealand’s 15-year-olds performed about the same as their peers from *Singapore (522) and Japan (520), they performed better on this scale than their peers from Australia (513), the United States (500), the United Kingdom (492) and the average across the 34 OECD countries (494).
Mean scores by gender on non-continuous texts
- In all OECD countries (including New Zealand), girls outperformed boys on the two text formats. In New Zealand a gender difference of well over half a proficiency level (44) was found. Finland (54) was the only other top- or high-performing country with a gender difference of more than 40 points.
- On the non-continuous texts scale, New Zealand’s girls’ achievement (555) put it among the countries or economies where girls achieved 550 score points or more – Finland (562), Korea (559), *Shanghai-China (557) and *Singapore (553). The mean performance of girls from the remaining four top- or high-performing countries was lower than that of New Zealand’s girls: Canada (544), Australia (541), Japan (537) and *Hong Kong-China (536). This was also true for girls from the United Kingdom (518), the United States (514) and the average for OECD countries (511).
- Although New Zealand boys’ mean performance (511) on reading non-continuous texts was substantially weaker than that for girls, they achieved a similar result to boys from Canada (511), *Hong Kong-China (510), Finland (508), Australia (507) and Japan (499). Boys from Korea (527), *Singapore (524) and *Shanghai-China (522) all achieved a better result. As found for girls, New Zealand’s boys outperformed boys from the United Kingdom (492) and the United States (492), and scored above the average for OECD countries (475).
Mean scores by gender on continuous texts
- A similar gender pattern was found on the continuous texts scale as was found for non-continuous texts. Well over half a proficiency scale separated New Zealand’s boys’ (495) and girls’ (542) mean performances. Finland (56) and *Shanghai-China (45) were the only other top- or high-performing countries or economies to show a gender difference of more than 40 score points on this scale. However, across all the OECD countries, 42 score points separated boys and girls.
- Boys in Japan (501) and Australia (493) achieved a result that was about the same as New Zealand’s boys, while boys in the remaining six top- or high-performing countries or economies all gained a significantly better result: *Shanghai-China (541), Korea (520), *Hong Kong-China (520), Finland (507), Canada (506) and *Singapore (506).
- Only girls from the four top-performing countries or economies − *Shanghai-China (587), Finland (563), *Hong Kong-China (559) and Korea (558) − were better than New Zealand’s girls at reading continuous texts. In three of the four large English-speaking countries − Australia (532), the United States (513) and the United Kingdom (504) – girls’ mean performance was significantly lower than in New Zealand. In Canada (543), Japan (541) and *Singapore (538) it was about the same.
Reading literacy aspects/competencies
Access and retrieve, integrate and interpret, reflect and evaluate
The three reading literacy aspects are reported because they are processes that are integral to proficient reading. They identify how readers engage with a text.
Mean scores by all students on the three reading literacy aspects
- New Zealand students performed very strongly on the reflect and evaluate scale (531). The only 15-yearolds to outperform New Zealand students on this scale were those from three East-Asian participating countries or economies: *Shanghai-China* (557), Korea (542) and *Hong Kong-China (540). Finland (536), Canada (535) and *Singapore (529) achieved an average scale score that was similar to New Zealand’s.
- New Zealand’s 15-year-olds also performed strongly on the access and retrieve (521) and the integrate and interpret (517) scales.
- Students from the four top-performing countries or economies, along with *Singapore, achieved higher scores than New Zealand students on both the access and retrieve and the integrate and interpret scales.
- Canada showed a similar performance to New Zealand on the integrate and interpret and the access and retrieve scales, while Australian students’ mean performance (513) was significantly below New Zealand’s on the latter.
- The average performance on the three literacy aspects for 15-year-olds from the United Kingdom and the United States was significantly lower than in New Zealand. This was also true for the average across OECD countries: integrate and interpret 493, reflect and evaluate 494 and access and retrieve 495.
Mean scores by gender on the three reading literacy aspects
- Although New Zealand boys’ average score on the three reading literacy aspects was significantly lower than that of their female counterparts, this was also the case for all participating countries and economies
- New Zealand girls, on average, performed very strongly on the reflect and evaluate scale (556), a score that was 51 points greater than the boys’ (506).22 Finland (59) and *Shanghai-China (50) were the only other top- or high-performing countries or economies with a gender difference of 50 points or more. The average OECD gender difference was 44 score points, with boys averaging 472 and girls 517.
- Girls in New Zealand also performed strongly on the access and retrieve scale (546), a score that was significantly higher than the mean performance of boys (497). Finland (59) and New Zealand (49) were the only top- or high-performing countries or economies to show a gender difference of more than 40 points on this scale. The average gender difference across all OECD countries was 40 score points, with boys averaging 475 and girls 515.
- Again, on the integrate and interpret scale more than 40 score points separated New Zealand’s boys (497) and girls (539). This was also the case in Finland (with a 50 score points gender difference). The OECD average gender difference was 36 score points, with boys averaging 476 and girls 512.
Although girls in each of the participating countries or economies outperformed boys, New Zealand, along with Finland, showed particularly large gender differences on each of the three reading aspects and the two text formats.
Changes in reading literacy performance
Between 2000 and 2009
As noted earlier, 2009 was the first time reading was reassessed as a main focus of PISA. Because there have been no major changes in the reading literacy framework since PISA 2000, it is possible to measure reading trends from 2000 to 2009, such as whether reading outcomes or gaps between higher- and lower-performing students have changed over the nine-year period. Forty-two of the PISA 2009 participating countries or economies also took part in the first administration of PISA.23 However, four of these countries – Austria, Luxembourg, the Netherlands and the United Kingdom are not reported.24 Twenty-six of the 38 countries or economies reported here are OECD members. Consistent with other results in this report, this section reports the changes in reading performance at the high level.
Mean scores by country comparisons between 2000 and 2009
- Just over half of the 26 OECD countries, including New Zealand, showed no change in mean reading performance. Seven showed an improvement and four a decrease.
- Six of the seven OECD countries that showed an improved reading mean reading performance since PISA 2000 were those with a performance that was below the OECD mean in 2000. Among these countries, 15-year-olds in Poland (+21), Hungary (+14) and Germany (+13) increased their mean reading performance to the OECD mean, with Poland showing the most marked improvement.
- Among the countries with a PISA 2000 reading mean that was above the OECD mean, Korea (+15) was the only country to achieve an improved mean reading performance.
- The reading performance of 15-year-olds in *Peru and Chile was well below the OECD mean in both PISA 2000 and in PISA 2009, but they showed the most marked improvement of the 38 countries or economies in PISA 2009 (+43 and +40, respectively).
- Ireland, Sweden and Australia were the only countries with a performance that was higher than the OECD mean in 2000 to show a decline in performance over the nine years. The largest decrease was found in Ireland (-31), followed by Sweden (-19) and Australia (-13). Ireland and Sweden’s overall performance decreased to the OECD mean in 2009, whereas Australia’s reading mean result remained above the OECD mean.
Proficiency levels by country comparisons between 2000 and 2009
Level 5 or higher (scores above 625 points)
- Over the nine-year period only four of the twenty-six OECD countries and two partner countries achieved a larger proportion of students achieving Level 5 or higher.
- Two of the four top-performing countries or economies, Korea and *Hong Kong-China, had an increase in the proportion of students who achieved the highest levels. Korea had the largest increase (+7%), followed by Japan (+4%), *Hong Kong-China (+3%), Israel (+3%), Chile (+1%) and Brazil (+1%).
- In contrast, nine OECD25 countries and one partner country had a statistically significant decrease between the two PISA administrations (2000−2009), including New Zealand (-3%). Ireland showed the largest decrease (-7%).
- Three other top- or high-performing countries, two of which are also English-speaking countries, showed a decrease of three percent or more: Australia (-5%), Canada (-4%) and Finland (-4%).
- Three other Scandinavian countries − Denmark (-3%), Norway (-3%) and Sweden (-2%) − also had a smaller proportion of students achieving Level 5 or higher in PISA 2009 than in PISA 2000, along with the Czech Republic (-2%) and the partner country Romania (-1%).
Level 1a or below (scores below 407 points)
- Between the two PISA administrations there was no change in the proportion of students not reaching Level 2 in any of the top- or high-performing countries or economies. Japan did show an increase of four percent, but this was not statistically significant.
- Seven OECD countries − Austria (+8%), Ireland (+6%), Czech Republic (+6%), Sweden (+5%), France (+5%), Spain (+3%) and Iceland (+2%) − and the partner country *Thailand (+6%) all showed proportionally more students at or below Level 1a (ie, not achieving Level 2 reading proficiency) in 2009 compared to 2000.
- Eight OECD countries and five partner countries all showed a reduction in the proportion of students not reaching Level 2; Chile (-18%) showed the largest decrease.
- Switzerland (-4%) and Denmark (-3%) were the only countries with a mean reading performance in PISA 2000 that was at the OECD mean to show a reduction in the proportion of students at the lowest levels. Poland (-8%) and *Liechtenstein (-6%) were the only countries with a mean reading performance that was similar to the OECD mean in PISA 2009, but lower than the OECD mean in 2000, to show a decrease in the proportion of students at the lower levels.
- Four OECD countries26 and four partner countries with a mean reading performance lower than the PISA 2000 OECD mean also showed a reduction in the proportion of students at these lower levels.
Mean scores by gender between 2000 and 2009
- There was no change in the performance of New Zealand’s 15-year-old girls or boys in reading literacy between 2000 and 2009. There was also no change in the difference between the means for boys and girls over the nine years.
- No country showed a reduction in the reading gender gap, but some showed an increase. The largest increases in the gender gap were found in Korea (+21), Israel (+27) and *Romania (+29), where the difference doubled in all three. In each of these countries the increased gender gap was a result of girls’ improved reading performance; boys’ reading performance remained about the same.
In summary, New Zealand’s mean reading performance did not change over the nine year period, 2000 to 2009. Korea was the only top-or high-performing country or economy where mean reading performance increased. Six countries’ reading performance declined, including Australia’s. While New Zealand, along with Canada, Finland and Australia had proportionally fewer students at the top proficiency levels than in 2000, *Shanghai-China was the only country with proportionally more students than New Zealand at the top levels in this round of PISA.
Students’ engagement in reading
Reading achievement
Students taking part in the PISA assessment also completed a questionnaire, which included questions that focused on students’ reading habits (reading enjoyment, time spent on reading for enjoyment, reading for school, diversity of reading materials, and online reading activities) and reading strategies. This section provides information on students’ reading enjoyment. Further reporting that covers the other aspects noted above will be available next year.
Reading enjoyment
PISA’s index of ‘reading enjoyment’ was derived from students’ level of agreement with a series of questions on students’ reading activities.
- New Zealand 15-year-olds’ average reading enjoyment (0.13) was greater than the OECD average (0.00).27 Of the nine high-performing countries, Finnish (0.05) and Australian (0.00) students showed the lowest reading enjoyment.
- When comparing New Zealand 15-year-olds’ reading enjoyment with the enjoyment of their peers from the three other large English-speaking countries, New Zealand students were about as positive as Canadian students (0.13), but more positive than those in the United Kingdom (-0.12) and the United States (-0.04). Only students from four East-Asian countries or economies − *Shanghai-China (0.57), *Hong Kong-China (0.32),*Singapore (0.29) and Japan (0.20) − indicated a higher level of reading enjoyment, while Koreans (0.13) showed the same enjoyment level as New Zealand.
- Not surprisingly, enjoying reading was strongly related to students’ reading ability. This was particularly marked in New Zealand and Australia, with the top quarter of New Zealand students high on this index achieving a mean reading performance of 593 score points, while the quarter of students who enjoyed reading the least gained an average score that was 127 score points lower (466).28 Finland (121) was the only other top- or high-performing country or economy with more than 120 score points difference between these two groupings.
- Although New Zealand’s girls (0.44) were much more likely than their male counterparts (-0.17) to enjoy reading, the gender differences in Finland (boys -0.41, girls 0.50) and Canada (boys -0.28, girls 0.55) were larger. In Australia, the United States and *Singapore the differences were about the same as in New Zealand. Korea showed the smallest gender gap on this index, followed by three other East-Asian countries or economies (*Hong Kong-China, *Shanghai-China and Japan), and then the United Kingdom.
Reading activities from which the reading enjoyment index was derived
- Except for Australia (42%), New Zealand students (40%) were as likely as, or slightly more likely than, students from the other top-performing countries to agree that they only read books to get the information they need; close to half of students from the United Kingdom (48%) and the United States (47%) agreed with this statement.
- Just under a third of New Zealand students (31%) reported that they found it hard to finish [reading] books, a proportion that was broadly similar to Australia (33%), Korea (32%), the United States (31%) and the OECD average (33%), but smaller than the United Kingdom (37%) and *Singapore (35%). In *Shanghai-China, *Hong Kong-China, Japan, Canada and Finland, around a quarter of students found it hard to finish books.
- *Shanghai-China (11%), Finland (35%) and *Singapore (35%) were the only other high-performing countries or economies with students who were less likely than New Zealand’s students (38%) to report that they read only when they have to. More than half of the Korean students (55%) agreed with this statement.
- Proportionally fewer New Zealand (18%) students than their peers in the four large English-speaking countries − Canada (22%), the United Kingdom (23%), Australia (26%) and the United States (26%) − thought that reading was a waste of time. Students from *Shanghai-China students were the least likely to agree with this statement (6%).
- Although close to one in five New Zealand students reported that they cannot sit still and read for more than a few minutes (18%), only four of the high-performing countries or economies had a smaller proportion. In Canada, Australia, the United States, the United Kingdom and the average for OECD countries, around a quarter of students indicated that they cannot sit still and read for more than a few minutes.
- Students from two Chinese economies, *Shanghai-China (70%) and *Hong Kong-China (65%), were more likely to report that reading was one of their favourite hobbies than their peers from the six other top-performing countries, including New Zealand (38%). Nevertheless, among the other high-performing countries or economies, only students from *Singapore (54%) and Japan (42%), on average, were more likely than New Zealand students to report reading as a favourite hobby.
- Approximately half of New Zealand’s students liked expressing their opinions about books they had read (49%) and going to the bookstore or library (54%), and around forty percent of them reported that they liked talking about books to other people (43%) and exchanging books with their friends (38%). Students from three non-OECD partner countries or economies − *Shanghai-China, *Hong Kong-China and *Singapore − were all more likely to agree with these statements than New Zealand students.
- Well over half of New Zealand students reported that they liked receiving a book as a present (57%). *Shanghai-China (69%) was the only PISA participating country or economy where students were more likely to report that they liked receiving a book as a present.
- In relation to the four large English-speaking countries and the OECD average, New Zealand students were as likely or more likely to regard reading as a favourite hobby, and to like talking about books to other people and going to a book store or library. On average, proportionally fewer students from the four large English-speaking countries than in New Zealand (a third or less)
- The value and importance of reading and the complexities in mastering the skill are eloquently captured by the OECD: “[s]uccess in reading provides the foundation for achievement in other subject areas and for full participation in adult life. The ability to convey information in written form as well as orally is one of humankind’s greatest assets. The discovery that information can be shared across time and space, without the limits of the strength of one’s voice, the size of a venue and the accuracy of memory, has been fundamental to human progress. And yet, learning how to read and write requires effort because it cannot be achieved without mastering a collection of complex skills. The brain is biologically primed to acquire language, but writing and reading are relatively recent achievements in human history. Becoming a proficient reader is a goal that requires practice and dedication.”29
Footnotes
- The reading performance of New Zealand’s 15-year-olds over the nine-year period will be measured in more detail in a national report that will be published in 2011.
- Complex competency or aspect was assessed in the new PISA 2009 electronic reading assessment (ERA).
- The other two text classifications are used to ensure an adequate coverage of the definition of reading literacy.
- In this report, reading proficiency levels are only provided in the overall reading literacy scale.
- Shanghai-China is a new participant in PISA.
- * Denotes a non-OECD partner country or economy.
- Singapore is also a new participant in PISA.
- Figure 3 also provides, for each country, an estimate of the standard error of the mean. There is a 95% likelihood that a country’s mean score will lie within 1.96 standard errors of the mean.
- See the Appendix for New Zealand’s range of ranking on the mathematical and scientific literacy areas, and comparisons with other top-performing countries on the three literacy areas.
- Because the results are rounded to the nearest whole number, this difference appears inconsistent.
- As noted earlier, Level 5 or higher was the highest reading proficiency level in earlier PISA administrations. The term ‘top reading performers’ has been retained by the OECD to refer to students who are proficient in reading at Level 5 or higher.
- The reporting of total ethnicity data is consistent with the Statistics New Zealand standard, but differs from the prioritised classification method used in previous PISA reporting (2000, 2003 and 2006) and many other Research Division reports. As part of this transition, mean scores by ethnic grouping have also been analysed using the prioritisation classification method: Māori (478), Pasifika (446), Asian (526), and Pākehā-European (546).
- The proportions of Asian, Māori and Pasifika students achieving at Level 6 were less than 3%. With fewer than 30 students in each grouping at this level in the PISA sample, there were too few students for reliable comparisons to be made.
- 20 A large standard error for Japan’s 5th percentile score means that the difference between Japan and New Zealand is not statistically significant.
- Because the results are rounded to the nearest whole number, this difference appears inconsistent.
- PISA 2000 here includes the countries that participated in PISA 2000 and PISA 2000 plus (PISA 2000 plus was administered in 2001).
- The Netherlands, United Kingdom (response rate issues) and Luxembourg (linguistic issues) did not meet the PISA established technical standards for the quality of data sets, and their results have been treated as missing. For further detail, see PISA 2009 Technical Report (forthcoming).
- The comparability of Austria’s 2009 data with the data from previous cycles cannot be ensured. This is because of education disputes in Austria at the time of the PISA 2009 assessment.
- The United States also showed a 2% decrease, but the difference between 2000 and 2009 was not statistically significant due to a larger standard error (1.6).
- The new OECD member country, Israel, showed a decrease of 7%, but this decrease was not statistically significant due to the relatively large standard error (3.4).
- The OECD mean index was set at zero. If a country has a negative index value this is because the students responded less positively to the underlying questions than students, on average, across the OECD. Therefore a negative mean index does not necessarily imply that students responded negatively to the underlying questions. OECD (2010). Learning to Learn: Student Engagement, Strategies and Practices. Vol. 3. Paris: OECD.
- For Australia there was a difference of 134 score points from the bottom quarter (454) to the top quarter (588).
- Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science, Vol 1, p.18. OECD: Paris.
Navigation
Where to find out more
Contact Us
Education Data Requests
If you have any questions about education data then please contact us at:
Email:
Requests EDK
Phone:
+64 4 463 8065