PISA 2009: Our 21st century learners at age 15 Publications
Publication Details
This report provides a high level picture of New Zealand’s 15-year-old performance in reading literacy (main focus), mathematical literacy and scientific literacy. It compares New Zealand’s results with other top- and high-performing countries. In July and August 2009 4,643 New Zealand 15 year-old students from 163 New Zealand schools took part in PISA 2009.
Author(s): Maree Telford with Steve May [Ministry of Education]
Date Published: December 2010
Mathematical literacy (minor focus)
What aspects of mathematical literacy does PISA measure and report on?
PISA measures30 student performance on four mathematical knowledge domains (clusters of relevant mathematical areas and concepts): quantity (related to number), change and relationships (related to algebra), space and shape (related to geometry) and uncertainty (related to statistics). The competencies reproduction (simple mathematical operations), connections (bringing together ideas to solve straightforward problems) and reflection (wider mathematical thinking) are also assessed. Because mathematical literacy is a minor domain of PISA 2009, results are only reported on students’ mean performance on the mathematical literacy scale31 and mathematical proficiency levels.
The mathematical literacy framework was established in PISA 2003 when mathematics was the main focus of PISA.32 The results from this 2009 administration can be compared with those from PISA 2006 and PISA 2003, but not with PISA 2000 when only two content areas were assessed (change and relationships and uncertainty).
Students’ mathematical results are compared to the mean performance across the average for the 34 OECD countries, as well as to countries with a student mean performance that was better than, or not statistically different to, that of New Zealand students.
Figure 5: What the mathematical literacy proficiency measures
Source: OECD. (2004). PISA 2003: Learning for Tomorrow’s World – First Results from PISA 2003. Paris: OECD.
Student performance in mathematics
Mathematical literacy scale
To provide an ongoing high-level picture of student performance in mathematical literacy, students’ results on the tasks that cover the three dimensions (knowledge domains, competencies and context or situation) were summarised on an overall mathematical literacy scale.
The mean mathematical literacy performance of the 65 participating countries or economies and the percentage of students on each of the proficiency levels are shown in Figure 6.
Mean scores by country comparisons on the mathematical literacy scale
New Zealand’s 15-year-olds performed strongly in mathematical literacy, achieving a mean score of 519 points. This was better than the OECD average (496).
- Fifteen-year-olds in five OECD countries − Korea (546), Finland (541), Switzerland (534), Japan (529) and Canada (527) − and in six non-OECD partner countries or economies − *Shanghai-China (600), *Singapore33 (562), *Hong Kong-China (555), *Chinese Taipei (543), *Liechtenstein (536) and *Macao-China (525) − achieved higher mean scores than New Zealand’s students.
- The mean performance of New Zealand’s students was similar to four OECD countries: the Netherlands (526), Belgium (515), Australia (514) and Germany (513).
- New Zealand students outperformed their peers in 49 of the 64 other participating countries or economies, including the United Kingdom (492), the United States (487) and 22 of the other 33 OECD member countries.
Proficiency levels (six levels) by all students on the mathematical literacy scale
Mathematical proficiency is reported on a continuum spanning six levels. A full description of the types of tasks that students reaching a particular level in mathematical literacy can typically do is shown in Figure 5: What the mathematical literacy proficiency measures.
Level 6 (scores above 668 points) and Level 5 (scores from 607 to 668 points)
Level 6 students demonstrated that they are capable of advanced mathematical thinking and reasoning. For example, students at this level can model complex problem situations and have a mastery of symbolic and formal mathematical operations.
Students achieving Level 5 or higher are also described by PISA as top-achieving students because they demonstrated that they are capable of developing and working with models in complex situations.
- *Shanghai-China had the largest proportion of students at the highest proficiency level, with more than a quarter (27%) of students achieving Level 6. At least 10 percent of the students from *Singapore (16%), *Chinese Taipei (11%) and *Hong Kong-China (11%) performed at this level, and in Korea and Switzerland eight percent did so.
Figure 6: Mathematical literacy proficiency levels
Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science, Vol 1. Paris: OECD.
- In New Zealand, five percent of our 15-yearolds achieved the highest proficiency level. This proportion was similar to Japan (6%), Belgium (6%), *Liechtenstein (5%), Finland (5%), Germany (5%), Australia (4%), the Netherlands (4%), Canada (4%) and *Macao-China (4%). A smaller percentage of students were at this level in the United Kingdom (2%) and the United States (2%). The average for the 34 OECD countries was three percent.
- In *Shanghai-China (50%), half of their 15-year-olds were at Level 5 or higher, and at least a quarter of students were at these levels in *Singapore (36%), *Hong Kong-China (31%), *Chinese Taipei (29%), Korea (26%) and Switzerland (24%).
- In the remaining top- or high-performing countries, including New Zealand (19%), around twenty percent were top performers (ie, at Level 5 or 6 mathematical proficiency): Finland (22%), Japan (21%), Belgium (20%), the Netherlands (20%), Canada (18%), *Liechtenstein (18%), Germany (18%), *Macao-China (17%) and Australia (16%).
- Proportionally fewer students were at Level 5 or higher in the United Kingdom (10%) and the United States (10%) than in New Zealand. The average across OECD countries was 13 percent.
Level 4 or higher (scores above 545 points)
Students achieving Level 4 were required to demonstrate that they can also work effectively with complex mathematical models, but only in explicit and concrete situations.
- At Level 4 or higher, 41 percent of New Zealand students were proficient in mathematical literacy. This proportion was similar to Japan (44%), the Netherlands (44%), Canada (43%), *Macao-China (42%), Belgium (42%), Germany (40%) and Australia (38%).
- In *Shanghai-China, close to three-quarters of students (71%) were at these levels, and in *Singapore (58%), *Hong Kong-China (56%), Korea (52%), *Chinese Taipei (51%), Finland (49%), *Liechtenstein (49%) and Switzerland (48%) there were around half or more.
- The proportion of 15-year-olds in the United Kingdom (28%) and the United States (27%) performing at Level 4 or above was substantially smaller than in New Zealand. This was also the case for the average of the 34 OECD countries (32%).
Level 1 (scores from 358 to 419 points) or below (scores below 358 points)
Students showing proficiency at Level 1 demonstrated that they can typically answer questions involving familiar mathematical contexts and perform actions that are obvious and follow immediately from the given stimuli.
- Fewer than 10 percent were at Level 1 or below in *Shanghai-China (5%), Finland (8%), Korea (8%), *Hong Kong-China (9%) and *Liechtenstein (9%).
- In seven of the other top- or high-performing countries or economies, between 10 percent and 13 percent of their students were at Level 1 or below: *Singapore (10%), Canada (11%), *Macao-China (11%), Japan (12%), *Chinese Taipei (13%), the Netherlands (13%) and Switzerland (13%).
- Fifteen percent of New Zealand students were at the lowest mathematical literacy levels, with 10 percent at Level 1 and five percent below Level 1. These proportions are similar to Australia (11% at Level 1, 5% below Level 1).
- Larger proportions were at Level 1 or below in Germany (19%), Belgium (19%), the United Kingdom (20%) and the United States (23%). The average for the OECD countries was 22 percent.
At least half of the top-performing countries or economies are East Asian, with Shanghai-China again outperforming the other 64 PISA participants.
Although New Zealand’s 15-year-olds continued to show a strong performance in mathematical literacy, as measured by PISA, New Zealand had a smaller percentage of students performing at Level 6 than six other top-or high-performing countries or economies. Nevertheless, the proportion of New Zealand students performing at the highest proficiency level was similar to the nine other top-or high-performing countries or economies.
At the other end of the scale, the proportion of New Zealand students who were not proficient in mathematics at the baseline (Level 2) was similar to Australia and smaller than the OECD average, the United Kingdom and the United States. However, the majority of the other top-or high-performing countries or economies had proportionally fewer students at the low levels.
Mean scores by gender on the mathematical literacy scale
- Although New Zealand’s boys (523) achieved a higher mean score than its girls (515), there was no statistically significant difference between the scores. In more than half of the participating countries or economies (35), boys outperformed girls, including Australia (boys 519, girls 509). The average across all OECD countries was boys 501, girls 490.
- In 10 of the 15 other top- or high-performing countries or economies, boys outperformed girls in mathematical literacy. In the other five there were no gender differences.
- Across all of the countries or economies participating in PISA 2009, a significant difference in favour of girls was found in five countries, all of which are non-OECD partner countries.
Proficiency levels by gender on the mathematical literacy scale
Level 6 (scores above 668 points) and Level 5 (scores from 607 to 668 points)
- *Shanghai-China (27%) and *Singapore (17%) had the largest proportions of boys performing at Level 6; *Hong Kong-China (13%) and *Chinese Taipei (13%) achieved more than twelve percent of boys at this level. These four East-Asian non-OECD partner countries or economies, along with Korea (boys 9%, girls 7%), achieved a substantially larger proportion of girls, compared to other countries, at Level 6 (*Shanghai-China 26%, *Singapore 14%, *Hong Kong-China 9% and *Chinese Taipei 10%).
- The proportion of New Zealand boys (7%) and girls (4%) performing at Level 6 was similar to the remaining top- or high-performing countries; examples are Japan (boys 8%, girls 5%), Finland (boys 6%, girls 4%) and Canada (boys 6%, girls 3%).
- Half of the boys and half of the girls in *Shanghai-China (boys 50%, girls 51%) were among the top performers who achieved at Level 5 or higher.
- In New Zealand, the proportion of boys (21%) and girls (16%) at Level 5 or higher was smaller than in *Singapore (boys 37%, girls 34%), *Hong Kong-China (boys 34%, girls 27%), *Chinese Taipei (boys 31%, girls 26%), Korea (boys 28%, girls 23%) and Switzerland (boys 28%, girls 20%).
Level 4 or higher (scores above 545 points)
- The proportion of New Zealand’s boys (43%) at Level 4 or higher (the combination of Levels 4, 5 and 6) was larger than that found for girls (39%). This was similar to seven other top- or high-performing countries, including Japan (boys 46%, girls 40%), Canada (boys 46%, girls 40%) and Australia (boys 41%, girls 36%).
- The proportions of New Zealand boys and girls at these levels were smaller than in seven of the top-or high-performing countries. *Shanghai-China (boys 70%, girls 72%) had the largest proportion of both boys and girls achieving at Level 4 or higher. *Singapore (boys 60%, girls 57%), *Hong Kong-China (boys 59%, girls 53%) and Korea (boys 53%, girls 51%) all had at least half of their boys and half of their girls at Level 4 or higher.
Level 1 (scores from 358 to 420 points) or below (scores less than 358 points)
- A similar proportion of New Zealand’s 15-year-old girls (15%) and boys (16%) were proficient at the lowest levels of mathematical literacy, with around five percent of these students from both gender groupings not achieving Level 1.
- The proportion of New Zealand boys at this level was similar to five other top-or high-performing countries, including Australia (15%), but was greater than in the remaining 10 ie remaining 10 top-or high-performing countries or economies, such as *Shanghai-China (5%) and Finland (8%).
- Among the top- or high-performing countries or economies, Germany (20%) and Belgium (21%) had higher proportions of girls at Level 1 or below compared to New Zealand. Australia (16%) and three other top- or high-performing countries had similar proportions of girls at this level, while the remaining top- or high-performing countries or economies had smaller proportions; for example, *Shanghai-China (4%) and Korea (7%).
Although there was no statistical difference between the mathematical performance of New Zealand’s boys and girls, proportionally more boys than girls showed an advanced level of mathematical proficiency. In PISA 2006 there was a small but significant difference in favour of boys (11 points).
Mean scores by ethnic group/s on the mathematical literacy scale34
As noted earlier, students taking part in PISA were asked to provide information about the ethnic group/s they belonged to. Students who belonged to more than one group were counted in each group they identified.
- Students who identified as belonging to Pākehā/ European (537) or Asian ethnic groups (529) performed strongly in mathematical literacy, achieving a score above the OECD average.
- The mean mathematical literacy score for students who identified as Māori (476) or Pasifika (446) was lower than
- the OECD mean (496).
Proficiency level by ethnic groups on the mathematical literacy scale
Level 6 (scores above 668 points) and Level 5 (scores from 607 to 668 points)
- Seven percent of Asian students and six percent of Pākehā/European students were proficient at the highest level of mathematical literacy, Level 6. A small proportion of Māori (2%) and Pasifika (1%) students were also successful at this level.
- Over 20 percent of Asian (23%) and Pākehā/ European (22%) students were proficient at Level 5 or higher. Eight percent of Māori and five percent of Pasifika students belonging to these groups reached this level of performance.
Level 4 or higher (scores above 545)
- Close to half of Pākehā/European (48%) students reached Level 4 or higher; the proportion of Asian students (44%) was slightly less. Proportionally fewer Māori (22%) and Pasifika (14%) students reached Level 4 or higher than the average for the OECD (32%).
Level 1 (scores from 358 to 420 points) or below (scores less than 358 points)
- Ten percent of Pākehā/European and 14 percent of Asian students were at Level 1 or lower. Twenty-seven percent of Māori students and 40 percent of Pasifika students were at these levels.
Changes in mathematical literacy performance
Between 2003 and 2009
Because the measure for mathematics has remained consistent since PISA 2003, it is possible to look at changes in 15-year-olds’ performance in mathematical literacy since 2003. Forty countries or economies participated in both PISA 2003 and 2009 administrations, and 28 of these were OECD member countries.35
Mean scores by country comparisons between 2003 and 2009
- There was no statistically significant change in New Zealand’s 15-year-olds’ mathematical literacy performance between 2003 and 2009.
- More than half (24 countries) of the 39 countries or economies that have participated in both PISA 2003 and 2009 did not show a statistically significant change in mathematical literacy over the six-year period. This includes the OECD and non-OECD partner countries or economies that performed better than New Zealand in PISA 2009 (Finland, Canada, Japan, Korea and Switzerland; and the non-OECD partner country and economies, *Hong Kong-China, *Liechtenstein and *Macao-China). There was also no change for the United States.
- The mean mathematical performance of 10 countries, including three PISA 2003 top-or high-performing countries Belgium (-14), the Netherlands (-12) and Australia (-10), showed a statistically significant decrease over the six-year period.
- In contrast, Germany’s 15-year-olds’ mean mathematical performance improved by 10 points. Seven other countries also showed a large improvement, although the mean performance of these countries in 2003 was far below the OECD average, and was still below the average six years later.
Proficiency levels by country comparisons between 2003 and 2009
Level 5 or higher (scores above 607 points)
- There was no change between 2003 and 2009 in the proportion of New Zealand students who were proficient in mathematics at Level 5 or higher.
- Portugal (+4%), Italy (+2%) and Greece (+2%) were the only countries to show a statistically significant increase in the proportion of students achieving the highest levels.
- Ten countries showed a statistically significant reduction since 2003 in the proportion of their top performers, with the Czech Republic (-7%), Belgium (-6%), the Netherlands (-6%) and Ireland (-5%) showing a reduction of five percent or more at these levels. Sweden (-4%), Denmark (-4%), Australia (-3%), Canada (-2%) and Iceland (-2%), and the non-OECD partner country *Latvia (-2%), all showed a smaller proportion of top performers. The average across the 29 OECD countries also showed a small, but statistically significant, reduction (-1%).
Level 1 or lower (scores below 420 points)
- At the lower end of the scale there was no change in the proportion of New Zealand students who did not achieve proficiency above Level 1. This was the case for all top- or high-performing countries or economies except for Belgium, where a larger proportion of students were at these lower levels in this administration of PISA than in PISA 2003.
- France (+6%), the Czech Republic (+6%), Austria (+4%), Ireland (+4%), Sweden (+4%) and Belgium (+3%) had proportionally more students at these lower levels than in 2003. Small increases in the proportion of students at the lower levels were also found in Luxembourg (+2%) and Iceland (+2%).
- Seven countries, all with a mean performance lower than the OECD mean in PISA 2003 and 2009, showed a reduction in the proportion of students who were not proficient in mathematics above Level 1. Mexico (-15%) and Turkey (-15%) showed a reduction of 10 percent or more, and Greece (-9%), Italy (-7%), Portugal (-6%), *Brazil (-6%) and *Tunisia (-4%) all had proportionally fewer students at the lower levels.
In summary, New Zealand’s 15-year-old mathematical mean performance did not change over the six years. Germany was the only country that performed above the 2003 PISA OECD mean where performance improved. Conversely, three high performing countries mathematical literacy performance decreased.
Footnotes
- The PISA mathematical literacy framework has three dimensions: knowledge domains, competencies involved (all noted above), and situations or context (mathematics for personal, educational, occupational, scientific and public use).
- The mathematical literacy scale was previously known as the combined mathematical literacy scale.
- Although mathematical literacy was assessed in PISA 2000, the mathematical literacy framework was expanded for the PISA 2003 administration.
- *Shanghai-China and *Singapore are new PISA participants.
- As noted earlier, the reporting of total ethnicity data is consistent with the Statistics New Zealand standard, but differs from the prioritised classification method used in previous PISA reporting (2000, 2003 and 2006) and many other Research Division reports. As part of this transition, mean scores by ethnic group have also been analysed using the prioritisation classification method: Māori (476), Pasifika (442), Asian (533) and Pākehā-European (542).
- Because the United Kingdom’s PISA 2003 sample did not meet the PISA established technical standards for the quality of data sets, their results have been treated as missing. For further detail see, OECD (2005). PISA 2003 Technical Report. OECD: Paris.
As for reading, the comparability of Austria’s 2009 data with the data from previous cycles cannot be ensured. This is because of education disputes in Austria at the time of the PISA 2009 assessment.
Navigation
Where to find out more
Contact Us
Education Data Requests
If you have any questions about education data then please contact us at:
Email:
Requests Data and Insights
Phone:
+64 4 463 8065