PISA 2009: Our 21st century learners at age 15 Publications
Publication Details
This report provides a high level picture of New Zealand’s 15-year-old performance in reading literacy (main focus), mathematical literacy and scientific literacy. It compares New Zealand’s results with other top- and high-performing countries. In July and August 2009 4,643 New Zealand 15 year-old students from 163 New Zealand schools took part in PISA 2009.
Author(s): Maree Telford with Steve May [Ministry of Education]
Date Published: December 2010
Scientific literacy (minor focus)
What aspects of scientific literacy does PISA 2009 measure and report on?
PISA measures student performance in scientific literacy in three scientific competencies – identifying scientific issues, explaining phenomena scientifically and using scientific evidence – and two scientific knowledge areas – knowledge of science and knowledge about science.36 Because scientific literacy is a minor domain in this round of PISA, results are reported only on students’ mean performance on the scientific literacy scale37 and scientific proficiency levels.
Scientific literacy was assessed in PISA 2000 and 2003 as a minor domain. In 2006, when scientific literacy was the main focus of PISA, the framework was further developed and established as a basis for measuring trends. Results from this PISA administration are therefore comparable with PISA 2006 results only.
Student results are compared to the mean performances across the average for 34 OECD countries, as well as the countries with a student scientific literacy mean performance that was either better than, or not statistically not different to, that of New Zealand students.
Figure 7: What the scientific literacy proficiency levels measure
Source: OECD. (2007). PISA 2006: Scientific Competencies for Tomorrow's World. Vol 1. Paris: OECD.
Student performance in science
Scientific literacy scale
Student performance on the various aspects of scientific literacy in PISA 2009 is summarised on the scientific literacy scale to provide an ongoing high-level picture of student performance.
The mean scientific literacy performance of the 65 participating countries or economies and the percentage of students on each of the proficiency levels are shown in Figure 8.
Mean scores by country comparisons on the scientific literacy scale
Overall, New Zealand’s 15-year-old students performed very strongly in scientific literacy, scoring 532 points. This was substantially better than the OECD average (501).
- Of the 65 countries participating in PISA 2009, only one OECD country, Finland (554), and three non-OECD partner countries or economies, *Shanghai-China38 (575), *Hong Kong-China (549) and *Singapore (542), achieved a significantly better mean performance than New Zealand.
- New Zealand students’ mean performance was not statistically different from that of six other OECD countries: Japan (539), Korea (538), Canada (529), Estonia (528), Australia (527) and the Netherlands (522).
- On average, New Zealand’s 15-year-olds achieved a statistically better performance than 54 of the other countries participating in PISA, including the United Kingdom (514), the United States (502), and 27 of the 33 other OECD countries.
Proficiency levels (six levels) by all students on the scientific literacy scale
Scientific proficiency is also reported on a continuum spanning six levels. A full description of the types of tasks that students reaching a particular level in scientific literacy can typically do is shown in Figure 7: What the scientific literacy proficiency measures.
Level 6 (scores above 708 points) and Level 5 (scores from 633 to 708 points)
Students at the highest scientific proficiency level, Level 6, clearly demonstrated that they have advanced scientific thinking and reasoning. For example, these students showed that they can use their scientific knowledge to provide solutions to unfamiliar scientific situations.
Students achieving at Level 5 are also described as top performers because they were required, for example, to demonstrate that they have the ability to identify the scientific components of many complex life situations.
- *Singapore achieved the largest proportion (5%) of students achieving at the highest level of scientific proficiency (Level 6), followed by *Shanghai-China, New Zealand (both with 4%), Finland, Australia and Japan (all with 3%).
Figure 8: Scientific literacy proficiency levels
Source: OECD. (2010). PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science. Vol 1, Paris: OECD.
- In the remaining top-or high-performing countries or economies, two percent of students achieved Level 6 in *Hong Kong-China and Canada, and one percent did so in the Netherlands, Estonia and Korea.
- Two non-OECD partner participants, *Shanghai-China (24%) and *Singapore (20%), along with Finland (19%) and New Zealand (18%), had at least twice as many students at Level 5 or higher than the average for the 34 OECD countries (9%).
- The other eight high-performing countries achieved 10 percent or more students at this level (Australia 15%, Canada 12%).
Level 4 or higher (scores above 559 points)
Students achieving Level 4 also showed a strong level of proficiency. For example, these students showed that they are capable of linking scientific knowledge appropriately and bringing critical insights to situations.
- Although 60 percent of students from *Shanghai-China achieved at least Level 4, there were over 40 percent of students at these levels in six of the eleven top-or high-performing countries or economies, including New Zealand (43%), Finland (50%), *Hong Kong-China (49%), Japan (46%), *Singapore (46%) and Korea (42%). In Australia (39%), Canada (38%), the Netherlands (38%), Estonia (36%), the United Kingdom (34%) and the United States (29%) there were fewer than 40 percent of students at these levels. The average for the OECD countries was 29 percent.
Level 1 (scores from 335 to 409 points) or below (scores less than 335 points)
Students at Level 1 have limited scientific knowledge. However, they demonstrated that they are capable of presenting scientific explanations that are obvious.
- Four percent of New Zealand’s 15-year-olds did not show scientific literacy at Level 1 (ie, they were below Level 1). Three other top-or high-performing countries − Australia (3%), Japan (3%) and the Netherlands (3%) − had a similar proportion of students as New Zealand not reaching Level 1.
- When compared with the OECD average (5%), New Zealand had a similar proportion below Level 1. In the remaining five top-or high-performing countries or economies, with the exception of Canada (2%) and *Singapore (3%), only one percent or fewer 15-yearolds did not achieve at least Level 1.
- Thirteen percent of New Zealand’s students did not show scientific proficiency at the baseline level (Level 2). Canada (10%), Japan (11%), the Netherlands (13%), Australia (13%), and *Singapore (11%) all had 10 percent or more students at this level.
- In *Shanghai-China (3%), Finland (6%), Korea (6%), *Hong Kong-China (7%) and Estonia (8%) a relatively small proportion were at Level 1 or below.
- Across the OECD countries and in the United States, 18 percent of students did not achieve above Level 1, and in the United Kingdom 15 percent did not achieve at this level.
Finland’s 15-year-olds were the only OECD students to outperform New Zealand’s students on average in scientific literacy. Nevertheless, *Shanghai-China’s 15-year-olds clearly stand out from every participating country or economy, with a mean performance that was at least 20 points higher than any other participating country or economy.
As found in the last administration of PISA, close to one in five of New Zealand students are doing exceptionally well in science (they are capable of advanced scientific thinking and reasoning).
The proportion of New Zealand students with low proficiency was similar to that in some other high-performing countries. However, as was also found in PISA 2006, when considering the proportion of New Zealand students achieving at the high levels of scientific proficiency, the number not achieving beyond Level 1 is disproportionately large.
Mean scores by gender on the scientific literacy scale
- Although New Zealand’s girls (535) showed a six-point advantage over boys (529), this difference was not statistically significant. The average difference between girls and boys across the 34 OECD member countries was also not statistically different (girls 499, boys 501).
- New Zealand’s boys achieved a similar mean science result as their peers in the six other high-performing countries, although boys from the four top-performing countries or economies achieved a better result.
- New Zealand’s girls achieved a significantly better result than girls from two of the high-performing countries, Canada (526) and the Netherlands (520), and, as found for boys, girls from the four topperforming countries or economies outperformed New Zealand’s girls.
- Eight of the thirty-four OECD countries showed an advantage for boys in science, including the United States (14 points) and the United Kingdom (9 points). One top-and one high-performing OECD country, Finland (15 points) and Japan39 (12 points), showed more than 10 score points in favour of girls. A further four OECD countries also showed an advantage for girls (Slovenia, Turkey, Greece and Poland).
Proficiency levels (six levels) by gender on the scientific literacy scale
Level 6 (scores above 708 points) and Level 5 (scores from 633 to 708 points)
- A slightly larger proportion of New Zealand’s 15-year-old boys (4%) achieved at the highest level (Level 6) than their female counterparts (3%).
- New Zealand, along with *Singapore (5%), *Shanghai-China (5%) and Australia (4%), had the largest proportion of boys achieving at the highest scientific literacy proficiency level.
- Three percent of New Zealand girls achieved Level 6. In *Singapore (4%), Finland (3%), *Shanghai-China (3%), Australia (3%) and Japan (2%), the proportions were similar to New Zealand’s. However, *Hong Kong-China (2%), Estonia (1%), the Netherlands (1%) and Canada (1%) had a smaller proportion than New Zealand at the highest level.
- Close to one in five of New Zealand’s boys (19%) were successful at Level 5 or higher; the proportion of girls (16%) was slightly smaller than for boys, but not statistically so.
- *Shanghai-China (23%), Finland (20%) and *Singapore (19%) were the only other countries or economies to have proportionally more girls than New Zealand at Level 5 or higher. Only the partner economy *Shanghai-China (26%) had a larger proportion of boys at these levels.
Level 4 or higher (scores above 559 points)
- There was no difference in the proportion of New Zealand’s 15-year-old boys (43%) and girls (43%) proficient in science at Level 4 or higher.
- Only *Shanghai-China (60%) and *Hong Kong-China (50%) had proportionally more boys achieving at Level 4 or higher than New Zealand (43%).
- *Shanghai-China (61%) and *Hong Kong-China (48%) also had a larger proportion of girls than New Zealand at Level 4 or higher, along with Japan (48%) and Finland (46%). However, in Australia (38%) and Canada (37%), there were smaller proportions of girls at these levels; this was not the case for boys.
- Smaller proportions of boys and girls were at Level 4 or higher across the 34 OECD countries (boys 30%, girls 28%) than in New Zealand. This was also the case in the United Kingdom (boys 36%, girls 31%) and the United States (boys 33%, girls 26%).
- Among the top and high-performing countries or economies, Canada was the only country to show a larger proportion of boys (40%) than girls (37%) at these levels. Conversely, Finland was the only top-or high-performing country or economy to show a larger proportion of girls (54%) than boys (46%) at these levels.
Level 1 (scores from 335 to 409 points) or below (scores less than 335 points)
- A larger proportion of New Zealand’s boys (16%) than girls (11%) did not show proficiency at Level 1 or below. Five percent of these boys and three percent of these girls did not show proficiency at Level 1.
- In four of the 10 other top- or high-performing countries or economies there were also 10 percent or more boys or girls who did not succeed above Level 1: Australia (boys 14%, girls 11%), the Netherlands (boys 14%, girls 12%), Japan (boys, 13%, girls 8%) and *Singapore (boys 10%, girls 12%).
- On average across the OECD countries, 19 percent of boys and 17 percent of girls were at Level 1 or below, and five percent of these girls and five percent of these boys did not achieve Level 1.
- Overall, a larger proportion of New Zealand boys than girls were at the lower scientific proficiency levels. However, at the high end of the proficiency continuum, boys were slightly more likely to display exceptional scientific knowledge and skills; that is, top performers who achieved at either Level 5 or Level 6.
Mean scores by ethnic group/s on the scientific literacy scale40
- Students identifying as Pākehā/European (mean score 555) had, on average, a very strong performance in scientific literacy. Students identifying as Asian (530) also performed strongly, achieving a mean score significantly above the OECD average.
- The mean performance of Māori (487) or Pasifika (448) students was lower than the OECD mean (501).
Proficiency levels by ethnic group/s on the scientific literacy scales
Level 6 (scores above 708 points) and Level 5 (scores from 633 to 708 points)
- Fewer than five percent of students from any ethnic group achieved Level 6, the highest scientific proficiency level: Pākehā/European 4 percent, Asian 4 percent, Māori 2 percent and Pasifika 1 percent.
- At least one in five Pākehā/European students (21%) successfully completed tasks at Level 5 or higher. A broadly similar proportion of Asian students (17%) were at this level. The proportions of Māori (8%) and Pasifika (4%) students achieving at these levels were smaller.
Level 4 or higher (scores above 559 points)
- Half of Pākehā/European (51%) students were proficient at Level 4 or higher. Forty-one percent of Asian students, 24 percent of Māori students and 15 percent of Pasifika students also achieved at Level 4 or higher.
Level 1 (scores from 335 to 409 points) or below (scores less than 335 points)
- Fewer than 10 percent of Pākehā/European students (7%) were at Level 1 or lower. The proportions of students identifying as Asian (14%) and Māori (22%) not reaching these levels were larger. Thirty-eight percent of Pasifika students did not demonstrate proficiency beyond Level 1.
- Only a very small proportion of Pākehā/European students (2%) and Asian students (3%) did not show proficiency at Level 1. Six percent of Māori and 13 percent of Pasifika students did not reach this level.
Changes in scientific literacy performance
Between 2006 and 2009
Because science was the main focus of PISA 2006, it is possible to look at changes in 15-yearolds’ scientific literacy performance since 2006. Fifty-seven countries or economies participated in both the PISA 2006 and 2009 administrations, including the 34 OECD member countries.41
Mean scores by country comparisons between 2006 and 2009
- New Zealand’s 15-year-olds’ performance in scientific literacy showed no significant change between 2006 and 2009.
- Over the three-year period, 11 countries or economies showed an improvement and five a decrease in 15-year-olds mean scientific literacy performance.
- Among the six countries that showed an improvement, only two were those with mean scores in PISA 2006 that were either at or above the OECD mean: Korea (+16 points) and Poland (+16 points).
- Four of the five countries or economies that showed a decrease in average performance – Finland (-9), *Chinese Taipei (-12), (both PISA 2006 top- or high-performing countries), the Czech Republic (-12), and Slovenia (-7) − were those with a science mean score in PISA 2006 that was either at or above the PISA 2006 OECD mean. Montenegro (-11) also showed a decrease.
Proficiency levels by country comparisons between 2006 and 2009
Level 5 or higher (scores above 632 points)
- There was no change between 2006 and 2009 in the proportion of New Zealand students who were proficient in scientific literacy at Level 5 or higher.
- Only two countries, Italy (+1%) and *Qatar (+1%), both with a PISA 2006 mean performance that was lower than the OECD average, showed a significant increase in the proportion of students achieving at the highest levels.42
- *Chinese Taipei (-6%), a high-performing country in PISA 2006, showed the largest decrease in the proportion of students reaching Level 5 or higher, followed by the Czech Republic (-3%), Slovenia (-3%), the United Kingdom (-2%), Canada (-2%) and Austria (-2%).43
Level 1 or lower (scores below 335 points)
- There was no change over the three-year period in the proportion of New Zealand students who were at Level 1 or below in scientific literacy.
- Sweden (+3%), Montenegro (+3%) and Finland (+2%) were the only participating countries or economies that showed significant larger proportions of students with a Level 1 or below scientific literacy proficiency in PISA 2006, compared to PISA 2009.44
- Korea (-5%) was the only country with a mean scientific literacy performance that was above the PISA 2006 OECD mean to show a reduction in the proportion of students at these levels in PISA 2009, compared to PISA 2006.
- Fifteen other countries also achieved a reduction in the proportion of students Level 1 or below, including the United States (-6%). The only countries to show a reduction of more than 10 percent of students at these levels were Turkey (-17%) and *Qatar (-14%).
In summary, New Zealand’s 15-year-old scientific literacy mean performance did not change over the three years. Only one of the top- or high-performing countries’ or economies performance improved, while two top- or high-performing showed a decrease.
Footnotes
- The PISA scientific literacy framework has three dimensions: knowledge domains, competencies involved (all noted above) and situations (personal, social and global).
- The scientific literacy scale was previously known as the combined scientific literacy scale.
- As noted earlier, both *Shanghai-China and *Singapore are new PISA participants.
- The gender difference in Japan was not significant due to the relatively large standard error.
- As noted earlier, the reporting of total ethnicity data is consistent with the Statistics New Zealand standard, but differs from the prioritised classification method used in previous PISA reporting (2000, 2003 and 2006) and many other Research Division reports. As part of this transition, mean scores by ethnic group have also been analysed using the prioritisation classification method: Māori (487), Pasifika (443), Asian (534) and Pākehā-European (560).
- As for reading and mathematics, the comparability of Austria’s 2009 data with the data from previous cycles cannot be ensured. This is because of education disputes in Austria at the time of the PISA 2009 assessment.
- Japan (2%) and Korea (1%) had larger increases than Italy and Qatar, but the increases were not statistically significant.
- Finland and *Liechtenstein also showed a 2% decrease, but the decrease was not statistically significant. Chile (1%) showed a small, but statistically significant, decrease.
- Indonesia also showed a 4% increase, but the increase was not statistically significant.
Navigation
Where to find out more
Contact Us
Education Data Requests
If you have any questions about education data then please contact us at:
Email:
Requests EDK
Phone:
+64 4 463 8065