PISA 2006: Mathematical Literacy - How ready are our 15-year-olds for tomorrow's world?
This report describes New Zealand’s results for mathematical literacy in the Programme for International Student Assessment (PISA) 2006, which covers 57 countries. It expands on information already released in international and national reports in December 2007. In 2006, mathematical literacy was a minor focus in PISA. This report also includes information on New Zealand results from 2003.
Author(s): Robyn Caygill, Nicola Marshall & Steve May [Ministry of Education]
Date Published: September 2008
|Figure 5: Achievement ||Figure 8: Ethnicity Grouping ||Figure 11: Language Proficiency Levels|
|Figure 6: Distributions||Figure 9: Performance||Figure 12: Immigration Classification|
|Figure 7: Proficiency ||Figure 10: Language at Home ||Figure 13: Immigration at Proficiency Levels|
Ethnicity, language and immigrant status
This section will examine the achievement of students in PISA across different ethnic groups, by language used at home, and by immigrant status.
Five broad ethnic classifications are used to describe ethnicity in New Zealand. They are: Pākehā/European, Māori, Pasifika, Asian, and ‘Other’ ethnic groupings. The majority of students in New Zealand are Pākehā/European (62%) or Māori (18%). Asian (11%) and Pasifika (7%) make up most of the rest of the ethnic groupings, with only two percent of students categorised in the Other ethnic grouping.
Previous international studies (for example, PISA 2000, see Sturrock & May 2002; TIMSS-02/03, see Chamberlain 2007) have shown that average mathematics achievement varies across ethnic groups. Although the variation in achievement is not caused by ethnicity per se, education policies have been introduced in an attempt to realise the potential of students. Specific areas of focus for the Ministry of Education (Ministry of Education 2007) include the achievement of Māori and Pasifika students. The results at the Year 5 level in TIMSS-02/03 (Caygill et al. 2007) have shown an increase in mathematics performance on average for Māori and Pasifika students since the first cycle in 1994/1995.
In PISA 2006, Asian (548) and Pākehā/European (539) students had significantly higher mean achievement than did their Māori (479) and Pasifika (463) counterparts. Māori students performed significantly higher in mathematical literacy than Pasifika students. No significant difference was observed between Pākehā/European and Asian students.
In comparison with 2003, there is no significant improvement in achievement for any group. However, Pākehā/European students in 2006 (539) performed significantly lower than their counterparts in 2003 (546). The average achievement difference between Pākehā/European students and their counterparts in other groupings has shrunk between 2003 and 2006, partially attributable to this drop in achievement for Pākehā/European students and partially attributable to small non-significant rises in achievement in the other groupings. The small changes in achievement can be seen in Figure 5, although the error bars suggest most of these changes were not significant.
Note that these 2006 data represent only the second data point for measuring change, because 2003 was the first PISA assessment to cover all content domains of mathematics. PISA 2009 (data released in 2010) will provide an opportunity to gather a third data point in this series.
Figure 5: Changes in mathematical achievement for ethnic groupings in PISA between 2003 and 2006
Although there is a difference in mean achievement between ethnic groupings, interestingly the range of achievement for Pākehā/European, Māori, and Pasifika students is remarkably similar (the range is defined as the difference between the 95th and 5th percentiles). Thus the picture of achievement within each of these three ethnic grouping is similar, albeit with a shift relative to the mean (see Figure 6).
Figure 6: Distributions of mathematical achievement in PISA 2006 for ethnic groupings of students
When examining the proficiency levels, a range of achievement was observed in all ethnic groupings. Within all ethnic groupings there were students who achieved at the highest proficiency level; that is, they demonstrated the ability to complete tasks requiring advanced mathematical thinking and reasoning. Similarly, within all ethnic groupings there were students who achieved at the lowest proficiency level; that is, they did not demonstrate the ability to complete a reasonable amount of the simplest mathematics tasks which PISA seeks to measure.
A higher proportion of Asian students were proficient at the highest level (11%) compared with any other ethnic grouping (Pākehā/European 7%, Other 7%, Māori 1%, and Pasifika 1%). Around half of all Asian (52%) and Pākehā/European (48%) students achieved at or above Level 4, as shown in Figure 7.
At the lower end of the spectrum, a greater proportion of Pasifika (11%) and Māori (8%) students performed below Level 1, compared with six percent of students in the Other grouping, and two percent each of Pākehā/European and Asian students. Combining the proportions of students below Level 1 and in Level 1, nearly one-third of Pasifika (30%) and one-quarter of Maori (26%) students performed at the lowest levels in the PISA assessment and were unable to demonstrate the ability to correctly complete much beyond simple mathematical tasks. In comparison, nine percent of Pākehā/European, 11 percent of Asian, and 16 percent of Other ethnic students demonstrated proficiency at or below Level 1.
Figure 7: Percentage of students in each ethnic grouping at each of the mathematical literacy proficiency levels
It is clear from the proficiency level proportions that Māori and Pasifika students were over-represented at the lowest levels of proficiency. However, in terms of actual numbers, Pākehā/European students made up the single largest group of low achievers. Figure 8 shows the ethnic composition of the 14 percent of students who achieved at Level 1 or below. Of these, well over a third, or 5.5 percent of all students, were Pākehā/European.
Figure 8: Percentage of students in each ethnic grouping in the group of lower achievers
Girls and boys within ethnic groupings
With the exception of the Pākehā/European grouping, there was no significant difference between the boys and the girls within the ethnic groupings. In the Pākehā/European grouping, the mean achievement of boys was significantly higher than that of girls. Figure 9 presents the mean mathematics performances for girls and boys within each ethnic grouping. This figure shows that for both boys and girls, the relative performances between ethnic groupings are the same.
Figure 9: Mean mathematics performance for girls and boys within ethnic groupings
Language spoken at home
Another factor influencing the performance of students of different ethnic groupings may be the language spoken at home. Students in PISA1 were asked: “What language do you speak at home most of the time?”, and this was then classified as either the language of the test (in New Zealand this was English) or other language for the purposes of international comparisons. Approximately 9 out of every 10 New Zealand students responded that English was the language they spoke most at home, while approximately 1 out of every 10 responded that it was a language other than English.2
There was no difference in mean mathematical achievement between students who usually spoke English at home and students who usually spoke another language at home in New Zealand (see Figure 10). This was different from the majority of OECD countries, where on average those who spoke the official language of instruction at home had a higher mean achievement than those who spoke another language at home (a difference of 50 score points across OECD countries). In Australia and Canada, like New Zealand, there was no difference in achievement between those who spoke the language of the test and those who spoke another language.
Comparing this result from 2006 with that found in 2003, a change is apparent. In PISA 2003 there was a significant difference between English speakers and those who mostly spoke another language at home, with English speakers demonstrating higher mathematical achievement on average. Figure 10 also appears to show a wider distribution of scores, although only the 95th percentile figure is statistically different when comparing the two language groupings.
Figure 10: Distributions of mathematical achievement in PISA 2006 for students, by the language spoken at home
Note: While the distribution looks wider for 'Other language' there is a lot of uncertainty for this result, as demonstrated by large standard errors, particularly for the 5th percentile (se of 15.0)
When examining the achievement of students at the different proficiency levels, as shown in Figure 11, small differences between students in language groupings can be observed, but none are statistically significant.
Figure 11: Percentage of students in each language spoken at home grouping at each of the mathematical literacy proficiency levels
Using reports from students on their country of birth and the country of birth of their parents, the OECD divided students into three categories to denote their immigrant status: native students, second-generation students, and first-generation students. The title native students was used where at least one of the students’ parents was born in New Zealand, second-generation students were those who were born in New Zealand but both of whose parents were not, while first generation was used for students where both they and their parents were born outside of New Zealand. The majority of students were native (79%), with seven percent of students second generation and 14 percent first generation.3 These proportions have not changed significantly since 2003.4
Mathematical literacy achievement was not significantly different for native New Zealand students compared with their first- and second-generation counterparts. However, first-generation students had significantly higher mean achievement than second-generation students; this is a similar finding to that observed in 2003. There are no statistically significant differences between the percentiles shown in Figure 12.
Figure 12: Distribution of mathematical achievement in PISA 2006 for students, by immigration classification
Note: While the distribution looks wider for ‘First generation’ students, differences are not significant.
Examining the proficiency levels for these three groupings, as shown in Figure 13, reveals small differences between the three groupings, but none of these differences are statistically significant.
Figure 13: Percentage of students in each immigration classification at each of the mathematical literacy
- Students who had received less than one year’s instruction in English and those in Māori-immersion classes were excluded from the PISA sample in New Zealand.
- Note that the figures presented here exclude missing or invalid responses – there were four percent of such responses in the sample. They also exclude the small proportion (0.2%) that mostly speak Māori at home.
- Adjusted percentages are shown. There were two percent of students with missing data for these questions.
- Note that the labels for these groupings have changed: what is now called first generation was called non-native; what is now called second generation was called first generation.
Where to find out more
For more information about this publication please email the: Research Mailbox