Definitions and technical notes

Mean

Student performances in PISA are reported using means, which is a type of average, for groupings of students. In general, the mean of a set of scores is the sum of the scores divided by the number of scores, and is often referred to as ‘the average’. Note that for PISA, as with other large-scale studies, the means for a country are adjusted slightly (in technical terms ‘weighted’) to reflect the total population of 15-year-olds rather than just the sample.

Minimum group size for reporting achievement data

In this report student achievement data are not reported where the group size is less than 30 students.

OECD mean or average

The OECD mean, sometimes referred to as the OECD average, includes only the OECD countries – no non-OECD (partner) countries are included in this average. The OECD mean is the average of the means for the OECD countries.

Percentile

The percentages of students performing below or above particular points on the scale are given in this report. The lowest outer limit of achievement is the 5th percentile – the score at which only 5 percent of students achieved a lower score and 95 percent of students achieved a higher score. The highest outer limit is the 95th percentile – the score at which only 5 percent of students achieved a higher score and 95 percent of students a lower score; thus 90 percent of the 15-year-old student scores lie between the 5th and 95th percentiles.

Proficiency scale

PISA developed proficiency levels to describe the range in literacy across 15-year-old students. The proficiency levels describe the competencies of students achieving at that level and are anchored at certain score points on the achievement scale. Note that students were considered to be proficient at a particular level if, on the basis of their overall performance, they could be expected to answer at least half of the items in that level correctly. Typically, students who were proficient at higher levels had also demonstrated their abilities and knowledge at lower levels.

Scale score points

The design of PISA allows for a large number of questions to be used in mathematics, science and reading; each student answers only a portion of these questions. PISA employs techniques to enable population estimates of achievement to be produced for each country even though a sample of students responded to differing selections of questions. These techniques result in scaled scores which are on a scale with a mean of 500 and a standard deviation of 100. The OECD mean score of 500 points was established as the benchmark against which performance has since been measured in the first cycle of PISA where each subject was the major focus: in PISA 2000 for reading, in PISA 2003 for mathematics, and in PISA 2006 for science.

Standard error

Because of the technical nature of PISA, the calculation of statistics such as means and proportions have some uncertainty due to (i) generalising from the sample to the total 15-year-old school population, and (ii) inferring each student’s proficiency from their performance on a subset of items. The standard errors provide a measure of this uncertainty. In general, we can be 95 percent confident that the true population value lies within an interval 1.96 standard errors either side of the given statistic.

Statistically significant

In order to determine whether a difference between two means is actual, it is usual to undertake tests of significance. These tests take into account the means and the error associated with them. If a result is reported as not being statistically significant, then although the means might be slightly different, we do not have sufficient evidence to infer that they are different. All tests of statistical significance referred to in this report are at the 95 percent confidence level.