Science literacy achievement: senior secondary schooling

Why this is Important

Scientific literacy assists students to participate as responsible and informed members of society, and as productive contributors to New Zealand's economy and future.

Attainment at senior secondary level contributes to preparation for successful participation in tertiary education, and the ability to contribute to, and participate in, a changing labour market and an increasingly knowledge-based society. Attainment level is also related to individual well being.


Scientific literacy was a minor domain in Programme for International Student Assessment (PISA) 2009, after being a major domain in PISA 2006. The science scores were summarised on a combined scientific literacy scale. The domain assesses 3 scientific competencies - identifying scientific issues, explaining phenomena scientifically, and using scientific evidence – and two scientific knowledge areas – knowledge of science and knowledge about science. Due to changes in the way scientific literacy has been assessed, no comparison can be made with the results for PISA 2000 and 2003.

The Item Response Theory (IRT) scaling approach and plausible values methodology is used in PISA.  This involved estimating the parameters for each item and examining the background characteristics of the students. From this, estimates of proficiency for each student and IRT scales for reporting student achievement were generated; in aggregate and for each major content domain. Finally, the resulting values were placed on a reporting scale in PISA 2006 with a mean of 500 and standard deviation of 100. The PISA 2009 scale was anchored against the PISA 2006 scale enabling change to be measured from 2006 to 2009.

The IRT analysis provided a common scale on which the performances of students within and across countries may be compared.

Each student has 5 estimates of ability called plausible value (PV1-PV5). The plausible values represent a set of random values for each student selected at random from an estimated ability distribution of students with similar item response patterns and backgrounds. They are intended to provide good estimates of parameters of student populations, for example, country mean scores, rather than estimates of individual student proficiency.

For any group of 15 year-old students, for example, the New Zealand Population, Māori, or Girls, the numerator and denominator are defined as follows:

Numerator: (Data source: OECD: Programme for International Student Assessment (PISA))
Sum of the mean mathematics literacy scores for each plausible value for that group. [Where the mean for each plausible value is defined as:

Numerator: Weighted sum of scores for that group.
Denominator: Sum of the weights for that group (equivalent to the estimated number of students in that group).]

Denominator: (Data source: OECD: Programme for International Student Assessment (PISA))
5 (number of plausible values).

Interpretation Issues

Due to changes in the way scientific literacy has been assessed, no comparison can be made with the results for PISA 2000 and 2003.

Mean PISA scores for the New Zealand population and sub-populations are based on scores generated using Item Response Theory. These scores are reported on an international scale with an international standard deviation of 100 so that approximately two-thirds of all students internationally have a score between 400 and 600.

The scientific literacy domain has undergone considerable expansion and change since being a minor domain in PISA 2000 and PISA 2003. It is not therefore possible to compare science outcomes from PISA 2006 or PISA 2009 with these earlier PISA assessments.

In PISA 2009 proficiency levels related to the difficulty of the tasks that students were assessed on, with each content area having its own set of proficiency levels. These range from Level 1 for the simplest tasks to Level 6 for the most complex. The following information on the proficiency levels for each content area was sourced from: OECD (2010). PISA 2009 Results: What Students Know and Can Do – Student Performance in Reading, Mathematics and Science – Volume I. Paris: OECD.

LevelLower score limit

Characteristics of tasks

6708At Level 6, students can consistently identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations. They can link different information sources and explanations and use evidence from those sources to justify decisions. They clearly and consistently demonstrate advanced scientific thinking and reasoning, and they demonstrate willingness to use their scientific understanding in support of solutions to unfamiliar scientific and technological situations. Students at this level can use scientific knowledge and develop arguments in support of recommendations and decisions that centre on personal, social or global situations.

5633At Level 5, students can identify the scientific components of many complex life situations, apply both scientific concepts and knowledge about science to these situations, and can compare, select and evaluate appropriate scientific evidence for responding to life situations. Students at this level can use well-developed inquiry abilities, link knowledge appropriately and bring critical insights to situations. They can construct explanations based on evidence and arguments based on their critical analysis.
4 559At Level 4, students can work effectively with situations and issues that may involve explicit phenomena requiring them to make inferences about the role of science or technology. They can select and integrate explanations from different disciplines of science or technology and link those explanations directly to aspects of life situations. Students at this level can reflect on their actions and they can communicate decisions using scientific knowledge and evidence.
3484At Level 3, students can identify clearly described scientific issues in a range of contexts. They can select facts and knowledge to explain phenomena and apply simple models or inquiry strategies. Students at this level can interpret and use scientific concepts from different disciplines and can apply them directly. They can develop short statements using facts and make decisions based on scientific knowledge.
2409At Level 2, students have adequate scientific knowledge to provide possible explanations in familiar contexts or draw conclusions based on simple investigations. They are capable of direct reasoning and making literal interpretations of the results of scientific inquiry or technological problem solving.
1335At Level 1, students have such a limited scientific knowledge that it can only be applied to a few, familiar situations. They can present scientific explanations that are obvious and that follow explicitly from given evidence.

 Copyright © Education Counts 2014   |   Contact for enquiries.