Indicators

Reading literacy achievement: senior secondary schooling

Why This Is Important

Reading literacy achievement at senior secondary level contributes to preparation for successful participation in tertiary education and training. Achievement level is also related to people’s well being and influences their ability to contribute to, and participate in, a changing labour market and increasingly knowledge-based society.

Literacy involves the ability of individuals to use written information to fulfil their goals, and the consequent ability of complex modern societies to use written information to function effectively.

The Programme for International Student Assessment (PISA) study assessed 15 year-old students’ reading ability on accessing and retrieving information, integrating and interpreting texts, and reflection and evaluation.

Indicator

As in PISA 2000, reading is a major area in PISA 2009. The reading scores were summarised on a combined reading literacy scale.

The Item Response Theory (IRT) scaling approach and plausible values methodology is used in PISA. This involved estimating the parameters for each item and examining the background characteristics of the students. From this, estimates of proficiency for each student and IRT scales for reporting student achievement were generated; in aggregate and for each major content domain. Finally, the resulting values were placed on a reporting scale in PISA 2000 with a mean of 500 and standard deviation of 100. Subsequent cycles (2003, 2006 and 2009) were anchored against the PISA 2000 scale. This enables a comparison to be made between the reading literacy achievement of 15 year-old students in each of 2000, 2003, 2006 and 2009.

The IRT analysis provided a common scale on which the performances of students within and across countries may be compared.

Each student has 5 estimates of ability called plausible value (PV1-PV5). The plausible values represent a set of random values for each student selected at random from an estimated ability distribution of students with similar item response patterns and backgrounds. They are intended to provide good estimates of parameters of student populations, for example, country mean scores, rather than estimates of individual student proficiency.

For any group of 15 year-old students, for example, the New Zealand Population, Māori, or Girls, the numerator and denominator are defined as follows:

Numerator: (Data source: OECD: Programme for International Student Assessment (PISA))
Sum of the mean reading literacy scores for each plausible value for that group. [Where the mean for each plausible value is defined as:

Numerator: Weighted sum of scores for that group.
Denominator: Sum of the weights for that group (equivalent to the estimated number of students in that group).]

Denominator: (Data source: OECD: Programme for International Student Assessment (PISA)
5 (number of plausible values).

Interpretation Issues

Mean PISA scores for the New Zealand population and sub-populations are based on scores generated using Item Response Theory. These scores are reported on an international scale with an international standard deviation of 100 so that approximately two-thirds of all students internationally have a score between 400 and 600.

In PISA 2009 proficiency levels related to the difficulty of the tasks that students were assessed on, with each content area having its own set of proficiency levels. These range from Level 1 for the simplest tasks to Level 6 for the most complex. For information on the proficiency levels for each content area see: OECD (2010). PISA 2009 Results: What Students Know and Can Do – Student Performance in Reading, Mathematics and Science – Volume I. Paris: OECD

Level
Lower score limit % of students able to perform tasks at this level or above Characteristics of tasks
67080.8% of students across the OECD can perform tasks at least at Level6 on the reading scaleTasks at this level typically require the reader to make multiple inferences, comparisons and contrasts that are both detailed and precise. They require demonstration of a full and detailed understanding of one or more texts and may involve integrating information from more than one text. Tasks may require the reader to deal with unfamiliar ideas, in the presence of prominent competing information, and to generate abstract categories for interpretations. Reflect and evaluate tasks may require the reader to hypothesise about or critically evaluate a complex text on an unfamiliar topic, taking into account multiple criteria or perspectives, and applying sophisticated understandings from beyond the text. A salient condition for access and retrieve tasks at this level is precision of analysis and fine attention to detail that is inconspicuous in the texts.
56267.6% of students across the OECD can perform tasks at least at Level 5 on the reading scaleTasks at this level that involve retrieving information require the reader to locate and organise several pieces of deeply embedded information, inferring which information in the text is relevant. Reflective tasks require critical evaluation or hypothesis, drawing on specialised knowledge. Both interpretative and reflective tasks require a full and detailed understanding of a text whose content or form is unfamiliar. For all aspects of reading, tasks at this level typically involve dealing with concepts that are contrary to expectations.
455328.3% of students across the OECD can perform tasks at least at Level 4 on the reading scaleTasks at this level that involve retrieving information require the reader to locate and organise several pieces of embedded information. Some tasks at this level require interpreting the meaning of nuances of language in a section of text by taking into account the text as a whole. Other interpretative tasks require understanding and applying categories in an unfamiliar context. Reflective tasks at this level require readers to use formal or public knowledge to hypothesise about or critically evaluate a text. Readers must demonstrate an accurate understanding of long or complex texts whose content or form may be unfamiliar.
348057.2% of students across the OECD can perform tasks at least at Level 3 on the reading scaleTasks at this level require the reader to locate, and in some cases recognise the relationship between, several pieces of information that must meet multiple conditions. Interpretative tasks at this level require the reader to integrate several parts of a text in order to identify a main idea, understand a relationship or construe the meaning of a word or phrase. They need to take into account many features in comparing, contrasting or categorising. Often the required information is not prominent or there is much competing information; or there are other text obstacles, such as ideas that are contrary to expectation or negatively worded. Reflective tasks at this level may require connections, comparisons, and explanations, or they may require the reader to evaluate a feature of the text. Some reflective tasks require readers to demonstrate a fine understanding of the text in relation to familiar, everyday knowledge. Other tasks do not require detailed text comprehension but require the reader to draw on less common knowledge.
240781.2% of students across the OECD can perform tasks at least at Level2 on the reading scaleSome tasks at this level require the reader to locate one or more pieces of information, which may need to be inferred and may need to meet several conditions. Others require recognising the main idea in a text, understanding relationships, or construing meaning within a limited part of the text when the information is not prominent and the reader must make low level inferences. Tasks at this level may involve comparisons or contrasts based on a single feature in the text. Typical reflective tasks at this level require readers to make a comparison or several connections between the text and outside knowledge, by drawing on personal experience and attitudes.
1a33594.3% of students can perform tasks at least at Level1a on the reading scaleTasks at this level require the reader to locate one or more independent pieces of explicitly stated information; to recognise the main theme or author’s purpose in a text about across the OECD a familiar topic, or to make a simple connection between information in the text and common, everyday knowledge. Typically the required information in the text is prominent and there is little, if any, competing information. The reader is explicitly directed toconsider relevant factors in the task and in the text.
1b26298.9% of students across the OECD can perform tasks at least at Level 1b on the reading scaleTasks at this level require the reader to locate a single piece of explicitly stated information in a prominent position in a short, syntactically simple text with a familiar context and text type, such as a narrative or a simple list. The text typically provides support to the reader, such as repetition of information, pictures or familiar symbols. There is minimal competing information. In tasks requiring interpretation the reader may need to make simple connections between adjacent pieces of information.

 Copyright © Education Counts 2014   |   Contact information.officer@minedu.govt.nz for enquiries.