PISA 2006: Reading Literacy - How ready are our 15-year-olds for tomorrow’s world?

Introduction

This report examines the reading literacy results for New Zealand students from PISA 2006. The international findings for PISA 2006 were published by the OECD in two volumes in 2007 (OECD 2007a and 2007b). A summary of key New Zealand results from this study was published in December 2007 (Telford & Caygill 2007). Other reports in this series will focus on mathematical literacy, scientific literacy, school contexts, and attitude and engagement factors.¹

The first part of this report provides an overview of the reading literacy domain, including what is assessed and how the results can be interpreted. The second part of the report focuses on the overall performance of New Zealand’s 15-year-olds in PISA 2006 in comparison with other participating countries and over time. Finally, results are examined for groups within the New Zealand population according to different characteristics: gender, ethnic grouping, immigrant status, language spoken at home and socio-economic status.

Definition of reading literacy

The PISA assessment frameworks (OECD 2006) define reading literacy as follows:

Reading literacy: An individual’s capacity to understand, use and reflect on written texts, in order to achieve one’s goals, to develop one’s knowledge and potential and to participate in society.

This definition focuses on the knowledge and skills required to apply reading for learning rather than on the technical skills acquired in learning to read. PISA does not seek to measure such things as the extent to which 15-year-old students are fluent readers or how well they spell or recognise words. Rather, PISA focuses on measuring the extent to which individuals are able to construct, expand and reflect on the meaning of what they have read in a wide range of texts.

Within the reading literacy domain, each problem in the assessment is defined by three dimensions: the format of the reading material, the type of reading task or reading aspects, and the situation or the use for which the text was constructed.

How reading literacy was measured in PISA 2006

Each student was assessed for two hours with a pencil-and-paper test containing both multiple-choice and constructed-response items. Background information was also collected by way of questionnaires completed by students, parents and school principals. Students were given one of thirteen assessment booklets with different combinations of science, mathematics and reading tasks. Less testing time overall was provided for the two minor domains, reading and mathematical literacy, than for the major domain, scientific literacy.²

The pool of reading items comprised a carefully selected mix of texts. In terms of format, 64 percent were continuous texts, typically composed of sentences organised into paragraphs, and 37 percent were non-continuous texts, such as diagrams, forms, maps and tables. In terms of situation or context, texts were drawn roughly equally from personal, public, occupational and educational situations.

The assessment of reading literacy was based on a range of tasks which students completed in relation to the texts. The tasks were structured so as to assess three reading aspects: some tasks (29% of the total) required students to retrieve information, other tasks (50%) required them to interpret texts, while the third type of task (21% of the total) required students to reflect on and evaluate texts.

Selected test questions

Appendix 4 contains reading test questions that were released after the 2000 cycle of PISA. The proportions of New Zealand students who correctly answered each question in PISA 2000 are given, as well as some international comparisons. Because these items were already available to the public they were not included in the PISA 2006 assessment. However, they do give an indication of the types of questions that students in 2006 are likely to have answered.

How is PISA reported?

In PISA 2000, student performance in reading was reported separately for each of the three aspects described above, as well as on a combined reading scale. In PISA 2003 and PISA 2006, however, because reading was a minor domain, a shorter testing time for reading meant it could only be reported on the single combined scale. An OECD mean score of 500 points was established for PISA 2000 as the benchmark against which reading performance has since been measured. Around two-thirds of students in OECD countries achieve scores between 400 and 600 points.

Proficiency levels

In 2000, PISA defined five proficiency levels to describe the range in reading literacy performance across 15-year-old students. These proficiency levels were anchored at certain score points on the achievement scale. They allow us to describe the kinds of reading tasks that students who have achieved a given score are likely to be able to complete. Table 1 gives an outline of the five reading literacy proficiency levels, along with the associated score points at the boundary of the levels. Note that students were considered to be proficient at a particular level if, on the basis of their overall performance, they could be expected to answer at least half of the items in that level correctly. Typically, students who were proficient at higher levels had also demonstrated their abilities and knowledge at lower levels.

What can PISA results tell us?

PISA allows us to compare the performance of New Zealand 15-year-olds in reading literacy against that of their counterparts in 55 other countries.³ The minor domain results offer an update on overall performance rather than the in-depth analysis permitted by major domain results. However, because the 2006 results represent the third occasion on which reading literacy has been assessed using the same assessment framework, they do allow trends in performance since 2000 to be examined.

Two main measures⁴ will be examined in this report:

• the mean scores of particular groups of students on the combined reading scale
• the proportions of students within particular groups achieving at each proficiency level.

Table 1: PISA reading literacy proficiency levels.  

Source: Adapted from OECD 2007a. See Appendix 3 for the detailed proficiency level map.

Footnotes

1. Only the mathematical literacy report was published at the time this report was released (Caygill et al. 2008). The rest are in press.
   
2. See Table A.2 for details of how the three-yearly PISA cycle is structured.
3. Due to an error in printing the test booklets in the United States of America, some of the reading items had incorrect instructions and the mean performance in reading cannot be accurately estimated for that country. Results in this report are for the remaining 56 countries.
4. Please refer to ‘Definitions and technical notes’ at the end of this report for further details.