PISA 2009: Our 21st century learners at age 15

Publication Details

This report provides a high level picture of New Zealand’s 15-year-old performance in reading literacy (main focus), mathematical literacy and scientific literacy. It compares New Zealand’s results with other top- and high-performing countries. In July and August 2009 4,643 New Zealand 15 year-old students from 163 New Zealand schools took part in PISA 2009.

Author(s): Maree Telford with Steve May [Ministry of Education]

Date Published: December 2010

An overview of PISA

What is PISA?

The Programme for International Student Assessment (PISA) is an international standardised study that assesses and compares how well countries are preparing their 15-year-old1 students to meet real-life opportunities and challenges.

What does PISA assess?

PISA assesses three key areas of knowledge and skills – reading literacy, mathematical literacy and scientific literacy – and has a focus on one of these literacy areas each time PISA is administered. The focus of PISA 2009 is reading. In each country students complete a two hour test booklet in the language of their instruction.2 The term ‘literacy’ is used to emphasise that the assessment is not restricted to assessing how well students, have mastered the content of a specific school curriculum. Instead, PISA focuses on assessing students’ ability to apply their knowledge and skills, and their ability to make decisions in real-life situations as they near the end of their compulsory schooling. PISA defines this approach as assessing the “things that 15-year-old students will need in the future and ... what they can do with what they have learned – reflecting the ability of students to continue learning throughout their lives by applying what they learn in school to non-school environments, evaluating their choices and making decisions.”3

What additional information is gathered?

Background information is also gained in each PISA cycle from questionnaires completed by students and school principals. In addition, in PISA 2009 parents completed a questionnaire. These questionnaires allow for the relationship between contextual information and achievement to be examined.

How often is PISA administered?

PISA is administered every three years, beginning in 2000. Reading was the main focus in the first cycle. In 2003 the focus was mathematical literacy, in 2006 the focus was scientific literacy and in 2009 it was reading literacy again. Rotating the main focus for each administration of PISA provides in-depth and detailed information on the subject of main focus along with an ongoing source of achievement data on the two minor subjects.

Who participates in PISA?

Around 470,000 15-year-old students from 654 countries or economies, including the 34 Organisation for Economic Co-operation and Development (OECD) member countries, participated in PISA 2009. In New Zealand 4,643 students from 161 schools took part. Students and schools were randomly selected. A two-tiered stratified sampling method was used to ensure the sample was representative. Schools were randomly selected based on the following characteristics: size, decile, location (urban or rural), authority (state or independent) and type (co-educational or single-sex). As a result, every 15-year-old had roughly the same chance of selection.

Why participate in PISA?

PISA assesses students who have completed around 10 years of compulsory schooling, which means the PISA results are an important source of information in New Zealand. PISA measures progress towards the Government’s goals of:

  • building a world-leading education system that equips all New Zealanders with the knowledge, skills and values to be successful citizens in the 21st century,
  • reducing systemic underachievement in education, and
  • driving the improvement of educational performance across New Zealand’s education system to improve education outcomes for all young New Zealanders.

PISA not only allows measurement of New Zealand’s progress on these goals over time, but also allows measurement of New Zealand’s performance relative to other countries in equipping students with skills and reducing disparities in achievement. The PISA data contributes to the evidence to shape the direction of teaching and learning in schools, focus on quality and effective teaching making our world-class education system even better.

Who organises PISA?

PISA is an initiative of the OECD and a collaborative effort of the participating countries. Two consortia were responsible for developing and overseeing PISA 2009 at the international level. The consortium responsible for development of the cognitive assessment and the implementation of the study was led by the Australian Council for Educational Research (ACER), and included Westat (USA), The National Institute for Educational Research in Japan, cApStAn Linguistic Quality Control, (Belgium), Unité d’analyse des systèmes et des pratiques d’enseignement (aSPe, Belgium) and the Deutsches Institut fūr Internationale Pūdagogische Forschung (DIPF, Germany). CITO Institute for Educational Measurement (Netherlands) led the consortium responsible for the development of the questionaires and included the University of Twente (Netherlands), the University of Jyvāskla’s Institute for Educational Research (Finland) and the Direction de l’Evaluation de la Prospective et de la Performance (DEPP, France). In New Zealand, the Comparative Education Research Unit within the Ministry of Education’s Research Division is responsible for PISA.

How did the OECD ensure the PISA data were of high quality?

A number of quality assurance procedures were put in place, both nationally and internationally, to ensure the data were as high a quality as possible. These included: rigorous training of staff; high-quality documentation; monitoring of sampling procedures; quality checks and monitoring at a number of stages, including during administration of the tests; multiple coding and data entry procedures; and data cleaning and checking procedures. Further details of international procedures will be found in the PISA 2009 Technical Report (forthcoming).

Footnotes

  1. Students are aged between 15-years-3-months and 16-years-2-months. As most students are aged 15, they are referred to as ‘15-year-olds’ for brevity.
  2. In New Zealand, PISA was administered only in English.
  3. OECD. (2009). Assessment Framework - Key Competencies in Reading, Mathematics and Science, p 9. Paris: OECD.
  4. The countries and economies participating in PISA 2009 are listed on the back page of this report.