Appendices

Appendix 1: Countries in PISA and structure of the PISA assessment cycle

Argentina *	Australia	Austria
Azerbaijan *	Belgium	Brazil *
Bulgaria *	Canada	Chile *
Colombia *	Croatia *	Czech Republic
Denmark	Estonia *	Finland
France	Germany	Greece
Hong Kong-China*	Hungary	Iceland
Indonesia *	Ireland	Israel *
Italy	Japan	Jordan *
Korea	Kyrgyzstan *	Latvia *
Liechtenstein *	Lithuania *	Luxembourg
Macao-China*	Mexico	Montenegro *
The Netherlands	New Zealand	Norway
Poland	Portugal	Qatar *
Romania *	Russian Federation *	Serbia *
Slovak Republic	Slovenia *	Spain
Sweden	Switzerland	Chinese Taipei *
Thailand *	Tunisia *	Turkey
United Kingdom	United States	Uruguay *

Note: * denotes non-OECD countries.

 

Table A.2: Structure of PISA assessment cycle

Year	Reading literacy	Mathematical literacy	Scientific literacy
2000 Total item pool	Major domain 270 minutes	Minor domain 60 minutes	Minor domain  60 minutes
2003 Total item pool	Minor domain 60 minutes	Major domain 210 minutes*	Minor domain  60 minutes
2006 Total item pool	Minor domain 60 minutes	Minor domain 120 minutes	Major domain 210 minutes

Notes: Each student is assessed on a selection of items from each domain, for a total of 120 minutes.

*In 2003, a separate problem-solving assessment area was included, which was allocated 60 minutes of the total testing time.

 

Appendix 2: Sample questions from PISA 2003

Figure A.1: Level 6 mathematics question – Carpenter

Content area:	Space and shape
Difficulty:	Linked to 687 score points
Scoring: Full credit: Partial credit:	Yes, no, yes, yes in that order Any three of the above correct.

                 

Country	Percent correct
Finland	22 (1.1)
Hong Kong-China	40 (1.5)
Korea	35 (1.4)
Netherlands	24 (1.3)
New Zealand	21 (1.1)
Australia	23 (1.1)
United Kingdom	15 (0.9)
United States	15 (1.0)

Note: Standard errors appear in parentheses.

 

Figure A.2: Level 5 mathematics question – Test scores

Content area:	Uncertainty
Difficulty:	Linked to 620 score points
Scoring: Full credit	One valid argument given. Valid arguments could relate to the number of students passing, the disproportionate influence of the outlier, or the number of students with scores in the highest level.

                 

Country	Percent correct
Finland	35 (1.3)
Hong Kong-China	64 (1.6)
Korea	46 (1.4)
Netherlands	41 (1.4)
New Zealand	42 (1.7)
Australia	43 (1.1)
United Kingdom	42 (1.4)
United States	40 (1.6)

Note: Standard errors appear in parentheses. 

 

Figure A.3: Level 4 mathematics question – Exchange rate, question 3

Content area:	Quantity
Difficulty:	Linked to 586 score points
Scoring: Full credit:	Yes, with an adequate explanation

Country	Percent correct
Finland	51 (1.4)
Hong Kong-China	53 (1.5)
Korea	40 (1.4)
Netherlands	48 (1.5)
New Zealand	42 (1.7)
Australia	46 (1.0)
United Kingdom	43 (1.2)
United States	37 (1.5)

Note: Standard errors appear in parentheses. 

 

Figure A.4: Level 3 mathematics question – Growing up

Content area:	Change and relationships
Difficulty:	Linked to 525 score points
Scoring: Full credit: Partial credit:	Gives the correct interval from 11 to 13 years or states that girls are taller than boys when they are 11 and 12 years old. Other subsets (of 11, 12, 13) not included in the full credit section.

Country	Percent correct
	Full credit	Partial credit
Finland	67 (1.4)	26 (1.2)
Hong Kong-China	54 (1.7)	34 (1.5)
Korea	80 (1.0)	4 (0.5)
Netherlands	67 (1.4)	23 (1.3)
New Zealand	55 (1.3)	34 (1.2)
Australia	54 (1.2)	36 (1.0)
United Kingdom	53 (1.4)	35 (1.3)
United States	39 (1.4)	43 (1.2)

Note: Standard errors appear in parentheses. 

 

Figure A.5: Level 2 mathematics question – Staircase

Content area:	Space and shape
Difficulty:	Linked to 421 score points
Scoring: Full credit:	18

Country	Percent correct
Finland	85 (0.8)
Hong Kong-China	87 (1.1)
Korea	81 (1.0)
Netherlands	85 (1.2)
New Zealand	79 (1.2)
Australia	78 (1.0)
United Kingdom	74 (1.4)
United States	70 (1.1)

Note: Standard errors appear in parentheses.

 

Figure A.6: Level 1 mathematics question – Exchange rate, question 1 

 

Content area:	Quantity
Difficulty:	Linked to 406 score points
Scoring: Full credit:	12 600 ZAR (with or without ZAR added)

Country	Percent correct
Finland	90 (0.9)
Hong Kong-China	89 (1.0)
Korea	81 (1.1)
Netherlands	85 (1.0)
New Zealand	80 (1.1)
Australia	81 (0.8)
United Kingdom	74 (1.3)
United States	54 (1.3)

Note: Standard errors appear in parentheses.

 

Appendix 3: Full detail of PISA mathematical literacy proficiency levels

Level	Lower score limit	What students can typically do
6	669.3	At Level 6 students can conceptualise, generalise, and utilise information based on their investigations and modelling of complex problem situations. They can link different information sources and representations and flexibly translate among them. Students at this level are capable of advanced mathematical thinking and reasoning. These students can apply this insight and understandings along with a mastery of symbolic and formal mathematical operations and relationships to develop new approaches and strategies for attacking novel situations. Students at this level can formulate and precisely communicate their actions and reflections regarding their findings, interpretations, arguments, and the appropriateness of these to the original situations.
5	607.0	At Level 5 students can develop and work with models for complex situations, identifying constraints and specifying assumptions. They can select, compare, and evaluate appropriate problem solving strategies for dealing with complex problems related to these models. Students at this level can work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representations, symbolic and formal characterisations, and insight pertaining to these situations. They can reflect on their actions and formulate and communicate their interpretations and reasoning.
4	544.7	At Level 4 students can work effectively with explicit models for complex concrete situations that may involve constraints or call for making assumptions. They can select and integrate different representations, including symbolic ones, linking them directly to aspects of real-world situations. Students at this level can utilise well-developed skills and reason flexibly, with some insight, in these contexts. They can construct and communicate explanations and arguments based on their interpretations, arguments, and actions.
3	482.4	At Level 3 students can execute clearly described procedures, including those that require sequential decisions. They can select and apply simple problem solving strategies. Students at this level can interpret and use representations based on different information sources and reason directly from them. They can develop short communications reporting their interpretations, results and reasoning.
2	420.1	At Level 2 students can interpret and recognise situations in contexts that require no more than direct inference. They can extract relevant information from a single source and make use of a single representational mode. Students at this level can employ basic algorithms, formulae, procedures, or conventions. They are capable of direct reasoning and making literal interpretations of the results.
1	357.8	At Level 1 students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined. They are able to identify information and to carryout routine procedures according to direct instructions in explicit situations. They can perform actions that are obvious and follow immediately from the given stimuli.