The Adult Literacy and Life Skills (ALL) Survey: Headline Results and Background

This fact-sheet provides the first headline results of the Adult Literacy and Life Skills (ALL) Survey together with some background information.

Author: Research Division, Ministry of Education
Date Published: December 2007

DOWNLOADS
Available from top right hand inset box. Hard copies are available by request via email: research.info@minedu.govt.nz.

The ALL survey is an investigation of the distribution of certain skills (such as numeracy and document interpretation) among the adult population. For developed countries, such skills are strongly linked with labour market outcomes.

The ALL survey is conducted across many countries. This allows for comparison with other countries as well as providing information specific to New Zealand.

The ALL survey follows a similar survey – the International Adult Literacy Survey (IALS) – conducted in 1996. Parts of the ALL survey are directly comparable to this earlier work. This comparison will provide a picture of some of the changes that have occurred over the previous decade.

For further information, please refer to The Adult Literacy and Life Skills (ALL) Survey: An Introduction.

Who does the ALL survey focus upon?

The ALL survey measures the distribution of certain skills through the adult population. Here, “adult” means any person between the ages of 16 and 65 living in a private household.

What does the ALL survey measure?

The ALL survey measures proficiency in four “domains”. Two of these are included here:

“Numeracy” is the ability to understand and process mathematical and numerical information.

“Document literacy” is the ability to read and understand discontinuous texts – such as graphs, charts and tables.

How is the ALL survey implemented?

The New Zealand implementation of the ALL survey has a random, geographically-based sample design.

Each survey was administered face to face by an interviewer of the National Research Bureau Ltd, according to the strict guidelines laid down by Statistics Canada. (See Who implements the ALL survey?)

What are the elements of an ALL interview?

The respondent provides the interviewer with two types of information.

The first type of information concerns the background of the respondent. This includes factors such as the respondent’s gender, ethnicity, age-group, labour-force status and employment status.

The second type of information concerns the proficiency of the respondent in one of the domains. The respondent provides this information by answering a sequence of written questions.

What do typical questions look like?

Here is a sample numeracy question.

Sample numeracy question:

Compare the percent of change in the Dioxin level from 1975 to 1985 with the percent of change from 1985 to 1995. Which percent of change is larger? Explain your answer.

How are an individual’s skills measured by the ALL survey?

To each individual, and for each of the domains mentioned, a score from zero to 500 is assigned. Zero indicates extremely low proficiency and 500 extremely high. In addition, based upon this score, one of five “cognitive levels” is assigned. The following list provides descriptions of typical tasks associated with each cognitive level.

Level 1 (0–225): Tasks in this level require the ability to read simple documents, accomplish literal information-matching with no distractions, and perform simple one-step calculations.

Level 2 (226–275): This level includes tasks that demand the capacity to search a document and filter out some simple distracting information, achieve low-level inferences, and execute one- or two-step calculations and estimations.

Level 3 (276–325): Typical tasks at level 3 involve more complex information filtering, sometimes requiring inference, and the facility to manipulate mathematical symbols, perhaps in several stages.

Level 4 (326–375): A level 4 task might demand the integration of information from a long passage, the use of more complex inferences, and the completion of multiple-step calculations requiring some reasoning.

Level 5 (376–500): Level 5 tasks incorporate the capability to make high-level inferences or syntheses, use specialised knowledge, filter out multiple distractors, and to understand and use abstract mathematical ideas with justification.

Who implements the ALL survey?

There are various parties involved in the implementation of the ALL survey.

The ALL survey is a joint project of the Government of Canada, the US National Center for Education Statistics (NCES) and the Organisation for Economic Co-operation and Development (OECD).

The survey was constructed by the Educational Testing Service (ETS) in consultation with the government of each participating country and the organisations mentioned. In the case of New Zealand, the government was represented by the Ministry of Education. For all participating countries, this construction process is monitored by Statistics Canada (representing the government of Canada). Such monitoring ensures that participating countries conduct the ALL survey in a standard way, guaranteeing internationally comparable results.

In New Zealand, the ALL survey was administered, under contract to the Ministry of Education, by the National Research Bureau Ltd. Again, this administration process was overseen by Statistics Canada.

Which countries are involved in the ALL survey?

The countries involved in the ALL survey are Canada, Italy, Norway, Switzerland, the United States of America, Bermuda, the Mexican State of Nuevo Leon, Australia, Korea, Hungary, New Zealand, and the Netherlands. Data from only some of these countries are currently available.

It is anticipated that results including all of the participating countries will be released by Statistics Canada near the end of 2008.

International Comparisons

In this section New Zealand is compared with the English-speaking part of Canada and with the USA.

These countries were chosen for comparison for several reasons: data availability; participation in both IALS and ALL; and English as language of survey administration.

Please note that in the following text “Canada” will refer to the English-speaking part of Canada.

Numeracy

Numeracy was measured by the ALL survey and its distribution among the adult population of New Zealand, Canada and the USA is shown in the following figures.

The first figure represents the distribution of numeracy scores using box-plots. The second figure shows the distribution of numeracy levels, i.e. the percentages of the adult population of New Zealand, Canada, and the USA at each numeracy level. These are anchored at the boundary of levels 2 and 3 to allow comparison of either “low numeracy” (levels 1 and 2) or “higher numeracy” (levels 3, 4, and 5) between populations. Note that here levels four and five are combined to give more robust statistical information.

All three countries had sizable subpopulations with low numeracy, and these subpopulations were proportionally similar in magnitude in New Zealand and Canada and larger in the USA.

Numeracy Score Distributions

For New Zealand, the 5th-to-95th percentile range extended from around 175 to around 360. The 5th-to-25th percentile range was longer than the 75th-to-95th percentile range indicating a skewed distribution, but the mean (270) sits slightly below the mid-point between the 25th percentile (235) and the 75th percentile (310).

The statistics for Canada were similar to New Zealand’s with a 5th percentile of around 170, a 25th percentile of around 240, a mean of around 275, a 75th percentile of around 315, and a 95th percentile of around 360.

All statistics for the USA were lower than those of New Zealand or Canada with a 5th percentile of around 165, a 25th percentile of around 220, a mean of around 260, a 75th percentile of around 300, and a 95th percentile of around 350.

Numeracy Level Distributions

For New Zealand, the percentage of the population at level 1 (around 20) was substantially lower than that for levels 2 (around 31) and 3 (around 33). The percentage of the population at levels 4 or 5 (around 17) was lower than that for the other levels.

For the USA, the percentage of the population at level 1 (around 27) was substantially higher than that for New Zealand or Canada (around 19). The three countries had similar percentages at level 2, but at level 3 the USA had around 29 percent and Canada around 34.

The percentage of the population of USA at levels 4 or 5 (around 13) was lower than that of Canada (around 18) or New Zealand.

Document Literacy

Document Literacy was measured by both the IALS and ALL surveys and its distribution among the adult population of New Zealand, Canada and the USA is shown in the following figures.

As for numeracy, the first figure represents the distribution of document literacy scores using box-plots and the second figure shows the distributions of document literacy levels. Again, levels four and five are combined to give more robust statistical information.

Document literacy has increased in all countries and the subpopulations with low document literacy in each country have shrunk.

Document Literacy Score Distributions

For New Zealand, the 5th-to-95th percentile range for ALL (from around 185 to around 355) was narrower than that for IALS (from around 155 to around 360). Canada followed the same pattern (its 5th-to-95th percentile range for ALL extended from around 175 to around 365 and that for IALS from around 130 to around 380), as did the USA (its 5th-to-95th percentile range for ALL extended from around 175 to around 355 and that for IALS from around 125 to around 370).

The 25th-75th percentile range was also narrower for ALL than that for IALS. For New Zealand, there was an increase at both the 25th percentile (from around 235 to around 250) and the 75th percentile (from around 310 to around 315). For Canada, the 25th percentiles were similar (near 250) while performance at the 75th percentile decreased (from around 330 to around 320). For the USA, there was an increase at the 25th percentile (from around 230 to around 235) while performance at the 75th was decreased (from around 320 to around 310).

The change in the document literacy means of the ALL and IALS surveys was largest for New Zealand (moving from around 270 to around 280) than for Canada (which was stable at around 285) or the USA (which was stable at around 270).

Document Literacy Level Distributions

The decrease in the percentage of the population at level 1 was more substantial for New Zealand (from around 21 to around 14) than for Canada (from around 16 to around 15) or the USA (from around 24 to around 20).

There was a decrease in the percentage of the population at level 2 for New Zealand (from around 30 to around 29), an increase for Canada (from around 22 to around 25), and an increase for the USA (from around 25 to around 32).

There was an increase in the percentage of the population at level 3 for New Zealand and Canada (both from around 33 to around 38), and for the USA (from around 32 to around 33).

For New Zealand, the percentage of the population at level 4 or 5 increased (from around 17 to around 19), while that for Canada decreased (from around 29 to around 22), as did that for the USA (from 20 to 15).

Search