Skills and education: How well do educational qualifications measure skills?

2. Executive summary

2.1: The focus of the study

This study uses the data from the Adult Literacy and Life Skills (ALL) survey to look at the relationship between New Zealanders’ educational qualifications and their literacy and numeracy skills. It builds on earlier explorations of the data (Earle, 2009a; Satherley, Lawes and Sok, 2008b).

It is widely assumed that, on average, people with higher qualifications tend to have higher literacy and numeracy skills. And conversely, those with higher skills are assumed to have gained higher qualifications. In part, these assumptions reflect the fact that one of the principal purposes of education is the development of skills. Information about skills in a population or in the labour market is difficult to find, while information on qualifications is reasonably readily available. As a result, in much research and analysis, qualifications are often used as an index of a population’s skill level.

However, there are significant numbers of people whose skill levels do not match their qualifications – some have high levels of education but lower skills or conversely, some may have high skills without having obtained higher educational qualifications. This study looks into the groups of people for whom skills and qualifications don’t match.

It addresses the following questions:

• How many people have degree-level qualifications but low skills in numeracy and literacy?
• What are the demographic characteristics of that group? What is their employment status and occupation? What is their immigration status? What is their first language?
• How many people with low qualifications have higher-level skills?

The findings of this study are presented in three sections. The first looks at the overall relationship between skill levels – as measured in the ALL survey – and the reported level of qualifications held. The second focuses on people who have degree qualifications or higher but who have literacy and/or numeracy at the lower levels on the ALL scale. The third looks at people without any qualification beyond secondary school level but who have high skills in literacy and/or numeracy.

2.2: Skills and education – the overall relationship

The ALL survey measured English-language based skills across four domains: prose literacy, document literacy, numeracy and problem solving.

Previous analysis of the ALL results shows that there is a moderate but definite relationship between skill levels in each of these domains and qualification level (Earle, 2009a; Satherley, Lawes and Sok, 2008b). The difference in score between those with no qualification and those with a bachelors degree represented just over one standard deviation of the score distribution for each domain. But there are groups for whom that relationship doesn’t hold. In his study of this matter, Earle (2009a) states:

While overall, people with no qualifications are likely to have low literacy and numeracy, there are a good number of people with no qualifications who have good literacy and numeracy. Similarly, there are some people who have bachelors degrees or higher who have low literacy and numeracy. This means that qualifications should only not be used on their own to determine the literacy or numeracy ability of an individual [but] qualifications do provide a limited means of determining the literacy and numeracy abilities of groups

The analysis in this report confirms this finding. It shows that people with higher qualifications are significantly less likely to have low literacy and numeracy skills. People whose schooling finished at year 10 or year 11 have much the greatest probability of having low skills. But there are significant numbers of people with high qualifications and low skills and with high skills and low qualifications.

2.3: High qualifications and low literacy and numeracy skills

The ALL dataset suggests that between 76,000 and 106,000 (of the 600,000) New Zealanders who have a degree or higher qualification have low English literacy and numeracy (out of a total of between 936,000 and 1,026,000 with low literacy and numeracy overall). Our analysis suggests that:

• The degree holders with low literacy are more likely to be in younger age groups, but this does not hold for numeracy.
• While female degree holders appear somewhat more likely to have low numeracy than males, the difference isn’t statistically significant.
• People with degrees who work in professional occupations are less likely to have low literacy – which may reflect the fact that they are more likely to have to ‘practise’ their skills but may equally be a ‘selection’ effect in that people without high literacy may not win professional jobs, despite holding a degree.
• While professionals and technicians with degrees are somewhat less likely than people in other occupational groups to have low numeracy, the difference is not statistically significant.
• People with degrees who were born in New Zealand were much less likely to have low literacy or numeracy than those born elsewhere. Established immigrants are less likely to have low literacy than recent immigrants, but that doesn’t apply to numeracy where the difference is not statistically significant.
• The main factor associated with low literacy and numeracy skills among degree holders is first language. Degree holders for whom English is a first language are much less likely to have low literacy and numeracy than others. Among people with degrees whose first language is an Asian language, there is significantly higher prevalence of low literacy than among those whose first language is neither English, nor an Asian language.
• About two-thirds of degree holders with low English prose literacy have a first language other than English, while about half of degree holders with low numeracy have a first language other than English.
• The importance of first language as a marker of people with high qualifications and low skills reflects the fact that ALL – across all four of its domains – uses tests in English to determine skills. And the New Zealand labour market and the wider society recognises skills in an English language context.
• The finding that a high proportion of those not born in New Zealand – especially immigrants from non-English speaking countries – are degree-qualified but have low English literacy and/or numeracy helps explain why many well-qualified migrants end up working in jobs unsuitable for their level of education, especially in their early years in New Zealand.
  

2.4: High literacy and numeracy skills and low qualifications

If we exclude students – who are working towards higher qualifications – between 83,000 and 119,000 New Zealanders with no qualifications beyond the level of senior secondary school have literacy and numeracy that is in the highest quartile of the New Zealand population. (There is a total group of 1,489,000 with no qualifications beyond the level of secondary school and there are between 400,000 and 477,000 New Zealanders with highest-quartile literacy and numeracy overall). Our analysis of the skills of those who are not students but who have no qualifications beyond the level of senior secondary school suggests:

• People with low qualifications who are in employment have a significantly higher prevalence of high literacy and high numeracy than those who have any other labour force status. This finding may reflect the fact that those in work are more likely to ‘practise’ their skills but may equally be a ‘selection’ effect in that people without high skills may be unable to sustain a job.
• People who were clerks and managers who had low qualifications were more likely to have high literacy and numeracy than those in most other occupations. This, too, may be a practice effect or a selection effect or a bit of both.
• People whose schooling finished at year 11 or earlier are much less likely to have high skills than people whose highest qualification was years 12 or 13 at school or who completed a level 1-3 tertiary education qualification. Those whose highest qualification was years 12 or 13 at secondary school were somewhat more likely to have higher literacy and numeracy than people whose highest qualification was a tertiary level 1-3 certificate, but the difference was not statistically significant.
• In the low qualification population as we have defined it, men were somewhat more likely than women to have high numeracy and women more likely to have high literacy, but the differences were not statistically significant.
• Virtually all (97 to 99 percent) of non-students with low qualifications but high literacy or numeracy had English as a first language, and 87 to 88 percent were born in New Zealand.
• People born in New Zealand who had no qualifications beyond the level of senior secondary school were more likely than those born overseas to have higher literacy and numeracy. This reflects the fact that ALL tested skills in the English language and immigrants have a high prevalence of people for whom English is an additional language.
• People in the 45-65 year age group were less likely than those in the 35-44 year age band to be in the high skill/low qualification group. The difference was statistically significant for numeracy but not for literacy. We verified that the same result holds for literacy in the International Adult Literacy Survey conducted in New Zealand a decade earlier.
• This last finding was perhaps surprising. Many would have expected that the high skill/low qualification population might largely comprise those who went into careers direct from school in an era when tertiary education was less of an option and when expectations of staying longer at school were lower. That this expectation is not borne out in the data may confirm that a tertiary education is an important means of facilitating the development of a person’s literacy skills.