Skills and education: How well do educational qualifications measure skills?

4. Skills and education

This first section looks at the strength of the relationship between qualifications and skills and attempts to estimate the size of the groups for which the ‘high education/high skills’ and ‘low education/low skills’ link does not hold.

4.1: Relationships between skill domains

The ALL survey measured literacy and numeracy skills across four domains. These domains are closely related, so that the four skill measures were highly correlated.

Table 1 shows the correlations between scores. It shows that the strongest correlation was between document literacy and prose literacy. The weakest correlations were between prose literacy and numeracy and between numeracy and problem solving.

Source: New Zealand results from the Adult Literacy and Life Skills Survey,Ministry of Education calculations

Similarly, Earle (2009b) shows that, for people in employment, prose literacy and document literacy tended to be closely related. Numeracy and prose literacy, and numeracy and problem solving, were the least closely related skills. For that reason, we follow Earle in focusing in this analysis on prose literacy and on numeracy.

4.2: Skills and highest qualification

In their analysis of educational participation and literacy, Satherley, Lawes and Sok (2008b) show that participation in education at higher levels tends to be associated with higher levels of prose literacy skill. This finding was derived from ALL data and also confirmed an earlier international study of literacy, the International Adult Literacy Survey, conducted in New Zealand in 1996. Earle (2009a) extended this finding by using ALL data to look at educational qualifications in more detail. Earle concluded that, overall, both literacy and numeracy levels are related to educational qualifications but that there are significant numbers for whom the linkage does not hold. In his subsequent ALL study, Earle (2009b) concludes:

for people in employment, there was a moderate but definite relationship between skill level and qualification level. The differences are more noticeable for people with lower levels of educational qualifications. The difference in literacy and numeracy skill between those with degrees and those with postgraduate qualifications was minimal.

The relationship between qualifications and skills is summarised in the graphs below.

Figure 1: Prose literacy and numeracy skill levels by highest qualification for total population aged 16 to 65 years

Source: New Zealand results of the Adult Literacy and Life Skills Survey. Ministry of Education calculations

Note: For classification of qualifications see Appendix A. The boxes indicate the range from the 25th to 75th percentiles. The line indicates the median. Bars indicate the range from the 5th to 95th percentile.

4.3: People with high qualifications and low literacy and numeracy skills

While Figure 1 shows a clear relationship between skills and qualifications at a population level, there are significant numbers for whom it breaks down. Figure 2 below shows the highest educational achievement of those with prose literacy at levels 1 or 2 in the ALL survey¹ – generally interpreted as being below what is needed for dealing successfully with life and work in a complex, advanced society (OECD and Statistics Canada, 2000). Roughly one in five holders of a bachelors degree or higher qualification has level 1 or 2 prose literacy, while more than a quarter of those who are tertiary qualified – at any level above level 3 on New Zealand’s qualifications register² – have low prose literacy.

Figure 2: People with low prose literacy (ALL levels 1 or 2) by highest level of education completed

Source: New Zealand results of the Adult Literacy and Life Skills Survey. Ministry of Education calculations

Note: For an explanation of the categories see Appendix A. The bars represent margins of error (at the 95% confidence level). The term ‘tertiary’ education means education that is above level 3 on the NZ Register of Quality Assured Qualifications

Likewise about a quarter of holders of bachelors degrees or higher qualifications had low (levels 1 or 2) numeracy, with a third of those with a tertiary-level qualification having low numeracy.

Figure 3: People with low numeracy (ALL levels 1 or 2) by highest level of education completed

Source: New Zealand results of the Adult Literacy and Life Skills Survey. Ministry of Education calculations

Note: For an explanation of the categories see Appendix A. The bars represent margins of error (at the 95% confidence level). The term ‘tertiary’ education means education that is above level 3 on the NZ Register of Quality Assured Qualifications

Applying these findings across the New Zealand population, we estimate that between 111,000 and 145,000 New Zealanders between the ages of 16 and 65 hold a degree or higher (out of 600,000 degree holders) but have low prose literacy. With numeracy, the estimate of the corresponding number is between 130,000 and 166,000. The overlap between these two groups -- that is, the number with a degree or higher but low prose literacy and low numeracy -- is between 76,000 and 106,000 (out of a total group of between 936,000 and 1,026,000 with low prose literacy and low numeracy). Taking the midpoints of these ranges, 71 percent of those with a degree or higher with low prose literacy also have low numeracy, while 62 percent of those with low numeracy also have low prose literacy.
 

4.4: People with low qualifications and high literacy and numeracy skills

The relationship between qualifications and skills is clearer for those with low qualifications. The numbers who had no qualification beyond year 11 at secondary school but whose prose literacy was at levels 4 or 5 in ALL were too small for analysis. Only about 8 percent of those who had no qualification beyond secondary school level³ had a prose literacy level of 4 or 5. In fact, only 15 percent of the entire ALL sample had prose literacy at level 4 or 5 (Satherley, Lawes and Sok, 2008a). This means that it is difficult to conduct any meaningful analysis of the people with ALL skill levels 4 and 5 who have low qualifications. Therefore, in this analysis, we take the low qualifications/high skills group to be those who had no qualification beyond school level but whose skills were measured as being in the top quartile⁴ in the ALL survey.

Figure 4: Prose literacy quartiles for people with lower educational achievement – year 13 at secondary school or tertiary qualifications at level 3 or lower

Source: New Zealand results of the Adult Literacy and Life Skills Survey. Ministry of Education calculations
Note: The bars represent the margins of error (at the 95% confidence level).

Around 15 percent of those with low educational qualifications have prose literacy in the top quartile. A similar result is found with numeracy, as shown in Figure 5.

Figure 5: Numeracy quartiles for people with lower educational achievement – year 13 at secondary school or tertiary qualifications at level 3 or lower

Source: New Zealand results of the Adult Literacy and Life Skills Survey. Ministry of Education calculations

Note: The bars represent the margins of error (at the 95% confidence level).

The ALL results suggest that something between 196,000 and 248,000 New Zealanders aged between 16 and 65 years have no qualification beyond school level (out of a total group of 1,489,000 with no qualifications beyond school level) but have high prose literacy, while the corresponding figure for numeracy is between 186,000 and 249,000. The overlap between these two groups, that is, the number with low qualifications but high prose literacy and high numeracy, is between 106,000 and 149,000 (out of a total group of 400,000 to 477,000 with high prose literacy and high numeracy). Taking the midpoints of these ranges, 58 percent of those with low qualifications but with high prose literacy also have high numeracy, while 59 percent of those with high numeracy also have high prose literacy.
 

Footnotes

1. The levels referred to here and subsequently in this report are ‘cognitive levels’ assigned on the basis of scores in the ALL tests. These cognitive levels are used in national and international comparison, essentially as a benchmark (Satherley and Lawes, 2007). Refer to appendix A for descriptors of the levels.
   
2. The term ‘tertiary-level qualification’ in this report refers to a post-school qualification that appears at Level 4 or higher on the New Zealand Register of Quality Assured Qualifications. Some post-school  qualifications are rated on this register as at levels 1-3 and hence, are at the level of a secondary school qualification – even though they are taught outside of schools. Refer to Appendix A for a more detailed explanation.
3. The people in this group may have had a post-school qualification but, if so, it was no higher than level 3 – which is the level of attainment of someone who had completed a high school qualification such as NCEA level 3.
   
4. The top quartile means the score that separates the highest quarter of the population from the rest.