Literacy and numeracy at work

Publication Details

This report looks at the use of literacy and numeracy skills at work, and how this relates to the skills and education of employees. It uses data from the Adult Literacy and Lifeskills (ALL) survey to look at how well employees’ skills match the literacy and numeracy practices that they undertake at work. It looks at how skills and education relate to different sets of practices, such as financial literacy and numeracy. It also identifies which groups of employees are more likely to have a skills shortfall or skills excess, and some of the barriers to further training for those with a skills shortfall.

Author(s): David Earle, Tertiary Sector Performance Analysis and Reporting, Ministry of Education.

Date Published: February 2011

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.

Appendix B: Modelling the job-practice scores

The data

The ALL survey included a set of questions about reading, writing and mathematics activities in the main job of the respondent. These include activities done on paper and on computer. The questions ask how often the activity was undertaken, with response options being "at least once a week", "less than once a week", "rarely" or "never". The activities listed were:

Read or use information from:

  • Letters, memos or e-mails
  • Reports, articles, magazines or journals
  • Manuals or reference books including catalogues
  • Diagrams or schematics
  • Directions or instructions
  • Bills, invoices, spreadsheets or budget tables

Write or fill in:

  • Letters, memos or e-mails
  • Reports, articles, magazines or journals
  • Manuals or reference books including catalogues 
  • Directions or instructions
  • Bills, invoices, spreadsheets or budget tables

Do any of the following:

  • Measure or estimate the size or weight of objects
  • Calculate prices, costs or budgets
  • Count or read numbers to keep track of things
  • Manage time or prepare timetables
  • Give or follow directions or use maps or street directions
  • Use statistical data to reach conclusions

The data was recorded in a four point scale. For analysis, 4 represented "at least once a week" and 1 represented "Never". Table 1 shows the descriptive statistics for the responses. Responses to most of the questions are heavily skewed towards the top end, that is "at least once a week". Most of them had quite large values for kurtosis, an indication that responses fail to fit a normal distribution.

Table 1: Descriptive statistics of variables
Note:
  1. Std*=Standard.
Variable Mean Std* Deviation Skewness Kurtosis
Read letters etc 3.35206 1.13416 -1.33401 0.00435
Read reports etc 3.16920 1.17773 -0.95786 -0.77241
Read manuals etc 3.11820 1.14654 -0.85687 -0.85164
Read diagrams 2.66186 1.25224 -0.18175 -1.61440
Read directions 3.30999 1.03349 -1.17956 -0.07970
Read bills & spreadsheets etc 2.76952 1.32003 -0.35267 -1.65455
Write letters etc 3.08319 1.26009 -0.82583 -1.11362
Write reports etc 2.62528 1.27366 -0.15527 -1.65478
Write manuals etc 2.09910 1.16009 0.56567 -1.17998
Write directions 2.76178 1.23307 -0.31246 -1.53208
Write bills & spreadsheets etc 2.52682 1.32531 -0.03374 -1.75814
Measure and estimate 2.59813 1.31980 -0.09349 -1.74761
Calculate prices 2.65913 1.31075 -0.20390 -1.70362
Count or read numbers 3.54200 0.94340 -1.88943 2.06530
Manage time 3.14714 1.21859 -0.92983 -0.89326
Give or follow directions or maps 2.75068 1.22730 -0.29878 -1.52689
Use statistical data 2.36445 1.23038 0.18036 -1.56747

Model development

The factor model was developed in two stages: an exploratory factor analysis to explore and model the data and a confirmatory factor analysis to validate the models and generate the scores.

Both the exploratory factor analysis and the confirmatory factor analysis used the full data set of 5939 records. There is a view that the data set should be divided, with each procedure run on separate subsets. This was not done for two reasons. The data set is very large, so the possibility of model misspecification through random chance is low. The complex survey design used in the survey can introduce unknown biases into subsamples.

In both stages, the analysis was undertaken from the polychoric correlation matrix. The polychoric correlations are more reliable with scale data that has less than 5 values.

Exploratory factor analysis

An exploratory factor analysis was undertaken (using PROC TCALIS in SAS). All 17 variables were used and solutions involving one through to four factors were tested for fit and interpretability. The four factor solution was chosen as providing good fit and interpretable factors. While the fit statistics continued to improve with more than four factors, each factor became more narrowly focused on specific questions, which decreased the usefulness of this approach as a data reduction technique.

Table 2: Measures of model fit for exploratory factor analysis
Note:
  1. Std*=Standard.
Statistic Desired
Level
One
Factor
Two
Factors
Three
Factors
Four
Factors
Standardised Root  Mean Square Residual <0.05 0.0873 0.0652 0.0333 0.0273
Root Mean-Square  Error of Approximation <0.05 0.1892 0.1562 0.1313 0.1156



The factors were extracted using maximum likelihood estimation obliquely rotated using the oblique varimax option.

Table 3 shows the factor loadings for the initial four factor solution. The highest loadings for each factor are highlighted in bold. Factor 1 relates to financial tasks, factor 2 to reading a range of material and writing letters and emails, factor 3 to writing substantive documents and factor 4 to practical literacy and numeracy tasks.

Table 3: Rotated factor loadings for initial exploratory factor analysis
Variable Factor 1 Factor 2 Factor 3 Factor 4
Read letters, emails etc 0.1115 0.9827 -0.0933 -0.0194
Read reports etc 0.0417 0.5989 0.3181 0.0269
Read manuals etc 0.0051 0.3860 0.4526 0.1756
Read diagrams -0.0027 0.2358 0.4353 0.2934
Read directions -0.1250 0.2378 0.4004 0.3494
Read bills and spreadsheets etc 0.8135 0.1680 0.0458 -0.0125
Write letters, emails etc 0.2829 0.6530 0.1681 -0.1303
Write reports etc 0.1121 0.2258 0.6811 -0.1027
Write manuals etc 0.2210 -0.0097 0.7210 0.0130
Write directions 0.1318 0.1171 0.5676 0.1476
Write bills and spreadsheets etc 0.9380 -0.0059 0.1373 -0.0697
Measure and estimate -0.0101 -0.1283 -0.0051 0.7354
Calculate prices 0.6623 0.0725 -0.0672 0.2902
Count or read numbers 0.2911 0.1602 -0.0844 0.6232
Manage time* 0.2156 0.1774 0.2432 0.3041
Give or follow directions or maps* 0.0587 0.1833 0.1631 0.3946
Use statistical data* 0.2149 0.2556 0.3046 0.2237


The last three items had low loading across all four factors, without much differentiation across factors. The factor model was rerun without these items. The results showed little change to the loadings of the remaining items on the factors. This suggests that these items are not essential to the formation of the factors.

Table 4: Rotated factor patterns for final exploratory factor analysis
   Variable Factor 1 Factor 2 Factor 3 Factor 4
Read letters, emails etc 0.1099 0.9686 -0.0761 -0.0007
Read reports etc 0.0420 0.5924 0.3231 0.0517
Read manuals etc 0.0040 0.3809 0.4614 0.2093
Read diagrams 0.0016 0.2422 0.4324 0.3090
Read directions -0.1171 0.2379 0.4093 0.3535
Read bills and spreadsheets etc 0.7962 0.1762 0.0609 0.0154
Write letters, emails etc 0.2822 0.6484 0.1772 -0.1250
Write reports etc 0.1172 0.2306 0.6721 -0.0961
Write manuals etc 0.2201 -0.0044 0.7199 0.0362
Write directions 0.1366 0.1239 0.5695 0.1498
Write bills and spreadsheets etc 0.9209 0.0063 0.1499 -0.0458
Measure and estimate -0.0105 -0.1152 0.0040 0.7470
Calculate prices 0.6499 0.0871 -0.0466 0.2911
Count or read numbers 0.2904 0.1746 -0.0582 0.5970


The covariance matrix for the final exploratory factor model is shows that there is low correlation between these factors, which suggests they have a good degree of discriminant validity.

Table 5: Rotated Factor Covariance Matrix

Factor 1 Factor 2 Factor 3 Factor 4
Factor 1 1.0000


Factor 2 0.5262 1.0000

Factor 3 0.3452 0.6462 1.0000
Factor 4 0.2895 0.2822 0.3441 1.0000

Confirmatory factor analysis

The exploratory factor analysis results were used to build a confirmatory model. In the first model, the variables with the highest loadings in the exploratory model were loaded to each factor, so as no variable was loaded to more than one factor. The four factors were covaried with each other. This produced a model that had moderately poor fit with the data and high correlations between factors (greater than 0.7).

The factor modification indices were then used to select variables to load onto more than one factor. This was done incrementally and resulted in small improvements to model fit. As it proceeded, correlations between most factors decreased to below 0.7, with the exception of factors 2 and 3, which increased to 0.9. 

The next stage was to treat factors 2 and 3 as one factor describing both reading and writing activities. The first version of the model used one to one matches only, collapsing the variables for factors 2 and 3 onto a single factor.  The factor modification indices were then used to select variables to load onto more than one factor.  The selection was also informed by the relationship of the variable to the factor. This process resulted in the three factor model displayed below. The factors have moderate correlations with one another. The final model has moderate fit with the data (SRMSR=0.0695, RMSEA=0.0797).

Table 6: Factor loadings for final confirmatory factor analysis model
Variable Factor 1
Financial
Factor 2
Intensive Literacy
Factor 3 Practical
Literacy  & Numeracy
Read letters, emails etc 0 0.8981 0
Read reports etc 0 0.8793 0
Read manuals etc 0 0.5567 0.3744
Read diagrams 0 0.2578 0.6137
Read directions 0 0 0.8270
Read bills & spreadsheets etc 0.9597 0 0
Write letters, emails etc 0 0.8968 0
Write reports etc 0 0.8148 0
Write manuals etc 0 0.7291 0
Write directions 0 0.4076 0.4396
Write bills & spreadsheets etc 0.9396 0 0
Measure and estimate 0 -0.3126 0.6583
Calculate prices 0.7685 0 0
Count or read numbers 0.3865 0 0.3892
Table 7: Rotated Factor Covariance Matrix
Variable Financial Intensive Practical
Financial 1.0000

Intensive 0.7440 1.0000
Practical 0.4567 0.6741 1.0000



The data was scored against the factor model to generate the practice scores. The scoring was done on the unstandardised values in the data. This produced scores in the range from 1.0 to 4.5. 1.0 can be read as meaning never undertaking the set of literacy and numeracy practices at work and 4.5 as undertaking all of the practices in the set at least once a week.

Footnote

  1. Residual indexes were used to test fit. The chi-square statistic was significant in all cases, reflecting the large size of the dataset.