Youth Training - Statistical Profile 1999 to 2008

10. Outcome modelling

In previous sections we have shown the observed labour market outcomes of Youth Training. However, these observations are not likely to show the full picture of what happens for each learner type because of the cumulative effects of different variables. This is where statistical modelling becomes useful. It enables us to determine the strength of the relationship between individual variables and the labour market outcome while controlling for all the other variables.

Using multinomial logit analysis, it is possible to calculate the contribution of each variable value with respect to the labour market outcomes recorded at the end of each placement. The dependent variable in this case is the category of outcome two months after the end of each Youth Training placement, and the independent or explanatory variables are those already described in preceding sections.

Logits were calculated for all variables in the model with respect to their contribution to a negative (an other) output over other outcomes. This makes it possible to calculate an odds ratio for that level of the variable for each possible output. For example, for the year variable, the logit shows how much more or less likely learners leaving placements in one year are likely to attain one type of labour market outcome compared with the other outcome.

The advantage of this approach is that it enables control of all the other variables within the model so that the contribution of each variable to the attainment of each labour market outcome can be more accurately assessed.

Simply put, we can show how powerful each variable is in predicting outcomes. We can also calculate the odds³³ of a positive outcome over a negative outcome for any single variable value. For example, we can assess the likelihood of an outcome for a person aged 16 years, controlling for the fact that they are also male, European, and for their previous qualifications and employment experience. This is important because any number of possible combinations of these variables are possible for a person, and various interactions between them may change the final result. For instance young learners who are males in Auckland may have different outcomes from young people who are female in the Southern region. If we calculate the odds for each iteration of each variable, while controlling for all other possible iterations of other variables, we can be more sure of what’s driving the attainment than by just looking at the observed results.

The model is limited by the data that is available to it, so its explanatory power may be relatively low.³⁴ However, this is common where analyses of education programmes are concerned. It is simply not possible to account for every variable that may have an effect on the outcome of Youth Training. The administrative dataset is a rich source of information, and by performing this analysis we are able to show with some confidence what some of the more powerful predictors of each outcome may be.

It is also not possible to include all of the administrative variables within the model, primarily because of collinearity issues. For example, trainee weeks and credit attainment are highly correlated, as are learner age and educational attainment. Where collinearity is indicated, one or more correlated variables are excluded to increase the precision of model estimates.

10.1 Summary of effects

Table 48 shows the summary of the model. The variables are ranked in the order of the amount of variation in the model each accounts for. The closer the variable is to the top of the list, the more important it is in terms of explaining how the dependent variable relates to the independent variable. In this case, the most important predictor is the number of credits attained variable, which explained 20 percent of the variance explained by the model. The second largest effect was Region, which explained 15 percent. The employment history of the learner accounted for 11 percent of the observed variance, while the ethnicity of the learner was the next most powerful predictor.

The model suggests that the number of credits attained in each placement, regional conditions / differences, ethnic group of learner and order of placement have the most influence on their success after leaving a placement. The employment and education history of the learner before entering Youth Training is also important. The field of study of the course, gender and the year have lesser but still important explanatory effects. In fact, all of the variables tabulated show statistical significance; that is they are shown to have an influence on outcomes. However, some (those closer to the top of the table) are more influential than others.

The tables in the remainder of this chapter compare the likelihood of different outcomes from Youth Training. They do this by comparing the probability of an outcome category with the probability of the outcome being unemployment/out of the labour force (which is referred to as an other outcome). This gives an ‘odds ratio’. If the odds ratio of an outcome category is greater than 1, this means that the outcome is more likely than unemployment/out of the labour force. If the odds ratio of an outcome is less than 1, then the outcome is less likely than unemployment/out of the labour force. These odds ratio apply to the average learner rather than to any particular reference category of learner.³⁵

10.2 Credit attainment

The number of credits attained in each placement was the largest predictor of programme outcome. The more credits learners attain, the greater propensity they have to move into employment or further training.

Table 49 shows that for learners who attained no credits, or 10 or fewer credits, an other outcome (i.e. unemployment/out of the labour force) was the more likely to occur of any of the other outcome categories.

For learners gaining 11 to 20 credits, the odds of any outcome were even. There was no statistical significance between any of the categories and the other outcome category, apart from further progressive training, which was less likely to occur.

Learners who gained between 21 and 30 credits were most likely to return to Youth Training, followed by employment and then by further progressive training (in order of odds of their occurring).

Learners attaining 31 or more credits were most likely to engage in further progressive training; be in employment, or return to Youth Training two months after leaving a placement.

Figure 28 – Odds ratio of labour market outcome to other outcome category by credits attained on placement

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.3 Region

Learners differed by geographical location in the likelihood of attaining each labour market outcome. An other outcome (i.e. unemployment/out of the labour force) is the most likely outcome in the Northland and Wellington regions. A return to Youth Training is most likely in Auckland where further progressive training and other outcomes are just as likely to occur.

Learners in the Bay of Plenty, Eastern Coast, Nelson-Marlborough-West Coast regions are most likely to be employed two months after leaving Youth Training. Learners in the Southern and Auckland Regions are most likely to return to Youth Training directly after leaving a placement while learners leaving placements in the Central and Waikato regions are the only ones who are more likely to go on to further progressive training directly after leaving a Youth Training placement.

It is not entirely clear whether this variable is showing a geographic isolation effect, an administration effect, a regional labour market effect or something else entirely. Generally learners placed in programmes in non-metropolitan areas have a higher propensity to gain an employment outcome over an other outcome than those placed in metropolitan areas. Further progressive training outcomes also follow this pattern, except for learners placed in the south island. This points to an isolation effect, but one that operates differently for employment than for training outcomes.

It may be that Youth Training participants in more densely populated areas face stiffer competition for jobs than learners in less densely populated areas. South Island participants may also face less competition for placement in further training establishments such as polytechnics and institutes of technology than learners in the more densely populated north island. Regional effects may also reflect the different concentrations of various industries within them, and differing work patterns. For example, some regions have high concentrations of agricultural and horticultural industry concentration, and employment in those regions may be more casually and seasonally-based as a consequence.

There is bound to have been some difference in the strength of the youth labour market between regions across the time period, and this is also likely to have contributed to the variance explained by this variable. It would be worthwhile modelling regional youth unemployment rates by year as a predictive factor to determine if this is the case.

It may also be that learners in metropolitan areas / certain regions have higher needs and therefore present more complex cases than those in less dense areas, and these needs are not necessarily correlated with other learner-related variables present in the model.

10.4 Employment history

The employment history of the learner prior to first entering Youth Training was the third most important predictor of labour market outcome, taking all of the other variables included in the model into account.

Table 51 shows the odds of an outcome over an other outcome for learners in each employment history category. For those who had never been in employment before first entering Youth Training, the odds of a return to Youth Training and further progressive training over an other outcome are higher than an employment outcome over an other outcome.

For those who have worked before entering (even part-time) the odds of employment after Youth Training are higher than for an other outcome. Learners with part-time work experience only are more likely to go into further progressive training than gain employment or return to Youth Training, but the odds of these are all greater than attaining an other outcome.

Employment is the most likely outcome only for those who have worked full-time prior to entering Youth Training, suggesting that their employment experience in conjunction with Youth Training that is most influential. It suggests that in the absence of work experience, Youth Training participation can only go some of the way to making a person employment ready.

Figure 29 – Odds ratio of labour market outcome to other outcome category by employment history of learner

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.5 Ethnicity

The ethnic group of the learner was the fourth largest predictor in the model of placement outcome. Employment is the most likely outcome for European learners, followed by a return to Youth Training. There is no statistical difference between the chances of going on to further progressive training and being unemployed two months after leaving Youth Training for European learners.

Māori and Pasifika learners are most likely to be unemployed on leaving over all other categories of outcomes (Pasifika are as likely to return to Youth Training as gaining an other outcome). Other learners have the highest odds of going on to further training, and are the only group where the odds of an other outcome are lower than all possible alternative categories.

Figure 30 – Odds ratio of labour market outcome to other outcome category by ethnic group of learner

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.6 Placement order

This variable simply shows the order of the placement for each learner, controlling for the other effects in the model. Overall, the odds of an employment outcome over an other outcome increase with the order of the placements until the learner reaches their third or more placement, when the odds of employment occurring begin to decline.

The odds of attaining a further training outcome decline when the learner leaves the second or further placement (but increase at the fourth or more), while the odds of a return to the programme consistently decline with each successive placement.

Learners leaving their first placement were most likely to attain a return to Youth Training outcome. They were less likely to attain an employment outcome than an other outcome and were just as likely to attain a further progressive outcome as an other outcome.

Figure 31 – Odds ratio of labour market outcome to other outcome category by placement order

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.7 Course major field

The major field of the course was the sixth largest predictor in the model of placement outcome. The two single largest fields represented in Youth Training are ‘Society and Culture’ and ‘Mixed field programmes’ (see table 27). Learners in courses in both of these fields are as likely to attain an other outcome than an employment outcome.

Employment is the most likely outcome where the majority of credits available are in applied vocational fields such as the ‘Agriculture Environmental and Related studies’ and ‘Engineering and Related Technologies’ fields. Further training is the most likely outcome of programmes in the ‘Education’ field.

Figure 32 – Odds ratio of labour market outcome to other outcome category by course major field

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.8 Education history

The education history of the learner prior to first entry into Youth Training was the seventh largest predictor in the model of placement outcome.

The odds of a positive outcome increase with the level of prior education. Unemployment or out of the labour force status (other outcome) is the most likely outcome for learners who had no or low qualifications on entering Youth Training.

Curiously, those with the highest level of education pre-entry into Youth Training have the highest odds of returning to the programme over an other outcome than any other group, while learners with no qualifications pre-entry are less likely to attain any other outcome than other.

Figure 33 – Odds ratio of labour market outcome to other outcome category by education history of learner

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.9 Gender

The learner’s gender was the seventh largest predictor in the model of placement outcome. Employment was the most likely outcome for males, followed by a return to Youth Training and further progressive training (this category may not be significantly higher than other).

Females were more likely to be unemployed than any other category of outcome, while males are more likely to attain an employment or return to Youth Training outcome.

10.10 Placement end year

The odds of attaining each placement outcome differs over calendar years. This is probably due to a number of unmeasured factors, such as a changing macro-economic environment from year to year, changing labour market conditions and the availability of other / further education options, and others.

The odds of an employment outcome over an other outcome gradually increased across years from 1999 so that by 2004 an employment outcome was more likely than an other outcome. The odds of a return to Youth Training are lower than the odds of an other outcome until 2007 when it becomes as likely.

Figure 35 shows the odds of each outcome over an other outcome by year, compared to the youth unemployment rate for low and no qualified fifteen to twenty-four year olds for each year.

The odds of an employment outcome increase inversely with decreases in the youth unemployment rate, and vice versa. The difference between the two is almost symmetrical: in the years where an employment outcome is more likely than an other outcome, the youth no / low qualified unemployment rate is below 15 percent.

This symmetrical relationship reinforces the finding that an employment outcome after a Youth Training placement is quite dependent on the prevailing labour market conditions at the time.

Figure 34 – Odds ratio of labour market outcome to other outcome category by year

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

10.11 Training provider type

The type of training provider was the tenth largest predictor in the model of placement outcome. For learners placed with charitable trusts, the most likely outcome is a return to Youth Training while learners placed with tertiary education institutions are most likely to go on to further progressive training.

This may reflect the differing trainee selection effects of each provider type, driven by their primary motivations for offering training. Charitable trusts for instance may choose people they perceive to be the most ‘in need’ to participate in Youth Training out of a conviction to assist the neediest members of society, and as such, may not attain as ‘good’ outcomes as frequently as other provider types. Private training establishments may be less inclined to do this as they operate as primarily as profit-making businesses. They may try to ensure they to fulfil outcome targets set on them by the TEC by choosing learners who are less ‘in-need’ to participate in their programmes. There are competing influences at work for them: they may want to assist those perceived to be most in need, but incentives built into the funding system may lead them to choose participants showing less complex learning requirements.

Learners placed with employers are most likely to have an other outcome, while for learners placed with private training establishments a return to Youth Training is the most likely outcome, followed by an employment outcome. Learners placed with tertiary education institutions are most likely to go on to further training over any other form of outcome.

Figure 35 – Odds ratio of labour market outcome to other outcome category by training provider type

Note: points above 1 indicate more likely to occur than an other outcome, while points below 1 indicate the outcome is less likely to occur than an other outcome.

Footnotes

33. In contrast, if the predicted probability were calculated, it would apply to a chosen reference category of learner.
34. A pseudo R Square statistic for the model was not produced.
35. See Allison, 1999 for a description of the outputs of generalised logit models.