1. Summary

Who participates in Youth Guarantee Fees-Free places?

Youth Guarantee Fees-Free places were established in 2010. The number of young people starting Fees-Free has increased each year, from 1,930 in 2010 to 9,000 in 2014.

Youth Training was amalgated with Youth Guarantee Fees-Free places in 2012, resulting in shifts in the types of providers and programmes. Following the amalgamation, q larger proportion of starters in Fees-Free Places were in private training establishments, at Level 2, and/or in non-trades programmes, as well as a greater proportion of learners entering the programme with a high risk of poor outcomes. The proportion of Māori starters increased from 35 to 50%.

Slightly more males than females started in Fees-Free places each year. The proportion of Pasifika starters has remained constant at around 20% each year. Around 60% of participants in each year had no school qualification prior to entry.

How effective are Youth Guarantee Fees-Free places?

Youth Guarantee Fees-Free places have been effective in keeping more young people in education during the period of the programme. The programme has also engaged some young people who would otherwise have been not in employment, education or training (NEET).

Fees-free participants were more likely to achieve NCEA Level 2 or equivalent than young people with a similar background. However, across all years, only around half of starters passed most of their courses, although the proportion failing most of their courses has decreased.

Fees-free participants were no more likely to progress to Level 4 and above tertiary education, or to gain full employment, than young people with a similar background. There is no evidence of any earnings premium from participation in the programme for those in employment.

Following the programme, participants were more likely to be NEET and/or receiving a benefit than young people with a similar background.

Who are Youth Guarantee Fees-Free places more effective for?

Analysis of employment and progression to Level 4 and above by groups of young people shows:

• Māori and Pasifika participants are no more likely to be in full employment or progress to higher level study than Māori and Pasifika with a similar background.
• Participants who had low or no school qualifications before they started, had lower education performance at school and/or had higher risk of poor outcomes were no more likely to progress to Level 4 and above or to full employment than similar young people.
• Participants who had a Level 2 qualification before they started, had higher education performance at school and/or had lower risk of poor outcomes were less likely to progress to Level 4 and above and more likely to progress to full employment than similar young people.

2. Youth Guarantee Fees-Free Places

Youth Guarantee Fees-Free places were implemented in 2010, as the first part of the Youth Guarantee policy. The Youth Guarantee policy provides opportunities for young people to achieve education success, and progress into further education, training and employment. It supports schools, tertiary education organisations and employers to work together.

Youth Guarantee Fees-Free provides full-time, fees-free tertiary study at New Zealand Qualifications Framework Levels 1 to 3 for 15 to 19 year olds who have left school with low or no qualifications. It is intended to re-engage young people in education and provide a pathway into further study, training and employment.

The programme was initially targeted to 16 to 17 year olds. The number of places and providers were expanded in 2012 by transferring funding from the Youth Training programme. The Youth Training programme had been established in 1999 and was aimed at providing training for employment and further education for 16 and 17 year olds who had become disengaged from education. The programme focused on providing short foundation skills courses. By moving the funding to Youth Guarantee Fees-Free places, a greater emphasis was put on providing programmes aimed at improving education outcomes, and pathways towards higher-level qualifications.

From 2012, 15 year olds with an early leaving exemption were eligible to participate in Youth Guarantee Fees-Free places. From 2014, the age eligibility was further widened to include 18 and 19 year olds.

Youth Guarantee Fees-Free places have been implemented in different ways by different providers. Some providers offer dedicated and tailored programmes for funded learners, others offer places within existing programmes, and many offer a mix of both. The results in this report show the average outcomes across all of the providers and programmes.

3. Monitoring and Evaluation

The purpose of monitoring and evaluating the Youth Guarantee programmes is to understand the extent to which the desired outcomes are being met.

The education outcomes of interest are:

• improved retention in school and/or tertiary education (including industry training)
• more students achieving NCEA Level 2, or equivalent
• increased progression to tertiary study at Level 4 or higher (including industry training).

The employment outcomes of interest are:

• obtaining sustained employment
• reduced incidence of not being in employment, education or training (NEET)
• reduced incidence of welfare benefit receipt.

This report includes results for young people who started the programme up to 2014. It uses data in the Statistics New Zealand Integrated Data Infrastructure (IDI). At the time of analysis, 2015 was the latest year for which most data was available. Employment data was available up to 2016, so an extra year is included for the employment indicator.

The outcomes are explored for the group of young people who started Youth Guarantee Fees-Free places in each year. We look at the effect of the programme for those who participated, compared with young people who had matching characteristics in the year prior to the programme but did not participate in Fees-Free places (the comparison group). The comparison group were similar to participants in terms of age, gender, ethnicity, education achievement and social background. Many of them also participated in school and/or other tertiary education settings. The matching was undertaken separately for each starting year within each programme, in order to control for changes in the composition of participants in the programmes over time. The participants and the comparison group had the same level of outcome on each measure in the year before the participants started the programme. A full description of the matching is set out in the appendix.

By comparing the outcomes of the participants to those of the comparison group, it is possible to estimate how much of the outcome was likely to be due to programme participation rather than the characteristics of the young people.The main area that we remain uncertain about is the effect of selection into the programme. For example, those participating in the programme may be more interested in gaining employment rather than going on to further study, than those in the comparison group. There is a possibility that further unmeasured factors (not available in the IDI) influenced both the choice to participate in a programme and the outcomes of participants. These factors could include family and peer influence, career and study preferences, as well as personality and other individual attributes. It is not possible to know if including these unmeasured factors would increase or reduce the apparent differences between the participants and the comparison group, or have no effect. However, given the wide range of variables used to match the individuals, we can be reasonably confident that the difference in outcomes is, to a large degree, related to the effect of programme participation.

4. Who Participates?

Age, gender and ethnicity

The number of people starting Youth Guarantee Fees-Free places has increased from 1,930 in 2010 to 9,000 in 2014. The majority of starters were aged 17 or 18 at the end of the year they started. In 2013 and 2014, around 5 to 6% of all 17 year olds in New Zealand and 4% of all 18 year olds started in Fees-Free places.From 2012, 15 year olds with an early leaving exemption were eligible to participate. Less than 100 have started in each year. From 2014, the age range for entry to the programme was extended to include 18 and 19 year olds. In that year, 22% of starters were aged over 18. Around 2% of 19 year olds and 1% of 20 year olds started in 2014¹.

Figure 1. Participation by age

Note: The height of the bars shows starters as a percentage of the population in each age group, and the labels show the number of starters.

In each year, there have been slightly more males than females starting. The proportion of female starters has increased from 41% in 2010 to 46% in 2014.

In 2010 and 2011, around 35% of starters were Māori. This increased to 50% in 2012, following the merging with Youth Training, and it reduced to 45% in 2014. In 2014, 10% of Māori 17 year olds and 7% of Māori 18 year olds started in Fees-Free places. This is nearly twice the rate of the total population. The proportion of starters who are Pasifika has been constant since 2010 at just under 20%. In 2014, 7% of Pasifika 17 year olds and 6% of 18 year olds started in Fees-Free places.

Figure 2. Proportion of age groups starting for Māori and Pasifika in 2014

Note: The height of the bars shows participants as a percentage of the population in each age group.

School achievement

From 2010 to 2014, around 60% of starters had no qualifications in the year prior to starting. In 2012 and 2013, 14% of starters had already achieved NCEA Level 2 and above. In 2014, 27% of starters had already achieved NCEA Level 2 and above and 7% had already achieved NCEA Level 3.

Figure 3. Distribution of starters by school-level qualification

Note: The height of the bars shows the proportion of participants with each level of qualification within each year. School-level qualifications include qualifications at the same level gained through tertiary education before starting.

Prior education performance provides a measure of how well students achieved in their Level 1 NCEA standards compared with all other students undertaking NCEA Level 1. Across all years, around 30 to 35% of starters were in the lowest 10% of performance and around 20% in the second lowest 10%. However, between 8 and 15% of starters in each year had above average performance (in the top 50% of all students).

Figure 4. Distribution of starters by prior education performance

Note: The height of the bars shows the proportion of starters with each decile of performance. The lowest 10% of all students by performance are represented by 1 and the highest 10% of all students by performance by 10.

Risk of poor outcomes

A risk score was calculated for each young person. The score was developed on the 1991 birth cohort and used the characteristics of the young people at age 15 to predict if they had had at least two years of being long-term NEET and not achieved NCEA Level 2 or above by age 20. The same weighting was then applied to subsequent birth cohorts. This provides a consistent risk rating of poor outcomes across birth cohorts.In 2010 and 2011, around 25% of starters were in the 10% of the population with the highest risk score. Following the amalgamation with Youth Training, over 40% of starters were in this highest risk group, reducing to 34% in 2014. In each year, a further 20% were in the second-highest risk group. However, in 2014 18% of starters were in the half of the population with lowest risk.

Figure 5. Distribution of starters by risk of poor outcomes

Note: The height of the bars shows the proportion of starters with each decile of risk. The 10% of the population with the highest risk is represented by 10 and the 10% of the population with the lowest risk by 1.

Provider and programme characteristics

This section looks at the characteristics of the providers and programmes that participants were enrolled in. It is based on the last Youth Guarantee Fees-Free programme that each person was enrolled in and is shown by starting year. The 2012 results are omitted due to inconsistencies around the data collection for that year.

The results contrast the composition of the programmes before the amalgamation with Youth Training (2010 and 2011) and afterwards (2013 and 2014). The graphs show the proportions by year. It should be kept in mind that the actual numbers of starters has increased substantially across years, so the numbers of students in each category may be increased even if the proportions have decreased.

Fees-free places are delivered by polytechnics, private training establishments and one wānanga. Figure 6 shows the distribution of participants between polytechnics and private training establishments. The wānanga has been included with the polytechnics due to confidentiality requirements of the IDI. In 2010 and 2011, more than two-thirds of starters were in polytechnics. In 2013 and 2014, over two-thirds of starters were in private training establishments. This is partly a result of the amalgamation with Youth Training, which was predominantly delivered through private training establishments.

In 2010 and 2011 nearly half of Fees-Free starters finished in a Level 3 programme. Some of these participants will have started in a lower level and moved to a higher level programme while in Fees-Free places. In 2013 and 2014, less than a third of starters finished in a Level 3 programme.

Programmes in Fees-Free places can be broadly grouped into trades, non-trades and mixed field programmes. Mixed field programmes cover employment skills. In 2010 and 2011, just under half of starters finished in trades programmes. In 2013 and 2014, less than a third of participants finished in trades programmes and around a third finished in mixed field programmes.

Across the period less than half of starters completed most or all of the courses they enrolled in. The proportion who failed most of their courses has decreased from 28% to 17%.

Nearly all Fees-Free participants were enrolled for at least half of an equivalent full-time year of study. Around half were enrolled for more than one full-time year. In 2013 and 2014 there was an increase in the proportion who were enrolled for more than one and a half years of equivalent full-time study, compared with 2010 and 2011.

Figure 6. Distribution of starters by provider and programme characteristics

5. Education Outcomes

Figure 7. Education outcomes by starting year

Note: Orange lines are the results for the participant group and grey lines are for the comparison group.

Youth Guarantee Fees-Free places were effective in keeping more young people in education during the period of the programme. Nearly all participants remained in education during their starting year, compared with only 70 to 80% of the comparison group. The size of this effect was largest in 2014, when the age range for Fees-Free places was increased. This result indicates that 20 to 30% of participants might not have been in education if Fees-Free places were not available to them.

Young people in Fees-Free places had higher achievement of NCEA Level 2 or equivalent after starting the programme than the comparison group. Across the years, the results indicate that around 15 to 20% of participants might not have gained NCEA Level 2 or equivalent if Fees-Free places were not available to them.

The NCEA Level 2 attainment rate, and differences between participants and the comparison group, dipped in 2012. This was a result of the merger with Youth Training. Both the rate and the difference has increased since then.Young people starting Fees-Free places in 2010 were more likely than the comparison group to enrol in tertiary education at Level 4 or above within two years of starting the programme. For 2011 starters there was no significant difference between participants and the comparison group. The rates for 2012 starters were not statistically significant at two years, and by three years the rates for participants were lower than for the comparison group. For 2013 starters, the rates were lower for participants after two years.

6. Destinations

Figure 8. Destinations by starting year

Note: Orange lines are the results for the participant group and grey lines are for the comparison group.

Young people who started in Youth Guarantee Fees-Free places in 2010 and 2011 were more likely than the comparison group to be in full employment two years after starting the programme. However, this result was not statistically significant for 2010 starters due to the smaller numbers. By four to five years after starting the programme there was no difference in employment rates for these starting cohorts.

Young people who started in Fees-Free places from 2012 to 2014 were no more likely to be in employment than the comparison group. By three years or more after the programme they were slightly less likely to be in employment.

Fees-Free places were successful in engaging a group of young people who might otherwise have been NEET during the period of the programme. During each starting year, around 15 to 20% of the comparison group were NEET. However, by two years after starting the programme participants were more likely to be NEET than the comparison group.

Young people in Fees-Free places were more likely than the comparison group to receive a welfare benefit during the starting year of the programme. The majority of the young people receiving benefits during the starting year, started in the year they turned 18 and therefore were entitled to a standard benefit. Young people who are receiving benefits can be required to attend training. This may explain the higher rate of benefit receipt for participants in the starting year. The increased proportion receiving benefits in 2014 reflects the increased age range of the programme.

In the years following the programme, a higher proportion of participants than the comparison group received a benefit. For the 2010 and 2011 starters there are signs of these rates converging after 3 or 4 years after starting the programme.

Figure 9 shows the average annual earnings from wages and salaries for people in Fees-Free places and for those in the comparison group. It is shown just for the years when the person was in employment and not in education, by the number of years in employment since the starting year.Participants in Fees-Free places had very similar starting earnings to the comparison group across the range of employment statuses. Their earnings continued to increase at the same rate as those of the comparison group. This suggests that Fees-Free places do not provide any earnings premiums for participants, compared to other people with similar characteristics.

Figure 9. Average real earnings by type of employment (in 2016 dollars)

Note: Shows earnings for people in employment and not in education. Time period refers to the number of years in employment, while not in education, since the starting year.

7. Who are Youth Guarantee Fees-Free places more effective for?

This section explores which groups of learners Youth Guarantee Fees-Free places are more effective for. The graphs show the proportions enrolled at Level 4 and above within two years of starting the programme, and the proportions in full employment two years later. These measures are not mutually exclusive.All results are based on those who started in 2012 and 2013. The rates have been derived from regression models. The graphs show the differences for each subgroup, when all other factors in the model are held constant. So they show if there is an advantage from the programme for that group having taken into account other differences associated with the group. Differences between groups are statistically significant when the error bars are not overlapping.

Student characteristics

In general, 17 and 18 year olds are more likely to have enrolled at Level 4 and above or be in full employment two years later than 16 year olds. There was no statistically significant difference in the proportion of participants and the comparison group enrolling at Level 4 and above by age group. However, young people starting the programme at age 16 were more likely to be in employment two years later than people of the same age in the comparison group.

Males and female participants had similar rates of enrolment at Level 4 and above. Male participants were slightly less likely to have enrolled at Level 4 and above than males in the comparison group. Male participants had much higher employment rates than female participants. The employment rates of male participants were slightly higher than those of males in the comparison group.

Māori participants were less likely to enrol at Level 4 and above than non-Māori participants, even after controlling for other factors. There was no significant difference between Māori participants and Māori in the comparison group on this measure. Māori participants were also less likely to be in employment two years later than non-Māori participants, even after controlling for other factors. There was no significant difference between Māori participants and Māori in the comparison group on this measure. However, non-Māori participants were more likely to be in employment than non-Māori in the comparison group.

There were no statistically significant differences in enrolment at Level 4 and above between Pasifika and non-Pasifika, or between Pasifika participants and Pasifika in the comparison group. Pasifika participants were less likely to be employed two years later than non-Pasifika participants, even after controlling for other factors. There was no significant difference between Pasifika participants and Pasifika in the comparison group on this measure.

Figure 10. Outcomes by age, gender and ethnic group

Note: Orange bars are the results for the participant group and grey bars for the comparison group. This figure shows the results of logistic regressions controlling for age, gender, Māori, Pasifika, prior education performance and risk of poor outcomes. Results are shown where other variables in the model are set their average or modal values. The error bars show the 90% confidence intervals.

School achievement

Figure 11 shows outcomes by the level of NCEA achieved prior to starting Fees-Free places. Those with higher levels of prior achievement were more likely to go into employment and/or enrol at Level 4 and above. However, participants with Level 2 prior to starting were less likely to enrol at Level 4 and above than similarly qualified people in the comparison group, and more likely to be in full employment.

Prior education performance shows how well young people did in their NCEA Level 1 assessment standards. Figure 12 shows that higher levels of prior education performance are associated with greater rates of enrolment at Level 4 and above, and lower rates of full employment, once other factors are controlled for. Participants with higher levels of prior education performance were less likely to enrol at Level 4 and above than people in the comparison group with the same level of performance. Participants with higher levels of prior education performance were more likely to be in full employment than people in the comparison group with the same level of performance.

Figure 11. Outcomes by NCEA attainment in year before starting

Figure 12. Outcomes by prior education performance

Note: Orange bars/lines are the results for the participant group and grey bars/lines for the comparison. Both figures show the results of logistic regressions controlling for age, gender, Māori, Pasifika, prior education performance and risk of poor outcomes. Results are shown where other variables in the model are set their average or modal values. The error bars show the 90% confidence intervals. Prior education performance is shown by deciles of the overall population. The graph shows the range that covers most of the participants in the programme. Those with the lowest performance are indicated by 1.

Risk of poor outcomes

A risk score was calculated for each young person based on their likelihood of having been NEET in at least 2 years and not achieving NCEA Level 2 by age 20. This score was calculated from factors not included in this model. Further information is available in the appendix.

Having a lower risk of poor outcomes is associated with higher rates of enrolment at Level 4 and above. Participants with a lower risk of poor outcomes were slightly less likely to be enrolled at Level 4 and above than those in the comparison group with similar levels of risk.

The relationship between the risk of poor outcomes and full employment is not as strong as for enrolment at Level 4 and above. Participants who had a lower risk of poor outcomes were more likely to be in employment two years later than people in the comparison group with the same risk level.

Figure 13. Outcomes by risk of poor outcomes

Note: Orange bars/lines are the results for the participant group and grey bars/lines for the comparison. This figure shows the results of logistic regressions controlling for age, gender, Māori, Pasifika, prior education performance and risk of poor outcomes. Results are shown where other variables in the model are set their average or modal values. The error bars show the 90% confidence intervals. Risk of poor outcomes is shown by deciles of the overall population. The graph shows the range that covers most of the participants in the programme. 10 indicates those with the highest performance

Provider and programme characteristics

The comparison group method cannot be used to provide valid measures of the different impacts by provider and programme characteristics. This is because it is not possible to associate participants to people in the comparison group who have the same education experience. The option of matching to similar students in other tertiary education was investigated. However, the majority of tertiary students at foundation level at ages 16 to 18 are in Youth Guarantee Fees-Free places and those who are not are doing quite different types of study.

In order to examine the effect of provider and programme characteristics for Fees-Free, regression models were run just for Fees-Free participants. The models compared the outcomes within Fees-Free, having controlled for the differences in background characteristics and the other provider and programme characteristics.

There was no statistically significant difference in enrolment or employment rates by provider type.

Students who finished in a Level 3 programme were more likely to enrol at Level 4 and above, and less likely to be in full employment two years later, than those who finished in a Level 1 programme.

There was no difference in enrolment rates at Level 4 and above across subject areas. Those who finished in a trades programme were more likely to be in full employment than other participants.

Participants who completed most of their courses were much more likely to enrol at Level 4 and above than other participants. There was no statistically significant difference in employment rates by course completion rate.

Employment and enrolment rates were similar for all participants who studied for up to one and a half equivalent full-time years. Those who studied for more than one and a half years had significantly lower employment rates and possibly lower enrolment rates. However, the reason that these students remained longer in Fees-Free may have been their difficulty in finding employment or lack of preparation for higher-level study.

Figure 14. Expected probability of progression to Level 4 and above and full employment for Youth Guarantee Fees-Free participants

Note: These are the results of logistic regression, controlling for age, gender, Māori, Pasifika, prior education performance, risk of poor outcomes and provider and programme characteristics. In each graph, the other variables in the model are set to their average or modal value. The error bars show the 90% confidence intervals.

Appendix: Methodology

Footnote

1. In this report, age refers to age at 31 December. So those who were 20 at the end of the year would have started while they were still 19.