Monitoring the Youth Guarantee policy 2013

Publication Details

This report focuses on the effectiveness of fees-free places and secondary-tertiary programmes at keeping young people in education, assisting them to attain NCEA Level 2 or equivalent and promoting higher level study in tertiary education. It also includes new information on employment and other destinations. It covers the period from 2010, when fees-free places were first set up, to 2013.

Author(s): David Earle, Ministry of Education.

Date Published: August 2015

Summary

The Youth Guarantee policy provides opportunities to support 16- and 17-year-olds to achieve education success, and progress into further education, training and employment. It supports schools, tertiary education organisations and employers to work together in new ways. From 2014, the age range was extended to include 18- and 19-year-olds.

There are a range of initiatives within the Youth Guarantee policy, including fees-free places at tertiary providers and secondary-tertiary programmes. The latter allow young people to remain enrolled at school, while participating in various forms of education delivered by tertiary education organisations. Since 2013, the Youth Guarantee initiatives have included Vocational Pathways as a framework to clarify the options for young people and identify the skills and knowledge valued by employers.

This report focuses on the effectiveness of fees-free places and secondary-tertiary programmes at keeping young people in education, assisting them to attain NCEA Level 2 or equivalent and promoting higher level study in tertiary education. It also includes new information on employment and other destinations. It covers the period from 2010, when fees-free places were first set up, to 2013.

Monitoring approach

The purpose of monitoring and evaluating the Youth Guarantee policy is to understand the extent to which the desired outcomes of the policy are being met. The outcomes are set out in the table below:

The first three outcomes are explored for fees-free places and secondary-tertiary programmes for the group of young people who started the programmes in each year. We look at the effect of each programme for those who participated, compared with young people with matching demographic and education characteristics who did not participate (the matched comparison).

By comparing the outcomes of the participants and matched young people, we can estimate how much the outcome is likely to be due to programme participation rather than the characteristics of the young people.

Initial results on the fourth outcome will be reported later in 2015. This will include early results from the implementation of Youth Guarantee networks and Vocational Pathways. These initiatives will start to have some effect from the 2014 academic year. In addition, this report contains new analysis on the destinations of young people following their participation in the programmes. This analysis used the Statistics New Zealand Integrated Data Infrastructure (IDI) to look at the proportions of young people in employment, not in employment, education or training (NEET) and receiving welfare benefits. The destinations of Youth Training participants are also included as these results have not been reported before and provide a comparison for the fees-free participants.

The comparison method used in the report makes use of educational factors only. It does not include other factors which have been shown to have a strong relationship to the probability of being NEET or on a benefit. These include family resources (including parental benefit receipt), peer influence and teen parenting. Nor does it take account of the fact that some people on a benefit are referred to educational programmes – including Youth Guarantee fees-free places. This factor may predispose the programme participants to be more likely to include beneficiaries. Therefore, it is possible that the method underestimates the programme effect with regard to reducing the proportion of young people who are NEET and/or receiving benefits.

Youth Guarantee is a fairly new policy. Youth Guarantee fees-free places were established in 2010 and extended further in 2012. Secondary-tertiary programmes were first established in 2011 and extended in 2012. The 2013 monitoring report provides information on programme effects for the period to 2013, by annual starting cohorts. At most, it provides information on effects three years after starting the programme. These findings need to be treated as indicative, early results, given the short time frame and the ongoing development of the programmes during this time period.

Participation in Youth Guarantee programmes

In 2012, fees-free places were expanded by transferring the funding from Youth Training. From 2012, the Youth Training fund ceased and most providers of Youth Training were funded to offer fees-frees places. In the following figures, Youth Training is included to provide a comparable view across the years.

In 2011, 10.6% of 18-year-olds had participated in one or more Youth Guarantee-related programmes, 8.1% had participated in Youth Training and 3.0% in a fees-free place.

In 2012, 12.0% of 18-year-olds had participated in one or more Youth Guarantee-related programmes, 6.1% had participated in Youth Training, 7.1% in a fees-free place and 0.9% in a secondary-tertiary programme.

In 2013, 13.7% of 18-year-olds had participated in one or more Youth Guarantee-related programmes, 2.7% had participated in Youth Training, 10.0% in a fees-free place and 2.8% in a secondary-tertiary programme.

Nearly 90% of young people who participated in Youth Guarantee programmes had NCEA Level 1 performance below the mean. Half of them had had at least one experience of disengagement from school – stand-down, suspension or serious truancy.

There were more males than females participating in Youth Guarantee programmes (57% were male). Māori also had higher participation rates, making up a third of participants in fees-free places and secondary-tertiary programmes.

Even though there was a shift in funding from Youth Training to fees-free places from 2011 to 2012 and the nature of the provision changed, the characteristics of the young people engaged in Youth Training and/or fees-free places has remained very similar over time.

Fees-free places

In 2013, half of those who had participated in fees-free places had both below-average performance at NCEA Level 1 and experienced disengagement from school.

Fees-free places were effective in improving retention in education during the year of starting the programme. This effect has improved in relative terms from 2010 to 2012 starters. However, for each starting cohort, the effect did not continue beyond the starting year. Young people who had been disengaged from school and/or had below-average performance in NCEA Level 1 had better retention rates in the starting year relative to their matched comparison group than other young people in fees-free places.

Table 1: Retention in Education for fees-free participants and comparison group during programme start year
Notes:
  1. Retention rate is the proportion enrolled in school or tertiary education for at least 75 days in the year.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
Year Total No Disengagement Disengagement
P C R P C R P C R
2010 96% 71% 1.34 97% 80% 1.21 95% 61% 1.55
2011 97% 71% 1.37 98% 81% 1.21 97% 60% 1.62
2012 92% 66% 1.40 95% 77% 1.24 89% 57% 1.55


Young people started fees-free places with a slightly lower level of NCEA Level 2 achievement than the matched comparison group. A similar to slightly greater proportion of those who started fees-free places, and were retained in education during the starting year, attained NCEA Level 2 than the matched comparison group. This effect was greater for young people who had experienced disengagement at school and/or had below-average performance in NCEA Level 1.

Table 2: Achievement of NC EA Level 2 or equivalent for fees-free participants and comparison group by the year following programme start
Notes:
  1. Achievement rates are for participants and comparison group who were retained in education during the programme start year.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
  5. 'NS' denotes that there is no statistically significant difference between participants and the comparison group, at 95 per cent confidence.
Year Total No Disengagement Disengagement
P C R P C R P C R
2010 68% 68% NS 78% 75% NS 57% 59% NS
2011 75% 70% 1.07 83% 78% 1.05 67% 59% 1.14
2012 63% 61% NS 765% 74% NS 52% 49% 1.07


Having attained NCEA Level 2 or equivalent, participants in fees-free places were less likely to progress to tertiary study at Level 4 and above than young people in the matched comparison group who had also attained NCEA Level 2 or equivalent.

Table 3: Progression to Level 4 and above for fees-free participants and comparison group by two years following programme start.
Notes:
  1. Progression rates are for participants and comparison group who attained NCEA Level 2 or equivalent.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
  5. 'NS' denotes that there is no statistically significant difference between participants and the comparison group, at 95 per cent confidence.
Year Total No Disengagement Disengagement
P C R P C R P C R
2010 42% 43% NS 39% 40% NS 41% 42% NS
2011 40% 46% 0.88 31% 40% 0.78 36% 44% 0.83


A higher proportion of participants in fees-free places were in employment following the programme than the matched comparison group. In particular, a higher proportion were in full employment and a lower proportion in part employment. Fees-free participants who were in full employment, and not in education, following the programme had lower average annual incomes than the matched comparison group ($30,850 compared with $32,000).

Fees-free places were effective in engaging young people in education who would otherwise have been NEET during the period of the programme. For 2012 starters, 10% of participants had NEET as their main activity during the starting year, compared with 20% of the comparison group. Across the starting cohorts, this effect was still evident one year after the start of the programme, but had largely disappeared two years after the start of the programme.

Young people on fees-free places could be eligible to receive an Independent Youth Benefit while on the programme. As a result, participants had a higher rate of welfare benefit receipt during the starting year of the programme than the comparison groups. For 2012 starters, 17% of participants received a benefit, compared with 10% of the comparison group, during the starting year. In the years following the programme, there continued to be a higher proportion of participants receiving a benefit than in the comparison group. As noted above, the comparison method does not take account of key factors that have a strong relationship to receiving a benefit and may underestimate the programme effect.

Table 4: Destinations of fees-free participants and comparisons group by two years following programme start
Notes:
  1. Progression rates are for participants and comparison group who attained NCEA Level 2 or equivalent.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
Year Full Employment Main Activity is
'Not in Education,
Employment or Training'
Receiving Benefit
P C R P C R P C R
2010 34% 32% 1.08 29% 25% 1.17 47% 37% 1.26
2011 37% 31% 1.19 29% 25% 1.17 41% 35% 1.19


The employment destinations for fees-free participants were better than for those who participated in the previous Youth Training programme. A lower proportion of Youth Training participants were in employment following the programme than their comparison group.

Secondary-tertiary programmes

In 2013, a quarter of participants in secondary-tertiary programmes had both below-average performance at NCEA Level 1 and experienced disengagement from school. A fifth had above-average performance at NCEA Level 1.

Young people starting secondary-tertiary programmes were already enrolled in education. The programmes were effective in maintaining education retention during the year of starting the programme. This effect did not continue beyond the first year. Young people who had below-average performance in NCEA Level 1 had better retention rates relative to their matched comparison group than other young people in secondary-tertiary programmes.

Table 5: Retention in education for secondary-tertiary programme participants and comparison group during programme start year
Notes:
  1. Retention rate is the proportion enrolled in school or tertiary education for at least 75 days in the year.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
Year Total No Disengagement Disengagement
P C R P C R P C R
2011 99% 91% 1.09 99% 90% 1.10 99% 92% 1.08
2012 98% 93% 1.05 97% 92% 1.05 98% 93% 1.05


Young people on secondary-tertiary programmes started with similar NCEA Level 2 attainment as the matched comparison group. A larger proportion of those who started the programme, and were retained in education during their starting year, achieved NCEA Level 2 or equivalent than the matched comparison group.

Table 6: Achievement of NCEA Level 2 or equivalent for secondary-tertiary programme and comparison group by the year following programme start
Notes:
  1. Achievement rates are for participants and comparison group who were retained in education during the programme start year.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
Year Total No Disengagement Disengagement
P C R P C R P C R
2011 78% 69% 1.13 88% 75% 1.17 60% 58% NS
2012 79% 71% 1.11 85% 76% 1.11 66% 58% 1.13


Having attained NCEA Level 2 or equivalent, participants in secondary-tertiary programmes were less likely to progress to tertiary study at Level 4 and above than young people in the matched comparison group who had also attained NCEA Level 2 or equivalent.

Table 7: Progression to Level 4 and above for secondary-tertiary programme participants and comparison group by two years following programme start
Notes:
  1. Progression rates are for participants and comparison group who were attained NCEA Level 2 or equivalent.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
  5. 'NS' denotes that there is no statistically significant difference between participants and the comparison group, at 95 per cent confidence.
Year Total No Disengagement Disengagement
P C R P C R P C R
2011 21% 33% 0.62 23% 36% 0.72 19% 29% NS


A higher proportion of participants in secondary-tertiary programmes were in employment following the programme than the matched comparison group. In particular, a higher proportion were in full employment and lower proportion in part employment. Secondary-tertiary programme participants who were in full employment, and not in education, following the programme had higher average annual incomes than the matched comparison group ($30,100 compared with $27,430).

Secondary-tertiary programmes were effective in engaging young people in education who would otherwise have been NEET during the period of the programme. For 2012 starters, 1% of participants had NEET as their main activity during the starting year, compared with 4% of the comparison group. Across the starting cohorts, this effect was still evident one year after the start of the programme, but had disappeared two years after the start of the programme for 2011 starters.

Secondary-tertiary programmes are targeted at secondary school students, very few of whom are eligible for a benefit during the period of the programme. For 2012 starters, 4% of both participants and the comparison group received a benefit, during the starting year. For the 2011 starters, participants were more likely to receive a benefit in the one and two years after the programme than the comparison group. For 2012 starters, there was no difference in the rate of benefit receipt for participants and the comparison group in the year following the programme. As noted above, the comparison method does not take account of key factors that have a strong relationship to receiving a benefit and may underestimate the programme effect.

Table 8: Destinations of secondary-tertiary programme participants and comparison group by two years following programme start
Notes:
  1. Progression rates are for participants and comparison group who were attained NCEA Level 2 or equivalent.
  2. 'P'= Participants
  3. 'C'= Comparison Group
  4. 'R'= Ratio of Participants Over Comparison Group (risk ratio).
Year Total No Disengagement Disengagement
P C R P C R P C R
2011 77% 72% 1.07 15% 13% 1.20 25% 18% 1.36

Conclusion

Youth Guarantee programmes, including Youth Training, have reached around 14% of young people by the age of 18. Most of those participating in the programmes had lower levels of achievement at NCEA Level 1 and half had experienced disengagement from school.

Fees-free places and secondary-tertiary programmes were effective in retaining young people in education in the year they started the programme. However, the programmes had no effect on retention after that.

The major effect of both programmes has been on increased attainment of NCEA Level 2 or equivalent. Young people on these programmes are more likely to attain this than young people with similar characteristics in other educational settings.

In general, it appears that fees-free places provide effective support for young people who have become disengaged from school. Secondary-tertiary programmes are targeted to young people who have remained in school and were more effective for young people had lower NCEA performance.

So far, neither programme has had any effect so far on increasing the proportion of young people with NCEA Level 2 or equivalent who progress to further study at Level 4 and above. Young people who had participated in secondary-tertiary programmes and achieved NCEA Level 2 were less likely to progress to Level 4 and above, compared with other similar young people. So while the programmes have had a positive effect on attaining NCEA Level 2 or equivalent, there is no evidence that they are providing a more effective pathway to further education and training than other educational choices for a similar group of young people.

There is some evidence that the programmes are providing a more effective pathway to employment, particularly to full employment. Secondary-tertiary programmes appear to be also providing a pathway to higher paid jobs.

Both programmes have been effective in engaging young people who would otherwise have been NEET. However, this effect appears to disappear within one or two years following the programmes.

There is no evidence from this study of either programme having any immediate effect on reducing the receipt of welfare benefits. As noted above, the comparison method does not take account of key factors that have a strong relationship to receiving a benefit and may underestimate the programme effect. It is too early to judge what effect, if any, the programmes have on benefit receipt in the long term.

Footnotes

  1. Each outcome area has a specific definition which is set out in Appendix C.4. These definitions are unique to this project and may differ from other definitions used by the Ministry of Education and the Tertiary Education Commission.
  2. These are young people who had participated in Youth Training up to age 17 by 2011.
  3. These are young people who had participated in Youth Training at age 16 in 2011.
  4. This is based on a performance score which takes account of the proportion of assessment standards with achieved, merit or excellence. Further details are set out in section 2.2.
  5. Further details on this are set out in section 2.2.
  6. Full employment includes those employed in jobs paying a pro-rated wage of greater than two thirds of the minimum wage for at least 6 months of the year. Part employment includes those who are not in full employment.
  7. The Independent Youth Benefit was replaced by the Youth Payment in 2013. It is a requirement of the Youth Payment that recipients are in, or available for, full-time education leading to NCEA Level 2 or equivalent.

Where to find out more

Contact Us

For more information about the content on this webpage, please email the:  Tertiary Mailbox