Monitoring the Youth Guarantee policy 2010-2012 Publications
This report focuses on the initial implementation of the Youth Guarantee policy in the period from 2010 to 2012 and looks at how effective the fees-free places and secondary-tertiary programmes were in improving educational outcomes for young people. Youth training is also included, as funding was transferred from this programme to fees-free places.
Author(s): David Earle, Tertiary Sector Performance Analysis, Ministry of Education.
Date Published: February 2014
Youth Guarantee provides new opportunities for 16- and 17-year-olds to achieve education success, and progress into further education, training and employment. It offers a range of programmes providing opportunities for young people to engage in education. It supports schools, tertiary education providers and employers to work together in new ways and has developed vocational pathways that clarify the options for young people and identify the skills and knowledge valued by employers.
The Youth Guarantee policy was implemented progressively from 2010. Fees-free places were established in 2010, providing opportunities for 16- and 17-year-olds, not currently engaged in education, to re-engage with education in a tertiary rather than school-based setting. The number of places was increased each year. In 2012, funding from the Youth Training programme was transferred to further expand the number of fees-free places. Secondary-tertiary programmes were established in 2011 and 2012. These programmes allow students to remain enrolled at secondary school, while participating in various forms of education delivered by tertiary education providers. Many of these programmes are known as 'trades academies'.
The Tertiary Education Commission and the New Zealand Qualifications Authority have worked with tertiary education providers to build their capability to deliver quality programmes through fees-free places and secondary-tertiary programmes. They have actively supported providers to move from delivering employment-based training under Youth Training to full qualifications through fees-free places.
The next phase of the implementation has been to influence the wider framework for transitions from school through tertiary education and into employment. Vocational pathways provide new ways to structure and achieve NCEA Level 2. They enable students, education providers and employers to see how their learning is relevant to a wide range of jobs and further study. Five pathways were finalised in 2013 for implementation in 2014. A sixth pathway will be developed by June 2014. The Ministry of Education is also leading Youth Guarantee networks of schools, tertiary education providers, employers and communities, to develop new ways in which education can be offered from 2014 onwards.
The purpose of monitoring and evaluating the Youth Guarantee policy is to understand the extent to which the education system is changing to meet the vision and expected outcomes of the policy. It is informed by an outcomes framework that identifies four key areas:
|Retention in education and training||What effect are Youth Guarantee policies having on more 16 and 17 -year-olds remaining in school and tertiary education?|
|Achievement of NCEA Level 2 (or equivalent)||What effect are Youth Guarantee policies having on more young people achieving at least NCEA Level 2 or equivalent?|
|Progression to tertiary study and skilled employment||What effect are Youth Guarantee policies having on more young people moving on to study in tertiary education and work-based training at level 4 and above?|
|Sustainable system level change which empowers young people to move successfully into a range of post-school education and employment options||Have the Youth Guarantee policy changes helped create an education system which actively empowers learners and employers to easily navigate the system and achieve successful outcomes?|
Progress towards the outcome areas is monitored by tracking year-of-birth cohorts of students through the education system and beyond. The first cohort are those born in 1993, who turned 15 in 2008 and were 17 in 2010 when the first Youth Guarantee programmes were introduced. Their experiences are compared with the next cohort, who had more experience of the Youth Guarantee programmes.
For each programme, we compare the outcomes for those who participated on the programme to a group of young people who did not participate who had similar characteristics and backgrounds. The comparison groups are specific for each programme and match the characteristics of the young people on the programme.
The characteristics used to choose the comparison group have strong associations with the outcomes being measured. This means that if the programmes had not existed, both groups would most likely achieved similar outcomes. There is still a possibility that a part of the difference may be due to further unmeasured factors that explain why some people participate in programmes and others don't. However, the approach sufficiently robust to attribute the difference to the presence of the programmes.
Participation in Youth Guarantee programmes
In 2011, 10.6% of 18 -year-olds had participated in one or more Youth Guarantee programmes (1993 birth cohort), 8.1% had participated in Youth Training and 3.0% in a fees-free place.
In 2012, 12.0% of 18 -year-olds had participated in one or more Youth Guarantee programmes (1994 birth cohort), 6.1% had participated in Youth Training, 7.1% in a fees-free place and 0.9% in a secondary-tertiary programme.
There were more males than females participating in Youth Guarantee programmes. Māori had higher participation rates in Youth Guarantee programmes, making up nearly half of Youth Training participants and a third of participants in other programmes.
Around 90% of young people who participated in Youth Guarantee programmes had NCEA level 1 achievement below the mean. More than half of those who participated in Youth Guarantee programmes had also had at least one experience of disengagement from school – stand-down, suspension or serious truancy.
Even though there was a shift in funding from Youth Training to fees-free places from 2011 to 2012 and the nature of the provision changed, the characteristics of the young people engaged across the two programmes has remained very similar.
Overall shifts in outcome areas
Looking at all young people, the proportion staying in education at 16 and 17 has been steady. Around 95% of all 16 -year-olds were in education and 88% of 17-year-olds. The overall rate of achievement of NCEA Level 2 or equivalent has been increasing. The proportion who achieved NCEA Level 2 or equivalent by age 18 increased from 70% in 2011 to 73% in 2012. By age 19 in 2012, 46% of all young people had enrolled in a level 4 or higher qualification after leaving school.
To get a better understanding of outcomes for different groups of students, we can look at young people by their level of achievement at NCEA level 1. For those with lower level 1 achievement, we can also look at whether they had experienced at least one instance of disengagement from school.
Of young people who had higher achievement at level 1 NCEA, 96% were in education at age 17 in 2010, 93% achieved NCEA Level 2 or equivalent by age 18 in 2011 and 67% had enrolled in level 4 and above qualifications by age 19 in 2012.
Where young people had lower achievement at level 1 NCEA, and had not experienced any disengagement from school, 83% were in education at age 17 in 2010, 58% achieved NCEA Level 2 or equivalent by age 18 in 2011 and 31% had enrolled in level 4 and above qualifications by age 19 in 2012.
Where young people had lower achievement at level 1 NCEA and had experienced some disengagement from school, 64% were in education at age 17 in 2010, 34% achieved NCEA Level 2 or equivalent by age 18 in 2011 and 18% had enrolled in level 4 and above qualifications by age 19 in 2012.
There has been some improvement in retention rates for young people with lower achievement at level 1 NCEA. The NCEA Level 2 achievement rates for these young people also improved by 5 percentage points from 2011 to 2012.
Two-thirds of participants in Youth Training had lower achievement at NCEA level 1 and had also experienced at least one instance of disengagement from school.
Youth Training was effective in increasing education retention for 16 and 17-year-olds who had experienced disengagement from school. However, for young people who had not experienced disengagement at school, being on Youth Training did not increase their rate of retention in education and may have encouraged them out of education.
Youth Training participants had lower levels of attainment of NCEA Level 2 or equivalent before starting a programme. Having starting the programme, they still had lower attainment at NCEA Level 2 or equivalent than others with similar characteristics and background. This was particularly noticeable for young people who had not experienced any disengagement from school.
Youth Training did not provide any pathway towards level 4 and above qualifications. For young people who had not experienced disengagement in school, it appears to have encouraged them away from study at higher levels.
Just under half (40-44%) of participants in fees-free places had lower achievement at level 1 and had experienced disengagement from school.
Fees-free places were effective in maintaining and improving retention for 16 and 17 years olds in education. Where young people had not experienced any disengagement from school, participating in a fees-free place maintained their retention in education. Where young people had experienced disengagement from schooling, fees-free places were effective in improving their retention in education.
Young people started fees-free places with similar levels of NCEA Level 2 achievement to other young people with the same characteristics and background. A greater proportion of those who started fees-free places attained NCEA Level 2 or equivalent by age 18 than in the comparison group. This was particularly notable for young people who had experienced disengagement at school.
Initially fees-free places made little difference to whether young people went on to study at higher levels. It would appear that as the programmes have developed, they are encouraging a larger proportion of young people to move into higher level study.
Only a quarter of participants in secondary-tertiary programmes had lower achievement at level 1 and had experienced disengagement from school. A quarter had higher achievement at level 1.
Secondary-tertiary programmes are aimed at young people who have remained at school. Participation in the programmes does have an effect of increasing their retention in education to age 18. The programmes are effective in improving retention for both those who had and who had not experienced disengagement from school.
Young people on secondary-tertiary programmes started with similar NCEA Level 2 attainment as other young people with similar characteristics and background. A larger proportion of those who started the programme achieved NCEA Level 2 by age 18 than those who did not.
It is too soon to measure progression to level 4 and above qualifications for secondary-tertiary programmes.
Youth Guarantee programmes, including Youth Training, have reached up to 12% of young people up to the age of 18. Most of those participating in the programmes had lower levels of achievement at NCEA level 1 and more than half had experienced disengagement from school. These two factors are strongly associated with lower rates of retention in education, attainment of NCEA Level 2 and progression to level 4 and above tertiary qualifications.
Youth Training, fees-free places and secondary-tertiary programmes were all effective in retaining young people in education. This effect has increased as the new programmes have been implemented.
Youth Training actually resulted in fewer young people attaining NCEA Level 2 or equivalent by age 18 than would be expected if they had not gone on the programme. By contrast, fees-free places are resulting in more young people attaining NCEA Level 2 or equivalent. Secondary-tertiary programmes are adding further to this contribution. It is expected in 2013 that the contribution of Youth Guarantee programmes to the attainment of NCEA Level 2 or equivalent will further increase.
Youth Guarantee programmes have had little impact so far on the proportion of young people going on to level 4 and above study after leaving school. It is likely this will improve from 2013 onwards. There are signs of improvement in fees-free places, and secondary-tertiary programmes are likely to add positively to this outcome. It is too early to measure the contribution of the latter at this stage.
These results are summarised in Table 1 below. The table shows for each programme the proportion of young people who achieved the outcome measure (in the rows labelled "programme") and the proportion in the comparison group not on the programme who achieved the outcome measure (in the rows labelled "comparison"). The difference is the amount that can be attributed to the programme. This is shown as a percentage point difference and as the number of young people.
|1993 Cohort||1994 Cohort|
|In education at 17 (2010)||NCEA Level 2 or equivalent by 18 (2011)||Enrolled at Level 4+ by 19 (2012)||In education at 17 (2011)||NCEA Level 2 or equivalent by 18 (2012)||Enrolled at Level 4+ by 19 (2013)|
|Transitional group: Youth Training to fees-free 2011/2012|
|Total difference to outcomes across all programmes|
|Cohort popn* at 15||63,125||63,125||63,125||61,831||61,831||-|
|As % of cohort||1.5%||-0.6%||0.5%||1.7%||0.1%||-|
The bottom rows of the table show the combined impact of the Youth Guarantee programmes (including Youth Training) on the performance of each age cohort as a whole in relation to the outcome measures. The impact of each programme, in terms of the number of people affected on the outcome, is totalled to provide the figure for the "additional number of people". This is then shown as a percentage of the cohort population which can be interpreted as the percentage point contribution of the Youth Guarantee programmes to the overall performance of the cohort.
The 'transitional group' are young people who started in Youth Training in 2011 and transferred to fees-free places in 2012. They were not included in the analysis for either Youth Training or fees-free places as they had partial experience of both programmes. They are included here to ensure that all young people in each cohort are counted.