A closer look at completion in higher education in New Zealand Publications
Publication Details
New Zealand has one of the lowest reported higher education qualification completion rates in the OECD, significantly below Australia. Why do so many New Zealand students not complete their qualification? This paper looks behind some of the numbers in an attempt to better understand and assess New Zealand's performance compared with Australia and internationally. It looks, for example, at the impact of part-time and partial qualification study on completion rates. New Zealand has the highest reported level of part-time study in the OECD, and one in eight bachelor’s-degree students in New Zealand pass every subject they've enrolled in, yet have not completed their degree after five years. What does this tell us about intentions and about how we should gauge success?
The paper takes another look at some international comparisons focussing on full-time students, and also looks at the impact of transfers, changing qualifications, and what happens to rates when a ten-year window is taken instead of a five-year window.
Author(s): David Scott, Tertiary Sector Performance Analysis & Reporting, Ministry of Education
Date Published: May 2009
Summary
Introduction
Qualification completion is an important indicator of higher education system performance. One question that arises with such indicators is how do we work out what rate of qualification completion represents good performance? In higher education, not everyone is able to, or wants to, complete a qualification. Some might argue that if there was no failure, then we would lose the value of qualifications to the labour market as indicators of relative skill. If 100% completion is not desirable or realistic, then what is a good level?
People naturally look for benchmarks or comparisons against which they can make such judgements about performance. Given the large amounts of money invested in tertiary education and the desire for high quality, it is natural to look for bases for comparison. It could be argued that relative performance is perhaps the only way to assess performance.
The most common argument against comparisons is that like is not being compared with like. This is often particularly so in institutional league tables, where more vocationally-oriented institutions, perhaps drawing from an older, more part-time student population, are compared with more academically-oriented institutions with higher academic entry requirements and a higher proportion of students straight from school who study full-time. In essence, comparisons pose a risk to the institution or department, that readers will draw false conclusions about their quality. To counter this, rates are disaggregated for different groups, or statistical methods are used to adjust for known demographic and study-related differences. Provided one has data for these adjustments, this can be useful in removing the effects caused by factors beyond the control of institutions.
The same issues apply to comparisons between countries – maybe more so. The Organization for Economic Co-operation (OECD, 2008) publishes an annual table of higher education qualification completion rates. New Zealand has consistently been near the bottom of this table. For tertiary-type A study (which includes study at bachelor's level and above, excluding doctorates), New Zealand has a completion rate of 58% compared with 69 per cent for the OECD mean, 72% for Australia, and 79% for UK. At 58%, New Zealand's completion rate is the fourth lowest of the OECD countries who report this figure.
On the surface then, there appears to be a problem. Why are so many New Zealanders not completing their degree? This paper explores New Zealand's low qualification completion rate from two particular perspectives – students studying part-time, and students who change and complete something different from what they started. The paper aims to show that both these features act to reduce apparent qualification completion rates, but not necessarily performance, if other achievement measures are used. The paper also aims to suggest, that once adjusted for part-time study, New Zealand's performance compares favourably with other countries.
Further adjustment for other factors is beyond the scope intended for this paper. In fact, current available OECD completion data does not permit adjustments for factors other than part-time/ full-time. Much has been written on why people fail to complete their study. While, discussion of some of these factors is, again, beyond the scope intended for this paper, for those interested, in New Zealand Prebble et al. (2005), and Scott & Smart (2005), provide a review of some of this literature, including secondary references to relevant international and New Zealand-based studies.
This paper also does not set out to support a case for moving away from qualification completion as an important indicator of system performance, but, rather, it sets out to provide some context on the New Zealand system to those who may use these comparisons to infer something about performance.
In the following analyses, unless otherwise referenced, all figures are derived by the author using Ministry of Education administrative data collected nationally from New Zealand tertiary education providers.
Full-time and part-time study
New Zealand has the one of the highest proportions of degree-level students studying part-time in the OECD. Around 60% of new entrants to tertiary-type A level study full-time in their first year, whereas just 42% of students study full-time continuously during their degree (OECD, 2008, Tables C4.5, A4.2).
Part-time students have a lower completion rate than full-time students. Table 1 shows bachelor's degree completion rates after nine years for full-time and part-time domestic students starting a bachelor's degree in 1998.
Part-time students naturally take longer to complete; hence typical five-year rates may then not be sufficient if a significant proportion is studying a few courses or subjects a year, or taking a one- or two-year break from study. This is the case in New Zealand, as shown in Figure 1, where 10% of part-time and 5 per cent of all students starting a degree complete in the sixth year or later. In terms of graduates, 20% of those graduates who studied part-time, and 13% of all graduates, completed in the sixth year or later. The effect is even more pronounced at PhD level, for which just as many PhD students take more than five years to complete as those who complete in five years or under.
Note:
| ||||||
Mode of Study | Percentage of Students | Nine-year Completion Rate | ||||
---|---|---|---|---|---|---|
Full-time | 39% | 75% | ||||
Part-time | 61% | 47% | ||||
All Students | 100% | 59% |
Figure 1: Qualification completion for bachelor's students by year and mode of study
However, even after allowing sufficient extra time for part-time students to complete, fewer do so in New Zealand. In fact, Scott & Smart (2005) showed that after controlling for a range of demographic and study-related factors, students who studied on a full-time basis throughout their degrees were over twice as likely as part-time students to complete the degree.
Not all students enrol with the intention of completing a qualification. New Zealand does not collect data on intentions, but in the United States, for example, around 10% of students commencing a bachelor's degree are enrolling for reasons other than to gain a qualification (US Department of Education, 2002). These students may, for example, include those who are already qualified and who are looking to gain extra skills or certification in a particular area.
It is not possible in New Zealand to exclude from the calculation of completion rates those students not intending to gain a degree. However, it is possible to measure enrolment and completion in the individual courses that make up a bachelor's degree, and from this perhaps make some inferences about the size and impact of this group of students. For example, one can measure the percentage of students who passed every course they enrolled in, but had not enrolled in sufficient courses or otherwise did not gain a qualification. This is sometimes referred to as partial qualification completion. It is not unreasonable to assume that some proportion of these students never intended to gain a degree. The term 'course' is used in this paper (and often, but not always, in the New Zealand context) to refer to the individual components of a qualification, variously called 'subjects', 'units', 'papers' or 'modules'.
Notes:
| ||||
Mode of Study | Degree Completion Rate1 | Partial Degree Completion Rate2 | Course/Subject Completion Rate3 | |
---|---|---|---|---|
With No Degree | Without any Qualification | |||
Full-time | 75% | 5% | 2% | 87% |
Part-time | 40% | 17% | 12% | 76% |
All students | 55% | 12% | 7% | 82% |
Table 2 shows degree completion rates, partial degree completion rates and course pass rates, by mode of study. The year 2001 is the first year for which course-level information is available, therefore six-year rates are the longest period currently available. Rates will differ slightly from those in Table 1 which are based on the longer nine-year window.
After six years around 12% of all students (and 17% of part-time students) had passed every course they had enrolled in, but had not enrolled in enough courses to complete their degree. Some of these qualified and exited with a lower-level qualification, such as a certificate or diploma. However, seven per cent of students (and 12% of part-time students) passed every course they enrolled in and left without a qualification at any level. This suggests that if performance were to be assessed on achievement at an individual course/ subject level, and not just on qualification attainment, then performance is being under-estimated in New Zealand by seven per cent.
Changing track
In the discussion to date, qualification completion has been measured in terms of those who complete a qualification at the same level as or higher than the one they started. This takes account of those students who change track during their study, failing (or choosing not) to complete their initial degree, but successfully completing another qualification at degree-level or higher. A high proportion of degree students in New Zealand change qualifications or providers during their course of their study. A recent study by this author, Scott (2008), estimated that around 20% of bachelor's students changed provider and 41% changed qualification during their study.
Furthermore that study showed that although four per cent of bachelor's students completed a higher-level qualification instead of a bachelor's degree, as much as seven per cent completed a lower-level qualification (such as a diploma or certificate) instead of a degree. Significantly fewer of those changing provider completed a degree. However, significantly more of these students were likely to complete a lower-level qualification. In fact, when lower-level qualification completion was taken into account, rates of qualification completion were about the same for both groups.
As with students who pass all their courses and leave without a qualification, those students who leave with a lower-level qualification instead of a degree are important to consider when assessing achievement. Most traditional measures of qualification completion exclude both these groups, and so if completion is measured solely on the completion of a qualification at the same level as or higher than what was started, then the full level of success in the system is being underestimated.
Figure 2: An expanded picture of outcomes after six years for students starting a bachelor's degree
In New Zealand's case, for example, Figure 2 suggests that if one were to consider those leaving with a lower-level qualification, as well as partial qualification completion, then New Zealand's performance is significantly under-estimated, by maybe as much as 25%. Although success at degree level (or higher) for those wanting degrees must remain the primary goal, nonetheless it is just as important to consider that the complement of this group should not be viewed as all failure.
International comparisons
For the first time, Education at a Glance 2008 contains a table of completion rates for full-time and part-time students separately (OECD, 2008, Table A4.2). The part-time effect is evident on New Zealand's data, where the rate of 58% for tertiary-type A study (bachelor's and above, excluding PhD) for full-time and part-time students together, is the fourth lowest for the countries reported, whereas the rate for full-time students, at 73%, compares much more favourably. This is shown in Table 3. However, although total rates are available for 24 OECD countries, full-time and part-time rates are available for only 12 countries.
Notes:
| |||||
Country | Qualification Completion Rate1 | Mode of Study2 | |||
---|---|---|---|---|---|
All students | Full-time | Part-time | Full-time | Part-time | |
Australia | 72 | 68 | 32 | ||
Austria | 71 | ||||
Belgium (Fl.) | 76 | 88 | 12 | ||
Canada (Quebec) | 75 | 79 | 38 | 75 | 25 |
Czech Republic | 68 | 96 | 4 | ||
Denmark3 | 81 | 81 | 92 | 8 | |
Finland | 72 | 57 | 43 | ||
France | 64 | 100 | a | ||
Germany | 77 | 96 | 4 | ||
Greece | 100 | a | |||
Hungary | 57 | 60 | 54 | 54 | 46 |
Iceland | 66 | 79 | 21 | ||
Ireland | 83 | 17 | |||
Italy | 45 | 45 | 100 | n | |
Japan | 91 | 91 | 85 | 89 | 11 |
Korea | |||||
Luxembourg | |||||
Mexico | 61 | 61 | 100 | a | |
Netherlands | 71 | 73 | 57 | 83 | 17 |
New Zealand | 58 | 73 | 48 | 60 | 40 |
Norway | 67 | 69 | 57 | 73 | 27 |
Poland | 63 | 66 | 61 | 55 | 45 |
Portugal | 73 | ||||
Slovak Republic | 70 | 64 | 81 | 62 | 38 |
Spain | 88 | 12 | |||
Sweden | 69 | 49 | 51 | ||
Switzerland | 70 | 90 | 10 | ||
Turkey | 100 | n | |||
United Kingdom | 79 | 72 | 28 | ||
United States3 | 56 | 56 | 65 | 35 | |
OECD average | 69 | 68 | 60 | 80 | 20 |
Figure 3 further explores the relationship between mode of study and completion rate, using recent OECD data. The graph plots tertiary-type A completion by the proportion of students in type A and advanced research programmes who are part-time. While this is not a strictly valid comparison, because of the inclusion of advanced research programmes, it does allow more countries to be compared, thereby providing a broad indication of the strength of the relationship across different countries. The fitted line is also included, based on ordinary least squares regression. This shows a correlation coefficient of -0.55 or an R2 of 0.30, suggesting a moderate relationship.
Figure 3: Completion rates for tertiary-type A students by percentage of students studying part-time
Conclusions
Qualification completion is an important indicator of system performance. Comparisons are an essential means by which to assess this performance. However, where comparisons are not disaggregated by, or adjusted for, demographic and study-related differences, there is a risk that inaccurate conclusions may be drawn about relative performance.
This present study aimed to explore New Zealand's relatively low standing in OECD rankings of qualification completion rates from two particular aspects – the impact of part-time study, and the impact of students who change and complete something different to what they started.
New Zealand has one of the highest rates of part-time study in the OECD. Studying part-time has been shown to be one of the most significant factors in degree completion in New Zealand. Part-time students don't just take longer to complete, even after sufficient time significantly more of them fail to complete. Part-time degree students are more likely than full-time degree students to be aged 25 years and over, currently employed, with previous tertiary education. Around 12% of part-time (and seven per cent of all) degree students successfully passed every degree-level course or subject in which they enrolled but left without any qualification (degree or otherwise). This, perhaps, suggests that, for some at least, gaining a qualification was not their intention.
Full-time students, on the other hand, are more likely to represent traditional younger students coming from school. New Zealand's overall degree-completion rate of around 58% belies the performance of these more traditional students, whose completion rate, at around 73%, compares favourably with other countries.
A further 12% of all degree starters leave with a lower-level qualification instead of a degree. Traditional completion rates tend not to recognise aspects of performance, such as completion of a lower-level qualification or partial qualification completion, which are otherwise counted as attrition. Both these aspects in New Zealand make up 13% of all entrants to bachelor's-level study. Latest OECD comparisons now include breakdowns of completion rates by both these factors, albeit for a few countries only. Making this adjustment, completion rates for New Zealand students compare more favourably with other OECD countries.
Qualification completion will continue to remain an important indicator of system performance in comparisons within and between countries. However traditional measures of completion often don't capture the full picture of success. This is particularly so in New Zealand because of the extent of those studying part-time and those changing track. This paper does not set out to support a case for moving away from these traditional measures, but, rather, to gauge the size of these aspects of performance not captured under traditional measures, and invites readers to consider their significance when drawing inferences about relative performance.
Acknowledgement
The author gratefully acknowledges Roger Smyth, (New Zealand Ministry of Education), who provided comments on this paper.
References
- OECD. (2008). Education at a Glance 2008. Paris: Organisation for Economic Cooperation and Development.
- Prebble, T., Hargraves, H., Leach, L., Naidoo, K., Suddaby, G., & Zepke, N. (2005). Impact of student support services and academic development programmes on student outcomes in undergraduate tertiary study: a synthesis of the research. Wellington: Ministry of Education.
- Scott, D. & Smart, W. (2005). What factors make a difference to getting a degree in New Zealand? Wellington: Min
- istry of Education.
- Scott, D. (2008). Different tracks, a look at the different ways New Zealanders get tertiary qualifications. Wellington: Ministry of Education.
- U.S. Department of Education. National Center for Education Statistics. (2002). Descriptive summary of 1995-96 beginning postsecondary students: six years later. Washington, DC: U.S. Department of Education.
Navigation
Author Posting
Author Posting. (c) Association for Tertiary Education Management and the L H Martin Institute for Higher Education Leadership and Management, 2009.
This is the author's version of the work. It is posted here by permission of Association for Tertiary Education Management and the L H Martin Institute for Higher Education Leadership and Management for personal use, not for redistribution. The definitive version was published in Journal of Higher Education Policy and Management, Volume 31 Issue 2, May 2009.
Contact Us
For more information about the content on this webpage, please email the: Tertiary Mailbox