Academic performance of first-year bachelors students at university

5. Results

This section presents the results of the data modelling. A discussion of the results is in the next section.

Logistic regression was used to model the likelihood of passing most (more than 75 per cent) bachelors-level courses by first-year students at university between 2006 and 2008. The independent variables considered were achievement at school (NCEA level 3 achievement score), timing of progression to tertiary study after leaving school, and the decile rating of the last school attended. Separate models were run for these variables together with the individual ethnic groups. Factors included in additional models were the student’s gender and ethnicity, and subject studied and study type.

Table 5 shows the goodness of fit statistics for the range of models used.

The data is mostly presented in graphical form, with means and 90 per cent confidence intervals. 90 per cent confidence intervals are used so that readers comparing the intervals between two means can be at least 95 per cent certain that the means are significantly different. The reasons why this apparently counter-intuitive approach is necessary can be found in Schenker and Gentleman (2001).

5.1. Academic achievement at school

In the study population, 77 per cent of students pass most of their courses. However, academic achievement at school has a strong bearing on this. Figure 4 shows the likelihood of passing most courses by achievement score. It can be seen that at the lower end of the achievement score range, about 2 to 4 out of every 10 students pass most of their courses, while at the upper end of the range, nearly all students pass most of their courses. At the average achievement score (about 50), students have an 83 per cent chance of passing most of their courses.

The next sections show how other variables modify this overall relationship between tertiary academic performance and success at school.

5.2. School decile

Overall, the decile of the last school the student attended plays a relatively minor role in academic performance once a student reaches university (figure 5), although the differences between decile categories are statistically significant. The results show:

• At the lowest achievement scores (20–25) there are no significant differences between the three decile categories, although the order (from highest academic performance to lowest) is mid- to low- to high-decile schools.
  
• For low to middle achievement scores (30-50), students from mid-decile schools have significantly higher levels of academic performance, and above 40, high-decile school students are also significantly higher than low-decile school students.
  
• For achievement scores above 50, mid- and high-decile school students’ tertiary academic performance is not significantly different, but students from low-decile schools show a significantly lower level of tertiary academic achievement at all but the highest NCEA level 3 achievement scores.
   

Figure 4: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score

Results are calculated using all ethnic groups, all study types, all subjects studied, all school decile categories, gap and no gap year students and both genders. Excludes extramural students.

Figure 5: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and school decile

Results are calculated using all ethnic groups, all study types, all subjects studied, gap and no gap year students and both genders. Excludes extramural students.

5.3. Study type

If a student studies full-time full-year, the chances of passing most of their courses overall is higher than if they have a lighter study load (see figure 6). It is only at the extreme ends of the achievement score range that study type has less bearing on academic performance.

Part-time full-year study is associated with the lowest academic performance, mostly in the upper part of the achievement score range, but over most of the range the differences between this and the other non full-time full-year study types are not statistically significant.

It is likely that study load itself does not directly affect the chances of passing most courses. More likely, it is the reasons why students do not study full-time full-year that affect academic performance. Specifically, paid work or family commitments, which preclude full-time full-year study, will take time and focus away from study.

5.4. Timing of progression

Taking a year off between school and starting university bachelors study significantly improves the likelihood of passing most courses for students with below-average school achievement scores (figure 7). It should be noted that few students take a gap year (8 per cent in the study population), and those that do, who have below-average achievement scores, are about half this total again. In the study population, just over 1,300 students showed improved academic performance after taking a year off before starting tertiary study. Overseas studies have generally attributed this improvement to increased motivation (for example, see Birch and Miller 2007). Holmlund et al (2008) suggest that students with lower school results may be waiting for better educational opportunities.

Figure 6: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and study type

Results are calculated using all ethnic groups, all subjects studied, all school decile categories, gap and no gap year students and both genders. Excludes extramural students.

Figure 7: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and whether or not the student took a gap year

Results are calculated using all ethnic groups, all study types, all subjects studied, all school decile categories and both genders. Excludes extramural students.

5.5. Subjects studied

The subject studied plays an important role in determining university performance for students with below-average school achievement; for students with above-average school achievement, the subject studied has almost no bearing on success (figure 8). This would suggest that some subjects are easier to pass than others.

Figure 8 shows that students in creative arts and education have relatively higher chances of passing most of their courses, even for students with the lowest achievement scores. This is in contrast to all other subjects shown, where students with below-average school achievement have very low likelihoods of passing most courses. Students in natural and physical sciences have the lowest likelihood of passing most of their courses.

Figure 8: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and subject studied

Results are calculated using all ethnic groups, all study types, all school decile categories, gap and no gap year students and both genders. Excludes extramural students.

Figure 9: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score, ethnic group and gender

Results are calculated using all study types, all subjects studied, all school decile categories, and gap and no gap year students. Excludes extramural students.

5.6. Ethnic group and gender

When controlling for gender, ethnic group and achievement score, there is little difference in the likelihood of passing most courses between male and female students (figure 9)¹³. Only for sole-European female students is the likelihood significantly higher, but then only for the middle range of achievement scores. Over all school decile categories, sole-Pasifika students with middle range achievement scores have a significantly lower likelihood of passing most of their courses, than similar ever-Pasifika students.

5.7. Ethnic group and school decile

Figure 10 shows the results by ethnic group and school decile. There are four pairs of graphs, arranged vertically, with each pair showing the results for the ever- and sole- ethnic group categories for each ethic group. In each graph the results for the three school decile categories are displayed.

A common pattern can be seen in all graphs, with much lower likelihoods of students passing most of their courses when their school achievement scores are low, and there being near certainty that most courses are passed for students with higher school achievement. This is the overall pattern seen in figure 4. There are also almost no differences within ethnic groups. However, there are some important departures from the common pattern.

Apart from the sole-European and sole-Asian ethnic groups, all students with higher achievement scores from low-decile schools have a lower chance of passing most of their courses. This varies from slightly less (but significantly) so for Māori, ever-European and ever-Asian students, to quite large differences for Pasifika students. On the other hand, students from middle and high-decile schools for all achievement scores show essentially the same pattern.

Sole-Asian students from low-decile schools have course pass likelihoods slightly lower but not significantly different from sole-Asian students from other schools. The more intriguing result is for sole-European students. Here, students with lower achievement scores, from low-decile schools, have significantly higher likelihoods of passing most of their courses compared to similar students from other schools. Students from mid-decile schools also have significantly higher likelihoods of academic performance than their peers from high-decile schools. At the upper end of the achievement score range, all sole-European students have essentially equal likelihoods. The possible reasons for this are considered in the discussion.

Figure 10:  Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and school decile and ethnic group

Results are calculated using all study types, all subjects studied, both genders, and gap and no gap year students. Excludes extramural students.

Figure 10 (continued) Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and school decile and ethnic group

5.8. School decile and gap year

It has been shown that students with below-average achievement scores have higher likelihoods of passing most of their courses if they take a year off before starting their tertiary studies (figure 7). However, the extent of the improvement in university performance depends on the school the student attended, and as will be shown, to which ethnic group they belong.

Figure 11 shows the expected likelihood of passing most courses by school decile categories. It can be seen that the improvement in performance with a gap year is significantly higher for students from low-decile schools. This improvement is dramatic. Comparing the top two graphs in figure 11, it can be see that the students from low-decile schools have a significantly higher likelihood of passing most of their courses for achievement scores below 50, in contrast to the results for students who do not take a gap year. Then, for students with achievement scores above 50, the students from low-decile schools match the academic performance of other students. Again, this is in contrast to the pattern seen for students who progress directly to tertiary study. The lower three graphs in figure 11, which present the same data, show the comparison by students who take a gap year or not. Again, the improvement for low-decile school students is quite clear, while the difference for mid- and high-decile school students is minor by comparison, and much the same as each other.

Figure 11:  Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score and timing of progression to tertiary study and school decile[click image to enlarge]

Results are calculated using all ethnic groups, all study types, all subjects studied, and both genders. Excludes extramural students.

There is a small but significant increase in the chance of passing most courses for each of the school decile categories. Taking a year off certainly improves the chances of passing. But the improvement is outstanding for students from low-decile schools.

It should be noted there are very few low-decile school students who take a gap year. This is partly because so few low-decile school students make it to tertiary study in the first place, and as was noted earlier, students from low-decile schools are less likely to take a gap year. In the study population, 8 per cent of students overall take a gap year, compared to 4 per cent for students from decile 1-2 schools. In the study population, 1,075 students come from low-decile schools, and of these, just 47 took a gap year. 

5.9. Gender and gap year

Birch and Miller (2007) also found that students with below-average achievement scores who took a gap year have higher marks in their tertiary studies. However, they also found that for female students, the gap year improvement occurred whether they were low- or high-performing students; for males it was only for low-performing students who benefited from the gap year. That is, Birch and Miller found there was a gender by gap year interaction.

This finding was tested with the study population of the present analysis. It should be noted, however, that the dependent variable was slightly different between this and Birch and Miller’s study. They considered marks, while the present study considers the proportion of courses passed. The results (not shown) indicate that while there are significant differences in the odds of passing most courses by gender, academic performance for male and female students does not vary with school achievement, school decile or the timing of progression to tertiary study. Birch and Miller offer no explanation for their finding. The results in the present study will be considered in the discussion.

5.10. School decile and gap year for each ethnic group

There are differences between ethnic groups in the likelihood of passing most courses by school decile, and whether or not the student took a gap year. Sample sizes for some of these categories are small, so there is less certainty about the results, and confidence limits are necessarily wider. Caution should therefore be exercised if using these results. Notwithstanding, the trends shown by these results are informative, and in spite of the wide confidence limits, significant differences can be seen for some groups.

European ethnic group

Contrary to the previous finding, European students do not show a significant improvement in their university performance after taking a year off before starting tertiary study (figure 12). As was seen previously (figure 10) sole-European students from low-decile schools out-performed their peers, in contrast to all other ethnic groups. This result is seen here again. For students from low-decile schools, the sole-European ethnic group shows the highest likelihood of passing most courses at the lower end of the achievement score range. While taking a gap year increases the likelihood, the increase is not significant. For students from high-decile schools, taking a gap year also marginally increases academic performance, but again, not significantly so.

Figure 12: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score, school decile and timing of progression to tertiary study for the European ethnic group [click image to enlarge]

Results are calculated for full-time full-year students studying Society and Culture subjects, for both genders. Excludes extramural students.

In addition, while there are some differences between ever-European and sole-European students from low-decile schools, again at the lower end of the school achievement range, there are essentially no differences within the European ethnic group for students from high-decile schools. In particular, the tertiary academic performance of ever-European students from low-decile schools is much the same as their counterparts from high-decile schools, and like these latter students, shows only a marginal improvement after taking a gap year.

For sole-European students from low-decile schools, the NCEA level 3 achievement score is not a good predictor of tertiary academic performance, since essentially all students along the achievements core continuum have a better than even chance of passing most of their courses.

Asian ethnic group

In contrast to the European ethnic group, the Asian ethnic group shows almost no within-ethnic group variation (figure 13). The only difference is that sole-Asian students from low-decile schools with lower school achievement have slightly higher likelihoods of passing most of their tertiary courses then ever-Asian students from low-decile schools, but the difference is not statistically significant. In addition, and again in contrast to European students, for students from low-decile schools, there is a significant improvement in tertiary academic performance after taking a gap year. While there is a slight improvement in academic performance for students from high-decile schools taking a gap year, it is not significant.

For Asian students from low-decile schools who took a gap year, school achievement is not as good a predictor of tertiary academic performance as it is for students who progressed directly to tertiary education, or for those from high-decile schools.

Figure 13: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score, school decile and timing of progression to tertiary study for the Asian ethnic group[click image to enlarge]

Results are calculated for full-time full-year students studying Society and Culture subjects, for both genders. Excludes extramural students.

The improvement after taking a gap year for students from low-decile schools occurs across almost the entire range of achievement scores. Only at the highest achievement scores, above 70, is the difference not statistically significant. The improvement in tertiary academic performance of students from low-decile schools who took a gap year surpasses that of all Asian students from high-decile schools.

Māori ethnic group

The Māori ethnic group shows essentially the same pattern as the Asian ethnic group (figure 14). There are no significant differences within the Māori ethnic group, that is, between the sole- and ever-Māori ethnic group categories. There is a significant improvement in tertiary academic ability after taking a gap year for students from low-decile schools. This improvement again occurs over almost the entire range of school achievement. And again, the students from low-decile schools who take a gap year surpass nearly all other Māori students in tertiary academic performance; they equal the performance of the students with the very highest school achievement.

Like the situation for Asian students, the achievement score is not as good a predictor of tertiary academic performance for Māori students who took a gap year and attended low-decile schools, compared to Māori students who progressed directly, or those from high-decile schools.

Pasifika ethnic group

This ethnic group, particularly the sole-Pasifika category, is the least represented in the study population, and any analysis at this level of breakdown necessarily will be less reliable, as evidenced by the much wider confidence limits. The results are presented nevertheless to show trends. In a few years time, when more data is available, this analysis can be strengthened.

Figure 14: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score, school decile and timing of progression to tertiary study for the Māori ethnic group [click image to enlarge]

Results are calculated for full-time full-year students studying Society and Culture subjects, for both genders.  Excludes extramural students.

The results for the Pasifika ethnic group, even allowing for the wide confidence limits, are quite different from the other ethnic groups (figure 15). There is no improvement in tertiary academic performance for the sole-Pasifika group after taking a gap year for students from low-decile schools, and there is a suggestion that for high-decile school students, taking a gap year has a detrimental effect on performance, across most of the school achievement range. Even with the limited data used here, there is a significant decline in the likelihood of passing most courses for these students at the upper end of the school achievement range.

Ever-Pasifika students’ results exhibit the pattern seen for Asian and Māori students; an improvement in academic performance for students from low-decile schools after taking a gap year, across nearly the whole range of school achievement. There are therefore quite marked differences within the Pasifika ethnic group.

For Ever-Pasifika students who took a gap year and attended low-decile schools, the achievement score is not as good a predictor of tertiary academic performance as those who progressed directly to tertiary education, or those from high-decile schools.

5.11. School decile and gap year for the sole-ethnic group category

The previous set of graphs (figures 12 to 15) highlights the comparison between students taking a gap year or not within ethnic groups. The same data, for the sole-ethnic categories, can be reproduced to show differences between the ethnic groups. This highlights the quite different outcomes from students of sole-Pasifika ethnicity. Between-ethnic group comparisons are only possible with the sole-ethnic group categories, since the ever-ethnic categories overlap.

Figure 15: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score, school decile and timing of progression to tertiary study for the Pasifika ethnic group [click image to enlarge]

Results are calculated for full-time full-year students studying Society and Culture subjects, for both genders. Excludes extramural students.

Figure 16 shows the results. Again, wide confidence limits can be seen for the Pasifika results, due to there being so few of these students studying at this level, particularly those who took a gap year. There are also relatively fewer low-decile school students.

Firstly, note that for students from high-decile schools who did not take a gap year, the results for each of the ethnic groups is very similar. There is a statistical difference between the sole-European and the sole-Asian and sole-Māori results for the middle achievement score range (not discernable in the figure), but the sole-Pasifika confidence limits overlap all the other ethnic group’s results. The Māori and Asian trend lines are so similar they are indistinguishable in the figure.

In contrast, there are quite divergent results for students from low-decile schools who did not take a gap year.  Sole-European students out-perform other students except at the very highest achievement scores, and while there is no statistical difference between the Asian and Māori results, Pasifika are significantly less academically successful.

Figure 16: Expected probability (and 90 per cent confidence limits) of passing most courses by NCEA level 3 achievement score, school decile, timing of progression to tertiary study and sole-ethnic group [click image to enlarge]

Results are calculated for full-time full-year students studying Society and Culture subjects, for both genders. Excludes extramural students.

Comparing the upper two graphs with the lower two in figure 16, the reader can again see the improvement in academic performance for students taking a gap year, especially for those from low-decile schools, and this occurs for all ethnic groups other than Pasifika. For the students from low-decile schools, sole-Asian students have the highest likelihood of passing most of their courses, but the Asian, Māori and European trend lines are not statistically different. As seen previously, the levels of academic performance for these groups, in the main, exceeds those of the students who did not take a gap year. There are also no significant differences between the Asian, Māori and European ethnic groups for students from high-decile schools who took a gap year.

It is clear that sole-Pasifika students’ academic performance is significantly different from other students, even when controlling for school decile, timing of progression, and academic achievement at school. Only for students from high-decile schools, and who do not take a gap year, is there parity in academic performance. Unfortunately, very few (12 per cent) sole-Pasifika students attend a high-decile school. In the study population, this equates to 84 students, 79 of whom did not take a gap year.

Footnote

13. The odds of female students passing most of their courses are significantly higher than the odds for males, for all ethnic groups other than Pasifika. Odds are ratios of probabilities, not probabilities themselves. The report confines itself to reporting probabilities, and in most cases, the results for odds and probabilities are much the same.