Are particular school subjects associated with better performance at university?

4. Effect of school subject on university performance

The previous two sections of this report showed that school achievement is associated with university performance, and that school achievement is also associated with the level 3 subjects studied at school. The fact that school achievement links the two factors of interest, school subject choice and university performance, makes it important that we control for school achievement when considering university performance.

School achievement can be controlled for in a variety of ways. In this report we consider four methods.

• First, we look at the university performance of students who did or did not take a particular subject at school. While this is not strictly controlling for school achievement, we are interested in contrasting the effect of taking a school subject on university performance against not taking that subject.
  
• Second, we model university performance using logistic regression, with school achievement in one subject used as a continuous variable (0–100), and a second subject regarded as a categorical variable (did or did not take the subject).
  
• We model university performance controlling for school achievement in two subjects, used as continuous variables, for students who took both subjects.
  
• Lastly, we model university performance for students who did or did not do a subject, controlling for school achievement in the subjects the two groups of students have in common.

The methods above are applied to university study considered in broad fields—physical and natural science, for example. We also consider study in narrow fields—mathematical sciences, and chemical sciences, for example, to test the hypothesis that the more similar the subject topic between a school subject and the university study, the more likely there is to be an association.

We could theoretically control for more than two subjects, but sample sizes become low and undermine the robustness of the models. Even with two subjects, not all subject combinations can be analysed because of low student numbers.

4.1. Effect of taking, or not taking, a school subject

This section considers whether taking a particular subject is associated with higher university performance, and, conversely, whether not taking a subject is associated with lower performance. We consider a range of tertiary fields of study, staring with degrees in engineering.

Engineering

Figure 4 shows the proportion of students passing most of their first-year engineering courses, by whether or not a student took a particular level 3 school subject.

The figure shows two aspects. Firstly, it can be seen that not taking physics, mathematics with calculus, or chemistry (but having taken any other subject) is associated with a significantly lower likelihood of passing most first-year courses in engineering. However, not taking any of the other school subjects is not associated with lower chances of passing engineering courses. Students who did not take statistics and modelling at school show a relatively higher likelihood of passing most courses, indicating that this group of students probably took mathematics with calculus, or physics or chemistry, given the height of the bar in the graph.¹⁰

Secondly, taking chemistry, physics, or mathematics with calculus is associated with higher likelihoods of passing most courses than for the other school subjects. This also applies to level 3 biology, but the difference is not significantly higher than not taking biology. Students taking statistics and modelling were significantly less likely to pass most of their engineering courses. Again, this is likely to be because they did not take those subjects that appear to be necessary for successfully studying engineering.

Figure 4: Effect of school subject on course pass rates for students studying engineering at bachelors-level at university

School subjects are sorted in order of increasing course pass rate for when the subject was not taken.
Error bars are 90 per cent confidence limits.
Results for where there were fewer than 50 students are not shown.

The difficulty with this analysis is that we are not actually controlling for school achievement. The higher pass rates are seen for subjects that have, on average, higher school achievement. Based just on this data, it is difficult not to come to the conclusion that physics, mathematics with calculus, and chemistry, overall, provide some benefit in studying engineering.

Later in this report we show results when controlling for school achievement. We find, in most cases, that when controlling for school achievement, most associations diminish or disappear entirely. Unfortunately, few students take engineering, and students who do not take mathematics, chemistry or physics at school, rarely go on to study engineering, so modelling performance in engineering to control for school achievement is problematic. However, our exploratory analysis suggests that once school achievement is controlled for, performance in engineering is independent of whether or not a student took physics, mathematics or chemistry at school.

Physical and natural sciences

Figure 5 shows course pass rates for students who studied physical and natural sciences¹¹ at university.

The first impression is that the results are not as striking as for engineering. The other difference is that the confidence bars are much shorter, reflecting the fact that the number of students studying engineering is relatively small, while science is one of the larger fields of study at university.

In spite of the smaller differences, it is clear that not studying chemistry, biology, physics, mathematics with calculus, and other languages, is associated with significantly lower likelihoods of passing most courses, compared to students who didtake those subjects. The difference is pronounced for chemistry, biology and physics, which happen to be science subjects. Of course, these subjects are again those selected by students with higher school achievement, although the effect for biology in figure 5 is higher than we might have expected given the difference in the average level 2 achievement score (table 1).

Unlike engineering, the probability of passing most courses for students not doing any of the school subjects ever falls below 0.5 for the study of science at university. Not doing chemistry alone results in a probability below 0.7, but students who do not take chemistry at school have a quite low average level 2 achievement score (table 1).

Figure 5: Effect of school subject on course pass rates for students studying physical and natural sciences at bachelors-level at university

School subjects are sorted in order of increasing course pass rate for when the subject was not taken.
Error bars are 90 per cent confidence limits.

Society and culture

Figure 6 shows the results for students studying society and culture.¹² Here, not studying any of the subjects is not associated with lower university performance, with all probabilities of passing most courses above 0.7. Certainly, taking English, statistics and modelling, other languages, chemistry, biology, mathematics with calculus, physics or accounting is associated with significantly higher course pass rates than not taking these subjects. But it would be wrong to conclude these subjects provide an advantage. Students who took these subjects have on average higher school achievement. On the other hand, these results suggest that not taking English may be associated with lower university performance, at least relative to not taking other subjects.

Figure 6: Effect of school subject on course pass rates for students studying society and culture at bachelors-level at university

School subjects are sorted in order of increasing course pass rate for when the subject was not taken.
Error bars are 90 per cent confidence limits.

Management and commerce

Figure 7 shows the results for students studying management and commerce¹³ at university.

Figure 7: Effect of school subject on course pass rates for students studying management and commerce at bachelors-level at university

School subjects are sorted in order of increasing course pass rate for when the subject was not taken.
Error bars are 90 per cent confidence limits.

The results are similar to that seen for society and culture (figure 6), with no likelihood of passing most courses below 0.7. However, the order of the subjects has changed. Performance in a management and commerce degree at university is lower for those students who didn’t take statistics and modelling, economics and accounting, which we find intuitively correct given the skills likely to be needed in this field of study. Yet we again see significant differences between students who did and did not take mathematics with calculus, chemistry and physics. And once more, taking other languages at school is also associated with better performance at tertiary level.

Summary

Had mathematics been the only school subject considered in this part of the study, it would be reasonable to conclude, like Sadler and Tai (2007) that mathematics is ‘an enabling science for a broad range of disciplines’. Interestingly, when we looked at the degrees in health, education or creative arts,¹⁴ mathematics did not show this association, although student numbers are low for these disciplines.

When a broad range of school subjects is considered, it is often mathematics, chemistry, physics, and other languages which are associated with higher levels of university performance, the same subjects which are associated with higher levels of school achievement. But for most fields of study, not taking these school subjects is not associated with ‘low’ levels of university performance, even though the difference between taking and not taking the subject may be statistically significant.

The exception appears to be engineering. It seems that not taking a science subject (mathematics, physics or chemistry) is a disadvantage when studying engineering at university. But this should not be surprising. In any discipline, if there are pre-requisite skills or knowledge required of a student, especially if these are fundamental to the particular area of study, then students with those skills and knowledge will have an advantage, and would be expected to do better. But there is a caveat; simply taking a mathematics class, and not gaining mastery of the skills taught, ought not to provide this advantage. As we will show in the next sections, a student must take the class and achieve a level of understanding that leads to proficiency in the use of those skills and knowledge, for there to be an association with higher levels of university performance.

A possible reason that engineering showed such strong results is that engineering, even when using the broad definition of university degrees as we have in our study, is quite a narrow discipline. Whereas physical and natural science encompasses studies in botany, zoology, chemistry, physics and genetics, for example, engineering is quite narrowly defined. It is possible that the closer the link between specific course requirements, and the particular skills and knowledge gained in studying a subject at school, the more likely we are to find an association between that school subject and performance in those university studies. We consider this question in more detail in section 4.5.

The requirement for specific skills or knowledge may also be evident in some of the other tertiary fields of study: chemistry, biology and physics for the physical and natural sciences; English in society and culture; statistics, economics and accounting for management and commerce. But the same caveat will apply. Simply taking the subject ought not to provide an advantage to a student—but doing that subject and doing it well, may do.

4.2. Controlling for achievement in a single school subject

The results presented in the previous section considered university performance as observed in the study cohort, measured as the proportion of students passing most —more than 75 per cent—of their courses in a broad field of study. An alternative method is to model the data using logistic regression. In this approach, the likelihood of a student passing most of their university courses in the broad field of study can be considered when controlling for other variables.

A problem that needs to be overcome in analysing school achievement is that there is no achievement information available in a subject for students who did not take that subject. While this is largely self-evident, it presents problems in an analysis controlling for school achievement. Including achievement in that subject means we have to exclude the students who did not take the subject. But if we do this, we do not have a control group of students who did not take the subject. This problem has rarely been considered in other studies.

The method we have adopted is to consider pairs of school subjects, where we control for achievement in one subject, and then contrast the results between students who did or did not take a second subject. For instance, we control for school achievement in English and see if there are differences in the university performance of those who took English and mathematics compared to the university performance of those who took English without mathematics. In this example mathematics is the control subject.

It is instructive to consider what the model results might look like if a particular subject were providing some benefit to a student in their tertiary studies. Figure 8 shows this hypothetical result.

Figure 8: Hypothetical result on the expected probability of passing most first-year bachelors courses where doing a subject provides a benefit

Our expectation is that if a particular subject is going to provide students with an advantage in their university performance, that performance ought to be more or less independent of school achievement. At the very least, it might be expected to ameliorate the effects of school achievement, such that students with below-average school achievement would perform better having taking the subject in question compared to those who did not. As school achievement increases, we would expect there might be a decreasing benefit, since there is an upper limit to university performance.

Controlling for English achievement, with and without mathematics with calculus

We chose English and mathematics since these are generally considered to be important subjects, and they represent subjects in the two main groups that students take. There are also sufficient numbers of students in these subjects to model reliably.

Only three tertiary fields of study are considered in this series of analyses: management and commerce, physical and natural sciences, and society and culture. These are the largest disciplines in terms of student enrolments. All of the other disciplines have too few students to model with any degree of robustness.

Figure 9 shows the results. What is immediately apparent is that the results do not resemble figure 8, our hypothetical expectation. Taking mathematics is generally not associated with better university performance for students who also took English.

Figure 9: Expected probability (and 90 per cent confidence limits) of passing most first-year courses by selected fields of study, by school achievement in English, with or without also taking mathematics with calculus [click image to enlarge]

What we find instead is that taking mathematics with calculus is associated with small but significantly higher university performance in management and commerce studies, at least in the middle range of school achievement scores. This mirrors the result seen in figure 7. But we would have expected a similar result for the other two fields of study, given the results seen for mathematics in figures 5 and 6. What we find instead, is that when controlling for achievement in English, taking mathematics does not make a statistically significant difference to students’ university performance in science, or society and culture degrees.

The more important finding is that the largest improvement in university performance is achieved by doing better in English, regardless of the university field of study. The improvement seen for students who took mathematics, where it does provide some advantage, is only marginal in comparison.

In other words, for students who are enrolled in management and commerce degrees, who have the same level of achievement in level 3 English, there is sometimes a small difference in university performance for students who also took level 3 mathematics. Mostly there is no difference in performance between students who did or did not take mathematics when controlling for achievement in English. The largest differences in university performance are between students of different levels of achievement in level 3 English.

With this technique, we are still not controlling for achievement in mathematics. Differences in university performance may still be due to differences in average achievement of students who took mathematics at school. And we are only considering students who took English at school. Table 2 shows that English students were generally less likely to also take mathematics and other science subjects. Students who took English, and also took mathematics, may not be ‘typical’ students, so the effect of mathematics may be different for these students, than say, a student who took chemistry and mathematics. These considerations need to be kept in mind when looking at these results. In this study, we have considered a number of combinations of school subjects, using a variety of analyses, so that overall, we can be confident of our findings.

Controlling for mathematics with calculus achievement, with and without English

Figure 10 shows the results when we consider English as the categorical variable, and school achievement in mathematics with calculus as the continuous variable.

Taking English is associated with a significant improvement in university performance in all three fields of study, at least in the middle range of school achievement in mathematics. But again, while the improvement seen with taking English is statistically significant, it is marginal when compared to the improvement in performance seen with increasing achievement in mathematics.

Figure 10: Expected probability (and 90 per cent confidence limits) of passing most first-year courses by selected fields of study, by school achievement in mathematics with calculus, with or without also taking English [click image to enlarge]

Controlling for chemistry achievement, with and without English

We now consider chemistry and English. We were curious to see if chemistry would produce an effect where mathematics did not. Chemistry appears to be important in a range of tertiary studies (figures 5, 6 and 7), and students who took chemistry showed the largest difference in level 2 achievement against those who did not take chemistry (table 1).

Figure 11 shows students who took level 3 English, and compares the university performance of students who also took chemistry to those who did not.

Figure 11: Expected probability (and 90 per cent confidence limits) of passing most first-year courses by selected fields of study, by school achievement in chemistry, with or without also taking English [click image to enlarge]

The results are similar to those seen previously. Taking chemistry is associated with higher university performance for science studies, but not for management and commerce, nor for society and culture. The results seen in figures 6 and 7 suggested otherwise. And again, the largest improvement in university performance is associated with increasing achievement in English.

Controlling for English achievement, with and without chemistry

This section considers how university performance varies with achievement in chemistry, for students who did or did not also take English.

Figure 12 shows the results. We see that if a student takes chemistry, also taking English is not associated with any significant improvement in university performance.

We might have expected a difference in society and culture, given the result in figure 6.

This result reinforces our previous conclusions; once school achievement is controlled for, differences in results between students who take or do not take a subject largely disappear. When there are differences, these are marginal when compared to the improvements obtained by increases in levels of school achievement in a second subject.

Figure 12: Expected probability (and 90 per cent confidence limits) of passing most first-year courses by selected fields of study, by school achievement in chemistry, with or without also taking English [click image to enlarge]

Controlling for achievement in the visual arts, with and without mathematics

In the introduction we indicated that our study is limited to school subjects taken by students who studied at bachelors-level at university. This means we cannot apply our conclusions too widely, particularly to the non-academic and vocational subjects that are taught in schools, which are not part of the list of ‘approved subjects’ for entry to universities.

Visual arts subjects is one group of subjects that are on the ‘approved subject’ list which may be regarded as requiring skills that are somewhat different to those in other subjects. In our study, we grouped together students who gained standards in photography, painting, design, sculpture and print making.

Figure 13 shows the effect of taking mathematics or not, on students who also took visual arts subjects. Confidence limits are wider than in previous results because of the small number of students who do visual arts with or without mathematics and calculus.

Figure 13: Expected probability (and 90 per cent confidence limits) of passing most first-year courses by selected fields of study, by school achievement in visual arts subjects, with or without also taking mathematics with calculus [click image to enlarge]

It can be seen there is no significant difference in university performance between students who did or did not take mathematics in any of the tertiary fields of study. We see the now familiar relationship with increasing school achievement and university performance, although the highest level of university performance is lower here than we have seen for other subjects. In spite of this, the results clearly show that increasing achievement in the visual arts is associated with increasing levels of university performance, in fields of study that we would presume would not benefit from having studied visual arts, and this occurs for students even if they did not also take mathematics. This occurs in spite of the results observed for mathematics in figures 5–7, and table 1.

Controlling for achievement in mathematics, with and without visual arts

Figure 14 shows the results when we consider achievement in mathematics with calculus, for students who also did or did not take a visual arts subject.

Figure 14: Expected probability (and 90 per cent confidence limits) of passing most first-year courses by selected fields of study, by school achievement in mathematics with calculus, with or without also taking visual arts subjects [click image to enlarge]

Here, we see that there is no difference in university performance in science. There is a suggestion that there may be a difference in management and commerce, and there is a clear difference, at least in the middle to higher levels of mathematics achievement, in society and culture. In contrast to the previous sections’ results, it is the students who did not take a visual arts subject who have the higher university performance, but this is to be expected, given that visual arts students have, on average, lower levels of school achievement (table 1). What we do find surprising is that there is no difference in science (compare with figure 5), and that there is only a marginal difference for management and commerce (compare with figure 7). Again, we are led to the conclusion that, when controlling for achievement in a subject, differences in university performance between taking another subject or not largely disappear. Moreover, it makes almost no difference which two subjects we care to consider.

Summary

In this section we looked at the relationship between pairs of level 3 subjects on university performance, considering one subject as a categorical variable (did or did not take the subject), and achievement in the other subject, treated as a continuous variable. The results show us that in some cases, taking a subject is associated with a statistically significant difference in the likelihood of passing most first-year courses at university. We saw this for students who took English, where also taking mathematics is associated with a difference in management and commerce degrees, but not for science, or society and culture degrees. We also saw that chemistry is associated with a difference in science degrees for students who also took English, but not for management and commerce, or society and culture degrees. English is associated with a difference for students who also took mathematics in all three degree categories we considered, but in none of the degree categories for students who also took chemistry. These rather inconsistent results provide no clear picture of a subject being associated with higher likelihoods of passing most first-year courses. What is consistent is that the likelihood of passing most first-year bachelors courses at university is associated with increasing achievement in a subject, and from the results presented, and others we reviewed but did not present, it appears this applies to any subject in the ‘approved list’.

4.3. Controlling for achievement in two school subjects

If increasing levels of school achievement in a subject—rather than simply taking that subject—is associated with university performance, then the next step is to consider what happens when we control for school achievement in two subjects.

Mathematics with calculus, and English

This section considers students who did both mathematics with calculus and English. Figure 15 shows the results, averaged over all university fields of study. The vertical axis is the same as in the previous sets of graphs, and shows the expected probability of passing most first-year bachelors courses. The two horizontal axes represent school achievement in mathematics with calculus (on the left) and English (on the right), ranging from 10 to 100.¹⁵  The vector AB represents the relationship between the probability of passing most courses with increasing levels of achievement in English, when the level of mathematics achievement is 10. The vectors parallel to AB (visualised in the three dimensional space) up to and including vector CD show this relationship for each step increase in mathematics achievement.

The vector AC, on the other hand, represents the relationship between the probability of passing most courses with increasing school achievement in mathematics, when the level of English achievement is 10. Again, the vectors parallel to this, up to and including BD, show the results for each step increase in English achievement. The curvilinear surface ABDC,formed by these intersecting vectors,represents the probability of passing most courses with varying levels of achievement in both subjects.

Figure 15: Expected probability of passing most first-year bachelors courses against school achievement in mathematics with calculus, and English

For students in their first year of bachelors study at university. All students achieved NCEA level 3 and met the university entrance requirement. Excludes extramural students.

The line ef in figure 15 (and those lines running parallel to it in the three-dimensional representation) represents lines of equal probability, much like a contour line on a map; in the particular case of ef,it is the isoline of 0.6 probability.¹⁶ These isolines assist in interpreting the diagram. For example, one can see from the figure that the probability of passing most courses at the lowest level of achievement in both subjects, at point A, is just below 0.4.

Lastly, the vector AD represents the probability of passing most first-year bachelors courses for the average achievement across both mathematics and English, and matches the result seen in figure 1, in two dimensions, where the results are averaged over all school subjects.

What do the results tell us? Firstly, because the results are almost symmetrical, we can say that university performance increases equally with increasing achievement in English or mathematics.

The results also show that doing well in one subject offsets lower achievement in another, but that doing well in both subjects is associated with the highest level of university performance. We conclude that simply taking English or mathematics is not what leads to better university performance (from figures 9 and 10), but that doing well in one or the other subject, and preferably both, is quite strongly linked to university performance.

It could be argued that the symmetry we see is the result of modelling the results over all tertiary fields of study, with differences between the fields averaging out. We consider fields of study separately in the next section.

We should point out that this symmetry is not seen with all pairs of subjects, as might have been expected given the results in figures 9 and 14.

Chemistry and English

This section considers the results when controlling for achievement in chemistry and English, for students studying society and culture (figure 16) and in the physical and natural sciences (figure 17). Again, we consider students who have taken both chemistry and English at school.

The results in figure 16 are also quite symmetrical, with students with high chemistry achievement and low English achievement performing slightly better (above 0.9 probability) than the complementary situation (below 0.9 probability). In general, about half of the response surface is above a probability of 0.9,¹⁷ indicating a variety of combinations of chemistry and English achievement can lead to high levels of university performance in studies in society and culture.

This symmetry is not unexpected, given the results in figures 11 and 12. There, when we control for achievement in one subject (English or chemistry), taking the other subject or not, in any combination, does not affect university performance in studies in society and culture.

The result when controlling for achievement in chemistry and English for students studying physical and natural sciences at university is shown in figure 17. The lack of symmetry is immediately obvious, and is expected given the results seen in figures 11 and 12.

Figure 16: Expected probability of passing most first-year bachelors courses against school achievement in chemistry and English, for students studying society and culture degrees at university

For students in their first year of bachelors study at university. All students achieved NCEA level 3 and met the university entrance requirement. Excludes extramural students.

Figure 17 shows that at low levels of achievement in both subjects, the likelihood of passing most science courses is just below 0.2. With increasing levels of achievement in chemistry, at the lowest level of English achievement, the change in probability rises to 0.9. Yet for the lowest level of chemistry achievement, increasing English achievement raises the probability to nearly 0.6.

Figure 17: Expected probability of passing most first-year bachelors courses against school achievement in chemistry and English, for students studying physical and natural science degrees at university

For students in their first year of bachelors study at university. All students achieved NCEA level 3 and met the university entrance requirement. Excludes extramural students.

Another way of looking at the results is to consider the relative achievement levels in chemistry and English for a student to have at least a 0.7 chance of passing most of their courses. A student with English achievement of 30 needs a chemistry achievement of 50 to have this chance of passing most first-year courses, whereas a student with English achievement of 60 needs to achieve a score of 30 in chemistry. In other words, doing well in English offsets poor achievement in chemistry in a science degree.¹⁸

While it is also clear that increasing achievement in chemistry is associated with a greater improvement in the likelihood of passing science courses, a higher likelihood occurs with higher levels of English achievement. In other words, improving achievement in chemistry above a score of 50 (about average) makes little difference to the likelihood of passing science courses once English achievement is also above average. And at the higher levels of English achievement (70 or higher), even students with below-average chemistry achievement (those with scores 30–50) have likelihoods of passing science courses mostly above 0.8.

4.4. Controlling for achievement across school subjects in common

The previous analyses considered the effect on university performance of single or pairs of school subjects. While this is a valid approach, a problem with this method is that a student will have taken a range of other subjects at school, in addition to the one or two being analysed. These other subjects will have provided the student with skills and knowledge, some of which may have been important in determining their performance at university. It is difficult to control for the effects of those other subjects.¹⁹

An alternative method of analysis involves looking at the results of students who did and did not do a subject, and to consider their school achievement in just the subjects they have in common.²⁰ For example, university performance can be compared for students who did not do mathematics with students who did do mathematics, controlling for school achievement across these students’ subjects except mathematics. If taking mathematics at school makes a difference to university achievement, there ought to be differences in university performance between these two groups.

Using this method, we considered the level 3 subjects mathematics, chemistry, accounting and English, and performance at university in management and commerce, science, and society and culture degrees. We modelled university performance as the likelihood of passing most first-year bachelors courses, against overall level 3 school achievement in the common subjects, with a separate model for each of the four school subjects. We also controlled for whether a student took the subject in question or not, and the university field of study. We included all possible interactions of these three variables in the models. Adjusted R2 values in the four models were over 0.25, and the models predicted the correct outcome for students in about 78 per cent of the cases. The models were therefore robust and reliable. Like our previous analyses, we also excluded students for subjects if the student gained less than 14 credits in that subject.

The results again show that for each school subject considered, overall school achievement was the strongest predictor of university performance. But in nearly every case, university performance was the same whether a student took the subject in question or not. In just four cases was there a difference. The strongest effect was for accounting, for students taking management and commerce studies (figure 18).

Figure 18: Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 accounting or not [click image to enlarge]

Very small, but still statistically significant effects were seen for English in society and culture, and science degrees (figure 19), but only for a narrow range of school achievement. There was also a small difference for students who had taken chemistry at NCEA level 3 and progressed to a science degree at university, again for just a narrow range of school achievement (figure 20). The difference in likelihoods between the two groups of students was generally extremely small, much smaller than the differences seen in figures 9 to 14.

There were no statistically significant differences in university performance between students who did or did not take mathematics in any of the three fields of university study considered (figure 21).

Figure 19: Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 English or not [click image to enlarge]

Figure 20: Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 chemistry or not [click image to enlarge]

Figure 21:  Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 mathematics with calculus or not

It is interesting to note that when we modelled the effect of taking mathematics at school and school achievement on university performance without controlling for the interaction between mathematics and school achievement, mathematics was significantly associated with higher university performance. But when we in included the interaction between taking mathematics and school achievement, university performance was found to be independent of whether a student took mathematics or not at school. We conclude that taking mathematics is associated with higher university performance only because mathematics students have higher average school achievement.

As a whole, these results confirm our earlier findings. The subject a student takes at school has little bearing on their university performance when we control for school achievement. The strongest association is always between university performance and school achievement. Where higher university performance is associated with a particular subject, it is often in subjects that have some relation to the area of university study. This is seen with NCEA accounting, and management and commerce degrees at university, and for chemistry and science degrees. But even where a subject is associated with higher levels of university performance, low levels of school achievement in that subject are associated with low levels of university performance.

4.5. Controlling for achievement in one school subject in narrow fields of university study

It is usually the case that first-year university students enrol in a broad range of courses, with specialisation occurring in the second year. For example, first-year science degree students may opt to take a subject offered by one of the non-science faculties, while business degree students may be encouraged or required to take statistics and computing, in addition to economics and management. Measuring university performance in broad fields of study, as we have done so far, is therefore appropriate for first-year students. However, some of these broad fields of study cover a range of different disciplines. For example, the broad field of study of society and culture contains degrees in arts, social sciences, law, and language and literature studies, while management and commerce includes accounting, management and finance, but not economics. It may be the case that, on average, a school subject has no association with university performance in a broad field, because a positive association in one of the component degrees may be balanced by no association in another. Therefore, we also analyse performance at course level at a more narrow level of definition of the field of study.

We considered courses in the fields of mathematical sciences, chemical sciences, accountancy, economics, law, and language and literature. We then modelled the likelihood of passing most courses (more than 75 per cent) in these courses in a specific field of study, against whether a student took a particular school subject or not, and the level of school achievement over all level 3 subjects.²¹ In effect, this shift explores the question: is the knowledge and skills acquired in a particular school subject a prerequisite for success in a university course? Students with less than 0.25 EFTS in a particular field of study are excluded, which corresponds to less than two papers in a year of study. The school subjects considered were mathematics with calculus, chemistry, accounting and English. Each school subject was modelled separately, and all two-way interaction effects were included in the models. Each model had an adjusted R2 of about 0.30, and a C statistic of about 0.80, with a total sample size of 15,267 students. Table 4 in the appendix shows the relative sample sizes for the different school subject/university degree groups for each model used in this section.

Accounting

For students who took accounting at school (figure 22), those who went on to study accounting at university show significantly higher levels of university performance when compared to students who did not take accounting with the same level of school achievement. This was the strongest association seen for any school subject/university course combination.

There is a weak association between taking accounting and performance in economics at university, even though there is a relatively strong likelihood that students who took accounting at school also took economics (table 2). There is also a weak association with performance in mathematical science courses. In these two latter results the higher performance occurs only for a narrow range of school achievement.

Figure 22: Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 accounting or not

There is no significant difference in performance in any of the other three fields of study, for students who did or did not take accounting at school, at any level of school achievement. However, there are substantial differences in performance between students with different levels of school achievement. Higher university performance is seen for students with higher levels of school achievement, regardless of whether they took school accounting or not. And for most students, this even occurs for those in accountancy. In other words, a student with higher NCEA achievement who did not do accounting at school is likely to do better in accountancy at university than a lower ability student who did do accounting at school. Simply taking accounting at school is not necessarily associated with higher levels of performance in accountancy studies at university—a student must take accounting and achieve well across their NCEA subjects. This finding can also be made for each of the school subjects considered in the following pages.

Mathematics with calculus

Figure 23 shows the results for students who took mathematics with calculus at school, for the same 6 fields of study at university.

Figure 23: Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 mathematics with calculus or not

The data shows there is a moderate association between performance in mathematics at school and performance in mathematical science studies at university, and a weaker association with performance in economics. Interestingly, there is no association with performance in accountancy despite the fact that having taken accounting has a small association with performance in mathematical sciences at university.

We also checked the results of taking mathematics at school on accountancy at university when not controlling for school achievement. Students who took mathematics at school showed a probability of passing most first year accountancy courses of 0.78±0.02 (probability and 90 per cent confidence limit), compared to 0.71±0.02 for students who did not take mathematics. Thye conclusion would have been that mathematics is positively associated with better performance in accountancy. Yet when controlling for school achievement, there is no difference in performance between these two groups of students.

Chemistry

Figure 24 shows the results for level 3 chemistry.

Figure 24:  Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 chemistry or not

It is immediately apparent that few students go on to study chemical sciences at university who have not taken chemistry at school, as shown by the wide confidence limits in the graph for chemical sciences in figure 24.

We see that, when controlling for school achievement, taking chemistry at school is associated with higher performance in chemical science studies at university, and also with higher performance in mathematical science studies. In both cases, the association only occurs in a limited range of school achievement, although the range would include the majority of students. There is also a weak association with performance in language and literature degrees, but again for a narrow range of school achievement.

English

The results for students who took level 3 English or not are shown in figure 25.

Figure 25: Expected probability (and 90 per cent confidence limits) of passing most first-year bachelors courses by selected fields of study, by school achievement, for students who took level 3 English or not

There are no strong associations between the performance at university and taking English at school, but a weak association can be observed for performance in law studies. Interestingly, no association is seen in language and literature studies. A weak association was seen for communication and media studies (but that result is not shown).

Summary

In this section we have shown that when considering specific subjects taken at school and narrow fields of study at university, particularly where the school subject and degree study are common subject areas, an association is found between better performance at university and the taking of those subjects. Associations were found for accounting at school and accountancy studies at university, mathematics and mathematical sciences, chemistry and chemical sciences, and English and law studies. However, accounting at school is also associated with higher performance in mathematical sciences and economics, while mathematics at school is associated with better performance in economics, but not accounting. Studying chemistry at school is also associated with better performance in mathematical sciences, and language and literature studies.

These results broadly mirror our earlier findings, when the university study was considered at a broad level. Accounting at school was associated with higher performance in management and commerce (figure 18), which includes accountancy studies; chemistry or mathematics at school were associated with better performance in science degrees (figures 20 and 21), which includes mathematical and chemical sciences; and English at school was associated with better performance in society and culture (figure 19), which includes studies in law.

Most of the associations are weak. For students with the same level of school achievement, there is little difference in performance. In most cases, there are no differences in performance across the entire range of school achievement. Accounting at school and accountancy studies at university is the exception; large differences in performance occur between students who did or did not do school accounting, and statistically significant differences occur across the entire range of school achievement. However, even for accounting, and in every other case, students with low school achievement have markedly lower performance at university compared to students with higher school achievement whether or not they have taken the preparatory subject at school. Where a school subject is associated with higher levels of university performance, the level of improvement is much less than the increase in performance seen for students with higher school achievement, irrespective of whether or not they took the particular school subject that is associated with the higher performance.

We analysed a wider range of university courses in fields of study and school subjects than we have reported in this section. An interesting result (for which we do not provide the data here), was that for students who took accounting at school, and studied creative arts at university, there was a negative association between taking the subject at school and university performance. This contrasted with the strong positive association found for students who took accounting at school and enrolled in accountancy studies. On the other hand, for students who took visual arts at school, the associations were reversed at university. For these students, the positive association was for performance in creative arts at university, and the negative association was found for performance in accountancy studies. These results suggest that preferences for a way of thinking and working— accuracy versus creativity, numeracy versus artistry—may be important in both the choice of school subject and degree study, and is clearly associated with student performance, although we are not suggesting these are mutually exclusive dichotomous preferences. Other studies have also found that university performance can be associated with personal preferences (Felder, Felder and Dietz 2002).

4.6. Controlling for achievement in two school subjects for accountancy students

This last section explores in a little more detail the association between taking accounting at school, and performance in accountancy courses at university. We were interested to see what impact a second school subject had on the relationship between accounting and accountancy, given it was the strongest association found between a school subject and a course of study at university. As noted previously in this report, when controlling for one school subject, it may be that another school subject taken by a student is more strongly associated with better performance at university. Checking how two subjects together are associated with university performance will enable us to see if there are interactions between them.

We modelled, using logistic regression, the likelihood of passing most first-year accountancy courses, controlling for school achievement, and investigating the effect of whether a student had taken accounting or not, and taken another school subject, or not. We used backward selection²² to limit the models to just those variables and interactions which were significant in a model. The other school subjects considered were: economics, physics, biology, statistics and modelling, mathematics with calculus, English and chemistry. A separate model was run for each of these school subjects. We tested the correlation between accounting and every other subject considered and found that the correlations were not strong enough to cause problems in the models.

In the results below, unless otherwise stated, school achievement was set to the average school achievement score for accountancy students.

• Economics and physics had no association with performance in accountancy when accounting was also taken at school, when controlling for school achievement. A student with average school achievement had a likelihood of passing most first-year accountancy courses of 0.57±0.04 (estimate and 90 per cent confidence limit) if they did not take accounting at school, versus a likelihood of 0.85±0.02 if they did. These likelihoods can be seen in figure 22.
  
• Taking biology at school was positively associated with better performance in accountancy courses, independent of the effect of taking accounting at school, when controlling for school achievement. That is, there was no interaction between these two school subjects. Students with average school achievement who took biology showed better performance in university accountancy, regardless of whether they took accounting at school or not. However, students who took accounting at school showed higher absolute levels of performance. The likelihood of passing most first-year accountancy courses for students who did not take biology and did not take accounting was 0.53±0.05, and 0.84±0.02 for those that did take accounting. For students who did take biology, the likelihoods were 0.66±0.06 and 0.90±0.03 respectively.
  
• Statistics and mathematics showed significant interactions with accounting at school, again after controlling for school achievement. Taking accounting at school was positively associated with better performance in accountancy at university, with or without also taking mathematics or statistics at school. On the other hand, taking mathematics was not associated with better performance, regardless of whether a student did or didn’t also take accounting at school. Taking statistics was associated with better performance in accountancy at university only when accounting was not also taken at school. Table 3 shows the expected likelihoods of passing most first-year accountancy courses for these school subjects.

• When controlling for school achievement, taking English at school is associated with better performance in accountancy at university when students also took accounting, but not when students did not take accounting. That is, taking English and accounting showed a stronger association with university performance in accountancy studies than for students who took accounting at school without English. For students who did not take accounting at school, the likelihood of passing most first-year accountancy courses was 0.57±0.06 for students who also took English, and 0.58±0.07 for those that did not. The likelihoods increased to 0.82±0.03 for students who just took accounting without English at school, and 0.88±0.02 if they took accounting and English.
  
• Taking chemistry at school was also positively associated with better performance in accountancy at university, independent of taking accounting at school, but unlike the other school subjects described above, the effect varied with school achievement. The expected likelihoods from the model for chemistry and accounting are presented in table 4. 
  • For students with average school achievement (which for these accountancy students was 54.6), taking either accounting or chemistry at school was associated with better performance in university accountancy studies, independent of whether the student also took chemistry or accounting at school, although taking accounting alone was associated with better performance than taking chemistry alone.
    
  • For below-average school achievement (set at 1 standard deviation below the average, or at a score of 41.2), taking accounting at school was associated with improved performance when chemistry was not taken, and chemistry was associated with better performance when accounting was not taken. Chemistry was not associated with improved performance when accounting was also taken, but accounting was associated with slightly better performance if chemistry was also taken.
    
  • For above-average school achievement (set at 1 standard deviation above the average, or at a score of 68.0), taking accounting at school was associated with better performance regardless of whether chemistry was also taken or not, but chemistry was associated with better performance only if accounting was also taken at school. Without accounting, taking chemistry was not associated with any improvement.
   

Summary

Accounting at school continued to show a strong association with performance in accountancy studies at university, but other school subjects variously modified the relationship. However, no consistent picture emerges. Some subjects do not affect the relationship, whereas others are related by second-order interactions (where the result depends on both school subjects), while others are related by third-order interactions (where the result depends on both school subjects and also on the level of school achievement). In practice, trying to predict what suite of school subject might be useful prerequisites for further study at university will be problematic. This is underscored by the results in this section: taking biology is associated with better performance in accountancy, while taking economics or mathematics at school is not. In our analysis we have only controlled for two subjects, yet it is likely that including a third or fourth school subject will result in even greater diversity of relationships.²³

In spite of the diversity in the relationships, the results show that taking accounting at school is strongly associated with better performance in accountancy at university, regardless of what other subjects are taken. It is interesting to speculate on why this relationship is so strong, given that the association seen with other subjects and university courses, when there was an association, was far weaker.

If the topics covered in the school and university studies overlap to a large extent, then we might presume that students who took the school subject, and continued on with that subject at university, would have an advantage over students who did not take the subject at school. Similarly, if there was little or no overlap between the school curriculum in a subject and the topics covered at university for that subject, then we might presume that having taken the subject at school might offer less advantage to those students. The determining factor affecting performance would be the propensity for a student to learn new material, which would be indicated by their ability to have done this previously—in other words, their level of school achievement. Of course particular skills or knowledge learned at school that are not taught at university, would also give a student an advantage.

While we cannot test this hypothesis, it may be that the reason we find accounting at school is more strongly associated with performance in university accountancy courses is that there is a large degree of overlap between the school and university course material. The smaller levels of association found between other school subjects and university courses might be due to some overlap in course material, but not enough to give students who had previously seen this material at school an advantage. Either way, students who had demonstrated an ability to learn new material, as indicated by their school achievement score, would perform better than those who had not.

This is supported by our results. The most consistent result of the analyses in this section was the strong association between school achievement and performance in accountancy at university. In each case, whatever variables or interactions were eliminated from the models using the backward selection process, school achievement remained in the models, and consistently it was this variable which had the highest association with the likelihood of passing most first-year accountancy courses, regardless of what other subjects were taken at school.

Footnotes

10. Refer to table 2 for subject choice correlations.
11. Physical and natural sciences includes studies in biological, earth and chemical sciences; physics and astronomy; and mathematical sciences.
12. Degrees in society and culture include studies in humanities and social sciences; law; political science; language and literature; philosophy; economics and econometrics; and sport and recreation.
13. Degrees in management and commerce include studies in accountancy; business and management; sales and marketing; tourism; and banking, finance and related fields
14. Results not shown.
15. Due to limitations in the graphing software, it is not possible to represent the origin as 0,0. This limitation does not affect the results or the interpretations of the graphs.
16. The value of the isoline is most easily determined by counting down form the topmost isoline, which is 0.9.
17. The 0.9 probability isoline runs almost from corner to corner of the response surface.
18. Of course, it may be that these English students are taking other science subjects at school, but as seen in table 2, this is not as likely as the English students taking other languages and humanities subjects.
19. The models could have included more subjects, but this would have reduced sample sizes considerably, since each student needs to have done all of the subjects in the model. And limiting the model to those subjects where there are sufficient students doesn’t solve the problem.
20. This approach was suggested by Dr. Michael Johnston of NZQA.
21. We have used school achievement measured over all level 3 subjects, rather than ‘subjects in common’, as we did in section 4.4, because the more complicated analysis produced essentially the same results as the simpler one.
22. Backward selection involves starting with a model which includes all variables and their interactions, and then iteratively removing those variables and interactions that are the least significant. The level of significance was set at 0.05. The number of students, and the number of starting parameters in the models, indicated that backward selection was an appropriate method for model selection.
23. Our explorations showed that this was likely, but student numbers in the extra categories become too small to model reliably.