Academic performance of first-year bachelors students at university

4. The study variables

This section considers the various factors used in this study. It discusses the nature and general characteristics of the factor in the study population, and any caveats or considerations in their interpretation.

4.1. Academic achievement at school

Academic achievement at school is measured with the NCEA level 3 achievement score. The achievement score variable has been used in other studies (Ussher 2008, Scott 2008, Earle 2008, Engler 2010). This measure of student achievement was developed for analysing NCEA results by Michael Johnston at the New Zealand Qualifications Authority (NZQA). Readers are referred to Ussher (2008) for a more detailed description of this variable.

The achievement score rates students’ grades in their NCEA level 3 qualification against other students in the same year, producing a score between 0 and 100. Students who gained level 3 credits with excellence and merit grades will score higher than students who gained credits with relatively fewer merits or excellencies, or with relatively more achieved grades. The score also adjusts for the level of difficulty within a standard. A student, who achieved an excellence in a standard where many people gained a merit or excellence, will receive a lower score for that standard, while a higher score is given to a similar student in a standard where most people received an achieved grade, for example.

For the whole study population, the mean achievement score was 51.7 (median 50.4), and ranged between 5 and 95. Most students had achievement scores between 20 and 90. Figure 1 shows the distribution of students across the range of achievement scores. It can be seen that the distribution is skewed slightly towards higher achievement scores, a result expected given this is a population of students who elected, and were selected, to study at bachelors level at university.

Figure 1: Distribution of students by NCEA level 3 achievement score

Recent research in New Zealand has shown that student motivation is strongly correlated with achievement in the NCEA system (Meyer et al 2009). Students who indicated in Meyer’s study that they were Doing their best were significantly correlated with the attainment of more credits, more internally passed standards with excellence, and externally assessed standards at all levels. Students who indicated Doing just enough were associated with lower achievement and significantly associated with higher total unit standard credits⁷. Since the relative number of unit standards and achievement standards, and how well a student performs in the latter are used to calculate the achievement score, it follows that the achievement score is correlated with motivation as well as ability.

Bowen et al (2009) concur. They argue that high school grades reveal more than mastery of content. They also reveal qualities of motivation and perseverance — as well as the presence of good study habits and time management skills — that say a great deal about the chances that a student will complete a tertiary programme. This is independent of the high school attended, since they found, as this study finds, that students’ high school grades predict tertiary outcomes regardless of whether the school had high academic standards or not. In the present study, given that the study population already met a minimum academic standard (NCEA level 3 and UE), it is not surprising that achievement score correlates so highly with passing first-year courses.

In 2007, the NCEA reporting system was changed to include endorsements on certificates of student achievement. Previously, a certificate only showed that a student had gained a particular NCEA level. With the change, the certificate also indicated whether the student achieved the NCEA level with merit or excellence. This had the effect of generally increasing student motivation (Meyer et al 2009); only about 10 per cent of students indicated the change did not matter to them. In the present study, this change to a student’s motivation would have affected students in the 2008 tertiary cohort (primarily those who left school in 2007). However, when the study population was tested, no significant difference between years was found. This may be due to the fact that the change to the NCEA reporting method was announced in mid-2007, so would have had a minimal affect on the motivation of the 2007 school-year students. Meyer et al (2009) found that only 53 per cent of students in 2007 knew about the planned changes. Whether this change has an impact on later cohorts of students needs to be tested in future.

4.2. Ethnic group

The four main New Zealand ethnic groups are considered; Asian, European, Māori and Pasifika. Ethnic groups are reported using the categories never-, ever- and sole-ethnicity.

In summary, the ethnic categories are based on the recorded ethnic identifications across the multiple data sources used in the study. If a student is only ever identified as Māori—across all years and across all data collections—they are counted in the sole-Māori category, and also in the never-European, never-Pasifika, and never-Asian categories. If a student is sometimes identified as Pasifika, and at other times as Pasifika together with another ethnicity, or as another ethnicity entirely, they are counted in the ever-Pasifika category, as well as in the other categories that apply. And if a student is never identified as European, either alone, or in combination with another ethnicity, they are counted in the never-European category, and in the other appropriate ethnic categories. For each student there are four ethnic variables, one for each of the main ethnic groups, and each variable holds one of three states (never-, ever- and sole-). For a complete description of these categories, and the reasons why they are used, see Engler (2010).

It should be noted that the never-, ever- and sole- descriptions are based on a six-year period from which the study population was drawn, not just the three years the study population were enrolled in tertiary education. The ethnic identification of a student might change again in the future. However, a six-year period provides a better indication of ethnic identification than a single snapshot in time.

Table 1 shows the sample sizes of the ethnic groups and their within-ethnic-group categories. It can be seen that sole-Māori and sole-Pasifika, as a proportion of the individual ethnic groups, are far less represented than either sole-European or sole-Asian groups. Comparing this table with table 2 in Engler (2010), which shows the same data for school leavers, it can be seen that the European and Asian sole-ethnic categories are virtually unchanged, while sole-Māori has dropped from 33 per cent to 21 per cent, and for Pasifika, 48 per cent to 38 per cent. This indicates significant shifts in the composition of these ethnic groups in the transition from school to tertiary study. This suggests that the students in these two sole-ethnic groups are more likely to have lower school achievement scores, or have attended low-decile schools, or both, which are associated with lower likelihoods of students electing to progress onto bachelors-level study (Engler 2010).

It should be pointed out that the never-, ever- and sole-ethnic method of reporting ethnicity is not a measure of, or a proxy for, the strength of a person’s cultural or ethnic affiliation. These categories simply represent the history of an individual student’s declarations on data capture forms over a period of time, and do not reflect the range of reasons a student might choose one or more particular categories. For example, a person who is regarded as ever-Pasifika may have ties to their culture as strong as, or stronger than, a person regarded as sole-Pasifika, and similarly for people in the other ethnic groups. The measure of ethnicity in this study, as in most administrative data, represents the identification of a person’s ethnicity⁸. It is what a person has said they are, when asked to indicate their ethnicity on a form or in a census. This is distinct from the identity of a person, which is the ethnicity they think they are. Two further facets of identity can be defined: attachment, which indicates to what extent a person can speak the language, knows the customs, and participates in their ethnic group’s cultural activities, for example, and orientation, which is a person’s ethnic identity in a given situation or context (this applies mostly to those people with multiple ethnic identifications). While these other facets of identity may (or may not) have an influence on educational and other outcomes, they were not able to be measured in this study.

This method of reporting ethnicity results in some students belonging to more than one ethnic-group category. Comparisons between ethnic groups must take this into account, although this does not affect within ethnic group comparisons. Specifically, a student will be counted in two (or, more rarely, three) of the ever-ethnic categories if the data shows they identified with two (or three) of the ethnic groups. Students in the sole-ethnic categories, by definition, can belong to one and only one of these categories. Students in the never-ethnic category for one ethnic group will belong to either the ever- or sole-ethnic group category of another ethnic group.

Figure 2 shows the distribution of students by achievement score and sole-ethnic group in the study population. It can be seen that the distributions differ slightly by ethnic group, although generally there is much overlap.

Figure 2: Distribution of students for NCEA level 3 achievement score and ethnic group

The study uses the ethnic groups at a relatively high level, combining the various sub-groups below this level of reporting. The Pasifika ethnic group contains students from the individual Pasifika nations; Tonga, Niue, Fiji and others. However, the small numbers of students in most of these individual nations preclude their being analysed separately. The same applies to the other ethnic groups; iwi are not separately analysed for the Māori ethnic group, nor are the individual countries that make up the European and Asian groups.

Analytically, this may not be a problem for Pasifika students. Madjar et al (2010) found that Pasifika students, whether New Zealand or overseas born, or of mixed or single ethnic heritage, had stronger connections to a generic Pasifika identity, and not to an individual Pacific Island nation. Madjar et al stated that the Pasifika participants in their study had a strong sense of Pacific Island identity, both in how they referred to themselves, and in how they perceived they were seen by others.

4.3. Timing of progression to tertiary study

The effect of students taking time off between leaving school and starting tertiary study has been little studied, although it is reported that the taking of a gap-year has been increasing in popularity in Australia (Birch and Miller 2007). In the population of the present study, between 7 and 8 per cent of students took a gap year in the years 2006 to 2008. This compares to 6 per cent for students at the University of Western Australia in the years 2002 to 2004 (Birch and Miller 2007), and 8 per cent in the UK in 2002 (quoted in Birch and Miller 2007). Holmlund et al (2008) report that among Swedish university entrants around the turn of the century, some 25 per cent had taken 2 to 4 gap years, and around 40 per cent had more than five years gap.

Previous studies on the effects of a gap year have produced conflicting results (Birch and Miller 2007). Some studies demonstrate that students benefit from a gap year, while others report that a gap year adversely affects academic outcomes and later labour market outcomes. Adelman (2006) advocates no delay in attending college in the US, with length of delay increasing their chances of not completing a degree. He notes however, that once a student begins tertiary study, their actual tertiary academic performance is more important in determining whether they gain a degree than whether they progressed directly to tertiary education after leaving school, or took some time off. Also in the US, Horn et al (2005) show that, compared to students who progressed directly, the likelihood of completing a post-secondary qualification, or still being enrolled after six years, was significantly lower only for students who delayed for no more than one year. The results for students who delayed for longer periods of time were not statistically different from the directly progressing students. The short duration of the data in the present study limits the analysis of gap-year students to those who only take a one-year gap. Future studies will determine the effect of a longer break between school and tertiary study in New Zealand.

Holmlund et al (2008) suggest that taking a gap year in Sweden is motivated by the desire to wait for better educational opportunities, and this is largely irrelevant for top performing students, where the stronger school credentials imply generally higher chances of being accepted in any university course.

Bornholt et al (2004) showed that students deferred their entry to an Australian university because of low confidence in their abilities and interest, the need for living expenses, the attitudes of parents, teachers and friends, and hesitancy about their course choice⁹. This suggests that those students who can overcome some of these concerns, or regain their interest, may go on to study at university.

The likelihood of taking a gap year varies. Birch and Miller (2007) found in Australia that taking a gap-year is positively correlated with being female, living outside a capital city, being born in an English-speaking country and speaking English at home. They also found a negative relationship between the probability of deferring university and students’ score on the university entrance examinations. They found no significant relationship with the socio-economic status of the student.

In the population of students in the present study, taking a gap-year was positively correlated with being sole-European. Being female was not significantly associated with taking a gap-year. A high achievement score (>75) was negatively associated with taking a gap-year, as was coming from a low-decile school. Pasifika and Asian students were less likely to take a gap year. Having taken a gap year, students were less likely to study full-time full-year.

In the present study it was not possible to determine what motivated a student to take a gap year, and what they did in their gap year. These are areas for future qualitative studies, particularly given the positive impact that taking a gap year has on first-year tertiary performance.

4.4. School decile

Few (3 per cent) of the students in the study population had attended a decile 1 or 2 school, with 44 per cent having attended a decile 9 or 10 school.

This difference reflects the proportion of students eligible, and wanting, to study at bachelors level. The previous study on this population of students (Engler 2010) showed that students from low-decile schools were significantly less likely to go on to study at bachelors level, in spite of having the credentials to do so. There are also fewer low-decile school students with the credentials to study at university. Using the cohort of students who left school between 2004 and 2007, who gained NCEA level 3 and progressed directly to tertiary study, 82 per cent of students from low-decile schools had gained UE, compared to 95 per cent from higher decile schools.

Table 2 shows the distribution of bachelors-degree students studying at university by achievement score bands and school decile categories. The data shows the disproportionate representation of students from high-decile schools. It also clearly shows that students from low-decile schools have on average lower achievement scores than their peers from other schools.

School decile is a limited but useful explanatory variable in education research. Many analyses show school decile having a significant association with educational outcomes. That it is important is quite clear; what is not as clear is what the variable actually measures.

A school's decile indicates the extent to which the school draws its students from low socio-economic communities, which is based on household income, parental occupation, receipt of benefits and parental education. Decile 1 schools are the 10 per cent of schools with the highest proportion of students from low socio-economic communities, whereas decile 10 schools are the 10 per cent of schools with the lowest proportion of these students. Deciles are used to provide funding to state and state integrated schools to enable them to overcome the barriers to learning faced by students from low socio-economic communities. The lower the school’s decile, the more funding they receive¹⁰. In education research, the school decile is often used as a proxy for socio-economic status in the absence of any better measure. Since the school decile is a property of the school (or the community or catchment from which the school draws its pupils), when analysing individual student data, each student from a school is given the decile rating of that school. This is problematic because all secondary school draw students from a wide range of socio-economic groups, so that any individual student may come from a different socio-economic group. This problem is usually accepted by researchers, given how important school decile is as a predictor variable, and conclusions reached by these studies often give caveats about the interpretation of the results. The error this introduces is less severe if the focus is on differences between low- and high-decile schools, that is, comparing decile 1 and 2 schools with decile 9 and 10 schools.

However, school decile is also likely to be a proxy for a number of school characteristics which are important in determining academic success. The quality and number of resources the school has available for its students may be correlated with the decile rating, as may factors associated with teaching, and teacher and parental expectations. This is not to suggest that only good teachers are to be found in higher decile schools, or that parents from all backgrounds do not have high aspirations for their children to succeed. But there are a number of factors that affect educational achievement that run parallel with the school decile rating. These are summarised for New Zealand schools by Ladd and Fiske (2001):

• Schools with high concentrations of disadvantaged children are forced to spend more time and effort establishing basic student routines, and devote more attention to pastoral and disciplinary activities, meaning less time is available for higher learning.
  
• High decile schools are often able to attract more capable teachers. Teachers with a given amount of experience receive the same basic pay regardless of where they teach. Some teachers find teaching in a high decile school more satisfying because they think the students are more motivated and ready to learn.
  
• High decile schools have better resources. Schools receive most of their funding from the government, but are permitted to raise additional funds from parents in the form of donations and other sources. Higher decile schools typically raise more local funds from parents and other sources than do lower decile schools. This does not offset the greater funding lower decile schools receive from government, but they represent only part of the contribution parents make. The greater the parents’ educational and financial resources, the more they can contribute in a variety of ways to the quality of that school. They also have access to parents from all walks of life who can organise and assist with school trips and cultural activities, and otherwise enrich the classroom environment.
  
• Higher decile schools will have a positive peer effect on student learning, where students are motivated to work harder in the presence of motivated and successful peers.

Thrupp and Lupton (2006) reiterate these same points. Socio-economic composition affects school processes in numerous ways which would cumulatively boost the academic performance of schools in middle-class settings and suppress it in low socio-economic settings.

Research has found other factors that, while not confined to low-decile schools, may be more prevalent in them. These include potentially high-achieving students’ suppressing their level of motivation so as not to be seen as ‘geeks’ or ‘nerds’ (Tyson et al 2005).

It is not possible to separate these factors or to include them individually in the analysis, but the salient point is that students from low-decile schools are more likely to leave school with lower levels of attainment, and have less experience learning in a motivated and motivating environment. What implications these have on the academic performance at university is taken up in the discussion.

4.5. Study type

Study type refers to the study load taken by a student. A student can be studying full-time or part-time for an entire year, or full-time or part-time for part of a year.

Most (94 per cent) of the study population were enrolled in full-time study for the full-year. The small numbers of students in other study types precluded a detailed analysis of this variable. The approach taken was to include all study types in the statistical analysis, to control for differences between them, but to only report the results for full-time full-year students, unless otherwise noted.

Scott (2006) found that the type of study was not a factor in passing courses, and that part-time students pass courses at the same or higher rate as students with larger study loads. In contrast, the data in the present study suggest that full-time full-year bachelors students had higher likelihoods of completing most of their courses¹¹.

4.6. Gender

The sexes are not evenly represented in the population of students in this study, with female students making up 62 per cent of the total. A similar ratio is found in each of the sole- and ever-ethnic categories for each ethnic group, for students taking a gap year or not, and across the school decile categories. As noted in Engler (2010), this imbalance reflects differences in achievement levels between females and males in attaining NCEA qualifications.

There were some differences by gender. Male Pasifika students were more likely to take a gap-year than female Pasifika students (although there were relatively few male Pasifika students), and overall, there were proportionally fewer male students with higher achievement scores. There were also differences with study type. Disproportionately more males were studying part-time full-year and full-time part-year. The full-time full-year and part-time part-year study-type categories showed about the same gender ratios as the overall study population.

4.7. Type of attendance

The type of attendance refers to whether a student studies intra- or extramurally. In the data available for this study, just 238 students (0.7 per cent) were studying extramurally. This reflects the fact that take-up of extramural study at degree level is overwhelmingly by older students, whereas the study population comprises young people. All analyses in this report exclude extramural students.

4.8. School credentials

In the wider population of students from which the study population was drawn (table 3), 92 per cent (32,602÷35,618) had achieved the NCEA level 3 qualification. Ninety seven percent of the NCEA level 3 students had also gained UE. These 31,706 students comprise the study population. Of those students with UE, 95 per cent (31,706÷33,238) had gained level 3. For students whose highest school qualification was NCEA level 2, just over half (53 per cent) had gained the UE standard.

The likelihood of a student passing most of their courses in tertiary education varies with the type of credentials gained at school. Table 4 shows the results. It can be seen that gaining NCEA level 3 improves the likelihood of passing most courses (27 percentage points higher than those without NCEA level 3), regardless of whether the student also has UE. Having UE also improves performance (14 percentage point improvement on average), but having UE only provides an improvement when combined with level 3, not level 2. There are several possibilities as to why this latter result may have occurred, but it is likely to be a limitation of the data, rather than a ‘real’ result¹². In any case, the differences in tertiary academic performance across the different combinations of school credentials are such that it is preferable to limit the study population to students who have both NCEA level 3 and UE.

4.9. Subject studied

In this analysis, the subject studied is based on the broad field of study that has been assigned to the qualification a student is enrolled in. A qualification field of study is an imprecise measure for two reasons. First, many generic qualifications cover several fields, and this not reflected in the field of study assigned to a student. For instance, all students taking a Bachelor of Science are assigned to the field ‘natural and physical sciences’. This means a student studying a BSc in computing science is assigned to ‘natural and physical sciences’, not to information technology. Secondly, most degrees allow students to take courses in several broad fields. A more precise measure would be based on the fields of study of the courses a student has enrolled in. A full discussion of these differences can be found in Scott (2009b). For the purposes of this study, field of study is analysed at qualification level and by broad categories only.

The largest group of students (31 per cent of the total) in the study population were enrolled in courses in the qualification field of study ‘society and culture’, which includes studies in:

• humanities and social sciences
  
• law
  
• political science
  
• language and literature
  
• philosophy
  
• economics and econometrics
  
• sport and recreation.

The next largest group (23 per cent) were enrolled in the ‘natural and physical sciences’, which includes studies in:

• biological, earth and chemical sciences
  
• physics and astronomy
  
• mathematical sciences.

The third largest group (20 per cent) were enrolled in ‘management and commerce’. Here studies include:

• accountancy
  
• business and management
  
• sales and marketing
  
• tourism
  
• office studies
  
• banking, finance and related fields.

The remainder of the students were enrolled in creative arts (10 per cent), health (5 per cent), education (4 per cent) and engineering (1 per cent), with a further 5 per cent in various other disciplines.

The small sizes of some of the subject fields precluded an exhaustive analysis using this factor. In this report, results are reported for all subjects studied, but in the more complex models used in the analysis, results are reported for ‘society and culture’, the largest group.

Figure 3 shows the distribution of students across the more popular subjects studied. Not all ethnic groups are equally represented in each field of study. Sole-Asian students are over-represented in management and commerce, and natural and physical sciences, and under-represented in society and culture. This preference of Asian students for subjects more aligned with mathematics, and less with literacy, has been found elsewhere (Ting 2000, Scott 2009b), and is attributed to them choosing to study in fields thought to lead to better-paid jobs, and because they are reputed to struggle with English. Sole-Māori and sole-Pasifika are over-represented in society and culture enrolments, and under-represented in the sciences.

Figure 3: Distribution of sole-ethnic students across the more popular subjects studied

This ethnic preference for particular subjects, and the fact that some subjects are traditionally more difficult to pass than others, will result in some confounding when ethnic group and subject studied is not controlled.

Footnotes

7. Details about the National Certificate of Education Achievement can be found at on the NZQA website [Accessed: 17 May 2010]. This site also has details about University Entrance.
8. These facets of identity were described by Tahu Kukutai in a paper presented at the University of Otago’s School of Medicine and Health Sciences seminar series,  21 August 2009 titled, "Exploring ethnicity: Concepts, tools and 'evidence'". They are used here with permission.
9. Bornholt et al (2004) considered students who had made an application to study at a university, but then deferred their enrolment once they were accepted. In the present study, it is not possible to distinguish between this type of deferment and one where a student takes a gap year before making an application to study at a university.
10. Quoted from Ministry of Education website [Accessed: 17 May 2010]
11. At the time Scott undertook his study, the study-type variable used in this present study had not been developed. Instead, Scott used the number of courses a student enrolled in as an indicator of study load. In addition, while Scott’s finding that studying full-time is not a success factor for passing courses is generally true across most levels of study,  his data suggests that a medium study load improves course pass rates for bachelors-level study (refer to Scott (2006) table 7, page 7).
12. The most likely reason is that students with NCEA level 2 without UE matriculated with Cambridge International Examination or International Baccalaureate qualifications.