9. NCEA assessment opportunities, choices, and issues

In Chapter 3, we reported the overall NCEA results of the study participants, and the factors among our school, family, and leisure activities-related variables that were linked to different levels of results in the senior school qualification. This chapter takes a closer look at how students thought about and approached their NCEA work. We discuss our findings in the light of challenges that NCEA is not motivating, and that it allows students to make easy choices.

Differences in the design of NCEA assessments are briefly outlined to provide a context for student decisions related to NCEA. We then look at differences in teachers’ experiences of students’ NCEA behaviour in most enjoyed and least enjoyed subjects and in English, and the variables that were related to teachers’ perceptions of student approaches to the NCEA.

The role of NCEA and students’ views of their subjects

The possibility for students to make decisions about which assessments they will undertake (at least in theory) has been a controversial feature of the NCEA. Early in the implementation phase of NCEA, some teachers were already concerned that students could choose to skip assessments (Hipkins et al., 2004) but other teachers actively helped students decide which assessments to skip as a means of managing their workloads. This controversy is interesting for several reasons:

• Although student decisions about whether or not to tackle assessments are more visible now than in the past, they are not new. Students could and frequently did leave out whole sections of examinations. Whereas in the past the consequence would simply be fewer marks from an overall possible total, aspects of a course in which students have chosen not to be assessed are now more apparent.
• Granting students agency to make considered choices about the assessments they will undertake can (again in theory) help them tailor an NCEA qualification to fit their personal learning pathway, rather than shaping up to a “one size fits all” qualification designed by others.
• If this agency is taken up by the student, with appropriate support and advice provided to help them choose well, they could be developing greater autonomy and aspects of lifelong learning skills in the process.

At the moment, however, these arguments seem rather academic. The Learning Curves research suggested that students were not skipping assessments as often as teacher concerns had led us to believe. In this study, we were interested to know if students were doing this, why they were doing it, and how they were making decisions. They were asked to respond to the following three statements about their most enjoyed and least enjoyed subjects and English.

In this subject:

• we can choose what assessments we want to do for NCEA
• the NCEA credits are easy to get
• I will get lots of NCEA credits in this class.

These items have already been discussed in the two previous chapters as part of the student factors attitude to work and relevant learning opportunities. They are repeated here to frame the picture of participation in NCEA that follows. The next table summarises the results, showing the percentage of students who agreed or strongly agreed with each statement in each type of subject. Very few students appear to think they can make an overt and considered choice about which NCEA assessments they will do, either in English or in their most enjoyed or least enjoyed subjects.

Note:

The largest percentages in each line are shown in bold.

NCEA and subject enjoyment and choice

Taken on their own, it would be easy to read the second two statements on the previous table as indicating that subjects are most enjoyed if they build an easy route to NCEA success. However, we have already seen that students seldom say this is why they chose subjects in the first place. They choose them because they are interesting to them, or serve a purpose for their future aspirations.

Students’ most enjoyed classes are ones in which they experience dynamic and interesting learning, with a teacher who holds high expectations of both their effort and ultimate NCEA success. It is likely that we see this reflected here in the students’ anticipation that they will get credits, and get them easily, when they are highly engaged in a positive learning environment.

Since we knew the names of students’ most enjoyed and least enjoyed subjects, it was also possible to check whether any particular subjects were more likely to be seen as a source of easy credits, or of lots of credits. No significant associations were found, suggesting that it is not subjects per se that are influencing expectations. Again this tends to reinforce the view that it is the nature of learning experiences, and learning success, that are influencing the patterns found here.

Achievement or unit standards?

No picture of a student’s NCEA behaviours and choices is complete without reference to those choices that teachers and schools make on their behalf. When discussing students’ choices, it is unfair to attribute agency to them for things over which they have no control. A specific example of this concerns the choice of whether their learning is to be assessed with achievement or unit standards. Assessment with unit standards has been seen as an indicator of a student’s desire to take the easy way through school (Meyer, McClure, Walkey, McKenzie, & Weir, 2006) but the choice of which standards to use in individual courses and other aspects of assessment are in fact decided by the school, and often by individual teachers within the school.

As part of the course design process, teachers select the type and number of standards they think will be appropriate for their students’ assessment. Sometimes this choice is determined by overall school policy, sometimes it is a free choice, and more often a mix of the two. The Learning Curves teachers were aware that many people see unit standards as inferior to achievement standards but they tended to take a much more sanguine view, and to value being able to choose the best assessment tool for the circumstances (Hipkins et al., 2005). There is a widespread perception that unit standards are always easier to achieve but this is not necessarily so. English teachers, for example, often include the unit standard for “wide reading” because this is something they value and there is no equivalent achievement standard. Full suites of “academic” unit standards were developed early in the National Qualifications Framework (NQF) years, and they still remain on the framework. In the Learning Curves schools other teachers besides the English teachers also selectively chose from amongst these to fill gaps in their programmes, or to feature aspects they thought were better assessed by the unit standard than the equivalent achievement standard (Hipkins et al., 2004).

The next table shows differences we found in the mix of assessment instruments being used in most enjoyed and least enjoyed subjects and English. English was most likely to be assessed mainly with achievement standards, or a mix of achievement and unit standards. Most enjoyed subjects were only somewhat more likely than least enjoyed subjects to be assessed mainly with unit standards. Thus there is no evidence that what makes enjoyed subjects enjoyed is that they use unit rather than achievement standards.

Note: 

The largest percentages in each line are shown in bold.

Resubmission for internal assessments

Taking up opportunities to resubmit work for internal assessments is another aspect of behaviour that could inform the motivation question. Not all schools offer this chance and not all students need it. However, where it is offered, critics have suggested that students are enabled to do a minimal amount of work on the first attempt, knowing that they can get a lot of feedback and improve on the second try. Both teacher and student data from this study suggest that this possibility is not as widely exercised as that criticism would indicate.

Reassessment opportunities were most likely to be offered in English (78 percent). They were offered in 64 percent of the most enjoyed subjects and 63 percent of the least enjoyed subjects. The lack of difference between most enjoyed and least enjoyed subjects suggests that options for reassessment are not a deciding factor in students’ nomination of most enjoyed subjects. This is an interesting finding given that the option for reassessment is seen by some NCEA critics as another aspect that encourages taking an easy path through learning. However, teachers of students’ most enjoyed subjects were more likely to say that the student had taken up opportunities offered for reassessment, and that the outcome was that the standard was achieved on the second attempt. This matches teachers’ generally more positive reports of students’ NCEA behaviours in the students’ most enjoyed subjects.

Students were mainly offered reassessment when they did not get a standard (91 percent of those teachers who offered this); they could also be offered reassessment when they had not done a standard they could do (34 percent), and to improve from an “achieved” level to merit or excellence (31 percent overall), and far more in English (39 percent) than in classes that were students’ least enjoyed (28 percent) or most enjoyed (25 percent)).

Yet few of the students said they had in fact been offered reassessment opportunities, and only 27 (6 percent) had used these opportunities:

• 19 students (2 percent) said they had been offered a reassessment opportunity in English and 14 of them took this up.
• Six students said they had been offered a reassessment opportunity in their most enjoyed subject and four of them took this up.
• 13 students said they had been offered a reassessment opportunity in their least enjoyed subject and nine of them took this up.

Skipping assessments

The extent to which Learning Curves students said they had personally skipped one or more assessments broadly matched teachers’ views of how likely it was that students would do this (see below). Often the students simply reported that they knew others who had skipped, while they personally felt they could not afford to squander any chance to gain credits. This feeling probably abated as students’ confidence and experience with the NCEA grew. In the final year of Learning Curves, a quarter of the Year 11 survey students had skipped at least one assessment, a third had done so by Year 12, and 40 percent by Year 13 (Hipkins et al., 2005).

As the next table shows, Competent Learners’ self-reported skipping of Level 1 standards is even lower than we might have anticipated, although some students said they could not remember or did not sit internal NCEA assessments in this subject. Responses to skipping external assessments referred to standards for which students had been entered but did not sit. Only Year 12 students could respond to this since students in Year 11 had yet to face external examinations for the first time when we interviewed them.  Skipping external assessments was also infrequent.  Note that though we report the figures in relation to their Year 12 classes, the figures refer to comparable subjects at Year 11, and it may be that there would be some differences related to the nature of their Year 12 class had we been able to gather information on the external assessments associated with Year 12.

Notes: 

The largest percentages in each line are shown in bold.

Numbers do not add exactly to 100% because of rounding.

Where we had the necessary records, we checked students’ responses to the question of whether they had skipped any English assessments against their NZQA Record of Learning. For 80 students, we were able to count all entries coded V (standard not attempted), Y (student absent), and Z (missing paper) for a Level 1 English achievement standard. It would have been a laborious manual task to check the most enjoyed and least enjoyed subjects since there were so many of them and there was no self-evident subject differentiation in the system of numbers that identifies specific achievement standards. At the time the analysis was done a mapping of codes identifying standards to subject areas was not available, but such mappings have subsequently been developed and can now be accessed by researchers.

Despite these limitations, this check suggested that students considerably under-reported their skipping of assessments. The students who had admitted to skipping an external assessment in English had a mean number of 4.82 credits coded V, Y, or Z with a range of 3–12 credits. This suggests some left just one standard in an examination while those at the 12 credit end skipped the entire paper. What about the students who said they had not skipped? Here the mean was 4.62 credits coded V, Y, or Z with a range from 2–14. It may be that students who left one aspect of the examination did not see themselves as skipping overall. However, those who had 14 credits coded V, Y, or Z must surely have been aware that they should have said they had skipped.

Patterns of skipping internal assessments

With the cautionary caveat that these data are almost certainly under-reported, we now look at self-reported patterns of skipping internal assessments. Very few students said they had skipped more than one internal assessment in English or in their most enjoyed subject, but it was almost as common to skip two or more as to skip one assessment in the least enjoyed subject—though the overall numbers were still low.

Student and teacher comments during the Learning Curves project gave us insights into the sorts of influences and events that could lead students to skip an assessment. We used these insights to shape a list of reasons to which the Competent Learners project students responded on a 4-point Likert scale: strong influence; some influence; not much influence; no influence at all. Some reasons (for example “had too many assessments at the same time” or “it was too much work for the number of credits”) might constitute a careful balancing of options when under pressure, but then again they might also be decisions made on the spur of the moment. Students who are absent for one-off internal assessment events are unlikely to be offered another chance, and so we included that as a reason. Of course some students may stay away deliberately, in which case the real reason for missing an assessment could lie elsewhere. Numbers of responses were too low to report as frequency data but broadly the trends were as follows:

• Two reasons stood out: “assessed a skill I was not confident in” and “I didn’t want to do task involved”. More students said they skipped for these than any other reasons, especially in their least enjoyed classes and in English. What is interesting here is that both reasons imply a level of personal responsibility for the choice.
• Students who skipped an assessment in their least enjoyed class tended to be more likely to pick reasons that can sound like excuses—the focus was not on them and their behaviour: “part of subject assessed was not important”. “assessment task was not easy enough”. “it was too much work for the number of credits”. “had too many assessments at the same time”; “wanted to do it but away at the time”.
• Very few students gave as a reason for skipping an assessment in English or in their least enjoyed subjects that they “had enough credits already” whereas several students said this for their most enjoyed subject. As we have seen, students anticipate readily gaining the credits they need when they are enjoying their learning.
  

The Learning Curves research found that avoiding the likelihood of failure or potentially embarrassing assessments such as speeches in English could be reasons to skip assessments. As anticipated we also found that the most commonly skipped type of English assessment was a speech or soliloquy. However, about a third of those who skipped in English avoided completing a research project—this might have been more to do with effort expended for credits gained. A quarter of students who skipped an English task chose not to do creative writing, which again is a performance of sorts. Reasons for skipping assessments in a most enjoyed subject were evenly divided between avoiding a performance and not completing a research task, and in the least enjoyed subjects almost every case involved a research task of some sort.

Patterns of skipping external assessments

Reasons Year 12 students gave for skipping external assessments mainly related to fear of failure: “I didn’t think I would pass the assessment”; “I did not study for that part”; “I couldn’t do it”. More students said they found it difficult to pace their time in the English examination than in their other subjects. By far the most common type of skipping in English was leaving some sections of the examination blank. Just five students said they chose not to go to the examination, or could not go because of illness/accident. We found a similar pattern for students’ least enjoyed subjects. Too few students skipped externals in their most enjoyed subjects for any pattern to be apparent.

Students’ tracking of their accumulating credits

There is another possibility to consider in relation to skipping assessments. If this is not a considered choice, it could be an impromptu choice, especially if there are no perceived consequences since students are likely to be offered far more credits than they need to succeed in NCEA. How carefully, then, did the students track their growing credit totals?

As well as obtaining results from NZQA for 256 of the students, we asked them to tell us how many Level 1 credits they had gained overall during their Year 11 studies and we checked what they said against their NZQA record for English. As the next table shows, many students responded to this question with reasonable accuracy, and those with the highest scores were most likely to be accurate in their recall.

The students who said they could not remember how many credits they gained are potentially an interesting group to explore. In the Learning Curves project we noted that students who are only loosely aware of how many credits they have gained, and who have no particular strategy for choosing which assessments they will undertake, are also very likely to:

• be interested only in the number of credits on offer and whether these are a fair trade for the work involved in passing
• see no qualitative difference between unit standards and achievement standards (“credits are credits”)
• avoid the possibility of failure by skipping assessments they do not feel confident of passing (Hipkins et al., 2005).

These are the types of beliefs and actions most likely to be cited by critics who believe NCEA encourages students to find the easiest possible pathway to a qualification (Meyer et al., 2006). Such beliefs and actions are not, of course, limited to students who do not track their credits, but they were almost always a feature of the thinking of this group. So what patterns did we find for our Competent Learner sample?

First, the numbers who said they could not remember how many Level 1 credits they got are relatively low, even for students in Year 12 when the data were gathered, who needed to think back to the previous year. Both overall and Year 12 only data are shown in the next table.

Very few students could not recall the number of credits they gained in their most enjoyed subject, even in the following year. English, which is compulsory in Year 11, follows a similar pattern here to least enjoyed subjects, although even for these the numbers are relatively low. This pattern provides one more piece to the picture that links learning and assessment behaviours to students’ perceptions of the quality and enjoyment of their learning experience.

Teacher view of student approach to NCEA assessment

How do teachers see students’ motivation in relation to their NCEA choices? Thirteen items made up the factor student approach to NCEA assessment. Teachers’ responses for these items are shown in the next table. All 13 items showed significant differences between most enjoyed and least enjoyed subjects, or in the case of the final two items, between English and the most enjoyed subjects. This is congruent with the students’ reports of being more likely to skip assessments in English, as outlined above.

Note the relatively low percentage of teachers who think students do the bare minimum or are not interested if there are no credits to be gained. This is a very different picture from the claims of widespread demotivation painted in negative media articles. It is food for thought that behaviours that seem to signal motivation challenges, which are commonly attributed to the design of NCEA itself, are more often identified by teachers of least enjoyed subjects, and that these stand in strong contrast to the much more positive and optimistic observations of teachers in most enjoyed subjects. This adds to the theme which comes through this report in a number of sections, that students do engage more in learning in classes that are taught in ways that encourage that engagement.

Notes: 

The largest percentages in each line are shown in bold.

(r) The item scale was reversed when the factor scale was calculated

In contrast to patterns of differences found for opportunities to learn, almost no differences were found for mathematics and science teachers’ responses. It seems that teachers’ experiences of students’ NCEA behaviour are relatively similar regardless of the subject. What is more likely to make the difference is whether a subject is seen by the student as most enjoyed. In those cases the teacher is likely to hold a positive view of that student’s choices and attitudes to NCEA.

Teachers’ views of how well they thought individual students were approaching NCEA were highly correlated with how they also thought students acted in their class as a whole; and more correlated with that than with the total number of Level 1 NCEA credits gained by that student, or their cognitive composite level. Students who engaged in risky behaviour—and were disengaged in school—were likely to have a lower rating for their approach to the NCEA work. There was a moderately strong correlation between teachers’ views of an individual student’s approach to NCEA with that student’s own reports of their engagement with school, their views of their teachers and what happened in their classes (the correlation with their view of their relationship with their teacher was slightly stronger than with their view of class practices), and their own level of intrinsic motivation. This adds to the indications from earlier chapters on achievement and engagement that students do respond to what is offered to them currently: their “assessment identity” draws on their past, and the weight they put on school, but this identity is also open to current positive learning experiences.

Note: 

Correlations stronger than ± 0.4 are shown in bold face.

Implications

Although NCEA has given some more flexibility about when students tackle the assessments of their work that contribute to the senior school qualification, students in this study did not seem to be making continual choices of whether to try for a standard or not. They were working within the framework of the courses they took, and therefore were not able to decide what kind of standard they might take (leaving aside the question of whether unit standards are easier than achievement standards). From the data we have in this study, we could not see any empirical confirmation that students were opting for ease at the expense of challenge, or the expense of qualifications that would leave all viable pathways open. As in the previous qualification regime, they were sometimes avoiding those assessments or parts of assessments where they felt least confident.

Students were more likely to skip assessments in their least enjoyed subjects. Students were most confident about getting lots of NCEA credits in their most enjoyed subjects, but we did not analyse the number of credits they gained in each of the subjects they told us about, so we do not know how their performance in NCEA compares across subjects. Certainly, we continued to see that students were responding differently in their most enjoyed and least enjoyed subjects; and from our data, what differentiates these is more to do with teaching practices than with the kinds of NCEA standards offered. Our data suggest that to make the most of NCEA, teaching practices are worth attending to as much as the structure of the qualification and the ways in which assessment occurs.