Publications

On the Edge of Adulthood: Young people's school and out-of-school experiences at 16

Publication Details

Competent Children, Competent Learners is a longitudinal study which began in 1993 and follows the progress of a sample of around 500 New Zealand young people from early childhood education through schooling and beyond. This is the main report from the age-16 phase of the study and details students’ participation in school, their experiences of learning, and their achievement in terms of the study’s competency measures and their NCEA results. It also describes overall patterns of family life, friendships and interests out of school at age 16.

Author(s): Cathy Wylie, Rosemary Hipkins and Edith Hodgen, New Zealand Council for Educational Research.

Date Published: May 2009


Please consider the environment before printing the contents of this report. This report is available as a download (please refer to the 'Downloads/Links' inset box, top right).
Please consider the environment before printing the contents of this report.

Achievement

In this chapter, we look at the performance of young people through a range of different lenses. We start with the measures used in this longitudinal study to describe cognitive and attitudinal competencies. Then we look at parent perceptions of their child’s behaviour in a different context from school. These parental perceptions can be different from teachers’ perceptions. Next we look at achievement in the sense of senior school qualifications, the NCEA, and we also report teacher views of the students’ overall ability and likely final educational qualification level.

Since the structure of the NCEA is different from the qualifications it replaced in 2002, we take a look at how the competency measures we have used in the study are related to NCEA achievements. Finally, we look at how other variables, such as engagement in school and relations with friends, relate to the number of Level 1 NCEA credits gained, and the student scores on the competency measures we have used: What kind of experiences and behaviours are associated with higher scores and credits gained?

Competency levels11

In the Competent Children, Competent Learners study, we have measured both cognitive performance (in reading, writing, maths, and a nonverbal test, the Ravens Standard Progressive Matrices) and attitudinal performance (these are dimensions related to the new key competencies in the revised New Zealand Curriculum) since we started to follow the study participants at age near-5. At age 16 we moved to a new test for literacy and numeracy, since we had reached a ceiling on the Progressive Achievement Tests (PATs), and we also wanted a shorter test to encourage the young people to stay in the study.

Cognitive competency measures—student assessments

At age 16, we measured literacy and numeracy by asking the students to answer questions in a subset of the International Adult Literacy Survey (IALS) that was drawn for the study by Statistics Canada, based on the pattern of age-16 New Zealand results when the IALS was first undertaken, in 1996. We continued to use the Standard Progressive Matrices to measure logical problem solving.

For analysis purposes, results from each of these three measures (literacy, numeracy and logical problem solving) were converted to 10-point scales, and scores for each scale calculated for each student. The average scores were 6 (s.d. 1.5) for numeracy, 6.4 (s.d. 1.2) for literacy, and 8 (s.d. 1.0) for logical problem solving. While students appeared to show the highest average levels of competency for logical problem solving it should be noted that many had reached a “ceiling” for achievement using these matrices compared to the larger gains they had made between each two-yearly interval when they were younger (Wylie & Hodgen, 2007). The standard deviations suggest there was slightly more variability for numeracy than for the other two competencies.

Attitudinal competency measures—teacher assessments

As in all the previous rounds of the study, we asked the participants’ teachers (n = 1,250, up to three teachers for each student) to respond to a range of items about the young people in order to measure their attitudes in school. These items included some we have asked throughout the study, as well as new items that give us further material that is relevant to the key competencies now included in the revised New Zealand Curriculum.In past rounds of the study, we have defined each competency before analysis. In this round, because we were adding new items to provide more insight into the key competencies, we undertook factor analysis to identify competencies. This means that we are using new labels for competencies, other than for social skills.

Four factors were developed from the patterns of responses to these items, which we have labelled thinking and learning; focused and responsible; social skills; and social difficulties. The individual items that make up the four factors are shown in Figures 2–5. In these figures the patterns of responses are averaged for each student across the three teachers who gave us their views. These were the English teacher (n = 418), those who taught subjects nominated as students’ most enjoyed (n = 415), and those who taught subjects nominated as the students’ least enjoyed (n = 417). The teachers did not know how the students categorised their subject.

Subjects nominated as the most enjoyed covered a wide range. Mathematics or a science was chosen as least enjoyed subjects by 62 percent of the age-16 students.

Teachers’ ratings of the same student did differ. Generally, teachers of the classes students nominated as their most enjoyed gave an individual student higher ratings than did their English teacher, or the teacher of that student’s least enjoyed class. The latter tended to give the lowest rating of the three teachers.

These differences between teachers’ perceptions and indications that students act differently in different classes relate to some extent to differences in the opportunities to learn that students experienced in their classes. These differences are explored in Chapter 7.

While each of these four factors was distinct, perhaps not surprisingly there was a very strong correlation between three of the competency factors: thinking and learning, and focused and responsible (r = 0.85), and between these two and social skills (r = 0.80 & 0.73 respectively). In other words, a student who received a high average rating for their thinking and learning behaviour was also likely to receive a high rating for their showing focused and responsible behaviour, and their social skills. However, the correlation between these three factors and the fourth, social difficulties, was lower (r = -0.48, -0.65, and -0.52 respectively), though still moderate to strong, indicating that some students with a high score for thinking and learning or being focused and responsible also experienced some degree of social difficulties.

Figure 2 shows the responses across all 1,250 teachers for the thinking and learning competency. The percentages refer to the percent of teachers giving a particular rating for one of the items. Seventeen percent of the 1,250 teachers thought that the student they were describing always takes on new ideas, for example. 12 Around half the teachers perceived that the student often or always showed openness to new ideas, an active curiosity, and an active role in ensuring that they understood things.

Figure 2: Thinking and learning competency

Figure 2: Thinking and learning competency

Responses for the four items ranked lowest on Figure 2 are interesting. These items illustrate metacognitive dimensions of the key competencies as defined in the revised New Zealand Curriculum (Ministry of Education, 2007). For example, reflecting on how one has learnt about something is a dimension of the thinking and managing self key competencies and has particular salience for lifelong learning. Working in a group together can alert students to the different ways their peers may perceive the question or concept being discussed, while providing opportunities to strengthen competencies in relating to others and participating and contributing. Being aware that there are different ways of interpreting knowledge potentially sits at an intersection of the key competencies thinking and using language, symbols, and texts, with the knowledge components of the eight learning areas of the curriculum. It is food for thought that the teachers perceived that students displayed these aspects of competency less often than the more traditional thinking and learning dimensions.

Figure 3 shows the responses across all teachers for the focused and responsible factor. On the whole, teachers’ views show that the age-16 students were reasonably well organised. More than three-quarters of the teachers reported that the target student often or always turned up to class on time, brought all the equipment they needed, and took responsibility for their own actions. They did not do everything asked of them: just over half often or always finished all their class work, and just under half, their homework. (Recall that half the students at age 16 said they did not like doing homework, for a range of reasons.) In terms of stretching or challenging themselves, 40 percent assessed their own work and made improvements to it before handing it in, and around a third chose work that allowed them to gain further knowledge or skills. Relatively few students were seen to act often or always without thinking of the consequences.

This factor illustrates many dimensions of the curriculum key competency managing self. However, as for thinking and learning, those items that illustrate deeper, more transformative “layers” of key competencies tended to be rated as happening less often. For example, persisting in the face of difficulties and choosing work that will be personally extending are both indicators of the dispositional aspects of managing self and they highlight intrinsic qualities of importance for lifelong learning. Voluntary self-assessment and improvement of work again points to reflective metacognitive dimensions of thinking.

Figure 3: Focused and responsible competencyFigure 3: Focused and responsible competency

Figure 4 shows the teacher responses across all 1,250 teachers for the social skills factor. Here there is a clear link to the curriculum key competency relating to others. Two-thirds of the teachers saw their student often or always showed tolerance (respects other points of view or different ways of doing things) but, overall, students were perceived to be less proficient at being able to present their own point of view in an appropriate manner even when there was a disagreement or at resolving any disputes that arose.

Figure 4: Social skills competency
Figure 4: Social skills competency

Figure 5 continues the key competency theme of relating to others and shows the averaged teacher responses for the social difficulties factor. Students who were seen by their teachers to have marked levels of social difficulties were a small minority. Less than 10 percent of the teachers reported a student who often or always mixed with antisocial peers, or were influenced by peer pressure to do something out of character, though this had increased since age 14. Involvement in bullying (that teachers knew about) was also uncommon, and much the same as at age 14.

Figure 5: Social difficulties competencyFigure5

Teachers’ overall judgements of ability and future achievement

We also asked teachers to give a global judgement about the ability of the students, and their view of their likely achievement in NCEA and post-school qualifications. Bear in mind that this sample has an over-representation of students from homes with higher levels of maternal qualification and income, and thus the picture below is likely to be somewhat more optimistic than would be found nationally. The frequencies given below are for all three teachers of English, most enjoyed subject, and least enjoyed subject (n = 1,250). The differences between the views of the three categories of teacher are discussed more fully in Chapter 8.

Overall achievement level in comparison to others in their class: 22 percent of the teachers indicated the student we asked about was performing at a very good/excellent level, 25 percent at a very good level, 28 percent at a medium level, 20 percent below average, and 4 percent at a minimal level.

Around half the teachers thought the student we asked about was likely to have received a Level 3 NCEA or scholarship qualification by the time they left school; this is much higher than the actual proportion of school leavers achieving at this level (36 percent in 2006).


Likely NCEA qualification level by the time they left school:

Likely NCEA qualification level by the time they left school
Scholarship:5 percent
Level 3:46 percent
Level 2:24 percent
Level 1:15 percent
None:5 percent

Just under half were also thought likely to go on to university.

Likely post-school qualification level
Postgraduate degree:14 percent
Undergraduate degree:35 percent
Tertiary diploma:19 percent
Trades qualification:10 percent
No post-school qualification: 1 percent
Don’t know:21 percent

Competencies at home

We asked all the parents (n = 438) to rate their child’s attitudes and behaviour on a slightly smaller range of items that were the same or similar to those asked of the teachers of the school stayers.

The factors that were identified here are different from the factors that were identified for the teacher views, possibly indicating differences in the relationships and contexts in which parents and teachers see behaviour. The three factors identified among parental responses were: self-confidence, self-efficacy, and responsibility. The 16-year-olds’ mean scores were highest for self-confidence, and lowest for self-efficacy.

Self-confidence

Three-quarters of the young people were seen by their parents as often or always confident in their interactions with adults; and over half were often or always clear in their communication, and open to what was happening around them. They were less likely to ask a lot of questions, however.

Figure 6: Parent view of their 16-year-old’s self-confidence Figure 6: Parent view of their 16-year-old’s self-confidence

Self-efficacy

Most of the young people’s parents thought they often or always showed adults respect. Round two-thirds thought their child met their goals, or were willing to learn from mistakes. Half thought that their child never or only occasionally acted without thinking of the consequences, and somewhat more, that they were not influenced by peer pressure to do things out of character.

In Figure 7, the (r) in the text for bottom two bars indicates that the item score was reversed when the scale measure was calculated.

Figure 7: Parental view of student self-efficacy Figure 7: Parental view of student self-efficacy

Responsibility

If they were interested in something, most of the young people had a good concentration span, though only half were seen by their parents to always or often persist with solving a problem, even when things went wrong. They were better at getting organised and passing on messages than finishing all their chores.

Figure 8: Parental view of student responsibility Figure 8: Parental view of student responsibility

A few items asked of parents were not sufficiently correlated with any others to make factors, indicating that the behaviours below, such as thinking outside the square, can occur with, for example, different levels of self-efficacy or self-confidence:

  • Sixty-one percent of the young people’s parents thought they often or always sought information before making a decision
  • 57 percent of the young people often or always thought outside the square
  • 45 percent often or always organised their time to get things done.

Comparing competencies at home and at school

There was only a moderate to low level of correlation between the views of teachers and parents, i.e., teacher and parent ratings of the same young person were usually not identical, but were also not wildly different. For some young people, the level of agreement between teachers and parents was good; for others it was partial, and for others it was poor. It seems that young people do respond differently to home and school environments, some more than others, and that they show different levels of responsibility and self-efficacy in these environments. The difference in these environments may also account for the levels of responsibility and self-efficacy they can show. For example, looking at overall percentages (rather than correlations for individuals), parents were more likely than teachers to think that their child often or always:

  • Had a good concentration span while working: there was a 37 percentage point difference in teacher/parent responses, but note that the parent question did add the phrase “when working on things that interest him/her”. Thus this difference may largely reflect concentration when working on a task freely chosen, compared with concentration on a task determined by someone else—in this case the teacher.
  • Meets any goals that s/he sets her/himself: 20 percent more parents than teachers said this happened often or always. Again, the difference might relate to students’ active engagement in such goals, including their reasons for setting them.
  • Clearly explains things so that you get a very good idea of what is happening and what s/he is thinking: 18 percentage points difference—the parent version said “clearly explains things he/she has seen and done so you get a good idea of what happened” suggesting this is about discussions of events, whereas in-school explanations are more likely to relate to abstract/conceptual matters, at least some of the time.
  • Follows what is being talked about in a conversation and stays on the same topic: this item was identical for parents and teachers but, again, there was an 18 percent difference in response frequencies. It is likely that conversations at home will cover a wider range of topics than classroom conversations, and perhaps will be more free-ranging.
  • Remembers and carries out instructions after hearing them once: here there was a 14 percent difference. Perhaps, again, the salience of the task makes the difference.
  • Presents his or her view in an appropriate manner even when there’s a disagreement: 13 percent more parents agreed this happened always or often. Their item added the qualifier that this was about discussions with an adult. Teachers could have been thinking also about peer-to-peer conversations.


However, there are also likely to be differences in the manner in which teachers and parents arrive at their views of young people’s competency levels, and the information they use to “measure” a young person against. Teachers could compare each student with a large number of young people in school settings, while parents might compare their child with a smaller number of young people known to them, or with how they see other young people behave in public settings.

There were no directly and fully comparable items on which the teachers were more likely to rate something happening than were the parents.

NCEA achievements

NCEA began in 2002, and had three years to bed in before students in this study reached Year 11, usually the first year when students undertake assessment for credits linked to one of the three levels of the NCEA.13 While the NCEA qualification was intended to provide more flexibility, the choice of which credits to work toward, and the number and kind on offer, is largely related to the structure of the courses students take. Chapter 6 outlines four clusters of subject combinations found at both Year 11 and Year 12.14

We found some interesting patterns that indicate that quite a number of Year 11 courses are being assessed for both Level 1 and 2 credits, with a somewhat lower number at Year 12 offering assessments for both Level 2 and 3 credits. It was possible for a student to have gained some Level 2 credits, but still be short of the 80 Level 1 credits needed for a Level 1 NCEA qualification. Seventy-two percent of the Year 11 students in this study gained the 80 credits needed for a Level 1 NCEA qualification. Just over half had also gained some Level 2 NCEA credits.

Most of the age-16 students gained a high proportion of the NCEA credits they attempted, though students taking mainly vocational subjects gained fewer than others. Subject clusters did offer different numbers of credits to students. On average, Year 11 students in the “vocational” and “contextual” subject clusters ended their year just short of the 80 credits needed for a Level 1 qualification; those in the “traditional arts” and “science” subject clusters ended their year with far more than they needed. Close to 90 percent of the Year 11 students in the arts and science clusters gained 80 credits or more, as did 42 percent of those in the “vocational” cluster and 14 percent in the “contextual” cluster. One percent of the Year 11 group (two students, both in the “traditional arts” cluster) had also gained sufficient Level 2 credits for a Level 2 NCEA qualification, with 53 percent gaining some Level 2 credits. Here the subject cluster difference was much less marked: 64 percent of those in the “contextual” cluster had gained some Level 2 NCEA credits, as had 60 percent of those in the “traditional arts” cluster, 49 percent of those in the “traditional science” cluster, and 46 percent of those in the “vocational” cluster. Eight Year 11 students had also gained some Level 3 NCEA credits: two from each subject cluster.

By Year 12 the number of students with sufficient credits for a Level 1 qualification had increased from 72 to 84 percent. All of the traditional arts students, 94 percent of the traditional science students, 78 percent of the “contextual” cluster students, but only 40 percent of the “vocational” cluster students were successful in gaining NCEA Level 1 by the end of Year 12.

Looking at Level 2 NCEA, we find that 98 percent of the students in the “traditional arts” cluster in Year 12 had gained the 60 Level 2 credits needed for this, as had 86 percent of those in the “traditional science” cluster. Fifty-one percent of those in the “contextual” cluster also had this number of credits, as had 21 percent of those in the “vocational” cluster. One student (in the “traditional arts” cluster) had the 60 credits needed for a Level 3 qualification, and 28 percent had some Level 3 credits—with similar proportions here for each cluster.

The two tables below give the total number of NCEA credits gained. They show that, on average, most Year 11 students in the traditional and “contextual” clusters and most Year 12 students gained most of the credits they aimed for; they also show differences in the number of credits on offer related to subject clusters.

Table 4: Year 11 Competent Learners’ first year of NCEA credits (n = 156)
Subject clusterRange of credits attemptedMean no. credits attemptedMean no. credits gained% gained of those attempted
Traditional arts82–21414713893
Traditional science74–18813712187
Contextual subjects57–106797190
Vocational subjects23–1541027978



Table 5: Year 12 Competent Learners’ cumulative NCEA credits (n = 261)
Subject clusterRange of credits attemptedMean no. credits attemptedMean no. credits gained% gained of those attempted
Traditional arts219–35228626792
Traditional science133–36525321885
Contextual subjects124–29821016578
Vocational subjects22–30614011483


Although there was considerable within-cluster variability, it is clear that the more academically inclined students—those who are most likely to still be studying at Level 3—are gaining far more than the 80 credits needed at Level 1, when arguably they do not need these for any qualifications purpose. This must represent a considerable amount of assessment activity for them.

The situation is somewhat different for students who do not experience learning success quite so quickly or easily. The NCEA was designed as an award students could work toward progressively. The achievement patterns described do show evidence that most Year 12 students in “contextual” clusters who had not gained a Level 1 award by the end of Year 11 had succeeded in doing so by the end of Year 12. Students in “vocational” clusters were still less likely to have gained a Level 1 NCEA award, even after two years. However, the credit record of these students provides encouraging evidence that they are being given more opportunities than in the past to experience success in gaining qualifications from their learning, even if they take two years to gain an NCEA award. It is of particular interest that the “percentage achieved” success rate of the Year 12 students in the “contextual” and “vocational” clusters is almost as high as for those in the two more traditional academic clusters. This is also encouraging because it was an express intention of the reforms that students be assessed when ready and so come to see themselves as successful learners. This is one of the conditions necessary to encourage the development of “lifelong learning” dispositions, which has been at least an implicit policy intent of the NCEA assessment regime (Hipkins, 2005).

Other research suggests that the enhanced success rate will have been achieved by focusing more on internally assessed standards where teachers can support students to demonstrate their learning, and by limiting less confident students’ exposure to external examinations (Hipkins, R., Vaughan, K., Beals, F., Ferral, H., & Gardiner, B., 2005). One study has suggested that low achieving students who have been disengaged in earlier years might be even encouraged to re-engage in learning if they experience success in gaining unit standards credits in a context for which they can see personal relevance and practical value, early in the school year (Boyd, with McDowall & Ferral, 2006).

Consistency of NCEA results and the project competency measures

NCEA is a new way of measuring student achievement, and has proved to be somewhat controversial. We visit some of those controversies in Chapter 9, when we look at whether students are using NCEA to make easy choices, and at their decision making around specific assessments.

One of the issues raised is how well the NCEA measures student ability. The Competent Children, Competent Learners study provided us with an opportunity to see how consistent NCEA results were with competency measures that are more traditional in the sense of describing performance levels in terms of numbers on the same scale, and in the case of the cognitive composite, on multichoice tasks. We found considerable consistency, indicating that NCEA results are not giving a different picture than the traditional measures.

We looked first at the correlations between our cognitive composite (the average of scores for literacy, numeracy, and logical problem solving), the attitudinal composite (the average of scores for three attitudinal competency factor scores, focused and responsible, thinking and learning, and social skills), the four attitudinal competencies taken separately, and the total number of Level 1 NCEA credits gained (whether unit or achievement standards), 15 the proportion of achievement standards at the excellent level, the proportion of achievement standards at the merit level, and the proportion of achievement standards that were achieved. What we found was that the correlation levels for the cognitive composite, the attitudinal composite, and the focused and responsible competency were moderate to strong: between 0.53 to 0.61 for all but the proportion of standards gained at the achievement level, where the correlations were 0.39 to 0.43. Correlations between NCEA achievement and the two social skills measures were lower. Thus a student with a low level on both our cognitive and attitudinal competency composites was also likely (but not always) to achieve fewer Level 1 NCEA credits than a student with a medium or high level on our competency composites. But a student’s social skill level as we measured it was unrelated to their NCEA achievement.

Table 6 shows how much the average number of Level 1 NCEA credits can vary dependent on levels of performance on the cognitive composite, on showing focused and responsible attitudes, and the student’s approach to NCEA. For example, it shows that students in the lowest cognitive competency quartile group16 gained an average of 78.7 Level 1 NCEA credits, cf. the average of 154.9 credits for those in the top quartile group on the cognitive competency.

Table 6: Mean (and standard deviation) of total number of Level 1 NCEA credits gained by students with different competency levels
Quartile groupCognitive composite - Year 11Cognitive composite - Year 12Focused and responsible - Year 11Focused and responsible - Year 12NCEA approach - Year 11NCEA approach - Year 12
Lowest78.7 (29.6)146.2 (51.0)79.2 (28.9)142.8 (53.5)88.7 (35.1)149.9 (48.4)
Second lowest111.4 (31.6)179.9 (63.1)99.6 (35.8)186.0 (52.8)107.4 (28.7)185.8 (63.3)
Second highest121.3 (31.4)210.3 (60.0)127.4 (29.5)221.8 (51.9)121.9 (35.9)223.8 (51.5)
Highest154.9 (26.5)249.5 (60.0)145.9 (24.7)270.0 (43.3)147.5 (27.0)253.3 (60.3)


Comparing the standard deviations, we see more variability among the credits obtained by Year 12 students than Year 11 (e.g., the standard deviations in relation to cognitive composite scores are between 26.5 and 31.6 for all quartile groups at Year 11, cf. from 51 to 63.1 for all quartile groups at Year 12). But there is no consistent pattern in the variability of the number of credits gained in relation to different quartiles; i.e., there is no greater variability in credit numbers for students in the lowest quartile groupings than there is for those in the highest quartile groupings.

We then undertook multivariate statistical modelling17 to see if we could predict the total number of NCEA credits an age-16 student would get from their competency scores. Both cognitive and attitudinal composite scores proved to be reasonable guides to NCEA success.

A model that included the cognitive composite, focused & responsible, and English teachers’ views of the student’s approach to NCEA (“NCEA approach”) accounted for 68 percent of the variability in the total number of Level 1 NCEA credits gained by Year 11 students, and 60 percent of the variability in the total number of Level 1 NCEA credits gained by Year 12 students. So higher numbers of credits gained reflect cognitive composite levels. Higher number of credits were also associated with positive attitudes to work, in particular to work for the NCEA, and the ability to focus on the task in hand and take responsibility—and these attitudinal factors gained more weight at Year 12. The two school engagement factors18 were also included in this model, but did not show separate contributions to the variance in student scores, probably because of their correlation levels with the other variables, and because the number of Level 1 NCEA credits gained was more strongly correlated with the cognitive and attitudinal competencies than with school engagement.

Associations with the proportion of achieved, merit, and excellence standards

We could also account for a reasonable proportion of the variance in the proportion of standards19 a student gained that were at the merit level, and the proportion that were judged excellent—but our models were less successful in accounting for the variance in the proportion of standards that were achieved. We could only account for 20 percent of the variability in the latter, with the cognitive and attitudinal composite competencies the only variables remaining in the model.

A possible reason for this is that a student with a relatively high proportion of standards that were “achieved” has a less clear description in terms of their overall performance than does a student with a high proportion of standards that were at the “merit” (or “excellence”) level. So a student with 80 percent of their standards “achieved” who achieved “excellence” in the others, would most likely have a somewhat different overall profile from a student with 80 percent of their standards “achieved” who did not achieve the remainder of their standards attempted. Just using the percentage of standards that were achieved cannot differentiate between these two groups.

The model accounting for the proportion of NCEA standards gained at the merit level by a student accounted for 53 percent of the variability between students. In this model, the cognitive composite carried the most weight, followed by the student’s level of being focused and responsible, their level for their NCEA approach, and also on their level on one of the three school engagement factors, affirmed at school.

Students with a low level of performance on the cognitive and attitudinal competency measures (in the lowest quartile group) gained merit in just over 10 percent of their NCEA achievement standards, while students with a high level on these measures gained merit in over a third of their NCEA standards. The difference was slightly smaller in relation to levels on the NCEA approach.

Fifty-two percent of the variance between students in the proportion of the credits gained at the excellent level was accounted for in our model. Again, the cognitive composite carried the most weight, followed by the attitudinal composite, and NCEA approach. Students with a low level of performance on the cognitive and attitudinal composites and their approach to the NCEA managed to get excellence in 2 percent of their achievement standards, on average, where students with a high level of performance on the composite and attitudinal composites and their approach to the NCEA managed to get excellence in a fifth to a quarter of their achievement standards.

The gap between the highest quartile of performers on the composite and attitudinal composites and their approach to NCEA, and other students, was most marked in relation to the proportion of excellent standards received.

Factors relating to the competency scores and NCEA Level 1 credits

What differences in competency scores were associated with differences in current experiences, in patterns of some key experiences over time, and in social characteristics? In this section, we describe correlations between competency scores and other variables that are in scale form (e.g., scales relating to friendship, behaviour, school engagement)20, and the level of variance in student scores accounted for in relation to categorical variables (e.g., values, motivation levels at age 14), for each of the age-16 attitudinal competencies, the cognitive composite, and the number of NCEA Level 1 credits.

Attitudinal factors

Table 7 shows correlations between scores on the four attitudinal measures, and the other scaled factors in the age-16 data. The correlations with other teacher views, of students’ overall ability and their approach to NCEA, are very strong. Correlations with age-14 competency levels, and their then teachers’ view of their overall ability, are also moderate to strong. Correlations with student reports of their engagement in school are moderate, as are those with their cognitive composite score at age 16. The stronger correlations, according to a somewhat arbitrary cut-point of 0.4 (or -0.4), are shown in bold face in the table, and the weakest (between -0.2 and 0.2) by –.

Table 7: Correlations between the four attitudinal competency measures at age 16 and measures
MeasureFocused & ResponsibleThinking & LearningSocial SkillsSocial Difficulties
Ability to cope with NCEA0.910.820.65-0.58
Overall ability0.790.790.59-0.45
Attitudinal composite 140.65*0.62*0.54*-0.45*
Overall ability 140.630.610.48-0.46
Engaged at school0.550.460.42-0.36
Cognitive composite 160.540.540.42-0.43
Cognitive composite 140.53*0.54*0.41*-0.44*
Affirmed at school0.420.400.38-0.25
Positive learning environment0.360.340.30-0.20
Attitude to work0.350.390.29-0.22
Parent view of responsibility0.340.340.28-0.24
Internal markers of learning 160.330.370.32-0.20
Internal markers of learning 140.330.360.35*
Absorbed in learning0.330.290.26
Satisfied with subject mix0.300.280.23
Parent view of self-efficacy0.220.290.29*-0.23
Parent view of self-confidence0.27*0.22
Adverse events-0.27
Disengaged in learning-0.44-0.30-0.330.32
Friends with risky behaviour-0.45-0.31-0.230.29
Risky behaviour-0.51*-0.35*-0.30*0.34*
Note:
  1. * Variable is included in relevant model.

Looking at this table (and also thinking about factors that are not correlated at around 0.3 or more, i.e. show little correlation), the following patterns are worth thinking about:

  • Risky behaviour and having friends with risky behaviour are the only factors from the friendship and family factors that show correlations of 0.3 or more with the attitudinal competencies. They are correlated more with being focused and responsible than they are with social skills or social difficulties—indicating that risky behaviour and some kinds of friendship warrant attention when it comes to school achievement, and not just in relation to relationships with others.
  • Student views of their own school engagement, approaches to learning and their classes show moderate levels of correlation with teacher views of how they see students operating in the class and around school: again, more so for the thinking and learning and focused and responsible competencies than for the social competencies. This level of correlation between teachers and students, on somewhat different measures, is reasonable: teachers were not making their judgements on irrelevant things.
  • Age-14 competency levels, and seeing learning in terms of internal (intrinsic) factors rather than seeing it as something done (just) to gain external recognition, continued to play a part in how student behaviour appeared to their teachers two years later. We return to this after we look at the relationship between the categorical variables and the four measures of competencies.
  • The age-14 competency levels shown at school had a stronger association with student behaviour seen at school two years later than did student behaviour in the different context of home.


Other previous patterns are related to age-16 attitudinal competency levels, as we see in the following set of four tables. These tables set out the categorical variables that showed significant relationships with the competencies; the final column gives the proportion of variance in student scores accounted for by the factor, when we undertook single-factor (one-way) ANOVAs that examine each factor separately. These figures allow some comparison of the different weight of different factors; for example, in the next table we see that subject clusters and previous patterns of enjoyment of reading have stronger associations with scores on the thinking and learning scale than do gender or previous patterns of TV watching.21


Thinking and learning


Table 8: Scores on the thinking and learning scale and categorical variables
Other variablePattern foundR2 (% of variance explained)
Number of Level 1 NCEA creditsThe higher the number of credits gained, the higher the scale score31.0
Subject clusterHigher scale scores associated with being in either “traditional arts” or science cluster19.4
Enjoyment of reading ages 8–14The higher the enjoyment of reading, the higher the scale score12.7
Attendance at schoolSimilar levels for good–excellent attendance; lowest scores for those with poor attendance12.3
Motivation at 14The higher the motivation at 14, the higher the scale score10.7
Maternal qualificationThe more qualified the mother, the higher the scale score9.6
School decile 8–14The higher the decile attended across the years of school, the higher the scale score7.0
Student values at 16Students with “satisfying life” values likely to have higher scale scores, and those with “standing out” values to have lower scale scores6.9
Family income at age 14The higher the income, the higher the scale score6.2
EthnicityHigher scale scores more likely for Päkehä/Asian students4.5
Family income at age 5The higher the income, the higher the scale score4.3
Student interests at age 14Creative interests associated with higher scale scores, followed by wide interests, and then sports. Computer games/no interests associated with lowest scale scores4.0
TV watching ages 8–14The less time spent watching TV, the higher the scale score3.4
Family financial situationThe less likelihood of difficulty the higher the scale score3.3
GenderFemales likely to have higher scale score than males2.6
Involvement in bullying ages 8–14The greater the involvement, the lower the scale score1.2


Of particular interest here are how some previous opportunities, experiences, and attitudes continue to colour attitudes to school work at age 16. Most important of these is the enjoyment of reading—an indication that reading is not hard work, and is seen as being worthwhile, both key aspects of finding the reading necessary for senior secondary schoolwork a channel rather than a barrier. Opportunities to gain enjoyment of reading are linked to ways in which time is spent, which makes sense of the appearance of student interests, and TV watching in this table. Different opportunities are also linked to differences in family resources, including here family income levels before the young people started school, and linked to family income, differences in school social mix (indicated by decile).


Focused and responsible

Much the same set of opportunities, experiences, and attitudes appear linked to how well age-16 students were taking responsibility for themselves in the class setting. Not surprisingly, attendance shows more association with this factor than with thinking and learning; as do patterns of TV watching (perhaps indicating a somewhat larger propensity for passivity among heavy watchers).

Table 9: Scores on the focused and responsible scale and categorical variables
Other variablePattern foundR2 (% of variance explained)
Number of Level 1 NCEA creditsThe higher the number of credits gained, the higher the scale score36.3
Subject clusterHighest scale score associated with being in “traditional arts” cluster; lowest in “vocational” or “contextual” clusters21.1
Attendance at schoolThe more frequent the attendance, the higher the scale score18.8
Enjoyment of reading ages 8–14The higher the enjoyment of reading, the higher the scale score16.0
Maternal qualificationThe more qualified the mother, the higher the scale score11.2
Student values at 16Students with “satisfying life” values likely to have higher scale scores, and those with “standing out” values to have lower scale scores9.5
Motivation at 14The higher the motivation, the higher the scale score9.0
School decile, 8–14The higher the decile attended across the years of school, the higher the scale score7.7
TV watching ages 8–14The less time spent watching TV, the higher the scale score5.6
EthnicityHigher scale scores more likely for Päkehä/Asian students5.4
Family income at age 14The higher the income, the higher the scale score4.4
Family income at age 5The higher the income, the higher the scale score3.7
Involvement in bullying ages 8–14Higher scores for those with no involvement in bullying3.6
GenderFemales likely to have higher scale score than males3.5
Student interests at age 14Creative interests associated with higher scale scores, followed by wide interests, and then sports. Computer games/no interests associated with lowest scale scores3.0
Family financial situationThe less likelihood of financial difficulty, the higher the scale score2.3

Social skills

Compared to the thinking and learning and focused and responsible factors, the social skills factor showed less association with subject cluster, school attendance, or the number of NCEA credits gained, and a stronger association with gender.

Table 10: Scores on the social skills scale and categorical variables
Other variablePattern foundR2 (% of variance explained)
Number of Level 1 NCEA creditsThe higher the number of credits gained, the higher the scale score18.7
Enjoyment of reading ages 8–14The higher the enjoyment of reading, the higher the scale score13.7
Subject clusterLower scale scores associated with being in either “vocational” or “contextual” clusters10.1
Maternal qualificationHigher scores for those whose mothers had a tertiary/university qualification9.6
Motivation at 14The higher the motivation, the higher the scale score8.4
Attendance at schoolLowest scores for those with poor attendance, followed by those with fair attendance7.4
School decile, 8–14The higher the decile attended across the years of school, the higher the scale score7.0
Student values at 16Lowest scores for those with “standing out” values6.2
Family income at age 14Lower scores among those with family incomes less than $60,0006.2
GenderFemales likely to have higher scale score than males4.9
Family income at age 5Lowest scores among the low-income group4.3
Family financial situationThe less likelihood of difficulty, the higher the scale score3.3
Student interests at age 14Creative interests and wide interests clusters have higher scores than sports or computer games/no interests clusters3.2
TV watching ages 8–14The less time spent watching TV, the higher the scale score*2.1
Note:
  1. * Association is at the indicative level (0.01 < p < 0.05)

Social difficulties

The pattern of relationships seen here is similar on the whole to the pattern for the social skills factor, but the strength of the associations with age-14 motivation and the previous pattern of school social mix is lower, and the strength of association with gender stronger; involvement in bullying also appears here. The pattern here may point to some different trajectories: for some young people, higher scores than others for social difficulties at age 16 show a deepening of paths cut some time before; for others, higher scores may indicate current reactions to new events and experiences.

Table 11: Scores on the social difficulties scale and categorical variables
Other variablePattern foundR2 (% of variance explained)
Number of Level 1 NCEA creditsThe higher the number of credits gained, the lower the scale score19.4
Enjoyment of reading ages 8–14The higher the enjoyment of reading, the lower the scale score10.1
Subject clusterHigher scale scores associated with being in either “vocational” or “contextual” clusters9.7
Maternal qualificationThe more qualified the mother, the lower the scale score9.6
Attendance at schoolHighest scores for those with poor attendance, followed by those with fair or good attendance7.7
GenderMales likely to have higher scale score than females6.1
School decile, 8–14The higher the decile attended across the years of school, the lower the scale score5.5
Student values at 16Students with “satisfying life” values likely to have lower scale scores, and those with “standing out” values to have higher scale scores4.6
Motivation at 14Highest scores for those with low motivation at 144.5
Family income at age 5The higher the income, the lower the scale score4.3
Involvement in bullying ages 8–14The greater the involvement the higher the scale score2.3
TV watching ages 8–14The more time spent watching TV, the higher the scale score*1.8
Note:
  1. * Association is at the indicative level (0.01 < p < 0.05)

Intrinsic and extrinsic motivation

There were reasonable correlation levels between three of these four competency measures and our measure internal markers of achievement. In this study, we have tracked whether students are seeing learning in terms of effort and internal markers of achievement, supporting intrinsic motivation and habits of thinking and application that will support ongoing learning, or whether they are more reliant on extrinsic motivation: how well they are doing in comparison to others. We have found that many students find motivation in both intrinsic and extrinsic markers.

Internal markers of achievement

At least two-thirds of the students usually or always saw doing well at school as working really hard, solving problems by working hard—and also that learning gave new ideas, and made them think about things.

Table 12: Internal markers of achievement (n = 421)
Internal markers of achievement Strongly agree % Agree %
Neutral %
Disagree %
Strongly disagree %
I do my very best35372061
What I learn really makes sense3535265< 1
I solve a problem by working hard32422041
I work really hard31452031
I catch on quickly24472171
Something I learn makes me think about things23531941
I learn something interesting22502161
I get a new idea about how things work1948274< 1

 

External markers of achievement

While many students thought they got good marks, they were not so sanguine about not having to try hard to get those marks; and they did not think that to do well at school necessarily meant one had done better than others.

Table 13: External markers of achievement (n = 421)
External markers of achievement Strongly agree % Agree %
Neutral %
Disagree %
Strongly disagree %
I get good marks/results4831174< 1
I’m the only one who can answer questions142725268
I know more than other people112730248
Others get things wrong and I don’t82331298
I don’t have to try hard61832368
I don’t have anything hard to do619273710


Intrinsic motivation

Because intrinsic motivation showed correlations with three of the four competency measures, and because it is related to the ability to keep learning (after school years), we examined its associations with other variables. The table below shows those associations—and the comparative lack of associations for our variable external markers of progress. What’s particularly interesting here is the moderate to strong correlation with having a positive learning environment in current classes, as well as the moderate correlations with age-14 experiences, indicating that intrinsic motivation is both built up over time as well as supported with current experiences. It is also interesting to see how intrinsic motivation is related to aspects of family life and friendship that emphasise communication.

But developing an intrinsic sense of motivation is unrelated to other factors that we have seen related to three of the competencies, and that also thread their way through the patterns of achievement and engagement we describe next: those with risky behaviour or friends with risky behaviour are as likely to have developed intrinsic motivation in relation to learning as others.

Table 14: Correlations between internal and external markers of progress age 16 and measures of experiences and perceptions
MeasureInternal markers of progressExternal markers of progress
External markers of progress0.52
Absorbed in learning0.51
Positive learning environment0.48
Internal markers 140.41
Attitude to work0.39
Overall ability 140.350.28
Ability to cope with NCEA0.33
Overall ability0.33
Cognitive composite 140.330.28
Family communicates well0.32
Absorbed in learning 140.31
Extending friendships0.29
Cognitive composite0.280.25
Inclusive family0.28
Attitudinal composite 140.28
Confident at school 140.27
Praise & achievement0.26
Parent view of responsibility0.26
External markers 140.39
Disengaged in learning-0.28
Note:
  1. Correlations of over 0.4 are shown in bold face, those between -0.2 and 0.2 are shown as –.

The associations with language and communication that were evident in some of the correlations above also come through the associations that emerge when we analysed the categorical variables: time and effort put into activities that involve the use of language seem positively associated with building a sense that one’s own effort makes a difference, and that doing well is a matter of gaining understanding as well as gaining marks. It is worth noting, however, that variations in young people’s levels of intrinsic motivation are less likely than the competencies to be related to variations in achievement, as measured by the number of Level 1 NCEA credits, and they are less related to differences in subject cluster. We can see this positively, as an indication that what is a useful long-term learning attitude may not be dependent on external results, or subject hierarchies. It also helps our understanding of learning and achievement—in negative as well as positive spheres—in some of those whose achievements in later life are such that one would be surprised to learn they had left school without qualifications, or had passed through school unremarked by peers or teachers.

Table 15: Intrinsic motivation and categorical variables
Other variablePattern foundR2 (% of variance explained)
Number of Level 1 NCEA creditsThe higher the number of credits gained, the higher the intrinsic motivation level7.7
Subject clusterHigher intrinsic motivation associated with being in “traditional arts” or science clusters6.4
Motivation at age 14Intrinsic motivation levels increased with age-14 school motivation levels4.6
Enjoyment of reading ages 8–14Intrinsic motivation levels increased with level of reading enjoyment4.3
Maternal qualificationHighest intrinsic motivation levels for students with a university qualified mother; lowest for those whose mother had no qualification3.4
Student values at 16Students with “standing out” values had lower levels of intrinsic motivation3.1
Age-5 family incomeLowest levels of intrinsic motivation for those from low-income families*3.0
School decile, 8–14Lower levels of intrinsic motivation for those who attended mainly decile 1–2 schools*2.5
Student interests age 14Highest intrinsic motivation levels among those in the creative or all-round interest clusters*2.1
TV watching ages 8–14The less time spent watching TV the higher the level of intrinsic motivation at 16*1.9
Parent interests at 14Highest intrinsic motivation levels for students whose parents were in the “literate-involved” cluster*1.9
Involvement in bullying ages 8–14Higher levels of intrinsic motivation among those with no bullying involvement*1.8
GenderFemales had higher levels of intrinsic motivation1.7
Note:
  1. * Association is at the indicative level (0.01 < p < 0.05)

Associations with cognitive composite and Level 1 NCEA credits

Table 16 below gives the correlations that are likely to have non-neglible associations,22 that we found between the number of Level 1 NCEA credits and the age-16 cognitive competency measure, with the factors relating to school and out-of-school views and relationships. There was a moderate correlation between our cognitive composite measure (the average of the students’ scores for literacy, numeracy, and logical problem solving), and their achievement of Level 1 NCEA credits in a wider range of standards, indicating that a student who did well on our assessments would also be likely to do well on their NCEA assessments, and that literacy and numeracy levels are important to qualification success. But there was also a similar level of correlation with their performance on our attitudinal composite measure, indicating that these dispositions—key competencies—are also important to NCEA success. Age-14 levels on our cognitive and attitudinal measures showed much the same correlation with NCEA Level 1 success as age-16 levels, indicating the importance of previous experiences and habits.

What else might have a bearing on success with Level 1 NCEA? Teacher views of how students approach it, as we have already discussed; but not far behind that are student views of their level of school engagement (0.57). School engagement levels were more strongly correlated with Level 1 NCEA success than were feeling affirmed at school. Current school engagement levels (0.57) were much more strongly related with Level 1 NCEA success as age-14 levels of school engagement (0.30).

Risky behaviour played a part, as did having friends with risky behaviour, at a low to moderate level. These two variables showed a similar degree of association with the cognitive composite. But the cognitive composite has a much lower association with school engagement and feeling affirmed at school.

Table 16: Correlations between the number of Level 1 NCEA credits and age-16 cognitive composite, with measures of experiences and perceptions
MeasureNo. of Level 1 NCEA creditsAge-16 cognitive composite
Cognitive composite 160.57
Approach to NCEA0.640.50
Overall ability0.640.64
Attitudinal composite 160.620.54
Cognitive composite 140.610.88
Attitudinal composite 140.590.58
Engaged at school0.570.32
Affirmed at school0.360.22
Parent view of responsibility0.340.39
Attitude to work0.300.35
Engaged at school 140.300.31
Affirmed at school 140.290.21
Parent view of self-efficacy0.280.26
Friends with risky behaviour-0.32-0.27
Risky behaviour-0.35-0.30
Note:
  1. Correlations of over 0.4 are shown in bold face.

The next two tables give the results of our single-factor or one-way ANOVA analyses of the associations between categorical variables and first, the number of Level 1 NCEA credits, and second, scores on the cognitive composite measure. The variables that show the strongest associations are the subject cluster, and for NCEA credit totals, attendance, and for the cognitive composite, enjoyment of reading ages 8–14.

Besides the subject cluster, what are the similarities—the factors that account for similar proportions of variance in scores? Some of these are to do with opportunities, such as the current family financial situation, current family income, and family income at age 5. Some are to do with ways that students have spent time—such as their level of TV watching between ages 8 to 14, and their current values. School decile patterns between ages 8 to 14 also show similar levels of association for both the NCEA qualification totals, and the cognitive composite.

One difference is likely to relate to differences in the focus of the assessments. Enjoyment of reading between ages 8 to 14 plays more of a part in the cognitive composite (a third of which comes from a literacy test) than in Level 1 NCEA totals (which covers a range of subjects). Motivation levels at age 14 and maternal qualification levels also played a somewhat stronger part in the variability of the cognitive composite scores than they did in Level 1 NCEA totals.

But attendance and involvement in bullying between ages 8 to 14 (whether as victim, bully, or both) had much stronger associations with the total number of Level 1 NCEA credits than with the cognitive composite. Subject clusters also had a somewhat stronger association, and previous attitudes to school between ages 8 to 12 showed an association that was not apparent in relation to the cognitive composite. These differences might shed some light on how some entrepreneurs and others whose school records are patchy go on to do very well as adults in spheres that are of more importance or interest to them than school was, and where they are more prepared to use their knowledge, skills, and attitudes.

Table 17: Number of Level 1 NCEA credits and categorical variables
Other variablePattern foundR2 (% of variance explained)
Subject clusterHighest number of Level 1 NCEA credits for students in the “traditional arts” cluster; lowest in the “vocational” cluster33.0
Attendance at schoolLowest number of Level 1 NCEA credits for those with poor attendance or attendance difficulties because of ill health16.0
School decile 8–14The higher the decile, the higher the number of Level 1 NCEA credits13.8
Maternal qualificationThe higher the level of maternal qualification, the higher the number of Level 1 NCEA credits13.3
Enjoyment of reading ages 8–14The higher the enjoyment of reading, the higher the number of Level 1 NCEA credits10.8
Family income at age 5The higher the family income at age 5, the higher the number of Level 1 NCEA credits10.6
Motivation at 14Lowest number of Level 1 NCEA credits for those with low motivation levels at age 149.9
Student values at 16Highest number of Level 1 NCEA credits for students with “satisfying life” values; lowest for those with “standing out” values7.8
Family financial situation 14Highest number of Level 1 NCEA credits for those whose families were in comfortable financial situations at 14; lowest for those whose families were in difficult financial situations7.8
Family income 16The higher the current family income, the higher the number of Level 1 NCEA credits7.8
Involvement in bullying ages 8–14Lower number of Level 1 NCEA credits for those who were involved in bullying in at least two of the four study phases from ages 8–147.3
EthnicityHigher number of Level 1 NCEA credits for Päkehä/Asian students4.1
TV watching 8–14Highest number of Level 1 NCEA credits for those who had a low level of TV watching 8–14; lowest for those who had a high level3.5
Attitude to school 8–12Lower number of Level 1 NCEA credits for those who had been unhappy at school in at least one of the three study phases from 8–123.1
Student interests 14Lowest number of Level 1 NCEA credits for those in the computer games/no interests cluster at 14*1.9
GenderHigher number of Level 1 NCEA credits for females*1.1
Note:
  1. * Association is at the indicative level (0.01 < p < 0.05)



Table 18: Scores on the age-16 cognitive composite scale and categorical variables
Other variablePattern foundR2 (% of variance explained)
Subject clusterHighest scores associated with being in “traditional arts” cluster; lowest in “vocational” and “contextual” clusters25.2
Enjoyment of reading ages 8–14The higher the enjoyment of reading, the higher the scale score23.0
Maternal qualificationThe more qualified the mother, the higher the cognitive composite score18.1
Motivation at 14The higher the motivation level at age 14, the higher the scale score13.7
School decile 8–14The higher the decile attended across the years of school, the higher the scale score12.3
Student values at 16Students with “satisfying life” values likely to have higher scale scores, and those with “standing out” values to have lower scale scores8.7
Family income at 5The higher the family income at age 5, the higher the composite score8.7
Family income at 16The higher the current family income, the higher the cognitive composite score7.9
Family financial situation 14Higher scores for those whose families were in comfortable financial situations7.2
Attendance at schoolNo difference between good, very good, or excellent attendance, or those with poor attendance because of ill health or participation in sports/arts; lower scores for those with fair attendance, and lower still for those with poor attendance4.8
Involvement in bullying
ages 8–14
The greater the involvement, the lower the scale score4.4
EthnicityHigher cognitive composite scores for Päkehä/Asian students3.7
TV watching aged 8–14The less time spent watching TV, the higher the scale score3.7

Further insights from multivariate analysis

We used the correlations and associations reported in the tables above to form models to see how much of the variance in student scores we could account for, and which of the factors and categorical variables were the strongest—which would remain in the model, and make a separate contribution to it. In these models, we find that age-14 competency scores account for most of the variance, and as it were “soak up” other related factors. Some of the factors that remained in the model are not the ones that show the strongest associations when analysed as single factors; so it may be that they remain because they account for some unique part of the variability.

As an example, risky behaviour appears in these models for the cognitive competency and all the attitudinal competencies; and attendance in the teacher-rated competencies related to attitudes to class work (focused and responsible, thinking and learning), but not in the model for the number of Level 1 NCEA credits. However, factors that show some association with risky behaviour (attendance and previous involvement in bullying) do appear in this model.

These models where performance or attitude levels two years earlier play such a strong role do account for a reasonable level of the variance in student scores at age 16—but not all of it. That tells us two things. First, that when it comes to the years when students encounter qualification assessments, their reaction to what they are offered does draw from the habits and attitudes and knowledge they have developed previously, in what we can think of as a “learning identity”. But these models also show that prior performance levels are not the only thing that determines how students act in class, how they respond to the learning opportunities they are offered there, and how well they do on qualification assessments.

Table 19: Results of multivariate models to predict age-16 competency and Level 1 NCEA credits using age-14 and age-16 factors
Competency/NCEAPattern foundR2 (% of variance explained)
Cognitive compositeDominant factor: cognitive composite age 14, followed by attitudinal composite 14; and then risky behaviour78.0
Thinking & learningStrongest factors: attitudinal & cognitive composites age 14; followed by parent perception of self-confidence; risky behaviour; attendance50.0
Focused & responsibleStrongest factors: attitudinal composite age 14, risky behaviour; followed by cognitive composite age 14, attendance; family communicates well58.0
Social skillsStrongest factor: attitudinal composite age 14; followed by cognitive composite age 14, risky behaviour, student values, praise & achievement, parent perception of self-efficacy, internal markers of progress at age 1436.0
Social difficultiesStrongest factors: age 14 attitudinal & cognitive composites; followed by risky behaviour30.0
Level 1 NCEA creditsStrongest factors: cognitive & attitudinal composites age 14; followed by attendance, family income age 14, involvement in bullying 8–14; then family communicates well, parent perception of responsibility, students working alone in English class, year level56.0


We also undertook a multivariate model that ignored current (age-16) experiences but started with age-8 competency levels and added social characteristics with some of the key indicators (from the models that are reported in the next chapter on school engagement) of how a learning identity has formed before students tackled senior secondary school. These key indicators were: enjoyment of reading, age-14 motivation levels, and school decile-pattern from age 8 to age 14. The purpose of this model was to see just what changes are possible over schooling, and whether these might occur differently for students with different social characteristics: or, to put it another way, what weight does school experience have between age 8 and age 16, and, therefore, how important is it for students to keep engaging with school and learning after their first three years of school?

The importance of performance levels before students reach school, and of their gains in the first three years of school has become more and more evident with each phase of this study. Of those who were in the lowest quartile of performance for each of the relevant competency measures at age 5, only 11 percent scored at the median or above at age 14 for mathematics, 29 percent for reading, and 42 percent for the attitudinal composite. The window of opportunity to make gains by age 14 is even narrower after age 8: 9 percent of those whose scores put them in the lowest quartile at age 8 improved their scores to reach the median or above in mathematics, 15 percent in reading, and 23 percent on the attitudinal composite.

However, the next table shows that although age-8 performance levels can account for a reasonable level of the variance in age-16 performance on NCEA, and even more of the variance in scores on the cognitive composite, they also leave much of this variance unaccounted for. A model that includes age-8 cognitive composite scores (the average of the age-8 numeracy, reading, writing, logical problem solving scores) can account for 65 percent of the variance in the age-16 cognitive composite scores, leaving 35 percent of the variance unaccounted for. Thus, what happens in between age 8 and age 16—at school, at home, and in activities and friendships—does matter.

The social characteristics that have a bearing on what happens in these years are maternal qualification (in relation to the cognitive composite score), and family income (in relation to Level 1 NCEA credits). Linked to family income through housing or the affordance of different levels of voluntary donations (state schools) or school fees (integrated and private schools) we see that decile history is relevant to both: an indication too of the role of peers in the development of learning identities. The other variables that remain also point to differences in ways that individual students have spent time, and in what has become important to them.

Table 20: Results of multivariate models to predict age-16 cognitive competency level and number of Level 1 NCEA credits from age–8 competency levels
Age-16 performancePattern foundR2 (% of variance explained)
Cognitive compositeDominant factor: age-8 cognitive composite; followed by maternal qualification, school decile 8–14, enjoyment of reading 8–14, age-14 motivation; then year level65.0
Number of Level 1
NCEA credits
Strongest factor: age-8 cognitive composite; followed by school decile pattern 8–14, involvement in bullying 8–14, & family income at 14; then age-14 motivation levels, age-14 values; and then the attitudinal composite at age 841.0

Some implications in relation to policy

What are the main implications of the analyses we have summarised in this chapter? There are four main messages that we draw from our findings, in relation to the current policy environment, particularly the introduction of the NCEA as a major departure from the previous senior school qualification system, and the coming introduction of the New Zealand Curriculum.

In this chapter, we traversed a range of different perspectives on 16-year-olds’ performance. These perspectives included multiple-choice assessments of numeracy and literacy, teachers’ judgements of student approaches and behaviour based on what they had seen in their classes, parents’ judgements of student approaches and behaviour based on what they had seen at home and on shared occasions, and achievement of NCEA credits, some internally assessed, and some assessed through examinations. We showed that, while there was considerable consistency between these different perspectives, they did not always give the same picture. For example, while a high-scoring student on our study’s measures for the cognitive and attitudinal competencies was also likely to get high numbers of NCEA credits, that was not always the case. Another example of how differences in judgement can differ: parents saw their children operate in different contexts than did their teachers, contexts that were likely to be more individualised, and more fluid than class settings, so their ratings of their children’s attitudes were likely to be somewhat higher than teachers’—but not always. These differences in perspectives underline the importance of considering context when making judgements or decisions based on individual performance; they also underline the value of seeking additional information about individuals if we are concerned with lapses from previous performance or wanting to improve performance.

However, it was unusual to get a completely different picture of an individual 16-year-old from the different information we had on their performance. Our comparison of the results of our competency assessments with NCEA performance should reassure those who have wondered if the new qualification was too lenient, or softer than its predecessors: those who struggled with our more traditional assessments were also more likely than not to struggle to achieve sufficient numbers of NCEA credits to gain useful qualifications.

Our models indicate that student attitudes and behaviour are as important to school success in NCEA as cognitive levels, underlining the importance of integrating development of the key competencies with development of what we have thought of as “academic” knowledge and skills if we are to improve student qualification levels, and reduce achievement gaps.

As well as giving us insight into individual students’ approaches to their learning, teacher assessments also gave us some useful information on the prevalence of the skills, knowledge, and dispositions emphasised in the new key competencies now included in the New Zealand Curriculum. Many of the dimensions represented by the key competencies are not widespread among this sample, even though the study sample has an over-representation of young people from homes with high levels of parental education and income, i.e., those who are more likely to have had opportunities to develop these dimensions. Chapters 7 and 8 in this report take a closer look at the key competency dimensions in terms of teaching practice and opportunities to learn. Both sources of information, on the opportunities for learning and the existing levels shown by students, indicate that the key competencies will need considerable support if we are to make the most of their inclusion in the New Zealand Curriculum.

We found that differences in both the number of NCEA credits attempted and gained reflected differences in subject clusters, with more on offer and gained in the “traditional” arts and science clusters than “vocational” clusters, or those we termed “contextual” subject clusters. Thus some of the differentiation occurring with the previous system of senior school qualifications has continued with the new system. This raises some questions about what more might be needed to improve opportunities to learn and to gain useful qualifications. Chapter 9 looks at these questions in more detail, and explores the question of one unintended consequence of the NCEA, a focus on credit accumulation.

Entrenched or open?

When we consider the patterns we see over time, we can also sketch some implications, of a broader nature.

By age 16, when the young people in this study were undertaking assessments for senior school qualifications as part of their ongoing courses as well as in end-of-year examinations, much of their learning identities was already shaped. So how they responded to these assessments, as well as to their classes, did carry much of what they had gained from their previous experiences: the attitudes they took to school and learning, previous success at school (both attitudes and success reflecting the kinds of opportunities they had had to learn). To succeed and make the most of secondary school years generally requires successful primary school years, and before that, rich early learning opportunities.

Most of the information we have at a national level about achievement gaps reports them in terms of social characteristics, particularly gender and ethnicity. Our analyses show other factors playing a larger part in accounting for differences in student performance. We look in Chapter 14 at differences related to social characteristics that might account for what we see in the national reports, and related to the factors that come through more strongly in this study, such as risky behaviour or other ways of spending time, developing habits and identities.

Some of the young people’s responses also reflected the regrettable fact that they had started to establish themselves as young people who gained a sense of themselves through risky behaviour and having friends who also made meaning of their lives through such behaviour, at the expense of making the most of what school could offer. Our analyses certainly point to risky behaviour in early and mid-adolescence as a key indicator of low performance, both in senior school qualifications and on our measures of cognitive and attiudinal competencies. Some of those who seemed to identify themselves as this kind of risk taker (as opposed to taking risks in new learning) had built up this identity over years; others seemed to have been attracted to this identity more recently, in early adolescence.

But the 16-year-olds’ performance was not just the sum of their previous experiences or their current ways of spending time out of school. We also found that current levels of engagement with school had some part in student success on senior school qualifications. In the next chapter, we focus on school engagement, and analyse the associations it has with different experiences, in and out of school, to see what we could be doing to improve engagement levels, and thus student success.

Footnotes 

  1. A fuller picture of the age-16 competency levels, and how they relate to previous competency levels is given in Wylie and Hodgen (2007), with the complete picture in the technical report by Hodgen (2007).
  2. Because we did not have exactly three teacher responses for each student (the number varied between none and three, with the vast majority having three), it is not quite true to say “Seventeen percent of the students are seen by teachers to always take on new ideas”, although this will, approximately, be true.
  3. Seven students were taking the Cambridge examination, and did not undertake any NCEA assessments.
  4. Reflecting the nature of the subjects in each cluster these were given the names: traditional arts; traditional science; contextual subjects; and traditional vocational. See the appendix for listings of subjects most likely to be taken by students in each cluster.
  5. We compared the total number of Level 1 NCEA credits gained, rather than the total number of credits, as both Year 11 and Year 12 students had all had the opportunity to attempt Level 1 credits. Also, while both Year 11 and Year 12 students have approximately equal numbers of Level 1 credits (on average, Year 12 students have slightly more), they have very different total numbers of credits.
  6. The different levels of competency used are based on the four quartile groups (the bottom quarter, second-to-bottom quarter, second-to-top quarter, and top quarter of the students) for that competency. Approximately a quarter of the students are in each quartile group (97 to 110 students, depending on how many students had a measure equal to one of the cut-points for a score).
  7. For full details, see Chapter 9 in the technical report accompanying this report (Hodgen, 2008).
  8. There were two factors related to student engagement evident in student responses about their behaviour at school, and how they felt about school. Engagement in school includes items about liking teachers, enjoying learning, and conversely (reverse scored to make the factor score) not getting bored, feeling restless, and wanting to leave school. Affirmed at school includes items about feeling safe, feeling that the student belongs, being treated as an individual, the fairness of school discipline and rules.
  9. As these are proportional measures, the proportions were calculated as, for example, the total number of standards at the merit level (Level 1–3) out of the total number of standards that were achieved at any level (levels 1–3) by the student.
  10. These are described in Appendix 1, and in the chapters that focus on each of these variables in turn.
  11. The R2 is for each variable taken individually and since there are overlaps between each of these—e.g., those who had good or better school attendance were also more likely to gain more NCEA credits—this table and others in this format do not show what difference each of these factors might make for student performance if other things were accounted for. For example, if attendance was in the model, accounting for approximately 12 percent of the variability, how much more of the variability would the number of Level 1 NCEA credits account for? Probably less than 31 percent, and this was explored in the more complex models, and on ‎ the *s indicate that one of the better possible models to predict age-16 levels of focused and responsible from age-14 competencies and age-16 attendance and out-of-school variables included the age-14 attitudinal and cognitive competencies, and age-16 parental view of self-confidence and attendance.
  12. We have included all those of 0.2 and above (in absolute value), and have paid particular attention to those that have higher correlation levels than 0.4.
 

 Copyright © Education Counts 2014   |   Contact information.officer@minedu.govt.nz for enquiries.