Summary Report: Motivation and Achievement at Secondary School - The Relationship between NCEA Design and Student Motivation and Achievement: A Three Year Follow-Up
This Longitudinal Research on the NCEA and Student Motivation and Achievement reports the results of the final two years of a four year research project to investigate relationships between the National Certificate of Educational Achievement (NCEA) and how students think about their learning and achieve academically in secondary school. This is the summary report, the full report from which this summary report is derived will be available from mid-July.
Author(s): Luanna H. Meyer, John McClure, Kirsty F. Weir, Frank Walkey, and Lynanne McKenzie
Date Published: June 2009
This Longitudinal Research on the NCEA and Student Motivation and Achievement reports the results of the final two years of a four year research project to investigate relationships between the National Certificate of Educational Achievement (NCEA) and how students think about their learning and achieve academically in secondary school.
Background to the Research
The project started in 2005, coinciding with the full implementation of New Zealand’s NCEA across all three years of the senior secondary school. Its purpose was to investigate the influences of this assessment system and its various design components on student motivation, study behaviours and achievement outcomes.
The research has followed several large student cohorts from a nationally representative sample of 20 secondary schools across their high school careers, beginning in year 10 prior to starting NCEA. Using a purpose-designed motivation survey focused specifically on aspects of the NCEA, we found that student motivation orientations in years 10-11 were related to future academic achievement in years 11 to 13. Nearly 1,200 students in Year 10 in 2005 were tracked to year 13 in 2008; records for 1,500 students overall had been tracked to Year 12 in 2007. From the 2006 sample, over 2,100 year 10 or 11 students were tracked to year 12 or 13 in 2008; 3,200 students had been located in Year 11 or 12 in 2007.
Based on the results using the full motivation measure, we developed a brief screening measure for Years 10-11 that was completed by nearly 4,000 students in 2007 and nearly 5,500 students in 2008 from 19 of the original 20 schools. We were able to replicate earlier findings that the two motivation orientations Doing My Best and Doing Just Enough were not only highly correlated with current achievement but were strong predictors of achievement 1-2 years later. A major goal of creating this brief screening measure was to provide a tool for school use to assess student motivation.
This Survey of NCEA Goals also measures student attributions for their best and worst marks in any subject, as well as self-ratings of the influences of teachers, peers, and family/whānau on achievement. We also asked the students to report whether they knew about the newly introduced certificate endorsements and whether these mattered to them. Finally, students reported information about non-school activities such as their involvement in sport, part-time work, family childcare, and other extracurricular activities.
In 2008 as in previous years, students also participated in focus group interviews. In all, 220 students participated in 23 focus groups at ten secondary schools, five from our national sample and five schools new to the study. In collaboration with Māori and Pacific colleagues, two focus groups comprising Māori and Pacific whānau provided insights into how well these parents thought the NCEA was working for their children, strategies they used to encourage achievement and study behaviour, and their opinions of various NCEA design issues.
This short summary report has been prepared for distribution to schools and other interested parties. The full report of this study will be available on the Ministry of Education’s Education Counts website http://www.educationcounts.govt.nz/publications enabling readers to access a copy or read more about particular sections of the research.
The research revealed the value of a concise measure of student motivation prior to beginning NCEA study in relationship to predicting achievement. Being able to predict achievement results early could inform teacher and school efforts to enhance teaching interventions and promote positive student study behaviour. Major overall findings emerging from the research are:
- NCEA Certificate Endorsements: The introduction of the NCEA certificate endorsements was followed by increases in motivation by students to do their best in their work—at all levels of achievement, not only for high achieving students. A significant number of students reported not knowing about the endorsements in both 2007 and 2008. Students who reported knowing about the endorsements showed stronger motivation to do their best than students who said they did not know.
- Teacher Relationships: Students’ perceptions of teachers’ attitudes towards them and their learning were significantly related to their motivation to do well on the NCEA. For Māori students, these perceptions were significantly related to how many credits they attained.
- Assessing Motivation: The short motivation assessment measure developed by the research can be used diagnostically to detect student motivation patterns that without intervention are likely to lead to low achievement. See Appendix A, on page 110 of the full report, available from: http://www.educationcounts.govt.nz/publications/schooling
- External Influences on Achievement: Students who do small amounts of part-time work, sport or other extracurricular activities show higher motivation and achievement patterns than students who do not participate in these activities or who do so more than 10 hours weekly. Students who spend more than a few hours weekly caring for other children in the family show lower patterns of motivation and achievement.
Relationship of Motivation Orientations to NCEA Achievement
- Self-reported motivation orientations for Doing My Best and Doing Just Enough in 2005 showed strong relationships with achievement 2-3 years later in 2007-2008 on total credits, internal credits with Excellence, and the number of external credits with Achieved, with Merit, and with Excellence.
- High scores on Doing Just Enough in 2005 significantly predicted higher numbers of Unit Standards in 2007 and 2008.
- Whereas achievement in Year 11 in 2005 was a significant predictor of achievement in Year 13 in 2007, positive motivation orientations added significantly to the accuracy of prediction of future achievement. Motivation orientations were better predictors of total credits achieved two years later than predictions made based solely on credits attained in previous years.
- There were significant differences in achievement related to whether students increased, decreased or stayed the same in self-reported motivation orientations from 2006 to 2007, with those whose motivation was stable or increased achieving more credits whereas those whose motivation declined attained fewer credits in 2007.
Effects of NCEA Certificate Endorsements on Motivation and Achievement
- Nearly half of Year 10-11 students reported not knowing that NCEA Certificates could be endorsed with Merit or Excellence both late in 2007 (47%) and in 2008 (45%).
- Of those who knew about the endorsements, students reported overwhelmingly that they mattered to them, with only 1 in 15 saying that the endorsements didn’t matter at all. The percentage who said endorsements did not matter was constant across the two years regardless of school decile.
- Self-reported knowledge of the endorsements was positively related to gaining NCEA Level 1 in Year 11 and was also related to gaining either a Merit or Excellence Level 1 endorsement. A large majority of students attaining NCEA Level 1 with Merit said the endorsements mattered to them mostly or definitely, and virtually all students attaining the certificate with Excellence said the endorsements mattered mostly or definitely.
- Comparing student self-reports about the endorsements in 2007 (only three months after these were announced) with 2008 (a full year after implementation) revealed relatively few differences in student ratings by school decile with two notable findings: (a) a pronounced increase from 2007 to 2008 in the percentage of students reporting that endorsements definitely mattered occurred in low decile schools; and (b) the percentage of students reporting that endorsements mattered to them at different levels remained constant across 2007 to 2008 for middle decile schools.
Student Attributions for their Best and Worst Assessment Results
- Students’ most highly ranked attributions for their best and worst marks were their own effort, their ability, the difficulty of the assessment task, and teacher influences. They rated the influences of luck and family/whānau lower. Students’ attributing their best marks to effort was a significant positive predictor of NCEA achievement.
- Girls were more likely to attribute their best marks to effort and their worst marks to lack of ability and difficulty of the assessment task in comparison to boys. Girls also attributed best and worst marks to teacher influences more than boys, who were more likely to attribute worst marks to bad luck.
- Pacific students rated both family and friend influences as more important to both best and worst marks than did NZ European, Asian and Māori students. Māori and Pacific students attributed their best marks less to ability, effort and assessment task difficulty than did European and Asian students.
The Relationship of Non-School Activities to Motivation and Achievement
- In 2007 and 2008, between 40-50% of Year 10-11 students reported they had part-time jobs, and those who worked up to 10 hours weekly generally attained more total credits than those who did not work at all or who worked 10 or more hours weekly.
- Over 60% of Year 10-11 students reported participation in sport, and those who reported playing sport attained more credits than those who did not.
- Nearly half of Year 10-11 students reported some weekly involvement in caring for siblings and other children in the family, with students from low decile schools doing proportionately more childcare and students from high decile schools doing less. NZ European students were less involved in childcare than others, and Māori and Pacific students reported more childcare than others. Childcare was negatively related to achievement in 2007 but showed no significant relationship in the 2008 data.
Teacher and School Influences on Motivation and NCEA Achievement
- Students in both the junior and senior secondary school overwhelmingly indicated they need more information from their schools and particularly from their teachers about the NCEA. They indicated that some teachers seem very well informed and provide excellent support, but most teachers do not and seem to leave this responsibility to the school overall. Parents echoed that their children would benefit from more information about the NCEA that was easily accessible in a range of ongoing ways.
- Many students asserted that it was teachers who had the single biggest impact on their learning, and they like teachers who make learning fun, catch their interest in the subject, and use humour in their teaching.
- Students value teachers who show a personal interest in them and their learning. They want teachers who set boundaries and express high expectations for them. They commented on the negative impact of teachers who communicate low expectations, show impatience or demonstrate favouritism. They want teachers who “believe in you and say you can do it.”
- Students prefer teachers who are knowledgeable about the subject but also how it relates to real life and future career goals. They value teachers who acknowledge the experiences students bring with them to the classroom and what they will take away with them for use later in life.
Parent and Student Attitudes about the NCEA and Design Changes
- Students and parents in the focus groups expressed overwhelming support for the Certificate endorsements. They felt that this external motivator added to existing internal motivators and created additional incentives within one’s peer group and school. They also felt that employers would value the endorsements so having these would enhance their career options. They were less sure about whether the endorsements would have an impact on university study.
- Māori student and parent focus groups interviewed at two schools were extremely well informed about the endorsements and how the NCEA worked generally. The students reported that they were highly motivated to do well, that their teachers cared about them and encouraged them to do their best, and that they anticipated getting Merit or Excellence certificate endorsements.
- Māori parents felt that the NCEA allowed their children to demonstrate achievement better than “previous ways” and was a “good system for Māori.”
- Many students continued to argue in favour of more grade bands and supported the development of subject endorsements in recognition that some students will excel in individual subjects but not in a number of areas as required for a certificate endorsement.
- There was strong support from students and parents in favour of having a single system of Achievement Standards rather than continuing with Unit Standards. Students in particular commented on the inability to earn Merit and Excellence for most Unit Standards and expressed concern over consistency and credit parity issues.
- Parents described a broad range of strategies to motivate their children and were generally positive about teacher influences, but they lacked specifics about what teachers were doing to support high achievement. They believed that the NCEA system had become more motivating with the introduction of endorsements for Merit and Excellence.
2. Implications of the Research for Supporting Student Motivation and Achievement
This longitudinal research on the impact of the NCEA on student motivation and achievement lays the foundation for intervention research to enhance how students think about their learning and to improve educational outcomes for all. Recently implemented design changes to NCEA are viewed positively by students and their families. This research has direct implications for the nature of communications with students and how teachers, parents and schools in general can encourage students to take on educational challenges to meet high expectations.
The NCEA and School Influences on Motivation and Achievement
More needs to be done at secondary level to provide students and their families with information about the NCEA. Knowing about the NCEA and how it works will enable young people to approach their senior years with determination to meet high expectations, not just accumulate credits. Students need better understandings of the impact of their motivation orientations on their academic achievement and how motivation relates to reaching life goals. Doing well on aspects of the NCEA involves mastering important subject area content and key learning skills. But as an integrated system of internal and external assessments of standards, the NCEA could also provide students with multiple opportunities to build better understandings of how their motivation orientation affects their achievement. This also links with development of the key competency of Managing Self in the New Zealand Curriculum (2007).
Teacher Influences on Motivation and Achievement
Teachers have both direct and indirect influences on student learning. Obviously, teachers organize the teaching and learning process for students towards achieving standards within the various subjects. But teachers also communicate to students whether or not they expect students to succeed, and students may perceive that some teachers are not interested in them or their learning. Teachers can enhance the learning environment for students by examining their relationships with students to ensure that they communicate caring about how each individual student is doing in school. Students’ perceptions of whether their teachers were interested in them and their learning were significantly correlated with achievement. For Māori students, the relationship with teachers was particularly important.
Non-School Influences on Motivation and Achievement
Families too have a critical role to play in monitoring their children’s out-of-school activities. Some part-time work and participation in sport and other extracurricular activities is associated with higher achievement than none at all or too much. This evidence supports encouraging young people to participate in activities outside school provided that these activities are kept to a manageable level. Too much time spent caring for other children in the family, however, detracts from doing well in school. Childcare can be an important part of acquiring a sense of family responsibilities for the future, but it may be that we sometimes expect too much from young people at a time when they need to develop their academic strengths so that they are prepared for positive futures.
Building Motivation to Enhance Future Achievement
We demonstrated that scores on a brief measure of students’ motivation towards Doing My Best or Doing Just Enough in Year 10—before students have started the NCEA—significantly predict students’ future achievement on the NCEA. By using this simple motivation measure, educators could assess how students think about their learning with a view to developing more positive motivation patterns so that students can meet high expectations.
These findings lay the groundwork for intervention initiatives to encourage students to adopt more positive motivation orientations towards their learning. As a standards-based assessment model, the NCEA can be conceptualised as a series of criterion-referenced achievement tasks—opportunities to improve one’s learning skills. More work is needed to demonstrate the kinds of messages that enhance Doing My Best motivations rather than Doing Just Enough approaches to school in general and NCEA assessments in particular.
Student and Parent Voice on NCEA Design Issues
Finally, the students and parents participating in this research strongly supported the introduction of Certificate Endorsements for Merit and Excellence, the review of Unit Standards, and further work on consistency across subjects and schools. There is concern that Unit Standards are disadvantaging students where they cannot strive for Merit and Excellence. As in previous years, there was also encouragement to address other aspects, such as subject endorsements and grade bands. The endorsements hold a positive relationship to motivation and achievement, not just for high achievers but for all students. This demonstrates that improvements can be made to a standards-based system towards more recognition of hard work and excellence without disadvantaging students generally. Having high expectations and rewarding students for meeting those expectations—across the curriculum—could only be a good thing for all students and their communities.
3. Research Approach
The purpose of the research was to investigate the influence of aspects of the NCEA on student motivation and achievement. During the first two phases of the research (2005-2007), we were able to show student motivation patterns related to particular design features of the NCEA and heard from students about their thoughts on how the NCEA had an impact on student study behaviour and achievement. By following students for two more years, we can now relate early indications of students’ approach to their learning and the NCEA to their actual achievement in the senior secondary school.
Measuring Motivation & Other Influences on Learning
We used the results from statistical analyses of an earlier, longer form of the self-report survey to develop a screening measure focused on motivation orientations and other influences shown to be related to aspects of the NCEA and student achievement. Among the descriptive questions, we asked students to tell us which levels of the NCEA they expected to complete, whether or not they knew that certificates could be endorsed with Merit or Excellence, and to rate on a 4-point scale how much the endorsements mattered to them.
One section of the Survey of NCEA Goals assesses motivation orientations shown to be significantly related to achievement—Doing My Best (DMB) and Doing Just Enough (DJE)—which we measured using items applying to aspects of the NCEA. This section also includes items to investigate influences of teachers and peers. Students rate the 16 items in this section on a Likert scale from 1-4, where 1 = “not me” and 4 = “definitely me”. Table 1 illustrates the types of items that assess the two motivation orientations and teacher and peer affiliation influences.
We were also interested in other personal, social and external influences on achievement. Another section of the survey comprises self-ratings of students’ attributions for their “best marks” and “worst marks” on “assessments in any subject”. For this section, students rated the influences of ability, effort, luck, task difficulty, family/whānau, the teacher and friends on a Likert scale from 1-4, where 1 = “no influence” and 4 = “big influence”. Students also estimated the number of hours spent weekly in part-time work, sport, caring for younger children in the family, other activities (e.g., music, scouts, volunteer work) and attending paid tutorials outside school.
|Dimension being measured||Sample item|
|Doing My Best|| |
|Doing Just Enough|| |
|Teacher Affiliation|| |
|Peer Affiliation|| |
The survey items were analysed statistically to reveal relationships among the different experiences and dimensions. We investigated the strength of the relationships and predictive power of the different significant relationships to future achievement on the NCEA. These results should not be used to make individual judgments, but they do reveal patterns of motivation orientations, relationships with others (including teachers and peers), and external commitments that predict likely achievement patterns. Patterns predictive of low achievement based on specific goal statements about the NCEA and relationships with teachers are two key areas where intervention could make a difference.
Details regarding initial development of the survey are available in the 2006, 2007 and 2009 reports, and more information about the statistical analyses is available in the full reports and in Meyer, McClure, Walkey, Weir, and McKenzie (2009).
Measuring Achievement on the NCEA
Students gave consent for us to access their achievement data in their official Records of Learning for Levels 1 to 3 of the NCEA; these data were provided to us directly from the NZQA. Achievement data that we analysed include: (a) total credits; (b) total Achievement Standard credits; (c) total Unit Standard credits; (d) credits Not Achieved, Achieved, achieved with Merit, and achieved with Excellence for Achievement Standards; (e) credits Not Achieved (available for 2008 only), Achieved, and—if available—achievement with Merit for Unit Standards. We also recorded whether the student attained particular levels of the NCEA certificate that year (1, 2, or 3) and whether the certificate was endorsed with Merit or Excellence. Finally, we recorded whether or not students had met requirements for University Entrance (UE).
Perspectives of Students & Parents about NCEA
Student Focus Group Interviews
Focus group interviews were conducted with Year 10 and with mixed groups of students from the senior secondary school, generally Years 12-13 so that they would have more experience with the NCEA. Students were asked what they knew about the NCEA; what they knew and thought about changes to the NCEA and the endorsements; what they liked and did not like about the NCEA; and how their schoolwork is influenced by friends, teachers and family/whānau. They were asked to tell us things they would like to change and things they want to stay the same. Senior students who were engaged with NCEA work were asked what they thought about having both Achievement Standards and Unit Standards.
In all, 23 focus groups with a total of 220 students from 10 different schools participated. While Māori and Pacific students were included in most groups, several of the focus groups conducted at schools with bilingual or immersion units were comprised of only Māori or Pacific students.
Focus groups were conducted by two researchers working as a team, with one researcher serving as facilitator and the other as note taker. This allowed us to “member check” with each group by reading out the list of contributions made by the group directly after each question, thus giving students the opportunity to correct or add to the list. We identified major themes emerging from the data, and we also cross-referenced these with results from the 2006 and 2007 studies (see especially the 2009 full report for more information).
Whānau/Family Focus Groups
In previous reports, we discussed input from parent interviews across a range of schools but parents of New Zealand European descent were overrepresented in the parents referred to us by the participating schools. Thus, in 2008, in partnership with James Graham (Ngai Te Whatuiāpiti) and Tolo Pereira, we conducted four parent focus group interviews with Māori and Pacific parents in separate groups. Following appropriate cultural protocols, the interviews and analyses followed similar procedures described for the student groups.
For further information about the Research Approach, see also pages 14-18 of the full report, available from: http://www.educationcounts.govt.nz/publications/schooling
Report of the Results
The following sections summarise results from the research, including the relationship between motivation and achievement; impact of NCEA design changes; influences of non-school activities; influences of teachers, peers and family; and student attributions for their assessment results.
4. Relationship between Motivation and Achievement on the NCEA
Two key motivation orientations Doing My Bestand Doing Just Enough, measured in relationship to aspects of the NCEA, significantly predict future achievement. The patterns of these two motivation orientations also differ by gender and by ethnicity, and these different motivation orientations are relatively stable over time. Our results support the validity of a brief screening measure predictive of future achievement that could drive intervention efforts.
Motivation as a Predictor of Achievement
High scores on Doing My Best significantly predicted more total credits, internally assessed standards with Excellence, and externally assessed standards at all levels—Achieved, Merit and Excellence. Doing Just Enough scores predict lower achievement across two years and more total Unit Standard credits.
One might think that these two motivations are simply reversals of one another, but our analyses showed that they were not. Students high on one were likely to be low on the other (thus they were moderately correlated) but it is not uncommon for students to have a mixed motivation pattern—being moderately high or low on both; thus the correlation was not so high as to indicate they measured the same construct. It is likely that students with these mixed patterns are also underachievers, in striving to do well on some things but giving up on others. Knowing how a student scores on these two motivation orientations provides information about future achievement that is over and above that provided by previous achievement alone.
Females reported Doing My Best slightly more than males, and males reported Doing Just Enough slightly more than females. As in previous years, we also found once again that Asian students showed the most positive motivation orientation, high on Doing My Best and low on Doing Just Enough.
For most students, motivation was quite stable across time, but there were also changes in motivation that are related significantly to future achievement. We constructed composite Doing My Best/ Doing Just Enough scores to describe six overall motivation categories, from low to high. We found that one-third of students stayed in the same category, one-third showed a minor motivation shift, and one-third showed larger shifts either positively or negatively, moving two or even three categories up or down from one year to the next. Students whose motivation stayed the same or shifted upwards achieved more total credits in subsequent years compared to those whose motivation shifted downwards.
More males than females moved up two or more motivation categories from Year 11 to Year 12. Asian students were most likely to increase in motivation across years, and European students disproportionately decreased most in motivation in comparison with all other ethnic groups. School decile zone was not related to motivation changes.
Validity of a Motivation Screening Measure
A goal of the longitudinal research was to develop and validate a motivation orientation screening measure that would be predictive of future achievement independently of information on prior achievement. Such a measure could have considerable utility for use in the junior secondary school, prior to having useful evidence on student achievement. Further intervention research could then investigate the effectiveness of different strategies to shift motivation patterns from Doing Just Enough to Doing My Best, including mixed patterns that may be related to underachievement where students are somewhat high on Doing My Best but also high on Doing Just Enough.
The Survey of NCEA Goals screening measure was also revised for this project to incorporate other self-ratings of factors shown to relate to motivation and achievement. We incorporated items to assess teacher affiliation (whether the student believes teachers care about them and their learning) and peer affiliation (whether the student prefers working in groups and being able to help one another, or working alone). We found that teacher affiliation in particular relates positively to student motivation and achievement, and this was particularly so for Māori students. For Māori students, perceptions that the teacher cares about the student and his/her work were correlated with total credits achieved.
The importance of teacher influences was further supported by students’ attributions for their best and worst assessment results. Other attributions that were related to achievement include ability, effort, task difficulty, and luck as well as peer and family/whānau influences. In addition, we examined how non-school activities relate to achievement and motivation. These findings are discussed in Sections 6 and 7 of this report.
The Survey of NCEA Goals has been shown to be a valid and useful measure that could be used as part of a programme designed to maximise student motivation and achievement. The measure can easily be completed within a form period in secondary school, in 5-8 minutes, and students report that it is easy and interesting for them to do.
5. Impact of NCEA Design Changes
In July 2007, the NZQA announced design changes to the NCEA that were driven by discussions about motivation and achievement. It was also announced that there would be a major review of the standards for consistency, and more systematic moderation across schools and subjects was initiated including the appointment of a larger team of specialist personnel to carry out moderation. One major design change signaled in our earlier reports was the need for additional recognition for high achievement across all standards. The introduction of the certificate endorsements for Merit and Excellence in 2007 enabled us to investigate their impact empirically.
NCEA Certificate Endorsements for Merit and Excellence
Our screening measure asks students if they are aware of the certificate endorsements and whether these endorsements matter to them. We were surprised by the relatively low percentage of students of the total completing our surveys in October 2007 (nearly 4,000) and 2008 (nearly 5,500) who reported knowing about the endorsements. While more Year 11 students were aware—which is logical as they should be in the thick of NCEA work—approximately one in four students in Year 11 said they did not know about endorsements as late as October 2008 (see Table 2).
|Year||Total %||Males||Females||Year 10||Year 11|
The patterns of reported awareness by school decile zone were also of interest to us, illustrated in Table 3. Roughly 1 in every 2 students at middle decile schools reported being aware of the certificate endorsements, and this figure declined slightly in 2008. In comparison with 2007, students in high decile schools showed a slight increase in awareness, and students in low decile schools reported the largest percentage increase in awareness of the endorsements in 2008.
|Year||Low Decile||Middle Decile||High Decile|
Based on these self-reports from students, it is clear that students need more information about the endorsements from their schools across all decile levels.
We also asked students whether the endorsements mattered to them, and students reported overwhelmingly that they do matter. Of those who reported they knew about the endorsements, only about 6.5% said endorsements did not matter, and this figure remained constant across both 2007 and 2008 data. This group of students may be those most difficult to motivate, and they appear to be evenly distributed across school decile zones in our sample. About one in four students said the endorsements mattered sometimes, 1 in 3 said they mattered mostly, and more than one-third said endorsements definitely mattered to them.
We were interested in whether there were any changes from 2007 to 2008 in whether students said endorsements mattered by school decile. Figure 1 illustrates these data (note that the stable 6% who said they did not matter are not included in the figure as this group showed no change across the two years or deciles).
Figure 1: 2007-2008 student responses on whether endorsements mattered, by school decile
Figure 1 reveals some interesting findings. It shows that the percentage of students from middle decile schools who say that the endorsements matter to them at all levels remained relatively constant from 2007-2008. There was a slight increase towards the endorsements mattering more in 2008 compared with 2007 for students in high decile schools, though the increase is small. In contrast, one in 10 students in low decile schools shifted ratings of how much the availability of endorsements mattered to them into the highest rating in 2008. With 30% of students in low decile schools marking this highest rating, they are nearly as high as those in middle decile schools and only 10% below those in high decile schools.
Our data from the student and parent focus groups offer further evidence that some schools appear to be doing more than others to encourage students to achieve. Māori and Pacific students were particularly adamant about this issue:
We learnt about it last year in Māori. The majority of us passed—all capable of passing and getting high achievement. Merit and Excellence endorsement makes the standard better. (Māori student)
I know you can get Merits and Excellence credits, and that you need it to get a better job. (Pacific student)
You’re always encouraged to achieve Excellence and Merit at this place. (Pacific student)
On some exams, you can’t get Merit and Excellence—everyone gets the same number of credits. We would like the opportunity to get Excellence/Merit. (Māori student)
Relationship of Endorsement Knowledge to Motivation and Achievement
Knowing about the endorsements also seemed to act as a “buffer” in keeping student motivation higher and preventing a decline across the senior secondary school years. For Doing My Best, the reduction in motivation from 2006-2007 was larger across time for those who said they did not know about the endorsements in comparison with those who said they did. Furthermore, students who knew about the endorsements significantly increased their motivation score on Doing My Best from 2007 to 2008 in comparison with those who said they did not know. Students who reported knowing about the endorsements also showed a more positive pattern in their Doing Just Enough scores.
Perhaps the most important question is how knowledge of the endorsements related to achievement one year later. 2007 results showed that nearly 70% of students who said they knew about the endorsements attained NCEA Level 1 in Year 11, whereas only 49% of those who said they did not know attained Level 1.
For those who reported knowing about the endorsements, 34% received a certificate endorsement in comparison to only 7% of those who said they had not known about them. Of the students who received a certificate endorsed with Merit and Excellence, 80% and 98% respectively reported that the endorsements mattered to them mostly or definitely.
Of course, another important issue is whether knowledge of the endorsements is motivating some students but not others, so we analysed whether the endorsements would have differential relationships with achievement for students achieving at different levels. We created three achievement groups—low, medium, and high—based on an equal division of total credits divided into thirds across the sample. The lowest third achieved 79 credits or less, the middle third between 80-115 credits, and the highest third more than 115 credits. Reported knowledge of the endorsements was related to more positive patterns on both Doing My Best and Doing Just Enough in comparison with not knowing about them for all achievement groups. While students in the highest achievement group showed the most positive patterns, patterns were also positive for those in the lowest achievement group.
Other NCEA Design Considerations
In the focus group interviews, students and parents had a great deal to say about various other design features of the NCEA. Complete results from the student and parent focus groups interviews are available in the full report (Meyer, Weir, McClure, Walkey & McKenzie, 2009).
Internal and External Assessments
Most students continue to express positive attitudes about the mix of internal and external assessments in the NCEA. They felt that the internal assessments allowed them to spread their workload across the year and use feedback to improve future academic performance. There was also support for external assessments as a kind of “quality control” to ensure consistency of standards across subjects and schools. Some students mentioned that the external exams gave students a good last minute opportunity to achieve credits needed for the certificate.
Consistency, Fairness and Credit Parity
Students commented about having additional assessment opportunities such as re-sits and re-submitting assignments:
Doesn’t make sense to be able to re-take internals over and over again, shouldn’t be able to have everyone pass for the reputation at the school.
Students expressed concern about inconsistencies and discrepancies in effort and recognition for their work:
Some credits aren’t the right amount for the amount of work you do—easy subjects like tourism in comparison to English—unfair.
Some exams can’t get Merit and Excellence—everyone gets the same number of credits. [We] would like the opportunity to get Excellence and Merit.
Unit Standards and Achievement Standards
There were mixed opinions about Unit Standards, though, as noted in our earlier reports, the overall perception is that they represent an easier route for lower achieving students:
It’s good to have both—gives you choice. Unit Standards are for people who just want to pass, Achievement Standards for people who want to do Merit or Excellence.
There were concerns about Unit Standards being unfair to students in offering neither incentives nor recognition for working hard and doing well:
There’s no advantage to studying harder for Unit Standards.
We’ve described the strong support for certificate endorsements, but students also wanted subject endorsements as they saw this additional change as being fairer to students. They commented that subject endorsements would be highly motivating for everyone but especially so for those who might not be able to attain the overall Certificate endorsement yet could achieve Merit or Excellence in particular subject areas where they have strengths. In these discussions, however, the students expressed concern that some students are disadvantaged by the lack of availability of Merit and Excellence opportunities for Unit Standards:
Stupid—for some students who do Unit Standards, [they] can’t aim for Excellence in Unit Standards and sometimes if it’s your best subject, it can be disheartening.
Possibility of a Single NCEA System of Achievement Standards
In response to the question about the one thing they would like to see changed on the NCEA, many comments focused on having one system of Achievement Standards rather than continuing with a dual system including Unit Standards:
We are given Unit Standards throughout the year and get Achievement Standards at the end of the year in externals. It’s a disadvantage; we want to aim for Achievement Standards.
Having both is confusing—why not just one?
As in previous years, there continued to be student comments in favour of finer grade bands, such as the availability of a larger range of grades or percentages. There were also comments focused more broadly on the value of the three certificates and the existing grade range:
Having the grades A, M & E—better than A, B, C & D and it’s better than having a percentage. 55% doesn’t sound good, but Achieved looks like an A..
Students also commented on a variety of issues that were not so much about NCEA design but had more to do with how it was working in their schools and for them personally. Comments were both positive and negative:
The cost should be reduced—too expensive. $75 is a lot to pay to sit each time.
No exams at the end of the year, but rather at end of Term—would reduce stress to know whole subject at the end of the year.
Parents don’t know anything about it—still think back to School C. More information is necessary.
The credits—by getting credits, it’s easier to get a job in the future.
Good to be able to access the Record of Learning.
Parents especially supported ready access for students and their family/ whānau to information from their child’s record of learning:
Overall Satisfaction with the NCEA
In comparison to our interviews with students during the first two years of the project, there were far fewer comments in 2008 about the need for improvements to the NCEA overall. In this summary report, we have highlighted particular issues raised by the students, but students largely supported the NCEA. A comprehensive presentation of student comments is available in the full report (Meyer, Weir, McClure, Walkey & McKenzie, 2009).
We reported the results of parent and student focus groups in our 2006 and 2007 annual reports, and these align closely with the kinds of comments reported here about aspects of the NCEA. Many of the concerns expressed by both parents and students are being addressed by changes such as the introduction of endorsements, the review of standards, and increased moderation to assure the validity and consistency of assessment decisions across schools and subjects. Other issues offer opportunity to continue to improve the system further.
The 2008 focus group interviews with Māori and Pacific parents revealed high satisfaction with the way that the NCEA was working and can work for their children. These parents were particularly positive about the change from the previous system to the NCEA:
[It’s a] good system for Māori who can achieve while learning—can see it working throughout the year and can understand what the student is doing. (Māori parent)
School C [6th Form Certificate] was a problem; many failed. NCEA is different for those who fell apart under exam conditions—internals are good. (Māori parent)
The NCEA works well for my granddaughter. The old system failed half of the students and wasn’t fair and I had to wait until the end of the year [to find out]. (Pacific Parent)
Furthermore, the 2007 introduction of NCEA design changes were largely supported by students and families alike as necessary improvements to what is increasingly being seen as a positive system to guide student learning.
ONGOING NCEA DESIGN:
Between 2007 and 2010 the Ministry of Education and the New Zealand Qualifications Authority are working with subject associations on the Alignment of Standards (AoS) project. This is part of the review of standards on the National Qualifications Framework (NQF).
The standards review comprises four stages and the overall goal of the review is to address issues of duplication, credit parity, consistency, fairness and coherence for all achievement and unit standards on the NQF.
This will be achieved through the application of a set of principles for the standards review. These principles were developed during Stage 1 of the review process (now complete). The set of principles addresses the broad issue of what constitutes a fit-for-purpose standard. These principles will underpin future standards development and review across the entire National Qualifications Framework.
Stage 2 of the review process, which is now underway, will fulfil the Cabinet NCEA improvements directive to align the achievement and curriculum-related unit standards with the New Zealand Curriculum (2007).
6. Influences of Non-School Activities on Motivation and Achievement
Nearly 3,500 students in 2007 and 5,400 students in 2008 in Year 10 or Year 11 answered our five questions about non-school activities. Nearly half reported working part-time, and there were also high percentages of students engaged in sport and caring for younger children in the family. Our results generally support a “threshold effect” whereby motivation and achievement are higher for students engaged in part-time work or extracurricular activities up to 10 hours weekly in comparison with students spending 10 or more hours weekly or no time at all in these activities. Participating in child care for more than a few hours weekly was negatively related to total credits achieved.
The percentages of Year 10-11 students participating in these activities for differing amounts of time weekly are very similar across the two years (see Table 4).
|Activity||Not at all||5 or less||6-10 hours||11-15 hours||15+ hours|
|Part-time work 2007||53||20||16||6||5|
|Part-time work 2008||60||20||12||5||3|
|Caring for children 2007||52||33||8||2.5||4.5|
|Caring for children 2008||53||32||8||3||4|
a Percentages are calculated based on those who answered these questions
Influences of Part-time Work and Extracurricular Activities
Our analyses revealed that part-time work had a significant relation to total credits attained in both years. Those who did not work achieved significantly fewer credits than those who worked between 6-10 hours, in both years. Those who worked more than 15 hours also achieved fewer credits than those who worked between 6-10 hours. Further analyses showed that this pattern held across gender, ethnicity and school decile level: Working part-time for a few hours each week was associated with higher achievement than either not working at all or working more than 10 hours weekly.
For paid tutorials, students who reported participation for up to 5 hours weekly achieved more credits than those who were not involved in paid tutorials.
Playing sport also showed a positive relationship to achievement; in 2007, the group that played sport up to 5 hours weekly achieving significantly more credits than those who did not play sport. The 2008 data revealed that those who reported no participation in sport achieved fewer credits than those who played sport up to 15 hours weekly. Again, this pattern held regardless of gender, ethnicity, and school decile zone.
The relationship of participation in other extracurricular activities was similar. Those who reported no involvement in other activities achieved fewer credits than those who reported between 6-10 hours, and those who spent more than 15 hours weekly in other activities achieved fewer credits than those who reported 6-10 hours. There was a trend whereby students from middle decile schools were less motivated than students from either high or low decile schools if they participated in other activities for more than 15 hours.
Influences of Childcare
The achievement pattern for those who reported looking after younger children in the family/whānau was different. Those who reported no involvement doing childcare or who spent five or fewer hours weekly in childcare achieved more credits at NCEA Level 1 in Year 11 than those who spent six or more hours doing childcare.
We looked more closely at Year 11 students who reported spending 6 or more hours weekly in childcare and found:
- A higher proportion from low decile schools were engaged in childcare (23% in comparison with being 16% of the total sample)
- A lower proportion from high decile schools were engaged in childcare (13.5% in comparison with being 27% of the total sample)
- A lower proportion of New Zealand European students were engaged in childcare compared with other groups (9% in comparison to being 63% of the total sample)
- A higher proportion of Māori (22%) and Pacific (44.5%) compared with other groups of students (and in comparison to each being approximately 10% of the total sample)
To the extent that certain sub-groups of students are more likely to be involved in taking care of younger siblings for the family, it would appear that their achievement could be affected negatively. A student in one of our focus groups had this to say:
In summary, it would appear that involvement in part-time work, sport or another activity related to student interests does not have a negative impact on school achievement—it may even be beneficial provided that the time spent in out of school activities does not exceed 6-10 hours weekly. The one exception to this is childcare which appears to have a negative relationship to achievement if the time commitment is more than several hours weekly. Childcare is not specifically tied to the student’s interests and is unrelated to school, which could explain why the relationship to achievement is different.
7. Teacher, Peer, Family/Whānau and Other Influences on Motivation and Achievement
The research provides support for the influences of Teacher Affiliation and Peer Affiliation on motivation and achievement. Positive relationships with teachers relate significantly to positive motivation and achievement. Conversely, students who reported their teachers were not interested in their learning showed more negative motivation orientations and lower achievement. Students attributed their best and worst assessment results to these social influences as well as ability, effort, luck and task difficulty attributions. There were significant differences for boys versus girls as well as for different ethnic groups.
In the final two years of the project, we tested additional items on the Survey of NCEA Goals to assess the influences of perceived support from teachers and others. Consequently, the motivation screening measure now reflects the values of collectivist as well as the more individualistic perspectives on motivation to enhance student achievement that have historically dominated the international literature.
Teacher Affiliation and Peer Affiliation Influences
Students in our focus group discussions commented both positively and negatively on the influences of their teachers on achievement and motivation:
Teachers do motivate us; they help us try to fix our mistakes. (Student)
I love my teachers; they encourage us to work and try our best and have fun. They push you—they may not be showing it but they are pushing you. (Student)
The teachers give us too much, don’t explain it well enough. They expect us to know. (Student)
The teachers are here to help us. They are supportive like a parent, push us to do well and really want us to pass. It’s good to know they are there for us—they are really caring about us totally as people, all aspects of our life. (Student)
[Teachers don’t always] tell us the honest truth—but rather say doing good without detail. (Parent)
We want them to give us a chance. (Student)
We examined statistically the relationships among Doing My Best, Doing Just Enough, and the Peer and Teacher Affiliation subscales. High Doing My Best scores were significantly positively correlated with Teacher Affiliation and Peer Affiliation scores. Conversely, students scoring high on Doing Just Enough were low on Teacher Affiliation but high on Peer Affiliation; these correlations were significant but small.
Teacher Affiliation and Peer Affiliation were not, however, related to achievement with one important exception: For Māori students, Teacher Affiliation was significantly correlated with the number of total credits achieved.
Attributions for Best and Worst Assessment Marks
Our research highlights how attributions relate to student achievement. Students rated four commonly cited attributions—ability, effort, task difficulty, and luck—and we also asked them to rate the influences of the teacher, family/whānau and friends on their best and worst marks in any subject. Attributions were strongly related to achievement outcomes, and there were interesting gender and ethnic patterns as well.
Across all students and both attributions for best and worst marks, the four most highly ranked attributions were effort, ability, assessment task difficulty, and teacher influences (see Figure 2).
Figure 2: Students’ attributions for their own best and worst marks in 2008 (range 1-4)
Gender had a significant influence on student attributions with girls more likely to attribute their best marks to effort and their worst marks to lack of ability and assessment task difficulty in comparison with boys. Girls were also more likely to attribute both best and worst marks to teacher influences than boys. Boys were more likely to attribute their worst marks to bad luck than girls.
Pacific students rated their family, teacher, luck and friends as the most important influences contributing to their best marks more so than Asian, Māori, and New Zealand European students. Pacific and Māori students were less likely to attribute their best and worst marks to ability, effort and task difficulty in comparison with Asian and European students. The largest difference across all influences was the difference in attribution for one’s best marks to family, with Pacific students significantly higher than European students.
Attributing one’s best marks to ability and particularly effort was significantly related to achieving more total Achievement Standard credits, more credits with Merit and Excellence, and fewer total Unit Standard credits. Attributing one’s best marks to good luck, family/whānau, and friends was associated with gaining more Unit Standard credits and fewer Achievement Standard credits as well as fewer credits with Merit and Excellence. Attributing one’s worst marks to ability, effort, task difficulty, and teacher influences was associated with attaining more Achievement Standard credits and fewer Unit Standard credits.
Attributing assessment results to Teacher influences significantly predicted total credits attained. These findings offer further evidence that student motivation and achievement are strongly related to how the student perceives teacher influences and his or her relationship with the teacher.
See also pages 73-100 of the full report, accessible from here for further information on the focus groups and on the key findings of the research.
Meyer, L.H., McClure, J., Walkey, F., McKenzie, L., & Weir, K. (2006). The impact of the NCEA on student motivation: Final Report to the Ministry of Education. Wellington: Ministry of Education and Victoria University.
Meyer, L.H., Weir, K., McClure, J., Walkey, F., & McKenzie, L. (2007). Longitudinal research on the relationship between the NCEA and student motivation and achievement: Final Report to the Ministry of Education. Wellington: Ministry of Education and Victoria University.
Meyer, L.H., McClure, J., Walkey, F., Weir, K.F., & McKenzie, L. (2009). Secondary student motivation orientations and standards-based achievement outcomes. British Journal of Educational Psychology, 79, 273-293.
Meyer, L., Weir, K.F., McClure, J., Walkey, F., & McKenzie, L. (2009).Longitudinal research on the NCEA and student motivation and achievement: A three-year follow-up. Final Report to the Ministry of Education. Wellington: Ministry of Education and Victoria University.
Ministry of Education (2007). New Zealand Curriculum. Wellington: Author.
For more publication-related information, please email the: Information Officer Mailbox