Executive Summary

Introduction

The National Monitoring Study of Student Achievement (NMSSA) – Wānangatia Te Putanga Tauira is designed to assess and understand student achievement across the curriculum at the primary level in New Zealand's English-medium state schools. The main purposes of NMSSA are to provide a snapshot of Year 4 and Year 8 student achievement and factors that are associated with achievement; to assess strengths and weaknesses across the curriculum and to monitor change over time.    NMSSA also has a specific focus on Māori and Pasifika students and students with special education needs.

NMSSA is a long-term project that commenced in 2012. In this first year of NMSSA it is possible to provide a baseline or snapshot of student achievement in two learning areas of the New Zealand Curriculum (NZC) – science and writing. Data in subsequent years will provide information about student achievement and strengths and weaknesses across the whole curriculum, including key competencies. It will also provide information about literacy and mathematics across the curriculum. In subsequent cycles, when NMSSA repeats its focus on each learning area, NMSSA will be able to report on any changes in achievement and monitor trends over a longer term. Thus, NMSSA is a national monitoring programme that will evolve and develop over time to assess and understand student achievement in New Zealand.

NMSSA follows on from the National Education Monitoring Project (NEMP) that was conducted between 1995 and 2010. NMSSA has built on and extended the design of NEMP to make use of more advanced psychometrics for reporting student achievement and exploring factors associated with that achievement. Thus, NMSSA is able to draw on findings from four cycles of NEMP assessments to retain continuity in monitoring national achievement and trends.

A focus on English: writing

Writing is creating meaning appropriate to the purpose and audience. According to the Literacy Learning Progressions, "Students use their writing to think about, record, and communicate experiences, ideas, and information" (p.6). The NZC presents a series of achievement objectives in the English learning area for each curriculum level that describe how students create meaning for themselves through speaking, writing and presenting. As students progress as writers they develop increasing levels of knowledge, skills and understandings related to creating and conveying meaning. They engage with tasks and texts that are increasingly sophisticated, and do this in increasing depth to meet the demands of their purpose for writing and their audience.

1. This report presents the findings about the achievement and attitudes of Year 4 and Year 8 students in English: writing and factors that are associated with that achievement. Hereafter, English: writing is referred to as writing. The components of the 2012 writing assessment programme include:
2. Writing for a Variety of Purposes is based on the e-asTTle framework and is a measure of students' writing over seven elements: ideas, structure and language features, paragraphing, vocabulary, sentence structure, punctuation and spelling. This was a paper-and-pencil assessment completed by approximately 2000 students at each year level.
3. Process of Writing is a measure of students' understanding of how to create, shape and refine text. This comprised seven elements: audience awareness, planning, crafting/ writing, revising and editing, proofreading, feedback and publishing. This was a series of individual assessments using one-to-one interviews and performance activities completed by approximately 700 students at each year level.
4. Student attitudes and learning opportunities in writing, including a measure of their self-efficacy and engagement with writing, were obtained via a questionnaire.
5. Teacher perspectives on writing teaching and learning in the school, including their confidence as writing educators and professional support for teaching writing, were obtained via a questionnaire.

Several of the writing measures including both achievement measures were developed using Item Response Theory to report on a scale common to Year 4 and Year 8 students. This allowed comparisons to be made between the two year levels.

The report also describes the achievement of subgroups of students (by gender, ethnicity, school decile and type of school) and the achievement of the key population groups (Māori, Pasifika and students with special education needs) and student and teacher perspectives on the learning and teaching of writing.

Key findings from the report

National student achievement

The Writing for a Variety of Purposes scale is divided into four broad bands, each describing the qualities (ideas, structure and language features, paragraphing, vocabulary, sentence structure, punctuation and spelling) of student writing associated with that part of the scale. The Process of Writing scale is divided into two broad bands describing the progression in seven different elements involved in the assessment (audience awareness, planning, crafting/writing, revising and editing, proofreading, feedback and publishing). The descriptors provide an indication of the progression of writing knowledge and competencies found between Year 4 and Year 8.

An alignment process was used to link the Writing for a Variety of Purposes scale to the descriptions of writing competency described by the Literacy Learning Progressions of the NZC. The process took advantage of the link that already existed between the e-asTTle writing scale and the Literacy Learning Progressions. The exercise allowed performance on the Writing for a Variety of Purposes assessment to be reported in terms of curriculum expectations at different year levels.

In this report, English Curriculum Level 2 is used as the expected level for students at the end of Year 4, and Level 4 as the expected curriculum level for students at the end of Year 8.

• Year 4 students' writing scores ranged across curriculum Levels 1 to 3 with the greatest proportion scoring in Level 2. Year 8 students' writing scores ranged across curriculum Levels 2 to 4 with the greatest proportion achieving in Level 3. The Year 4 result is in line with end of year NZC expectations, while the Year 8 result is below NZC expectations.
• The results show that Year 4 students' writing typically involved writing simple ideas using brief but coherent text, with language features appropriate to the purpose. Their writing showed they were usually successful with beginning and end of sentence punctuation, and their sentences often showed variety in structure. They used correct spelling for a range of personal and high frequency words, and vocabulary sometimes included a few precise words.
• Year 8 students' writing typically included ideas that show some complexity and elaboration. Text was generally coherent with basic paragraphing, and ideas that flowed. The structural and language features were appropriate to purpose and showed some development and control. They typically used a variety of precise vocabulary that added information and enhanced meaning. They wrote sentences that had correct beginning and end punctuation and some correct usage of other punctuation. Sentence structures showed variety, extension and a sense of control. A wide range of high frequency words were spelled correctly, and approximations were made for difficult words.   Of the five different writing purposes (prompts) used (to explain, persuade, describe, recount and narrate), Year 4 students did least well, on average, at writing a persuasive piece and best, on average, at writing a recount. Year 8 students did least well, on average, when writing an explanation or a descriptive piece than when writing for the other three prompts.

• Of the seven elements in Writing for a Variety of Purposes (ideas, structure and language, organisation, vocabulary, sentence structure, punctuation and spelling), students at both year levels scored, on average, the highest on spelling and the lowest on punctuation. The second strongest element for Year 4 was ideas, while for Year 8 it was sentence structure.
• Year 4 and Year 8 students did not differ markedly, on average, in their understanding of the process of writing. About half of Year 4 students and most Year 8 students were typically able to talk about the audience for their writing, and discuss a pre-writing/planning phase for writing. They were able to talk about crafting of writing, describing at least one aspect such as making choices about ideas, structure of language features, etc. These students generally acknowledged that proofreading is part of the writing process, and were able to talk about making changes to their writing although they were unlikely to articulate why changes should be made. They typically did not mention feedback as a part of the writing process, nor acknowledge publishing as part of the writing process. The lower scoring 50 percent of Year 4 students had not yet reached the level of competence with Process of Writing described here. A very few Year 8 students (5 percent) demonstrated more developed levels of knowledge and skill on the Process of Writing scale.
• As expected, Year 8 students achieved higher scores, on average, than Year 4 students with an average annual effect size of 0.36 on Writing for a Variety of Purposes. This level of growth is towards the higher end of the range found for other curriculum areas (Hattie, 2009)¹ and is significantly higher than that for Process of Writing (0.18). It is also significantly higher than that for NMSSA Science achievement².
• Progress from Year 4 to Year 8 is very similar for all subgroups (e.g. boys and girls, ethnicity, decile and types of school).
• There was a wide distribution of scores at both year levels and some overlap in the achievement of Year 4 students and Year 8 students, particularly in the Process of Writing.
• Results showed that, on average, achievement varied by   gender, ethnicity and school decile. For both year levels and both measures of writing, average achievement was higher for girls than boys, lower for Māori and Pasifika students than for non-Māori and non-Pasifika students respectively, and was lower for students from lower decile schools. Findings reported by NEMP in 1998, 2002 and 2006 indicate ongoing disparities between subgroups over this period. NMSSA indicates that the differences continue and are statistically significant.
• There is a complex relationship between ethnicity, school decile and writing achievement. Both factors are significantly associated with results for Writing for a Variety of Purposes, although not for Process of Writing. However, when controlling for decile, the differences between the average scores of NZ European, Māori and Pasifika students disappear for students from decile 1 schools (there were insufficient Pasifika and Māori sample sizes for comparisons at the other decile levels). This contrasts with the finding for NMSSA Science (2012) where differences between ethnicity groups persisted for students from decile 1 schools.

• Feedback about their writing appeared to be a relatively common practice involving teachers and peers. Year 8 students had a greater understanding of the relationship between feedback and improving their writing. Feedback given by students or received from teachers focused primarily on the deep and surface features of writing³.

Factors associated with achievement

A number of factors associated with achievement were examined. These included a measure of student attitude to writing, the amount of English spoken at home, writing learning opportunities at school reported by students and teachers, teachers' confidence as writing educators, and the level of professional and curriculum support provided within school and by professional learning and development (PLD) programmes.

• Overall, Year 4 students were more positive about writing than Year 8 students. Girls were generally more positive than boys at both year levels, and the difference was similar at both year levels. These findings are consistent with those from NEMP since 1998 to 2006.
• Pasifika students scored higher, on average, on the Attitude to Writing scale than European and Māori students at both Year 4 and Year 8 which is also consistent with NEMP findings.
• Attitude to Writing was weakly related to Writing for a Variety of Purposes particularly for students with low Attitude to Writing scores and was more strongly related at Year 8 than Year 4. There was no relationship between Attitude to Writing and Process of Writing.
• Year 4 students reported more frequent involvement in a range of different writing experiences than Year 8 students. The most frequent activities at both year levels were teacher-led; sharing their writing with the teacher, and writing about something their teacher had asked them to write about. A sizable proportion of students in both year levels reported infrequent involvement in many of the experiences.
• Teachers were very positive about writing and their confidence as teachers of writing. This supports the findings by the Education Review Office (2007) that 87 percent of teachers were effective teachers of writing in some or all six quality of teaching indicators⁴.
• Teachers of Year 4 students reported the use of remedial activities outside the classroom more often than those who taught Year 8. Year 8 teachers were more likely to report the use of extension activities outside of the classroom.
• Most teachers reported that they were regularly involved (once a term or more) in a range of professional interactions that supported their teaching of writing. This included working together to plan and prepare, discussing useful approaches to teach writing, and discussing samples of students' work.
• Over 80 percent of Year 4 teachers and 75 percent of Year 8 teachers reported that they were involved in professional development and learning focused on writing in the last 12 months. This is substantially higher than the level of science PLD reported by teachers in NMSSA Science (2012).

Achievement of Māori students

Students could identify with up to three ethnic groups. All students who identified as Māori were included in the Māori analyses. The national sample at Year 4 included 423 Māori students and at Year 8, 353 Māori students. We compare Māori student subgroups to all students in the national sample. When making these comparisons the national sample is referred to as 'All Students'.

• Year 4 and Year 8 Māori students tended to achieve at a slightly lower level than NZ European students (Chapter 3) and some features of Māori student achievement followed similar patterns to the national samples.
• Between the year levels, as expected, Year 8 Māori students, on average, achieved higher scores than Year 4 Māori students. However, there was a wide distribution of scores at both year levels and considerable overlap in the achievement of Year 4 and Year 8 Māori students. The overall difference in average scores between Year 4 and Year 8 students was greater for Writing for a Variety of Purposes than for Process of Writing as was the case for the national sample.
• For Writing for a Variety of Purposes, the average of Year 4 Māori students was just within Curriculum Level 2. As performance at Level 2 is the expectation described in the NZC, roughly half of Year 4 Māori students might be viewed as achieving at or above curriculum expectations, with the other half below such expectations. In Year 8, the average score was within the upper portion of Level 3. As was also the case with All Students, this was below the expectations outlined in the NZC (Level 4).
• At both year levels, average scores of Māori students on Writing for a Variety of Purposes differed by school decile and gender. This was the measure that assessed a range of writing skills using a piece of the student's work. On average, Māori girls scored higher than boys and Māori students at high decile schools scored higher than those from low decile schools. In both cases the difference between these groups was similar at both year levels. Differences by school type were not notable at either year level.
• Gender and decile differences were also observed at Year 4 on Process of Writing, the measure that assessed student's awareness and understanding of a range of processes involved in writing. These differences were not significant at Year 8.
• The percentages of Year 4 and Year 8 Māori students who achieved above the respective national averages were lower than for All Students.

• Approximately 40 percent of Māori students at Year 4 and Year 8 scored above national averages for both writing measures. At Year 4 the above average group included more girls than boys and came evenly from across the full range of school deciles. At Year 8 the same pattern held for gender, but a majority of students came from mid decile schools.
• Just over 80 percent of all Māori students attended low and mid decile schools. This contrasts with just over 50 percent of NZ European students attending low or mid decile schools. When these figures are accounted for, they show that, as for All Students, a higher proportion of Māori students attending high decile schools scored above the benchmark than from mid or low decile schools. However, at Year 8 this difference is less pronounced.

Achievement of Pasifika students

All students who identified as Pasifika were included in the Pasifika analyses. The national sample at Year 4 included 262 Pasifika students and at Year 8, 206 Pasifika students. We compare Pasifika student subgroups to all students in the national sample. When making these comparisons the national sample is referred to as 'All Students'.

• On average, Pasifika students scored lower than All Students at both year levels. However, the difference between Year 4 and Year 8 Pasifika scores was similar to that for All Students, and notably so for the Writing for a Variety of Purposes measure. This suggests that Pasifika students showed a similar rate of progress to All Students.
• Year 8 Pasifika students achieved higher scores, on average, than Year 4 Pasifika students. However, there was a wide distribution of scores at both year levels and overlap in the achievement of Year 4 students and Year 8 students.
• Writing achievement varied at both year levels for Pasifika students depending on the amount of English spoken at home. Students who spoke English at home 'always' or 'often' tended to achieve at a higher level than those who spoke English at home 'sometimes' or 'never'.
• There was little difference in average scores for Pasifika students at Year 4 with respect to the type of school they were attending. Year 8 average scores, however, were higher for those Pasifika students attending full primary schools than for Pasifika students in intermediate schools.
• For Writing for a Variety of Purposes, a little over 50 percent of Year 4 Pasifika students achieved in Level 2 of the NZC or above, compared to 65 percent of All Students. Performance by Pasifika students was, on average, in line with expectations outlined in the NZC. A third of Year 8 Pasifika students achieved within Levels 4 and 5, similar to the All Students group. This was below the expectations outlined in the New Zealand Curriculum.
• The percentages of Year 4 and Year 8 Pasifika students who achieved above the national averages were lower than for All Students (51 and 53 percent respectively).
• While 37 percent of Pasifika students at Year 4 scored above the national average, a greater percentage of Pasifika students at Year 8 scored above the national average (48 percent) –almost at the same level as All Students. About 70 percent of these students were girls, a greater percentage than in the All Students group.
• More than 85 percent of all Pasifika students at both year levels attended low and mid decile schools. This contrasts with just over 50 percent of NZ European students attending low or mid decile schools. When this is accounted for, results show that a greater proportion of Pasifika students at high decile schools scored above the national benchmark. This reflects the same relationship between achievement and school decile that was found for All Students.

Achievement of students with special education needs

For the first time, students with special education needs were identified in national monitoring. This represents a major step forward in the inclusion of children with special education needs in reporting national level assessment.

Overall, the numbers of students reported on in Chapter 7 are relatively small and the findings should therefore be interpreted with caution. This is particularly true with regard to the high special education needs group from which many of the special education needs student withdrawals are likely to have come. As such, this group cannot be considered a statistically representative sample.

Participating schools were asked to identify students who had special education needs as:

• High special education needs: For example, ORS funded, Supplementary Learning Support, severe behaviour or communication assistance from Special Education
• Moderate special education needs: For example, provided with a teacher aide from school funds, on the case load for Resource Teachers: Learning and Behaviour (RTLB), or Child Youth and Family Services (CYFS)
• On referral: For example, to Special Education or CYFS with action pending.

Students not falling into any of the above categories were assigned to a no special education needs group.

Although the number of students with high special education needs was very small, students with moderate special education needs made up 8 percent of All Students at Year 4 and 5 percent at Year 8.

• On average, Year 8 students with special education needs scored higher than Year 4 students with special education needs. As with All Students, there was some overlap in the achievement of Year 4 and Year 8 students with special education needs.
• At Year 4 all students with high special education needs, and nearly three quarters of students with moderate special education needs achieved within curriculum Level 1. The remainder achieved at Level 2 or 3. Students identified as being on referral performed in very similar ways to the national sample.
• At Year 8, over one third of students with high special education needs achieved within curriculum Level 2 or 3. Just over one third of students with moderate special education needs achieved within curriculum Level 3 or 4. In contrast, about one third of students in the on referral and the no special education needs groups achieved at Level 4 or above.
• At both year levels, students with high special education needs or moderate special education needs achieved, on average, at a lower level than those with no special education needs or on referral. However, there was a wide range of achievement among the students with moderate special education needs and their scores overlapped with the students with no special education needs or those students on referral. The difference between the average scale scores of students with moderate special education needs and no special education needs groups was about 1.0 scale score units.

• The difference in scores between Year 4 and Year 8 (progress) for students with moderate special education needs and for students on referral was about 25 scale score points (an effect size of about 1.3). This difference is equivalent to that observed for All Students (Chapter 3).
• On average, Attitude to Writing scores were similar across all groups of students within each year level. As with the All Students group, the average Attitude to Writing score declined from Year 4 to Year 8. The no special education needs group showed a smaller decline in average Attitude to Writing between Year 4 and Year 8 than the moderate special education needs group.
• Students with special education needs reported having a similar range of opportunities to learn to write in school as the students with no special education needs. Students with high special education needs reported writing using a computer more often than other groups of students. Year 4 teachers reported using specialist advice to adapt the curriculum for learners with special needs more often than Year 8 teachers did.

• Fifteen percent of Year 4 students and 17 percent of Year 8 students with moderate special education needs scored above their respective national averages.
• Over half of the students on referral (55 percent at Year 4 and 58 percent at Year 8) scored above the benchmark at each year level. This was slightly higher than for All Students.

Footnotes

1. Hattie, J. (2009) Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, London & New York: Routledge, Taylor & Francis.
2. National Monitoring Study of Student Achievement, Science 2012, Educational Assessment Research Unit, Otago
3. Deep features of writing include ideas/content, structural and language features, organisation, vocabulary, audience awareness/engagement. Surface features of writing include spelling, punctuation, grammar, neatness.
4. The Teaching of Writing: Good Practice in Years 4 and 8 (July-2007)