Executive Summary

Introduction

The National Monitoring Study of Student Achievement (NMSSA) – Wānangatia Te Putanga Tauira is designed to assess and understand student achievement across the curriculum at the primary level in New Zealand's English-medium state schools.

The main purposes of NMSSA are to provide a snapshot of Year 4 and Year 8 student achievement and factors that are associated with achievement; to assess strengths and weaknesses across the curriculum; and to monitor change over time. NMSSA also has a specific focus on Māori and Pasifika students and students with special education needs.

NMSSA is a long-term project that commenced in 2012. In this first year of NMSSA it is possible to provide a baseline or snapshot of student achievement in two learning areas of the New Zealand Curriculum (NZC) – science and writing. Data in subsequent years will provide information about student achievement and strengths and weaknesses across the whole curriculum including key competencies. It will also provide information about literacy and mathematics across the curriculum. In subsequent cycles, when NMSSA repeats its focus on each learning area, NMSSA will be able to report on any changes in achievement and monitor trends over a longer term. Thus, NMSSA is a national monitoring programme that will evolve and develop over time to assess and understand student achievement in New Zealand.

NMSSA follows on from the National Education Monitoring Project (NEMP) that was conducted between 1995 and 2010. NMSSA has built on and extended the design of NEMP to make use of more advanced psychometrics for reporting student achievement and exploring factors associated with that achievement. Thus, NMSSA is able to draw on findings from four cycles of NEMP assessments to retain continuity in monitoring national achievement and trends.

A focus on science

Science in the NZC is about exploring how the natural world, the physical world and science itself works so that students can participate as critical, informed and responsible citizens in a society in which science plays a significant role. The NZC provides a framework for schools to develop their own science curriculum. It does not prescribe what should be taught, except at a high level. The Nature of Science is the core strand in the NZC science curriculum, and is explored through four contextual strands - Living World, Planet Earth and Beyond, Physical World and Material World. Unlike other learning areas in the NZC, the achievement objectives for Levels 1 and 2 are the same, and Levels 3 and 4 are almost the same.

This report presents the findings about the achievement and attitudes of Year 4 and Year 8 students in science and factors that are associated with that achievement. The components of the 2012 science assessment programme include:

1. Knowledge and Communication of Science Ideas – a measure of students' knowledge, understanding and communication of ideas across the four content strands of the science curriculum. This was a paper-and-pencil assessment completed by approximately 2000 students at each year level.
2. Nature of Science – a measure of students' understanding of scientific thinking as applied to the content and competencies specified in the Nature of Science themes in the NZC. This measure was derived from a series of individual assessments using one-to-one interviews and performance activities completed by approximately 700 students at each year level.
3. Student attitudes and learning opportunities in science including a measure of their self-efficacy and engagement with science.
4. Teacher perspectives on science teaching and learning in the school including their confidence as science educators and professional support for teaching science.

Several of the science measures including both achievement measures were developed using Item Response Theory to report on a scale common to Year 4 and Year 8 students. This allowed comparisons to be made between the two year levels. The report also describes the achievement of subgroups of students (by gender, ethnicity, school decile and type of school), the achievement of the key population groups (Māori, Pasifika and students with special education needs).

Key findings from the report

National student achievement

For each science achievement scale a set of descriptors was developed that described the knowledge and competencies associated with three broad bands in the scale. The descriptors provide an indication of the progression of science knowledge and competencies found between Year 4 and Year 8. These descriptors provide valuable information not previously available about how students may be expected to progress through the science curriculum. As such, they are likely to be a valuable resource for the sector.

The NMSSA science achievement scores were also aligned with the science curriculum by a panel of New Zealand science education experts. The panel identified a series of cut-off scores on the Knowledge and Communication of Science Ideas scale that defined a series of boundaries (cut-off scores) where one curriculum level progressed into the next. The panel was able to make confident distinctions between scores associated with Emerging (entry) Curriculum Level 1 and 2 and Developed (advanced) Curriculum Level 1 and 2. A similar distinction was made between score levels associated with Emerging Level 3 and 4, and Developed Level 3 and 4.

• For both science measures, the average achievement of Year 4 students was within the Developed Curriculum Level 1 and 2 band, while for Year 8 students the average achievement was within the Emerging Curriculum Level 3 and 4 band. The average results for Year 4 students aligned with the expected level described in the NZC, while the average Year 8 results did not reach the expected curriculum level (Developed Curriculum Level 3 and 4).
• The results show that the middle 50 percent of Year 4 students drew on everyday experiences and observations rather than specific science knowledge to answer questions, were beginning to develop scientific vocabulary and recognise how scientists find things out. They knew how to carry out scientific investigations and could offer their own explanations for the outcomes observed.
• The middle 50 percent of Year 8 students were able to use basic knowledge of more abstract science, notice simple patterns in data and make basic inferences from these. They demonstrated a developing understanding of scientific thinking, process and vocabulary.
• As expected, Year 8 students achieved higher scores, on average, than Year 4 students with an average annual effect size of about 0.30. This level of growth is similar to that found for other curriculum areas (Hattie, 2009)¹.

• The progress between Year 4 and Year 8 was similar for all but one set of subgroups (e.g. boys and girls; types of school). Students in high decile schools showed greater progress than those in low decile schools.
• However, there was a wide distribution of scores at both year levels and some overlap in the achievement of Year 4 students and Year 8 students.
• The results showed that, on average, achievement varied by ethnicity and school decile. For both year levels and both measures of science, average scores were lower for Māori and Pasifika students than for non-Māori and non-Pasifika students respectively. Average scores were also lower for students from lower decile schools. Achievement in science was similar for boys and girls.
• The disparity between school decile and ethnicity subgroups found in NEMP from1999 to 2007 continued to be present in 2012. The decreasing disparity between boys and girls found in NEMP over the same period has reversed at Year 4 with there being a greater discrepancy between boys and girls than in 2007, but a decline in the disparity at Year 8.
• Apart from absence of gender difference at Year 8 the pattern of results for these subgroups is generally consistent with the TIMSS 2010/2011 Year 9 science results.

Factors associated with achievement

A number of factors associated with achievement were examined. These included a measure of student attitude to science, the amount of English spoken at home, science learning opportunities at school reported by students and teachers, teachers' confidence as science educators, and the level of professional and curriculum support provided within school and by professional learning and development (PLD) programmes.

• Overall, students at Year 4 reported a more positive attitude to science than at Year 8, which is consistent with the findings from TIMSS and has been a persistent finding from NEMP since 1995.
• Attitude to science was related to achievement particularly for students with low Attitude to Science scores and the relationship between attitudes to science and science achievement was stronger at Year 8 than Year 4.

• Students who always spoke English at home were more likely to achieve at a higher level at both Year 4 and Year 8 than students who spoke English sometimes or never. This difference held for both group-administered and individual assessments.
• There appear to be very few opportunities for hands-on science activities such as doing science experiments or using specialist science equipment in school. Year 4 and Year 8 students reported that they most frequently accessed science information by listening to their teachers, followed by independently accessing information or using information from their family and whānau.
• Most teachers who responded to the questionnaire reported that they were responsible for teaching science to their class, although at Year 8 about a third of the teachers who responded were specialist science teachers. Although the majority of teachers at both Year 4 and Year 8 liked teaching science, smaller proportions of teachers at both year levels felt happy about their teaching or confident in their ability to teach science, particularly to a diverse group of students. This lack of confidence mirrored reports of somewhat low levels of professional support within schools and limited access to targeted professional development. This finding reflects that reported in TIMSS, and presents a less positive picture of professional confidence and support than was found for NMSSA writing in 2012.²
• These findings are generally consistent with those of the Education Review Office evaluation of science teaching and learning in Years 5 to 8 (2012)³ and would support ERO's recommendations that the MoE investigates opportunities for support and ongoing professional learning and development for teachers, and that schools give priority to science teaching and learning in the curriculum, and to the quality of science teaching and learning.

Achievement of Māori and Pasifika students

Students could identify with up to three ethnic groups. All students who identified as Māori were included in the Māori analyses, and all students who identified as Pasifika were included in the Pasifika analyses. The Year 4 national sample included 423 Māori students and 262 Pasifika students. The Year 8 national sample included 353 Māori students and 206 Pasifika students. We compare Māori and Pasifika student subgroups to all students in the national sample. When making these comparisons the national sample is referred to as 'All Students'.

• Māori and Pasifika students were positive about how their culture, language and identity were valued at their school and were positive in their attitudes to learning science.
• Year 4 and Year 8 Māori and Pasifika students, on average, achieved at a lower level than NZ European students although the average annual growth between Year 4 and Year 8 was similar to that for NZ European students (Chapter 3).
• Between the year levels, as expected, Year 8 Māori and Pasifika students, on average, achieved higher scores than Year 4 Māori and Pasifika students respectively. However, there was a wide distribution of scores at both year levels and some overlap in the achievement of Year 4 students and Year 8 students.
• For both science measures, the average achievement of Year 4 Māori and Pasifika students was within the Developed Curriculum Level 1 and 2. This aligns with the level expectations described in The New Zealand Curriculum.
• However, the Year 8 average score for Māori and Pasifika students is below the expectations of Developed Level 3 and 4 described in The New Zealand Curriculum, and below that for All Students (although both were in Emerging Level 3 and 4).
• For both year levels and both measures of science, achievement, on average, was lower for Māori and Pasifika students from lower decile schools. The achievement in science was similar for boys and girls, and for Year 4 Māori and Pasifika students attending full primary and contributing schools. Achievement by school type showed less consistency at Year 8.
• The percentages of Year 4 and Year 8 Māori and Pasifika students who achieved above the national averages were lower than for All Students. In all groups fewer students achieved above the national average at Year 8 than at Year 4.

• At Year 4, 43 percent of Māori students and 23 percent of Pasifika students scored above the national average. At Year 8 the percentages were lower at 30 percent of Māori students and 19 percent of Pasifika students.
• Māori boys and girls at both year levels and Year 8 Pasifika boys and girls were equally represented in the above groups but a greater proportion of Pasifika girls than boys achieved above the national average at Year 4.
• Māori and Pasifika students attending high decile schools were more likely to score above average on the science measures than Māori and Pasifika students in middle or low decile schools. This reflects the relationship between achievement and school decile that was found for All Students. Just over 80 percent of all Māori students and almost 90 percent of Pasifika students attended low and mid decile schools. In contrast just over 50 percent of NZ European students attended low or mid decile schools.
• Achievement varied at both year levels for Pasifika students depending on the amount of English spoken at home. Students who spoke English more frequently at home achieved at a higher level although this was not consistent across all categories.

Achievement of students with special education needs

For the first time, students with special education needs were identified in national monitoring. This represents a major step forward in the inclusion of children with special education needs in reporting national level assessment.

Participating schools were asked to identify students who had special education needs as:

High special education needs: For example, ORS funded, Supplementary Learning Support, severe behaviour or communication assistance from Special Education

Moderate special education needs: For example, provided with a teacher aide from school funds, on the case load for Resource Teachers: Learning and Behaviour (RTLB), or Child Youth and Family Services (CYFS)

On referral: For example, to Special Education or CYFS with action pending.

Students not falling into any of the above categories were assigned to a 'no special education needs' group.

Although the numbers of students with high special education needs were very small, students with moderate special education needs made up 8 percent of All Students at Year 4 and 5 percent at Year 8.

Overall, the numbers in Chapter 7 are relatively small and the findings should therefore be interpreted with caution. This is particularly true with regard to the high special education needs group from which many of the special education needs student withdrawals are likely to have come. As such, this group cannot be considered a statistically representative sample.

• On average, Year 8 students with special education needs scored higher than Year 4 students with special education needs. As with All Students, there was some overlap in the achievement of Year 4 and Year 8 students with special education needs.
• At both year levels, students with high or moderate special education needs achieved at a lower level, on average, than those on referral or with no special education needs. However, the overlap between the groups indicated that there were students, particularly those with moderate special education needs, who were achieving at the same level as students with no special education needs. Students identified as being on referral performed in very similar ways to All Students.
• Students with moderate special education needs demonstrated a similar difference in average achievement between Year 4 and Year 8 as students with no special education needs. These results suggest that on average, students with special education needs are progressing from Year 4 to Year 8 at a similar rate to those with no special education needs.
• At Year 4 the average score for students with high special education needs was within Emerging Levels 1 and 2 of the curriculum. For students with moderate special education needs the average was just within Developed Level 1 and 2. At Year 8, the average score for both high and moderate special education needs groups was at the top end of Developed Level 1 and 2. About 30 percent of students with high or moderate special education needs were achieving at least at Level 3 and 4.
• At both year levels, students with moderate and high special education needs demonstrated as favourable an attitude to science as their peers in the on referral and no special education needs groups. Similarly to the national sample, attitude to science declined slightly between Year 4 and Year 8 for students with special education needs.
• At both year levels, 17 percent of students with moderate special education needs and about 50 percent of students on referral achieved above the national averages. There was a greater percentage of boys than girls in the special education group compared with the All Students group.
• Students with special needs who achieved above the national average tended to come from mid and high decile schools as was the case with All Students.

Footnotes

1. Hattie, J. (2009) Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, London & New York: Routledge, Taylor& Francis NMSSA, SCIENCE 2012 Executive Summary 9
2. National Monitoring Study of Student Achievement, Writing 2012, Educational Assessment Research Unit, Otago University and the New Zealand Council for Educational Research
3. Science in the New Zealand Curriculum: Years 5 to 8 (May 2012) Education Review Office website.