NMSSA 2017 Insights for Teachers: Science

Publication Details

This report is designed to support science teaching in primary classrooms. It draws on insights generated from the assessment of the science learning area carried out by the National Monitoring Study of Student Achievement (NMSSA) in 2017.

The report is in four parts. Part one introduces NMSSA and the NMSSA science assessment. Part two presents four practical insights into learning in science. Part three considers what we can learn from the NMSSA results about the literacy and numeracy skills students need to engage with their science learning. Part four discusses the relationship between students’ knowledge of science and the science capabilities they are developing.

Author(s): Educational Assessment Research Unit and New Zealand Council for Educational Research. Report for the Ministry of Education.

Date Published: May 2019

Summary

What is NMSSA?

NMSSA is designed to assess student achievement across the New Zealand Curriculum (NZC) at Year 4 and Year 8 in New Zealand English-medium state and state-integrated schools. The study is carried out over five-year cycles. Each year involves nationally representative samples of students from 100 schools at Year 4 and 100 schools at Year 8. The first cycle of NMSSA ran from 2012 to 2016. Science was assessed in 2012 and again in 2017.

The science learning area

The purpose statement for science in NZC says:

In science, students explore how both the natural physical world and science itself work so that they can participate as critical, informed, and responsible citizens in a society in which science plays a significant role.

To help teachers and students work towards this purpose, the achievement aims for the science learning area are organised into strands: the Nature of Science strand and the four contextual strands.

The Nature of Science (NOS) strand is about ‘what science is and how scientists work’. NOS is the overarching core strand and is required learning up to Year 10. There are four sub-strands to the NOS strand: Understanding about science; Investigating in science; Communicating in science; and Participating and contributing.

The contextual strands provide guidance about appropriate science knowledge to be developed. There are four contextual strands: Physical World; Material World; Living World; and Planet Earth and Beyond.

The NZC says that a school curriculum should integrate both the NOS and contextual strands.

The science capabilities were developed to supplement the description of science provided in NZC. They weave together the knowledge, skills, and practices that make up the NOS sub-strands with the four contextual strands to support students to develop the capabilities required to become critical, informed, responsible citizens.

There are five science capabilities. They are:

  • Gather and interpret data: making careful observations and differentiating between observation and inference
  • Use evidence: supporting ideas with evidence and looking for evidence that supports or throws into question other people’s explanations
  • Critique evidence: evaluating the trustworthiness of data
  • Interpret representations: thinking about how data is presented and asking questions about what it reveals
  • Engage with science: drawing on the other capabilities to engage with science in authentic contexts.

Table 1, shows the relationship between the five capabilities, the four NOS sub-strands and the four contextual strands.

Table 1: Relationship between science capabilities, the Nature of Science and the contextual strands

Figure 1: NMNSSA Science Capabilities & Context Strands

The NMSSA science assessment

NMSSA developed a framework based on the science capabilities for its science assessment. Each task in the NMSSA science assessment was designed to assess one or more of the capabilities.

The assessment was made up of two parts. The first part, which was completed by up to 25 students in each school, involved a group-administered paper-and-pencil assessment. Here the tasks included a mixture of selected response (such as multichoice) and short answer questions. Most tasks involved students interacting with stimulus material, such as different types of diagrams, science drawings, photos, or descriptions. The second part of the assessment involved ‘in-depth’ tasks, which were given to a subset of the students who took part in the group-administered assessment (up to eight students at each school).  Most of these in-depth tasks took the form of one-to-one interviews with a trained NMSSA teacher assessor. These tasks were usually open-ended. Students were often asked to use equipment to perform a task and prompted to explain their thinking and reasoning.

Figure 1: A description of the science capabilities scale

NMSSA Science Capabilities (SC) Scale

NMSSA science capabilities scale

NMSSA used students’ responses to both parts of the assessment to construct a measurement scale—the science capabilities (SC) scale. The scale allowed NMSSA to report the relative difficulty of the questions used in the assessment as well as each student’s overall level of achievement in science.

Figure 1 shows the SC scale. The descriptions show how students’ science capabilities increase in sophistication as the scale score increases, and outline the sorts of things students typically know and can do when they score at that part of the scale. For example, a student scoring about 120 units on the scale would typically be able to do the things described at that level. They would generally find the skills and knowledge described lower on the scale more straightforward. The descriptions further up the scale would typically involve knowledge they were less clear about or skills they were unable to demonstrate consistently.

How did students do on the science capabilities assessment?

Figure 2: The 2017 score distribution for Year 4 and Year 8 students on the Science Capabilities (SC) scale

Score Distribution for Year 4 and Year 8 students on the Science Capability (SC) scale

The 2017 study found that most students (94 percent) in Year 4 were achieving at or above curriculum expectations (developed level 1 and 2), while in Year 8 a minority (20 percent) were achieving at or above curriculum expectations (developed level 3 and 4).

The study also found that any changes in the overall average scores for Year 4 and Year 8 students between 2012 and 2017 were not statistically significant. However, there were statistically significant increases in average achievement scores for several population subgroups, including Year 4 Asian students, Year 8 girls, Year 8 Māori students, Year 8 Pasifika students, and Year 4 and Year 8 students in low decile schools.

Footnotes:

  1. Read more about the science capabilities here: http://scienceonline.tki.org.nz/Science-capabilities-for-citizenship/Introducing-five-science-capabilities
  2. Adapted from Junior Science: Thinking with Evidence https://www.nzcer.org.nz/tests/junior-science-thinking-evidence
  3. When NMSSA piloted a range of tasks it was found that more Year 4 students could answer when the focus was on three of the science capabilities: Gather and interpret data; Use evidence; and Interpret representations.     As a result, most of the tasks for Year 4 students in the NMSSA science assessment focussed on these capabilities. At Year 8, students were able to answer across the full range of the science capabilities.
  4. For more detailed information about the scale description go to the report, NMSSA Report 17: Science 2017– Key Findings.
  5. In the NZC, achievement objectives for science are the same for levels 1 and 2 and almost the same for levels 3 and 4. To differentiate between different levels of performance at levels 1 and 2, and levels 3 and 4, a curriculum alignment exercise undertaken in 2012 defined an ‘emerging’ and ‘developed’ expectation for the achievement objectives contained in each pair of levels.

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065