A focus on science achievement and engagement Publications
This paper summarises data on engagement and achievement in science across primary and secondary schools. This information has been collated across national and international studies undertaken over the last 15 years. This information raises key issues for policy and practice and these are discussed briefly.
Author(s): Research Division
Date Published: April 2009
The Ministry has put in place a number of mechanisms for monitoring student performance in science at the system level in primary and secondary education. The following information about science learning is drawn from system-level monitoring and is based on evidence from a range of national and international surveys that have been carried out during the last 15 years, starting with the first Trends in International Mathematics and Science Study (TIMSS) in 1994.
We are interested to see what the latest research results on science achievement might mean for policy, practice, and potential research areas. While overall the changes in the data on student achievement in science do not give cause for immediate alarm, some small downward movements as well as shifts in related measures (e.g. student interest and time spent in science) are starting to be observed. The meaning and potential impact on science achievement of such shifts and the extent to which they are likely to continue has still to be fully explored.
In this report, we aim to summarise key issues for policy and practice raised by data on engagement and achievement in science across primary and secondary schools. This information has been collated across national and international studies undertaken over the last 15 years. The most recent information for primary schooling comes from the National Education Monitoring Project (NEMP) in 2007 and from TIMSS 2006/07 (Year 5). At the secondary level, information comes from the Programme for International Student Assessment (PISA) in 2006, NCEA results, and TIMSS 2002/03 (Year 9). In addition there is information from the Competent Learners Study, the Transitions Study, and from a national survey of primary schools conducted by NZCER in 2007.
The collated evidence is being used as a basis for engaging with the Strategy and Implementation Board and the Leadership Team; as a means of collaborating with communities of interest in policy, research and practice; and as a broad context that will help develop key messages for the release of New Zealand data internationally in December 2008 from the implementation of TIMSS in 2006/07.
- It is important to remember that performance in science can reflect learning from within school, in the family and whānau, and in the broader community.
- As this summary draws on a range of evidence, it is important to bear in mind the different science assessment frameworks on which studies are based. These frameworks determine what aspects of science or scientific literacy are measured and the way they are measured. For example, PISA focuses more on the application of science knowledge and less on the content of science, whereas studies with younger students tend to emphasise both content and application. All studies use pencil-and-paper tests with NEMP also using performance-based tasks.
- Along with an overall science score, the international studies examine the branches of science – physics, chemistry, biology, and earth science – as well as the cognitive skills used to answer questions.
- The term engagement is used to mean different things in the current literature and in conversations within the Ministry. Engagement in this paper relates to students' attitudes towards science and towards school, and their general experience in those situations. Presence in science is measured in these studies by aspects such as teaching time devoted to science, the importance of science within the whole curriculum, and approaches to teaching science.
- Indices are often used in these studies to summarise students' responses to background questions. For example, where students are asked about things such as their interest in science, it is often measured by responses to a range of questions and then summarised as a single index of "interest".
- The intention of this paper is to examine the general picture of science gathered from a range of studies. Therefore the specific details such as numbers and proportions will not be presented. Instead, the appendices contain the details for your reference. In particular, Appendix B is numbered for cross-referencing with the paragraphs below and contains specific information related to each paragraph.
What does achievement in science look like at primary and secondary level?
When New Zealand student achievement in science and scientific literacy is examined relative to other countries, our students do better at the secondary level than at the primary level. At the primary level, New Zealand students overall achieve around the international mean – a level that has been maintained since the mid 1990s.1 At the secondary level, students overall achieve well above the international mean both in science (TIMSS) and scientific literacy (PISA). Given the number of countries participating in the international studies, it is more meaningful to compare New Zealand to English-speaking countries, which include Singapore, England, the United States, Australia, and Scotland. At the primary level, New Zealand students perform significantly lower than all English-speaking countries except Scotland. At the Year 9 level, Singapore and England significantly outperform New Zealand students, but the United States, Australia, and Scotland are about the same. In scientific literacy at the 15-year-old level, no English-speaking countries outperform New Zealand and only Australia performs similarly, although Singapore does not participate in this assessment.
The spread of achievement varies across the studies at the primary and secondary levels. At the primary level, New Zealand has one of the widest ranges of scores between the lowest- and highest-performing students. At the secondary level, for PISA, the range is wide only when compared to other high-performing countries, while in TIMSS at the Year 9 level, the range is comparable to many other countries and narrower than some of the high-performing countries.
In both primary and secondary schooling, however, there are relatively high proportions of New Zealand students in the top science proficiency levels or benchmarks compared with high performing countries, and this is particularly apparent at the secondary level. At both primary and secondary levels, New Zealand also has a relatively high proportion of students at the lowest levels of science proficiency or benchmarks. At the primary level this has not changed over time, but at the secondary level in TIMSS improvements in the proportions of students in the lower benchmarks were observed between 1994 and 2002. The relatively higher proportions of students not reaching the lower benchmarks is a cause for concern as these students will struggle to complete even the most basic science tasks.
As mentioned earlier, science is comprised of physics, chemistry, biology, and earth science. At the primary level, Year 5 students achieved better at earth science questions and worse at physical science questions (which includes aspects of chemistry and physics). At secondary school, chemistry is generally an area of weakness relative to other areas (TIMSS Y9). NCEA results show that Year 11 general science is achieved by around 60 per cent of those that sit it. At Years 12 and 13, biology is the most popular subject and physics least. In terms of the cognitive skills students use when they answer questions, Year 5 students achieved better in questions requiring them to apply their scientific knowledge, compared to questions that required reasoning or reproducing knowledge.
For all of the studies, science achievement varies more among students within schools than it does between schools. This reinforces the notion that there is a diverse range of student ability within schools, emphasising that students start school with different knowledge and skills, and learn at different paces. However, there is also a clear link between school decile (as an indicator of the extent to which schools draw students from low socio-economic communities) and science achievement, with students in higher decile schools having higher achievement, on average, than those in lower decile schools.
In all surveys there are high performers and low performers in all ethnic groupings. Proportionally more Pākehā-European and Asian students are among the high performers and fewer among the lower performers. Māori and Pasifika students are over-represented among the proportions of lower performing students. While the proportions of lower achievers are smaller for Pākehā-European, the large size of the grouping, in terms of numbers, means there are as many, or more, Pākehā-European in this low-performing group as there are Māori or Pasifika students.
There is consistent data across studies at both the primary and secondary level that students with a high use of English (the language of instruction and assessment) in the home tend to out-perform those students with a low use of English in the home.
There is no overall gender difference at either the primary or secondary schooling levels. However, across some of the studies there does tend to be a higher proportion of boys who did not reach the lower benchmarks (TIMSS Year 5 but not Year 9) or lower levels of proficiency (PISA). In addition, boys and girls demonstrated their relative strengths and weaknesses in the content and cognitive areas. As seen in the TIMSS 2006/07 results, Year 5 boys tend to be stronger at earth science and girls at life science and they are about the same at physical science (physics and chemistry). At the secondary level, boys, on average, outperform girls on literacy measures involving earth and space systems and physical systems while girls and boys are about the same on living systems (PISA 06).
As mentioned earlier, the cognitive skills required to answer questions are examined in the studies. At the primary level, boys and girls perform about the same on questions that require them to reproduce or apply their scientific knowledge, but girls perform better, on average, than boys on questions that require them to reason. Evidence from PISA 2006 shows that, on average, girls perform better than boys in terms of knowledge about science, which is the aspect of scientific literacy that measures students' knowledge about how scientists obtain evidence (scientific enquiry) and how scientists use data (scientific explanations).
While PISA has assessed science each of the three times it has been implemented since 2000, trend data is not available because a new assessment framework was introduced when science became the major domain assessed in 2006. Trend information presented here includes NEMP, TIMSS, and NCEA.
At the primary level, results from NEMP show that the overall performance in science has remained fairly stable over the last 12 years. In 2007, however, NEMP showed a small downward shift between 2003 and 2007 in two of the curriculum areas measured (physical world and material world).
The picture for science in the latest TIMSS conducted late 2006 (mid 2007 in most other countries) is complicated. Although results from 1994, 1998, and 2002, showed a steady increase, this trend did not continue in 2006 when the results returned to the 1994 levels. The rise between 1994 and 2002 was largely accounted for by a positive shift in performance by Māori and Pasifika students, with an increase also observed for Asian students. Between 2002 and 2006 the shift for Māori and Pasifika students reversed to the previous lower levels of performances. Over the period 1994 to 2006 Pākehā-European performance has remained static whereas the performance of Asian students increased. Given there has been little intervention for science over the last few years, the general result from TIMSS is not particularly surprising.
At the secondary level, NCEA science results are equivocal with a small increase in the average number of science standards gained at levels 1 and 2 and a small decrease at level 3. Information from implementations of Year 9 TIMSS 1994 and 2002 also showed performance to be roughly the same over that period.
What does engagement look like at primary and secondary schooling and how is it linked to science achievement?
While there are some early indications from national and international surveys that there are decreases in the performance of New Zealand students on two science assessments at the primary level, there are stronger signals that student attitudes and engagement are starting to shift and that there are school factors (including time spent teaching science) that are changing.
Generally, primary students are more likely to express positive attitudes than secondary students. Around three quarters of primary students expressed a positive attitude to science, in terms of interest and enjoyment. On similar questions at the Year 9 level, fewer secondary students expressed these positive attitudes to science. Science and mathematics were the least enjoyed classes at age 16 for students in the Competent Learners Study.
Around half of the primary students reported high self-confidence (TIMSS Y5). In terms of scientific literacy rather than science, secondary students were more confident in their abilities. In PISA over half the students expressed real confidence about the extent they could complete a range of given scientific literacy tasks. At the secondary level, self-efficacy in science is higher for males than females (PISA and TIMSS).
There is often a positive link between student engagement when defined in terms of their interest, enjoyment, self-belief, and future motivation in science, and their attainment in these surveys. At both primary and secondary levels, the strongest relationship tends to be between achievement and self-efficacy. The notion of self-efficacy includes both a student's confidence in their ability to do science, as well as their self-belief in their ability to overcome difficulties when attempting scientific tasks. The reasons for this relationship are, however, a little more difficult to unpack as self-confidence can enhance performance and doing well can enhance confidence. Nonetheless, it is clear from the evidence that these two factors interact strongly.
There are some interesting relationships among ethnic groups when their attitudes to science are examined. At the secondary level, while Pasifika students expressed less confidence in their ability to complete a range of science tasks, they expressed more enjoyment than students from other ethnic backgrounds, with the exception of Asian students (PISA). Asian students expressed the highest level of self-confidence in their ability to complete science tasks (both PISA and TIMSS Y9). However, at primary level (TIMSS Y5), there was little difference among the ethnic groupings when enjoyment and interest were compared, but more Pākehā-European students expressed high levels of self-confidence.
There is evidence across different surveys that the teaching of science at primary school is starting to wane. In NEMP, primary students are less likely in the latest cycle than they were in earlier cycles to report they are doing interesting things in science and also less likely to report that they are learning about science. At the same time, they are more likely to say they want to learn more science in school and they are doing more science in their own time. Their interest in science does not appear to be diminishing but the extent to which they are engaging in interesting science learning at school does appear to be reducing. Background information gathered from school principals and teachers in these national and international surveys is consistent with this. In TIMSS, primary teachers reported fewer hours teaching science in 2006 compared with 2002 – approximately a one-third reduction in teaching time. When principals were asked in a recent NZCER survey where they were putting their main curriculum effort and emphasis during 2007, only 2 percent responded that science was an area of emphasis at their schools. Compared with other curriculum areas, this placed science instruction firmly at the bottom of the list.
While very limited, there is also information about the nature of science tasks being undertaken. There has been a reported increase in the use of internet searching and copying tasks in primary science. There are also indications that science at primary and secondary school are approached in quite different ways, which raises issues around transition from primary to secondary science. Students transitioning from primary to secondary were more likely to describe science at primary than secondary school "fun" and reported having more autonomy in primary science than secondary. In the same study Year 9 teachers reported they had to help students "unlearn" science from primary school, leading to more copying of formulae and hence boredom among some students.
What do we make of all this?
It appears from these results that science is becoming both less of a priority as a curriculum area than other subjects and less of a priority than it was in earlier years for primary schools (and there is evidence that it has not been a high priority in terms of time allocation in primary schools for some time). Secondary teachers seem to be telling us that they have to 'play catch-up' when students arrive at secondary school. Again, this is not new. This is leading to less of the 'interesting' science happening in secondary schools.
Fewer hours of science instruction along with the apparent reduction in 'hands-on' investigative work is occurring in conjunction with a relative reduction in science achievement compared with other countries. This raises questions about our practice in science. Are we happy with the status-quo or do we need to take a critical review of current practices in the light of this evidence?
Students are interested in doing science in the middle primary years and would like to do more than they are. While many of our 15-year-old students are doing well at the secondary level in scientific literacy compared to their international counterparts, TIMSS highlights some relative weaknesses with a new cohort at the primary level. In addition, PISA concentrates more on scientific literacy for generalists (your everyday person-on-the-street) rather than science for specialists. Do the results from TIMSS imply that our education is lacking for those who need a higher level of science knowledge and skills?
As a country, understanding of science and technology is important to our agricultural sector, health sector, protection of our environment, and to future business innovations (e.g. the relationship with technology – biotech, electronics etc.). In turn, the environment is important to one of our bigger earners as a country, tourism. Is the emphasis being placed upon science in our education system out of alignment with this need? Is the lack of emphasis on science caused by a crowded curriculum? Or is it rather, as some commentators are suggesting, an unintended effect of emphasis on literacy and numeracy? Should we re-emphasise, for example, that science offers many opportunities to apply and develop numeracy and also literacy skills?
There is evidence from surveys of youth that across the world students are becoming less interested in science as a career and are more likely to opt for social science. In addition, there are further reports coming out from the OECD on PISA 2006 that show top performers in science and students from low SES backgrounds who do well have strong self-belief and self-confidence in science. And also there is some evidence that even very successful senior secondary science students are not interested in pursuing science in their post-school studies.
- In TIMSS 2006/07 the international researchers no longer report an international mean because of the instability of this measure as countries move in and out of the study. Instead relativity to the scale score of 500 is reported – this number is fixed for all cycles.
Education Data Requests
If you have any questions about education data please contact us:
Email: Requests EDK
Phone: +64 4 463 8065