Science - Trends in Year 5 science achievement 1994 to 2006

Discussion of interactions

This report so far has presented results related to the science achievement of New Zealand Year 5 students from TIMSS 2006/07, mainly in the form of descriptive statistics. The focus has been on background characteristics of the students in isolation from other characteristics, when in fact, many of these characteristics are interrelated. For example, one could speculate that differences in achievement between students of different ethnic groups are interrelated with differences in home language. Indeed, earlier in this report it was demonstrated by analysing these factors together, that the difference in science achievement between those who spoke English at home regularly and those who did not was smaller when ethnic differences and immigration status were taken into account.

To investigate the possible interactions between characteristics of students and science achievement outcomes further, some statistical modelling was undertaken. This involved putting all of the factors that have shown a relationship with achievement together in a statistical model using an analysis tool called MLWin.¹ This tool carries out multi-level modelling analysis of this type of data. A more detailed discussion of the modelling work will be presented in a separate working paper later in 2008 or early in 2009.

The value of looking at such models is that the relative importance of different background characteristics can be determined. Modelling also allows for the elimination of factors that are unimportant or measure the same underlying trait as others. The multi-level aspect of this type of statistical model takes account of the fact that student learning takes place within classes that are part of schools and that all of these can impact on achievement. Thus this type of model allows for consideration of differences in results between schools, classes, and students.

The first step of the analysis was to examine variations in science achievement between schools, classes, and students, without including any background characteristics. Around 22 percent of the variation in science achievement was attributable to differences between schools, around 7 percent to differences between classes in the same school, and around 70 percent to differences between students. In other words, while there were some differences in science achievement between the schools, and also the classes within schools, the majority of differences were between the students themselves.

In the second step of the analysis, the following background characteristics were considered in the model: gender, ethnicity, speaking English at home, born outside of New Zealand, student age, books in the home, possessions in the home, attitudes, out-of-school activities, school decile and science instructional hours.
The model of influences on Year 5 students’ science achievement shows the following significant relationships with achievement when the other factors were taken account of:

• students with a greater number of books in the home had higher science achievement than those who had fewer;
• older students had higher science achievement than younger students;
• students born in New Zealand had higher science achievement than those who were not;
• students with a greater number of educational resources in the home had higher science achievement than those with fewer;
• students who spoke English frequently at home had higher science achievement than those who did not;
• students with high self-confidence had higher science achievement than those with lower self-confidence;
• students who felt safe at school had higher science achievement than those who felt less safe;
• students who engaged in the out-of-school activities television watching and playing computer games for a moderate amount of time had higher science achievement than those who never did or did these for a higher number of hours per day;
• students who read books regularly had higher science achievement than those who did not;
• boys had higher science achievement than girls;
• Asian students had higher science achievement than Pākehā/European students;
• Māori students had lower science achievement than Pākehā/European students;
• Pasifika students had lower science achievement than Pākehā/European students;
• Students who were in a higher decile school (living in a community with less economic disadvantage) had higher science achievement than those in lower decile schools.
  

After completing the statistical model, variations between schools, classes, and students were re-examined to see if the model helped to explain the variations initially observed. The model explained just under three quarters of the variation between schools, just under one half of the variation between classes, and about one quarter of the variation between students. In other words, the model explained much of the variation between schools, and between classes within schools, so that most of the variation that remained unexplained was between students.

Factors relating to the socio-economic status of the students such as decile, educational resources in the home, and books in the home all had a reasonably strong relationship with science achievement in the model. While it is not easy to effect change in these, self-confidence in science and reading books were two factors that had a strong relationship with achievement that might be influenced by teachers and parents.

This report has not examined all of the data collected in TIMSS. Further analyses will be undertaken including investigation of what is happening in science in schools from information collected from teachers and principals. The model presented in this section is an initial investigation and should be read as such, but it gives valuable insight into the factors explaining differences in the science achievement of Year 5 students in New Zealand.
 

Footnote

1. For further details see Goldstein (2003).