NMSSA 2016: Technology - Key Findings

Publication Details

In 2016, the National Monitoring Study of Student Achievement (NMSSA) assessed student achievement at Year 4 and Year 8 in two learning areas of the New Zealand Curriculum (NZC) - technology and learning languages. This report provides findings on an additional area of research in 2016 – the use of digital technologies for teaching and learning.

Author(s): Educational Assessment Research Unit and New Zealand Council for Educational Research. Report for the Ministry of Education.

Date Published: May 2018

Executive Summary

Technology

The NZC defines the focus of technology to be for 'students to learn to be innovative developers of products and systems and discerning consumers who will make a difference in the world' (p. 17).

The technology learning area comprises three strands: Technological Practice (knowing how to plan for practice, develop a brief and evaluate a range of outcomes); Technological Knowledge (knowing what key concepts underpin technological development and outcomes) and Nature of Technology (knowing why technology is influenced by, and influences, historical, social, environmental and cultural events).

Technological literacy is at the heart of technology education and is developed by exposure to a wide range of relevant learning experiences. Technological literacy enables students to live with, critique and contribute to technological developments that shape their lives.

Study features

NMSSA used a two-step sampling procedure to select 100 schools at each year level and up to 27 students within each school. The nationally representative sample at each year level was made up of about 2,300 students (see Appendix 1, Technical Information 2016 report).

A programme was designed to gain an understanding of achievement in technology using three data gathering components. Table 1 outlines the features of each component.

Table 1: Features of the NMSSA Technology Study Components
Note:
  1. * IP Component = Indicator of Progression Component
Assessment Programme in Technology
Component 1: Technological Literacy

Strands and IP Components*

Technological Practice (TP)
  • planning for practice
  • brief development
  • outcome development and evaluation
Technological Knowledge (TK)
  • modelling
  • products
  • systems
Nature of Technology (NT)
  • characteristics of technology
  • characteristics of technological outcomes

Assessment Approach & Students Participating

  • group-administered tasks mainly presented by computer
  • completed by all Year 4 and Year 8 students (about 2,300 at each year level)
Component 2: Student Questionnaire

Strands and IP Components*

  • attitudes to technology
  • opportunities to learn technology at school

Assessment Approach & Students Participating

  • computer-based questionnaire
  • completed by all Year 4 and Year 8 students (about 2,300 at each year level)
Component 3: Teacher and Principal Questionnaires

Strands and IP Components*

  • teacher and principal views of technology instruction in their school
  • teacher confidence as technology educators
  • professional learning and development in technology
  • provision for teaching technology in the school

Assessment Approach & Students Participating

  • paper-based questionnaires
  • completed by over 200 teachers and about 100 principals at each year level


The Technological Literacy (TELI) assessment, which assessed acrossthe three technology strands: Technological Practice, Technological Knowledge and Nature of Technology, covered the components listed in the Technology Indicators of Progression. Some aspects of the TP strand were not able to be asssessed in NMSSA given the extended developmental process required to fulfil some aspects of the strand. The relative weightings of the three strands in the TELI assessment was 22, 36 and 42 percent, respectively.

Achievement on the TELI assessment was reported on a measurement scale that covered both year levels. The scale was aligned to the levels of the NZC through curriculum alignment processes that defined minimum scale scores (cut-scores) associated with achieving, on balance, the achievement objectives that were able to be assessed at curriculum levels 2, 3 and 4. Alignment of an achievement scale to the NZC has not been attempted before in this learning area.

Other data were collected through questionnaires from students, teachers and principals.

Key findings about achievement in technology

Achievement on Technological Literacy (TELI)

Seventy-three percent of Year 4 students achieved above the minimum TELI score associated with achieving curriculum level 2 objectives. Fifty-three percent of Year 8 students achieved above the minimum score on the TELI scale associated with achieving curriculum level 4 objectives.

The average TELI scale score for Year 8 students was 38 units higher than Year 4 students. This difference indicates that students make about 10 scale score units of progress per year between Year 4 and 8.

There were significant differences in achievement by gender, ethnicity and school decile at both year levels.

The average TELI scale score for girls was about 6 units higher than boys at both year levels.

The average TELI scale score for Māori and Pasifika students, who were more likely than other students to attend mid and low decile schools, was 11 TELI units lower than non-Māori and about 14 TELI units lower than non-Pasifika students, respectively. These differences are roughly equivalent to the amount of progress associated with one year of instruction and one and a half years of instruction, respectively.

The average TELI scale score for students from low decile schools (deciles 1, 2 and 3) was 20 TELI units lower than high decile schools (deciles 8, 9 and 10) and about 14 TELI units lower than mid decile schools (deciles 4, 5, 6 and 7), respectively. The difference between high and low decile schools is roughly equivalent to the amount of progress associated with two years of instruction.

Achievement on technology strands

Overall, on the TELI assessment, the items representing each strand have very similar difficulties.

Learning and teaching in technology: attitudes, opportunites, resources

About the students

About 80 percent of NZ European and Māori students at both year levels reported that they always speak English at home, compared with about 60 percent of Pasifika students and about 40 percent of Asian students.

The amount of English spoken at home was related to achievement. At Year 4, the difference, on average, between students who always spoke English at home scored 10 TELI units higher than those who never did. At Year 8, the difference was 25 TELI units. These differences represent roughly the amount of progress associated with one year and two and a half years of instruction, respectively.

Students' attitudes to school and to technology

Overall, Year 4 students indicated that they liked school more than Year 8 students. About 80 percent of Year 4 students reported liking school heaps or quite a lot, compared with 70 percent of Year 8 students. Greater proportions of Pasifika and Māori students, and students in low decile schools, liked school heaps. This contrasts with  greater proportions of  NZ European students and students in mid and high decile schools liking school quite a lot.

There was a complex association between how much students liked school and achievement.  Students who did not like school at all had, on average, the lowest level of achievement. However, students who liked school heaps had lower levels of achievement, on average, than students who liked school quite a lot.

Year 4 and Year 8 students were positive about technology with the majority of students' Attitude to Technology (ATT) scale scores being in the positive and very positive categories. Year 4 students who had very positive attitudes to technology scored higher on the TELI assessment than those who had negative attitudes.  However, very few students had negative ATT scores.

Students' perspectives on learning opportunities and choices in technology

Overall, Year 8 students reported more frequent opportunities to learn technology at school than Year 4 students, with experiences related to technological practice being the most frequently experienced by students at both year levels. Teachers reported broadly similar opportunities to learn as students.

Year 8 students had different choices of technology that they could study. Just over 70 percent of students studied resistant materials (e.g. wood, metal) and food technology/biotechnology. About 50 percent studied textiles. Less than 30 percent took each of computer programming/robotics, electronics and media. There were no gender, ethnicity and school decile differences in the percentage of students taking each technology option.

Teachers' perspectives on teaching and learning in technology

Overall, teachers were positive about technology and being a technology educator with Year 8 teachers being more positive than Year 4 teachers.

The teaching of technology was generally integrated with other learning areas at Year 4 and taught as a separate subject at Year 8.

Year 8 students tended to learn technology for a greater number of hours over the year than Year 4 students did. About 36 percent of Year 4 students spent more than 20 hours learning technology compared with 65 percent of Year 8 students.

Technological Practice was taught for about half of the technology programmes at both year levels. The other half of the programmes were devoted almost equally to teaching Technological Knowledge and the Nature of Technology.

Teachers at both year levels used a number of instructional strategies to teach technology, with the most frequently used strategies being whole class activities and mixed ability group-based activities.

Teachers' perspectives on resources for teaching and learning in technology

Overall, Year 8 teachers indicated that they received more professional support for teaching technology than Year 4 teachers. They also rated the overall level of professional support they received more favourably than Year 4 teachers did with 6 percent of Year 4 and 33 percent of Year 8 teachers rating it as good or excellent, respectively.

The majority of teachers at both year levels were aware of 'Technology Online' on Te Kete Ipurangi (TKI) website, used it occasionally, and found it easy to locate the resources they needed.

Fewer teachers were aware of the technology 'Indicators of Progression' for assessing students' work, with less than half of the Year 8 teachers using them occasionally.

Just under half of Year 8 teachers and a fifth of Year 4 teachers had had technology-focused PLD in the last two years.

The majority of Year 8 teachers reported having access to suitable spaces, plentiful materials and good quality equipment. While two thirds of Year 4 teachers reported having access to suitable spaces, the majority reported not having access to plentiful materials or good quality equipment.

Principals' perspective on teaching and learning in technology

About two thirds of Year 8 schools had a technology centre, and about half of those had their technology specialist teach students from other schools.

The classroom teacher was largely responsible for teaching technology at Year 4 and some had support from a specialist technology teacher. At Year 8, in the majority of schools, a specialist taught all areas of technology, except for media, either entirely or with support from the classroom teacher. A mix of specialist teachers and classroom teachers taught media.

The majority of principals reported that teachers are implementing strategies to meet the needs of diverse students, and teachers have appropriate pedagogical content knowledge to identify and respond effectively to the learning needs of students in technology.

Overall, Year 8 principals rated their school's provision for learning in technology more highly than Year 4 principals. About 90 percent of Year 8 principals and 60 percent of Year 4 principals rated it as good, very good or excellent.

Two thirds of Year 4 principals indicated that teachers in their school had either no access or little access to technology PLD. In contrast, nearly two thirds of Year 8 principals reported that access to PLD was moderate or extensive.

About 60 percent of Year 4 principals and 30 percent of Year 8 principals indicated that technology had not been a focus for development in the last five years.

Priority learner groups

The percentage of Year 4 and Year 8 students from priority learner groups achieving at curriculum levels 2 and 4 objectives, respectively, are summarised in Table 2.

Table 2: Percentage of Year 4 and Year 8 students from each priority learner group achieving at or above curriculum level 2 and level 4, respectively

Year Level and
Curriculum Level
Percentage of Students from each Priority Learner Group
Māori
Students
Pasifika
Students
Students with Special
Education Needs
Year 4, Level 2 and Above 57 51 57
Year 8, Level 4 and Above 34 26 21


At both year levels, the pattern of differences in achievement related to gender and school decile for Māori and Pasifika students was similar to the national sample.

The annualised progress for the national sample was similar for Māori and Pasifika students (10 TELI units) and a little lower for students with special education needs (7 TELI).

Overall, Māori, Pasifika and students with special education needs indicated similar levels of positive attitudes to technology as the national sample at both year levels.

The national average on the TELI assessment at each year level was used as a benchmark for comparison purposes. At each year level, lower percentages of students from the priority learner groups scored at or above the respective national averages than the national samples. The average TELI scale scores for students achieving above the benchmark score were lower for students from the priority learner groups compared with all students in the national sample.

Footnotes

  1. Ministry of Education. (2007). The New Zealand Curriculum. Wellington: Learning Media.
  2. Te Kete Ipurangi Technology Online Indicators of Progression.
  3. Te Kete Ipurangi (TKI)

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065