Outcomes for Teachers and Students in the ICT PD School Clusters Programme 2006-2008 - A National Overview

Publication Details

This report focuses on the effectiveness of the 2006-2008 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first six ICT PD programmes. It is the last report of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.

Author(s): Selver Sahin & Vince Ham - CORE Education [Report to the Ministry of Education]

Date Published: May 2010

Effects of the ICT PD Programme on Student Learning

The ICT PD clusters contracts identify several performance indicators related to the expected downstream student learning effects of the professional development. At a general level, there is an expectation that the ICT PD programmes would “facilitate improvements in students’ learning, engagement and achievement.” This was to be evidenced through:

  1. increases in reported ‘instances of teaching using ICT with classes to facilitate the learning of students’, and
  2. identifying ‘instances of teachers aligning ICT use with student learning’

The surveys provide both quantitative evidence of the extent of such increases in classroom use of ICTs in the cluster schools, and some qualitative evidence of the nature of the ‘alignment’ of ICT use with a range of student learning objectives.

Increased Classroom Usage of ICTs

One measure of increased classroom use of ICTs during the programme is the change in proportions of ‘high usage’ teachers (those who integrated ICT based activities into “all” or “most” of their units of work) at the beginning and end of the programme. In this cohort, the proportion of ‘high usage’ teachers increased from 18% at the start for the programme to 47% at the end. Conversely, the proportion of teachers who did not integrate ICTs at all decreased from 20% to 4% (Table 11). These figures were almost identical to those for the previous (2005-7) cohort.

Table 11: The proportion of units of work in which ICTs were incorporated before and after ICT PD, as reported in the end of project surveys (n= 2601 Before, 2577 After)
 Percentage of teachers before the programmePercentage of teachers after the programme
All or almost all units5%20%
Most units13%27%
Several units20%30%
One or two units42%18%
No units20%4%

The increase in the frequency of participants’ usage of ICTs with classes over time was correlated with gender, sector and length of time in the ICT PD programme. In particular, female teachers reported a greater increase (from 17% to 49%) than male teachers (from 20% to 43%) in their integration of ICT-based learning activities into the classroom. In the same way, primary teachers increased their classroom usage significantly more than secondary teachers. At the end of the programme, well over half (54%) of primary teachers were using ICTs in most or all of their units of work, compared with just over a third (37%) of secondary teachers. Those who had been in the programme longer also tended to report use of ICTs in a higher proportion of units than those in the programme for shorter periods. At the end of the programme, over a third (37%) of those who had been actively involved in the programme for less than a year were ‘high usage’ teachers. By comparison, well over half (56%) of those who had been active in the programme for more than two years were ‘high usage’ teachers at the end of the programme.

Aligning ICT Use with Student Learning Outcomes

In the surveys, we asked teachers to identify the specific learning activities their students had taken part in during the PD programme, the Essential Learning Areas covered by these activities, and the specific student learning outcomes they expected from or observed during those activities. These observed learning outcomes provide a window into what the teachers saw as the ‘quality’ in the ‘quality learning experiences’, that they provided for students during the programme.

Qualitative analyses of data from previous cohort surveys (Ham, Toubat & Williamson-Leadley, 2006) has suggested that, in teachers’ minds at least, the learning outcomes most often demonstrated when students use ICTs in learning activities, can be grouped into four main categories:

  1. Student motivation and engagement.
  2. Generic thinking skills – (Presentation & Communication, information processing, higher order thinking, and creativity).
  3. Generic social / collaborative skills
  4. Specific curriculum content knowledge and objectives, including technical (ICT) skills and knowledge.

The 2006 cohort’s identification of the alignment of ICT-based classroom activities with these student learning outcomes is outlined below under these same key categories.

1.  Increased use of ICT-based activities for student motivation and engagement with learning tasks.

The routine involvement of students in ICT activities with motivation/engagement learning intent or outcomes doubled over the period of the programme (Table 12).

Table 12: Proportion of teachers reporting frequent use of ICTs in classrooms for motivation/reward/engagement outcomes before and after the ICT PD programme (n= 2445 Before, 2430 After)

      Activity focus
Avg. daily or almost dailyAvg. once or twice a weekAvg. once or twice a termAvg. once or twice per yearNot at all
Motivation/Reward/ EngagementBefore ICT PD3%12%19%15%51%
After ICT PD10%20%22%12%35%

2.  The acquisition and demonstration of a range of generic communication and cognitive skills

Communication skills

The use of ICTs such as Web2 sites or emailing for interactive, topic-related communication, for example making inquiries of experts outside the classroom or engaging with social networking websites for classroom learning purposes, did not increase dramatically for most teachers over the period of the programme at a national level, though we note that the proportion of teachers whose students had ‘not at all’ engaged in these kinds of activities did reduce from 69% of teachers to 40%. The proportions that reported student use for this purpose at least a few times a term increased noticeably (Table 13, below).

By contrast, the majority of teachers were reporting studentuse of ICTs quite regularly for other communication activities by the end of the programme, notably by way of presentations of their work to teacher or peers. By the end of the programme, for example, the proportion of those whose students had ‘not at all’ used ICTs for multimedia presentations had decreased from 49% to 17%. Conversely, by the end of the programme the students of well over half (59%) of the teachers were engaging in this at least once or twice a term. Student use of ICTs for static print presentation had been rather more common prior to the programme, but more routine use of this too increased over the period of the programme. By the final year of ICT PD, the students of almost a half of teachers were using ICTs for static print presentations on a weekly or more frequent basis, and about four fifths on a termly or more frequent better basis.

In all three of these aspects of ICT use connected to communication skills, primary students were likely to be involved in these activities more regularly than secondary students.

Table 13: Frequency of students’ engagement in ICT based activities connected to communications skills, before and after the programme (n= 2475~2494)
Classroom activityFrequency
Avg. daily or almost dailyAvg. once or twice a weekAvg. once or twice a termAvg. once or twice per yearNot at all
Static print production / presentationBefore5%15%%31%30%19%
Multimedia presentationBefore1%5%16%28%49%
‘Online’ interaction with others (email, Web2 etc)Before4%6%8%12%69%
Information Processing

As had been the case with previous cohorts, the increase in student use of ICTs was most dramatic in relation to searching for, gathering or processing information, especially from the Internet. Teachers reported a significant increase in students’ engagement in such activities over the programme. For example, 46% of teachers reported that their students had never or only rarely accessed or searched for information on the Internet before the programme. By the end of the programme, the students of over four fifths (84%) of the teachers in the programme were using ICTs for this purpose on a termly or weekly/daily basis (Table 14).

Primary students were more likely to be involved in using ICTs for information processing as routine (better than termly) users than secondary students (63% cf. 41%).

Table 14: Students’ engagement in ICT-based activities related to a variety of cognitive skills before and after the programme (n=2437~2463)
Classroom activityFrequency
Avg. daily or almost dailyAvg. once or twice a weekAvg once or twice a termAvg. once or twice per yearNot at all
Information gathering/
Higher Order Thinking, problem solving etc.Before2%5%12%17%64%
Higher Order and Critical Thinking Skills

As can also be seen in Table 14, when teachers were asked about ICT activities associated with higher order thinking skills such as problem solving or synthesis and evaluation, the results show that the proportion of termly or better users of ICTs for these outcomes more than doubled over the period. However, such increases were less than those noted for information gathering and were lower also than those reported for communications-based activities.


In relation to ICT activities specifically aligned with creativity as a learning outcome the proportion of low frequency-users of ICTs for creative activity reduced noticeably (from 80% to 51%) over the programme. Conversely, the proportion of routine users increased from 8% to 19%. The greatest student use of ICTs for creative activities was in the termly rather than weekly/daily categories. Proportionally more primary teachers also used ICTs with students for creative activities than secondary teachers. This possibly reflects the relatively greater role of activities such as ‘creative writing’ and ‘story telling’ in the primary sector.

3.  ICTs for collaborative or social learning

The frequency of student engagement in ICT based activities related to collaborative learning, social interaction, and a sense of being part of a learning community, such as working in groups to solve a problem, collaborating on desktop publishing projects etc, also increased over the period of the programme, though there was much less emphasis on these learning outcomes in ICT use than on other outcomes such as communication, information processing and the like. Two thirds (66%) of teachers said their students had ‘not at all’ engaged in ICT based activities connected to collaborative learning and social interaction during the year prior to the programme, though by the end of the programme this had reduced to 43%. The proportion of teachers who reported high levels (daily or once/twice a week) of their students’ engagement in collaborative ICT based activities increased from 6% to 18%.

Table 15: Students’ engagement in ICT-based activities related to collaborative and social learning, before and after the programme (n=2428 Before, 2416 After)
Classroom activity     Frequency
Avg. daily or almost dailyAvg. once or twice a weekAvg. once or twice a termAvg. once or twice per yearNot at all
Collaborative learning and social interactionBefore1%5%13%15%66%

4.  Curriculum Practice & Technical Skills

There was also an increase in the frequency of students’ use of ICTs related to reinforcement of content knowledge, practice at rule application and concept learning through the use of drill and practice, educational games, tutoring software, and the like. The proportion of teachers whose classes had rarely or never used these technologies for these purposes reduced significantly over the programme, and conversely the proportion of teachers who reported daily or weekly student engagement in such classroom activities increased. By the end of the programme well over a third of teachers, and proportionally more primary (51%) than secondary (20%) teachers, were reporting routine (weekly or daily) use of ICTs for curriculum practice of this type.

As also had been the case with previous cohorts, a significant proportion of identified learning outcomes of ICT use, and a significant amount of regular use of ICTs by students, related to student acquisition of ICT skills per se. The tendency to view technical skills acquisition as a legitimate outcome for computer based learning activities was especially strong amongst primary teachers, some 60% of whom reported that they routinely used ICTs for such a purpose at the end of the programme (cf. 37% of secondary teachers).

Table 16: Students’ engagement in ICT-based activities related to curriculum practice and technical skills, before and after the programme (n=2424~2445)
Classroom activity    Frequency
Avg. daily or almost dailyAvg. once or twice a weekAvg. once or twice a termAvg. once or twice per yearNot at all
Curriculum practiceBefore3%12%18%24%43%
Technical skillsBefore8%16%22%22%31%

Essential Learning Areas

The largest proportion of ICT-based student activities reported by teachers related to the English (20%) and Maths (17%), followed by Social Science (13%), Science (11%) and Cross-curricular (10%).

Figure 4: Students’ use of ICTs by Essential Learning Area (n=9762 activities)

Image of Figure 4: Students’ use of ICTs by Essential Learning Area (n=9762 activities).