Outcomes for teachers and students in the ICTPD School Clusters Programme 2005-2007: A national overview
This report focuses on the effectiveness of the 2005-2007 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first five ICT PD programmes. It is part of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.
Author(s): Vince Ham, CORE Education. Report prepared for the Ministry of Education.
Date Published: July 2009
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Section 6: Comparisons with Previous Cluster Cohorts
The reported increases in skill levels over all five of the ICT PD programmes that have been completed to date were considerable for all cohorts, especially for female and for primary teachers.
Similar proportions of teachers in the early 2001and 2002 cohorts reported 'moderate' or 'high' skill levels at the end of their programmes. Noticeably higher proportions have reported such levels since the 2003 cohort, except in the case of graphics skills, which have remained relatively static across all cohorts (Figure 7). There has been a noticeable 'levelling off' of end of project skill levels at 'high' or 'very high' levels across all ICTs measured since the 2004 cohort.
Figure 7: Proportion of teachers reporting moderate to very high skill levels at the end of ICT PD programmes, across a range of ICT skills, 2001-2005 cohorts
Reported gains in ICT skills during the programme were also greater in the earlier cohorts than in the more recent cohorts. This may be largely explained by increasing entry-level skills among each successive cohort. The noticeable 'shift' that occurred between 2003-2005 may be the effect of the laptop scheme, which was 'rolled out' during the 2003 and 2004 cohort programmes.
The particular skill areas where the greatest and lowest gains in competence were reported over the period of the 2005 cohort programme, were similar to those reported by both earlier cohorts.
For all cohorts, reported increases in confidence as a consequence of the ITCPD programmes have been significant, both with regard to teachers' personal confidence with ICTs and their confidence about student use of ICTs in their classes (Figure 8). Large proportions of teachers from all cohorts have reported moderate and high levels of confidence about ICT usage at the end of the programme. The relative increases in classroom confidence are significant for all cohorts and may be slightly decreasing in magnitude over time. However, for all cohorts gains in and levels of personal confidence continue to exceed those in confidence about classroom use of ICTs.
Figure 8: Proportions of teachers reporting confident to very confident levels of confidence with ICTs before and after ICT PD programmes, 2001-2005 cohorts
Increased classroom usage
Figure 9: Proportion of teachers whose students routinely* used ICTs for classwork before and after the ICT PD programme, 2001-2007
- (*ie: ICTS were incorporated in 'all' or 'most' of their units of work over the previous year.)
The extent to which teachers integrated ICT-based activities in ICT PD cluster classes increased significantly from quite low entry points, and to similar extents, for all four cluster cohorts for which there is comparable data. For all of the last three cohorts, about half of the teachers at the end of the programme were routinely incorporating ICTs into 'most' or 'all' student units of work, and three quarters or more were doing this at least 'regularly' ('several units' or more).
Student learning activities
On those indicators where direct cross-cohort comparison is possible, there seem few significant differences among cohorts with regard to the types of learning outcomes reported for classroom use of ICTs, or in the reported increases of such use over the period of the programmes.
Generally, and certainly since 2003, each cohort seems to be achieving slightly higher rates of usage of ICTs for various student outcomes than the previous cohort. It is noted, for example, that entry point student use of ICTs for online communication (mostly emailing) and problem solving have stayed relatively stable over more recent cohorts, while exit levels of such use have shown slow increases since 2002.
Student use of ICTs for information processing (predominantly Internet use) continues to show the greatest increases across the cohorts. Use of ICTs for problem solving activities, reported previously as declining from 1999 to 2002 cohorts, is still comparatively low, but has recovered to exceed earlier levels in more recent cohorts. Most of this 'problem solving' use is explained by secondary student use of spreadsheets, data loggers and the like, in the 2003, 2004 and 2005 cohorts. The same 'recovery' trend is shown in respect of post-programme routine use of ICTs for curriculum practice (Drill & Practice, computerised tutorials, multimedia books etc).
The increases made during the programmes in were similarly large for all cohorts. Increased student use of multimedia production tools for presentation continues to be a major contributor to increases in ICT use for 'static communication', especially in primary schools.
Figure 10: Proportions of teachers' reporting frequent* classroom usage of ICTs for various learning outcomes, before and after the ICT PD programme, 1999-2007
- (*At least once or twice per term on average in the previous year)
The distribution of student usage across Essential Learning Areas at the end of projects shows few clear trends across the cohorts, except perhaps for the continued predominance of ICT use for Language objectives and a slight decline in the proportion of ICT activities for Mathematics. Language activities account for the highest proportion of classroom ICT activities in all cohorts. The proportion of activities in the Language area increased noticeably between the 2003 and 2004 cohorts, apparently at the expense of Social Studies and Mathematics activities but this trend was reversed in the current (2005-7) cohort (Figure 11).
Figure 11: Proportion of ICT activities by Essential Learning Area, 1999-2007
Participant satisfaction with programme
Finally, we note that levels of goal achievement and expectations met were similar to, but not quite as high as, those achieved in the 2004 cohort, but were higher than those than achieved in the earlier 2003 cohort, across all of the groups of goal identified. We attribute the lower satisfaction levels reported by the 2003 cohort to the fact that there were proportionally more secondary teachers, and more teachers for whom ICTs were a low priority on entry in the 2003 cohort compared to the two more recent cohorts. Both these groups tend to report lower end-point satisfaction levels.
Figure 12: Proportions of teachers reporting their PD goals were 'largely met', 'fully met' or 'exceeded'
For all cohorts that were asked this question, participant satisfaction has varied by both sector and length of time in the programme, with both primary teachers and those in the programme for longer stating higher levels of satisfaction and goal achievement than secondary teachers and those in the programme for less time.