Outcomes for teachers and students in the ICTPD School Clusters Programme 2006-2008: A national overview
This report focuses on the effectiveness of the 2006-2008 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first six ICT PD programmes. It is the last report of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.
Author(s): Selver Sahin and Vince Ham, CORE Education. Report prepared for the Ministry of Education.
Date Published: May 2010
This report is available as a download (please refer to the 'Downloads' inset box). To view the individual chapters please refer to the 'Sections' inset box. For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Section 6: Comparisons with Previous Cluster Cohorts
The reported increases in skill levels over all six of the ICT PD programmes that have been completed to date were considerable for all cohorts, especially for female and for primary teachers.
Similar proportions of teachers in the early 2001 and 2002 cohorts reported 'moderate' or 'high' skill levels at the end of their programmes. Noticeably higher proportions have reported such levels since the 2003 cohort, except in the case of graphics skills, which have remained relatively static across all cohorts over time (Figure 7). There has been a noticeable 'levelling off' of end of project skill levels at 'high' or 'very high' levels across all ICTs measured since the 2004 cohort.
Figure 7: Proportion of teachers reporting moderate to very high skill levels at the end of ICT PD programmes, across a range of ICT skills, 2001-2006 cohorts
Reported gains in ICT skills during the programme were also greater in the earlier cohorts than in the more recent cohorts. This may be largely explained by increasing entry-level skills among each successive cohort. The noticeable 'shift' that occurred between 2003-5 may be the effect of the Ministry of Education's TELA laptop scheme, which was rolled out during the 2003 and 2004 cohort programmes.
The particular skill areas where the greatest and lowest gains in competence were reported over the period of the 2006 cohort programme, were similar to those reported by earlier cohorts.
For all cohorts, reported increases in confidence as a consequence of the ICT PD programmes have been significant, both with regard to teachers' personal confidence with ICTs and their confidence about student use of ICTs in their classes (Figure 8). Large proportions of teachers from all cohorts reported moderate and high levels of confidence about ICT usage at the end of the programme. The relative increases in classroom confidence are significant for all cohorts and may be slightly decreasing in magnitude over time. This again can be explained by the increasing entry level skills that teachers reported in the last three cohorts.
For all cohorts, levels of personal confidence continue to exceed those in confidence about classroom use of ICTs.
Figure 8: Proportions of teachers reporting confident to very confident levels of confidence with ICTs before and after ICT PD programmes, 2001-2006 cohorts
Increased Classroom Usage
The extent to which teachers integrated ICT-based activities in ICT PD cluster classes increased significantly from quite low entry points, and to similar extents, for all six cluster cohorts for which there is comparable data. As Figure 9 shows, for all of the last four cohorts about half of the teachers at the end of the programme were 'regularly' incorporating ICTs into 'most' or 'all' student units of work.
Figure 9: Proportion of teachers whose students regularly* used ICTs for classwork before and after the ICT PD programme, 2001-2008 (*i.e.: ICTS were incorporated in 'all' or 'most' of their units of work over the previous year.)
Student Learning Activities
On those indicators where direct cross-cohort comparison is possible, there seem few significant differences among cohorts with regard to the types of learning activities reported for classroom use of ICTs, or in the reported increases of such use over the period of the programmes.
The increases made during the programmes were similarly large for all cohorts. Increased reported student use of multimedia production tools for presentation continues to be a major contributor to increases in ICT use for 'static communication', especially in primary schools.
It is noted that entry point student use of ICTs for online communication (mostly emailing) and for problem solving have stayed relatively stable over more recent cohorts, while exit levels of such use have shown slow increases since 2002.
Student use of ICTs for information processing (predominantly Internet use) continues to show the greatest increases across the cohorts. Use of ICTs for problem solving activities, reported previously as declining from 1999 to 2002 cohorts, is still comparatively low, but has recovered to exceed earlier levels in more recent cohorts. Most of this 'problem solving' use is explained by secondary student use of spreadsheets, data loggers and the like, in the 2003, 2004, 2005 and 2006 cohorts. The same 'recovery' trend is shown in respect of post-programme routine use of ICTs for curriculum practice.
There is a clear tendency for students in the last two cohorts to be reported as attaining higher frequencies of use of ICTs than previous cohorts with respect to all of: static presentation (mostly word processing and slide shows), problem solving activities (mostly through spreadsheet use), information processing activities (mostly through Internet use), online communication (email, social software), and curriculum practice activities (drill and practice, computerised tutorials, multimedia books, etc.).
Figure 10: Proportions of teachers' reporting frequent* classroom usage of ICTs for various learning outcomes, before and after the ICT PD programme, 1999-20 (*At least once or twice per term on average in the previous year)
The distribution of student usage across Essential Learning Areas at the end of projects shows few clear trends across the cohorts, except perhaps for the continued predominance of ICT use for Language objectives. Despite a slight decline over time, Language activities2 continue to be significantly higher than all the other learning areas, while the gap between other areas has been narrowing (Figure 11).
Figure 11: Proportion of ICT activities by Essential Learning Area, 1999-2007
Participant Satisfaction with Programme
Finally, we note that the 2004 cohort remains the highest in terms of the levels of goal achievement that participants reported at the end of the programme. Figure 12 shows that levels of goal achievement, although were higher than those achieved in the earlier 2003 cohort, have declined over the last two cohorts.
Figure 12: Proportions of teachers reporting their PD goals were 'largely met', 'fully met' or 'exceeded'
For all cohorts that were asked this question, participant satisfaction has varied by both sector and length of time in the programme, with both primary teachers and those in the programme for longer stating higher levels of satisfaction and goal achievement than secondary teachers and those in the programme for less time.
- For consistency with the previous cohorts, the proportions of activities engaged in the areas of English and Languages were combined together under the Language heading when analysing the data.