Outcomes for teachers and students in the ICTPD School Clusters Programme 2005-2007: A national overview

Publication Details

This report focuses on the effectiveness of the 2005-2007 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first five ICT PD programmes. It is part of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.

Author(s): Vince Ham, CORE Education. Report prepared for the Ministry of Education.

Date Published: July 2009

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.

Section 2: Methodology

Research questions

The brief for the research on the 2005-7 cohort of ICT PD clusters was to provide a broad national profile of the impact of the programme across the country as a whole. It has not been our role to evaluate specific clusters or their particular PD models. To this end the core research question being addressed is not so much to identify which particular models of PD are most effective, but rather to conduct a survey-based study to evaluate how well, and in what respects, the national ICT PD School Cluster initiative has been meeting the objectives of stakeholders and participants across the cohort. In doing this we addressed the following core research question:

How effective overall was the 2005-7 ICT PD programme in meeting its goals of:

  • increasing teachers' ICT skills, confidence and knowledge related to the educational applications of ICTs, and
  • promoting quality classroom learning experiences for students?

Research strategy and data collection

Survey data were gathered from all 20 of the 2005-7 cohort clusters. These comprised responses from both pre- (Baseline) and post- (End of Project) programme surveys of all participants. In order to maximise validity of comparison across cohorts, the Baseline and End of Project instruments were based on the equivalent questionnaires developed for the previous cohorts in the programme and reported separately in previous reports to the Ministry.

Table 1: Timetable of research surveys in ICT PD cluster cohorts, 1999-2008 (Shaded surveys are those relevant to this report)
Notes:
  1. Key: BL=Baseline survey
  2. EOP = End of Project survey
Cohort 1999 2001 2002 2003 2004 2005 2006 2007 2008
1999 BL
EOP

       
2001
BL
EOP
       
2002     BL
EOP
     
2003       BL
EOP


2004      
BL   EOP    
2005      

BL   EOP
2006      


BL:

2007      



BL
2008      




BL


The profile of the 2005 cohort on entry to the programme, as established by the baseline survey of 2005, along with entry-point comparisons between this and previous cluster cohorts, has been reported separately to the Ministry (Ham et al. 2005). This report focusses on the effects of the programme evidenced through retrospective self-report data gathered in the End of Project survey (October 2007).

Respondent demographics

Table 2: Number of responses to 2005-2007 Baseline and End-of-Project surveys, by cluster
Cluster EoP
AVRACS Cluster - Ardgowan School 51
Big River Cluster - Clutha Valley School 58
Heretaunga Futures Cluster - Oxford Crescent School 78
Kerikeri Schools ICT Cluster - Kerikeri Primary School 29
Kiri Kiri Mangu Cluster - Parakai School 66
Nga Tii Roa Cluster - Paengaroa School 44
Onehunga-Howick Secondary - Onehunga High 102
Rotorua Secondary ICT Cluster - Rotorua Girls' High 52
South 3 Cluster - James Hargest  High 83
South and Rural Wairarapa ICTPD Cluster - Greytown 72
South Otago Net - Tokomairiro High School 54
Tai o Rapuwai Cluster - Port Chalmers 54
Tapatoru Cluster - Halswell School 78
Te Ihutai Cluster - Redcliffs School 98
Te Waka Cluster - Kaiapoi North School 35
The Ignition Project - Hillmorton High School 52
Waikowhai Cluster - Waikowhai Intermediate 8
Waimea South Cluster - Brightwater School 74
Wanganui  Cluster - Wanganui East School 65
West Auckland Intermediates - Henderson Intermediate 93
Grand Total 1,246


In all, 1246 valid End of Project survey responses were received from teachers in the 20 cohort clusters. This represents an estimated response rate of c.60% for the End of Project survey. The End of Project response rate is consistent with response rates for the other 'online' surveys of ICT PD cluster teachers conducted prior to 2007.

It appears that the demographic distribution of the respondent group for the End of Project survey was dissimilar to that of the broader teacher population in the ICT PD programmes in terms of gender and in terms of sector. About 75% of respondents were females and 25% were males, which, when broken down by sector represents a ratio of female to male teachers among the primary sector respondents of 78%:22% and in the secondary sector of 65%:34%. Compared to the general gender distribution of teachers in the respective sectors, therefore,  male teachers were notably underrepresented in the 2005-7 ICT PD cohort and surveys, as they have been in most previous cohorts and surveys.

About 72% of responding teachers taught at primary school level, 27% at secondary, and 1% taught both primary and secondary students. As had been the case for all but one of the previous cohorts, the proportion of primary participants in this ICT PD cohort was significantly larger than that of secondary. In this cohort primary teachers are over represented compared to secondary teachers by margins higher than the two previous cohorts but similar to those in the first three (1999-2001, 2002-4, 2002-5). The 2003-5 cohort remains the only ICT PD cohort since 1999 in which secondary and primary teachers, and males and females, have taken part in similar proportions to those in the general teaching population.

Table 3: Respondents by school sector (end of project survey, n=1241)
School Sector Total %
Primary 892 72%
Secondary 335 27%
Both 14 1%


A strong correlation was evident in studies of previous ICT PD cohorts between the length of time that individual teachers were actively engaged in the programme and their subsequent levels of confidence and classroom usage of ICTs. This may account for the tendency for respondents to have been actively engaged in their programmes for longer rather than shorter lengths of time. In the 2005-7 cohort some 55% of teachers' were actively engaged in the ICT PD programme for more than 2 of the 3 years of the programme. Only 7% of teachers had been in a programme for short-term periods of less than six months (Table 4). Overall these proportions are roughly comparable with those for the two prior cohorts, but higher for the specific proportion taking part for more than 31 of the 36 months.

Table 4: Respondents by length of time actively involved in the ICT PD programme
Length of Active Involvement Total Percentage
0-6 Months 83 7%
7-12 Months 167 13%
13-18 Months 113 9%
19-24 Months 195 16%
25-30 Months 122 10%
31-36 Months 552 44%
N 1,232

Where to find out more

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065