Outcomes for teachers and students in the ICTPD School Clusters Programme 2005-2007: A national overview
This report focuses on the effectiveness of the 2005-2007 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first five ICT PD programmes. It is part of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.
Author(s): Vince Ham, CORE Education. Report prepared for the Ministry of Education.
Date Published: July 2009
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Section 5: Participant Evaluations of the Programme
Teacher satisfaction with the programme
Generally, teachers indicated high levels of satisfaction with their programme in relation to meeting their goals and, even though c.18% of teachers entered the programme stating that ICTs were a low PD priority for them, the great majority by the end felt it had been a positive force in their recent development as teachers.
Substantial majorities of the participating teachers indicated that their goals had been met to a large extent or better. Overall, about a fifth of teachers felt their goals had been fully met or exceeded, and about 42% felt their goals had been largely met. A small group of 5% of teachers felt their goals had not been met, the majority of these being secondary teachers.
Teachers were asked about three kinds of goals: technical skills, practical ideas for classroom uses of ICTs, and improved quality in teaching and learning. For all three goal types, just under two thirds of the teachers said that these goals had been largely met, fully met or exceeded (Table 16).
|Level of Achievement||Goal|
|Technical skills||Classroom ideas||
Quality of Teaching|
In this cohort, the differences between primary and secondary teachers in terms of levels of goal achievement were present, but less significant than in previous cohorts. Primary teachers reported higher achievement levels compared to secondary teachers for goals related to classroom use and ideas and improving teaching and learning generally, but not in relation to technical skill acquisition.
Figure 5: Proportions of teachers reporting extent to which their goals/expectations of the ICT PD programme had been met, by length of time in the programme (with trendlines)
As has been found in relation to many other effects of the ICT PD programmes, there was a significant correlation between the length of time individuals were involved in the programme and their level of goal achievement. Across all three goal types, and for both sectors, the longer the period of involvement, the higher the levels of goal achievement. Figure 5 shows this relationship between satisfaction with programme and length of involvement. In the first six months the proportions of teachers largely/fully satisfied and those only partially or not satisfied are virtually equal. By the end of the three years well over twice as many teachers were largely/fully satisfied their goals had been met as felt they had only partially or not been met.
The ICT PD programme's contribution to teachers' increased ICT usage
Teachers were asked directly about the extent to which their increased use of ICTs with classes over the three years of the programme were directly attributable to the programme. The great majority of teachers (93%) indicated that the increase in their ICT usage was attributed either completely, largely or partly to the ICT PD programme. Some 57% said that their increased use of ICTs with classes was either largely or completely attributable to the programme (Figure 6).
Figure 6: The extent to which increased use of ICTs with classes is attributable to the ICT PD programme
In a triangulating question, teachers were asked how significant the programme had been in contributing to their overall development as professionals. If anything they were more positive on this than they had been in the question about attribution to usage, a similarly small minority (6%) said it had not been significant to their overall development. But a rather larger proportion of two thirds (66%) of teachers indicated that the programme had been either 'significant' or 'very significant' as a factor in their overall development as teachers. Primary teachers were rather more likely than secondary teachers to attribute general professional significance to the programme.
Teachers' evaluations of the delivery of the ICT PD programme
When teachers were asked to comment on the particular aspects of their particular programmes that they most and least appreciated, they often commented on similar issues. The feedback in relation to the most and least appreciated aspects are summarised in the exemplary statements in the boxes below, categorised by:
- Programme content (32% of statements of appreciation). Statements by teachers about the content of their ICT PD programmes and their particular cluster models centred around learning new technical skills; gathering and sharing practical ideas for ICT integration into classroom curriculum programmes; and learning about principles of effective teaching and learning in general.
- The skills and effectiveness of the Facilitators and other leadership elements (17%). As has been the case in most cohorts, many of the comments highlighted the pivotal role of able, knowledgeable and professionally skilled facilitators. Teachers required their facilitators to be technologically and pedagogically knowledgeable, willing to adopt a supportive, empathetic attitude, and always 'available'.
- Programme delivery (18%). The organizational aspects of programme delivery appreciated most were the personalized attention many facilitators were able to build into their programmes, small group sessions, one to one help, in-classroom visits, arranged visits among schools, and practical (ie: hands on and/or interactive) workshops.
- The teachers' involvement in various 'Learning communities' (33%) as part of their programmes. Teachers often commented on the various opportunities that accompanied the programmes to share learning and experiences both within and beyond the cluster, and the contribution these made to developing a 'community of practice' around the programme. Most prominent among these opportunities were the national conferences, outside/overseas visitors and experts, meeting other teachers from other schools, and sharing expertise and knowledge with colleagues through various 'celebrations' and events both within and outside the cluster context. This, along with programme content, was the aspect of the programmes most often mentioned when teachers were asked to identify the most appreciated aspects of the programme. More appreciative comments related to the notions of collegiality and learning community in the programmes than to the other categories identified.
Examples of comments on the 'most appreciated' aspects of the programmes that were coded as being about opportunities for sharing, celebrating and presenting
- "The opportunities to attend conferences, share ideas with like minded people, the online sharing have all contributed. And also the great opportunities to network and form long lasting professional friends all over the nation"
- "Collegiality across schools opportunity to access live experts both local and international"
- "To be able to share ideas with other teachers and visit schools to see how they do things has probably made the biggest impact on my teaching."
- "Collegial support within the school and cluster. Opportunities to be part of whole school PD in eg. thinking workshops/ speakers "
- "Sharing within the community.
- "collaboration with other schools and teachers, access to conferences and visiting speakers"
- "Attending Lead Teacher Days. Assistance from Facilitator. Attendance at ULearn. Success of Cluster Wide exhibition."
- "The chance to look at other schools and see that we are doing extremely well. Also the PD with Jamie McKenzie - very inspiring. Going to ULearn."
- "ULearn, GDITE course, Cluster share, networking."
- "Collegiality and networking"
- "Study tours PD involving the Wairarapa cluster networking among staff on a similar level to me"
- "The highlight was the ICT conference in Rotorua at the beginning of the year...lots of great ideas, networks, skills etc."
- "it was a good challenge for me to teach teachers and share knowledge I had gained by attending learning@schools conference."
In terms of suggestions for improvement of the programmes, the most often identified suggestions related to:
- giving more time for teachers to practice what they have learnt, and perceived work overload to fulfill the demands of the PD,
- continuing the PD opportunity after the contract period,
- providing greater variety in ICT sessions,
- increasing the opportunity for one-to-one, in-class support from facilitators,
- tailoring the PD better to the needs of individuals or doing more effective needs analysis
- improving technical support in schools to avoid ICT failure,
- improving student access to computers and software,
- providing even more opportunity for collegial networking and building learning communities.
Most of these paralleled the 'concerns' about integrating ICTs discussed in an earlier section.