Outcomes for teachers and students in the ICTPD School Clusters Programme 2006-2008: A national overview

Publication Details

This report focuses on the effectiveness of the 2006-2008 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first six ICT PD programmes. It is the last report of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.

Author(s): Selver Sahin and Vince Ham, CORE Education. Report prepared for the Ministry of Education.

Date Published: May 2010

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.

Section 5: Participant Evaluations of the Programme

Teacher goal Achievement in the Programme

Generally, teachers indicated high levels of satisfaction with the programme in meeting their goals and the great majority, by the end, felt it had been a positive force in their recent development as teachers.

Teachers were asked about three main kinds of goals: technical skills, practical ideas for classroom uses of ICTs, and improved quality in teaching and learning. For all three major goal types, over half of teachers reported that their goals had been largely met, fully met or exceeded (Table 17). 

Table 17: Levels of ICT PD goal achievement reported by teachers (n= 2560 (T), 2428 (C), 2471 (Q))
Goal Level of Achievement Technical
Skills (T)
Classroom
Ideas (C)
Quality of Teaching
and Learning (Q)
Exceeded 3% 4% 3%
Fully met 8% 9% 9%
Largely met 44% 43% 42%
Partially met 40% 38% 40%
Not met 5% 6% 6%


In this cohort, sector differences were more marked than in the previous cohort of 2005-7. Primary teachers reported higher goal achievement levels compared to secondary teachers in relation to all three of technical skills acquisition, producing classroom ideas, and enhancing teaching and learning quality.

There was also a noticeable gender difference among teachers in terms of their goal achievement. For all three goal types, female teachers reported higher levels of achievement than male teachers.

As has been found in relation to many other effects of the ICT PD programmes, there was also a clear correlation between the length of time individuals were involved in the programme and their level of goal achievement. Across all three main goal types, and for both sectors, the longer the period of involvement, the higher the levels of goal achievement. Figure 5 shows this relationship between goal achievement and length of involvement.

Figure 5: Proportions of teachers reporting extent to which their goals/expectations of the ICT PD programme had been met, by length of time in the programme (with trendlines)

Image of Figure 5:   Proportions of teachers reporting extent to which their goals/expectations of the ICT PD programme had been met, by length of time in the programme (with trendlines).

The ICT PD Programme's Contribution to Teachers' Increased ICT Usage

Teachers were asked about the extent to which their increased use of ICTs with classes over the three years of the programme was attributable to the programme. The great majority of teachers (89%) indicated that the increase in their ICT usage was attributed either completely, largely or partly to the ICT PD programme. Some 52% stated that their increased use of ICTs with classes was either largely or completely attributable to the programme (Figure 6).

Figure 6: The extent to which increased use of ICTs with classes was seen as attributable to the ICT PD (n= 1533 (P), 954 (S) programme

Image of Figure 6:   The extent to which increased use of ICTs with classes was seen as attributable to the ICT PD (n= 1533 (P), 954 (S) programme.


The other, non-programme, factors most often identified as contributing to their increased use of ICTs with students included:

  • Recent entry into the programme.
  • Their own prior knowledge or interest and self-teaching.
  • Availability of high levels of technology in and of itself.
  • A change of role or responsibility (e.g.: promotion) within school which required more attention to ICTs.
  • Informal learning and mentoring from students or their own children, or from individual colleagues who were not Lead Teachers or cluster Facilitators.
  • Other PD initiatives within the school that were not directly part of the ICTPD programme but which fostered the occasion to use ICTs with classes (e.g.: EHSAS initiatives).
  • Professional learning from conferences.
  • Completing ICT or learning related courses for formal qualifications at a local university, polytechnic or the like.

In a triangulating question, teachers were asked how significant the programme had been in contributing to their overall development as professionals. While only a small minority (11%) of teachers said it had not been significant to their overall development, well over half (59%) of teachers indicated that the programme had been either 'significant' or 'very significant' as a factor in their general development as teachers. Primary teachers were much more likely than secondary teachers to attribute general professional significance to the programme (70% and 42% respectively).

Teachers' Evaluations of the Delivery of the ICT PD Programme

Teachers were asked to comment on the particular aspects of their programmes that they most and least appreciated. The feedback in relation to these most and least appreciated aspects are summarised in the exemplary statements in the boxes below, categorised by:

  • Programme content. Statements by teachers about the content of their ICT PD programmes and their particular cluster models centred around learning new technical skills, gathering and sharing practical ideas for ICT integration into classroom curriculum programmes, and learning about principles of effective teaching and learning in general. Prominent among critical comments on programme content were those about tailoring the PD better to the needs of individuals or doing more effective needs analysis, providing greater variety in ICT session content and activities, and improving the connection and balance between skills acquisition and quality classroom practice with ICTs as a focus of workshops.
Sample of appreciative comments on content Sample of critical comments on content
  • "Learning about the thinking maps and using Solo Taxonomy as a tool for assessment (and using it as incentive for more quality work from children!"
  • "That the programme covered a wide range of skills and topics, starting at a very basic level and moving on through to a higher level."
  • "Different resources you can use on the laptops, different thinking skills you can use in the class i.e. thinkers keys, HOMS, thinking hats..."
  • "Learning new skills."
  • "Ideas for integrating ICT into classroom."
  • "Exposure to new software and programmes."
  • "New ideas, new software and resources."
  • "Different learning activities."
  • "Lead teacher workshops that were about something you would never use in the classroom or school."
  • "PD was directed too much at IWB teachers."
  • "Late after hours courses of a highly technical nature."
  • "The focus was so much on classroom practice that our administration needs have not yet been covered"
  • "Action research was added pressure at times."
  • "Being made do skills tests that I could already do - complete waste of time."
  • "Being given a limited and unrelevant (for me) list of school-based workshops to attend, rather than doing workshops that I really wanted/needed to do."

 

  • The skills and effectiveness of the Facilitators and other leadership elements. As has been the case in most cohorts, many of the comments highlighted the pivotal role of able, knowledgeable and professionally skilled facilitators. Teachers required their facilitators to be technologically and pedagogically knowledgeable, willing to adopt a supportive, empathetic attitude and always 'available'.
Sample of appreciative comments on facilitation and leadership Sample of critical comments on facilitation and leadership
  • "Our current facilitator being available and willing to be hands on and help in school with staff and students."
  • "Help from the coordinator in establishing the class blog."
  • "Having the ICT specialist available to teach, inspire and walk along side me so I gained first hand knowledge."
  • "Motivation through a super tutor and giving me confidence to know I can do this too!"
  • "The accessibility of our cluster leader - and his expertise."
  • "Individual teachers who trained, gave good ideas for relevant teacher practice."
  • "The ongoing PD support and motivation given by the adviser."
  • "Support, inspiration and ideas from our facilitators."
  • "As ICT coordinator it can be difficult to get everyone on board in your school!"
  • "Need to accommodate different levels of skills across the cluster"
  • "Lack of time for staff PD within school PD structures, due to so many other curriculum needs."
  • "Not putting enough focus on good pedagogy using ICT"
  • "The key note speaker at our cluster ICT conference."
  • "Staff not willing to participate in IT PD"
  • "Extra work load."
  • "One tutor knew it all and assumed that we knew it also, where most of us were very hesitant with new ideas so they ended up getting lost."
  • Programme delivery. The organisational aspects of programme delivery appreciated most were the personalised attention many facilitators were able to build into their programmes, small group sessions, one to one help, in-classroom visits, arranged visits among schools, and practical (i.e.: hands on and/or interactive) workshops. Many of the least appreciated aspects to do with programme design and delivery were about timing arrangements for training and other PD activities, about the need to balance workload and still have time to practise what they have learnt, about reducing paperwork and compliance requirements, and about ongoing access to equipment and facilities.
Sample of appreciative comments on programme delivery Sample of critical comments on programme delivery
  • "PD sessions and hands on activities."
  • "Having someone to ask, knowing there is an expert there if you needed help."
  • "The small group seminars that focused on a limited number of skills."
  • "Workshops and conferences to upgrade skills."
  • "Variety of after school holiday courses that were short, practical and available to everyone."
  • "One on one tutorials based on personal need."
  • "Practical hands-on sessions like how to use the data display, how to use the department distribution files."
  • "Step by step guidelines and assistance."
  • "The milestone reporting requirements!!"
  • "Inservice not advertised to our school in time/PD at other schools during busy time of year? TIME for 1-to-1"
  • "Stress caused by termly commitments to showing proof of our goals"
  • "Time to sit at the computer to get to play!!!! Not enough time!!!!"
  • "PD held in the holidays or after school."
  • "All the holiday workshops."
  • "The difficulty to have access to computer suite at times -- many classes wanting it at same time or the system down!"
  • The teachers' involvement in various 'Learning communities' as part of their programmes. Teachers often commented on the various opportunities that accompanied the programmes to share learning and experiences both within and beyond the cluster, and the contribution these made to developing a 'community of practice' around the programme. Most prominent among these opportunities were the national conferences, outside/overseas visitors and experts, meeting other teachers from other schools, and sharing expertise and knowledge with colleagues through various 'celebrations' and events both within and outside the cluster context. Critical comments about the community aspects tended to be about a perceived lack of real inter-school collaboration in the cluster, the levels of coordination and organisation present in the cluster, working in mixed ability groups, and, seeing the same rather than new or innovative things happening in watch others' schools.
Sample of appreciative comments on learning community elements Sample of critical comments on learning community elements
  • "Being able to network with other lead teachers. The chance to share ideas, resources and strategies has been invaluable. As well as working with a facilitator to plan and approach problems. And opportunities to attend conferences - this has been incredible."
  • "Being able to learn, listen, understand concerns and share ideas with others."
  • "The collegial nature of the project. Teachers form all the schools getting together to learn from one another. Children and teachers presenting their learning at conferences. The sharing has been fantastic."
  • The ability to link with others in the network to find out what I need to know."
  • "Sharing ideas with other teachers in the cluster."
  • "Meeting other teachers and getting more personal skills together."
  • "Collegiality between schools."
  • "Sharing cluster days. Visiting other schools to see best practice."
  • "The ICT( Northern Bays and ULearn) conferences really helped to innovate."
  • The cluster meetings, although sounding good in theory, from what I experienced were very disorganised and ended up being a waste of time.  
  • "frustration of working with other schools when we all have different priorities when organising collaborative events for teachers or students."
  • "Contrived get together with other schools does not work. There has to be a common interest for productive learning and sharing to take place."
  • "Forced or unnatural sharing of finished products."
  • "Seeing repeated activities at different schools."
  • "working with a partner at different level of confidence to me meant I did not gain as much as I would have liked from face to face contact with [facilitator]."


Although arguably out of scope of the cluster programmes' aims, there was a comparatively large number of comments wanting improved technical infrastructure and support in schools to avoid ICT failure and to enable teachers to better put into practice what they have learnt. Comments along these lines included: "Lack of computers to actually implement these ICT ideas", "sometimes being shown something that would be great to use in your class but not being able to follow through due to having a different platform in the school", "not enough time to consolidate things learnt and put them into practise", and so on.

Most of these paralleled the 'concerns' about integrating ICTs discussed in an earlier section.

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065