National trends in the ICTPD School Clusters Programme 2003-2005
This report is part of an ongoing evaluation of the Information and Communication Technologies Professional Development (ICTPD) initiative. It covers the 2003-2005 cohort of ICTPD clusters and makes comparisons with previous evaluations.
Author(s): Vince Ham, Hasan Toubat and Sandra Williamson-Leadley, CORE Education.
Date Published: December 2006
This research report is submitted to the Ministry of Education as part of an ongoing evaluation of the Information and Communication Technologies Professional Development (ICTPD) teacher professional development initiative announced in the strategy documents Interactive Education: An Information and Communication Technologies Strategy for Schools (Ministry of Education, 1998) and Digital Horizons: Learning Through ICT: A Strategy for Schools (Ministry of Education, 2001, Revised 2003).
This report on the 2003-2005 cohort of ICTPD cluster teachers supplements, and makes comparisons with, the evaluations of the first three ICTPD School Cluster Programmes submitted to the Ministry in 2002, 2004 and 2005.
As outlined in the performance agreements between the Ministry and the ICTPD clusters, the ICTPD School Cluster programmes in New Zealand are aimed at increasing teachers' ICT skills and pedagogical understandings of ICTs, at increasing the use of ICTs for professional and administrative tasks in schools, and at increasing the frequency and quality of the use of ICTs in schools to support effective classroom teaching and learning.
The research reported here takes these performance criteria as its starting point to provide a national overview of the programme's impact on teachers, teaching and learning in New Zealand, by means of an analysis of the results of a pre- and post- survey of c.1,400 participant teachers.
The overall finding of the study is that the 2003-2005 ICTPD programme had a marked and significant effect on the teachers and students in cluster schools with respect to all of its key goals. There were high levels of goal achievement reported by participants, and marked increases or changes with respect to the Ministry's objectives as outlined in cluster performance agreements. At the national level, the programme seems to have achieved its overall goals of significantly increasing teachers' skills and confidence with ICTs, improving administrative efficiency in schools, improving teachers' understandings of the role of ICTs in teaching and learning, and providing quality ICT-mediated learning experiences for students. The programme for the 2003-2005 cohort seems to have increased teachers' ICT skill, confidence and understandings, and to have increased routine student use of a range of ICTs for learning in classes, in orders of magnitude at least similar to those of earlier cohorts.
While stating these as general effects, we note that overall the ICTPD programme continues to have a greater impact among primary teachers than among secondary teachers.
Specifically, we found that:
- The great majority of teachers expressed high levels of satisfaction and goal achievement at the end of the programme. Participant satisfaction, while high across the board, varied somewhat by sector and length of time in the programme, with primary teachers and those who had been in the PD programme for most of the 3 years stating higher levels of satisfaction and goal achievement than secondary and those who were in the programme for only a few months. The great majority of participants' goals related to the acquisition of technical skills, gaining ideas for ICT-based teaching/learning activities, increasing use of ICTs for school administration, and improved understanding of teaching and learning generally.
- The programmes were seen by the majority of participants as having been a 'significant' event in their overall development as teachers, contributing well beyond any ICT-specific issues of increased technical skill, to encompass improved understandings in relation to teaching and learning more generally. Some 60% of the teachers indicated that the programme had contributed new ideas about teaching and learning, including 8% who felt that the programme had provided them with a whole new approach to teaching and learning. The rest said that the programme had played a more 'confirming' role, consolidating current ideas/understandings about teaching and learning. Primary teachers were rather more likely to see the programme as contributing new insights and ideas in this regard than secondary teachers.
- Teachers generally appreciated all of the ways the programmes were delivered and the knowledge and expertise of particular facilitators. The content of the programmes, being for most programmes a mix of skills development, classroom ideas and principles of effective teaching and learning generally, was seen as relevant and effective. Prominent among the most appreciated aspects of the programmes too, were the various opportunities provided to share ideas and problems and reflect together on their use of ICTs, both within clusters and through the various regional and national conferences, suggesting that a learning community focused on critical reflection of classroom ICT practice is continuing to develop.
- At the end of the programme the greatest concerns for teachers with regard to their professional use of ICTs were a lack of student access to equipment, a perceived lack of time to keep up to date, lack of technical support, and some concern about the apparently never-ending need to self-upskill in using new software packages and technologies as they become available.
- There was a marked and significant increase in teachers' ICT skills over the period of the programme. Teachers' reported skill levels on entry to the programme were generally moderate, though still higher than those for previous cohorts, and still variable across different ICTs. There was a distinct tendency for males and secondary teachers to report higher entry skill levels than female or primary teachers, though such differences had reduced by the end of the programme.
- By the end of the programme impressive majorities of teachers were reporting moderate or high skill levels with regard to file management (92%), basic computer operation (95%), word processing (97%), graphics (77%), the Internet (94%) and telecommunications (95%). Lower but still relatively high rates of moderate and high skill were reported with regard to multimedia packages (72%), spreadsheets (72%) and databases (58%), and these still show significant increases compared to entry point proportions. The greatest proportional increase in skills was reported in relation to spreadsheets, which is most likely a reflection of the significantly higher than usual proportion of secondary teachers in the cohort.
- There was also a marked and significant increase in teachers' confidence about their professional use of ICTs over the period of the programme, both in terms of their confidence as personal users and in relation to students using ICTs in their classes. On entry the great majority of participants were either 'not confident' or neutral about their professional use of ICTs, female teachers and primary teachers being less confident than male and secondary teachers. By the end of the programme the great majority of all the demographic groups studied were reporting moderate or even high levels of such confidence. Moreover, the longer teachers were in the programme, the greater was the extent of their gain in confidence, and throughout the programme, levels of confidence as personal users were higher than those related to classroom use of ICTs.
- There was a marked and significant increase in teachers' use of ICTs for lesson planning and preparation and for general school administration over the period of the programme.
- The effect of the ICTPD programme on teachers' classroom practices has been substantial. Over a quarter acknowledged that over the programme period their classroom practices had changed to 'a large extent' or 'completely', while 49% felt their practices had changed to some extent. Female and primary teachers reported greater change in this regard than male or secondary teachers.
- On entry to the programme teachers were already generally positive about the value of using ICTs for teaching and learning, but were perhaps ambivalent or uncertain as to the exact nature of that benefit or value. At the end of the programme they showed a generally more positive disposition towards the value of ICTs in the teaching and learning process, and many stated a clearer conception of how its educational value might be judged. The most frequently reported effects of using ICTs with classes on their teaching included: teaching with increased confidence and enthusiasm, expanding their repertoire of teaching techniques, using a wider range of activities and catering for a greater range of student need across a broader range of curriculum objectives. The most prominent effects of the programme in terms of developing teacher understandings about teaching and learning were expressed as: a better understanding of student-centred teaching and learning, getting new ideas about establishing a resource-rich learning environment, increased knowledge of teaching and learning styles or approaches, increased awareness of 'quality' in teaching and learning, challenging pedagogical perspectives through sharing and discussion, and the accumulation of a variety of practical classroom ICT-based activities.
- There was a marked and significant increase in teachers' use of ICTs with their classes as a result of the programme. The great majority of teachers had either never used ICTs with classes prior to the programme, or had only used them once or twice a year. By the final year of the programme the majority of participating teachers reported that their students were using ICTs on a regular basis (i.e.: in most or all units of work over the year).
- While the proportion of units of work involving student use of ICTs increased significantly during the programme, the range of ICTs used by students for learning remained fairly limited. Word Processing (including digital images and pictures), the Internet, and slideshow presentations are by far the most frequently used ICTs in classes.
- Primary teachers reported using ICTs in a greater proportion of their units of work than secondary teachers. The other significant predictors of increased classroom usage of ICTs by teachers in the clusters were their rising levels of confidence with and about the technology, and the length of time they were actively engaged in the PD programme.
- In terms of the downstream effects of the ICTPD programme on enhancing the 'quality' of student learning experiences, the surveys provide some proxy evidence through such things as the learning outcomes reported by participating teachers as they observed students engaged in ICT-based activities, the variety of curriculum goals met and Essential Learning Areas covered as students engaged in ICT-based activities, and the range of ICTs used by participating teachers and their students.
- In terms of the impact of using ICTs with classes on students, teachers tended to conceive of the benefit potential of ICTs in terms of increased student-centredness in lessons, increased student motivation, coverage of a range of curriculum topics, student acquisition of ICT skills, and increased opportunity for learning activities which promoted communications skills, enquiry skills, high order thinking, creativity and a range of social skills.
- In cluster schools, students used ICTs most often to achieve Language, Mathematics or Science objectives.
- In most respects the effects and trends listed above for the 2003-2005 ICTPD national cohort are similar in both nature and magnitude to those reported for the three earlier cluster cohorts. In general, the same effects, of a similar size, were identified for all cohorts. The only notable exceptions or differences between the 2003 cohort and earlier cohorts in terms of effect or effect size were:
- The skill levels of teachers in the 2003 cohort on entering the programme, and also on exiting it, were generally higher than those of earlier cohorts. This would imply some personal upskilling effect in New Zealand schools generally, independent of the cluster programme. However, there also seems to have been a significant '2003 cohort effect' in regard to skills in that the gains reported also seem higher than for the previous two cohorts. By contrast, entry levels of the 2003 cohort in terms of classroom usage of ICTs were not markedly higher than for earlier cohorts, except in the case of Internet use. For this cohort, as for previous cohorts, any prior upskilling outside the ICTPD programme did not translate into increased use for teaching and learning until well into the ICTPD programme.
- There were some cohort differences in terms of the learning outcomes of ICT use by students in the 2003 cohort compared to earlier cohorts. Students in the 2003 cohort seemed, for example, to engage in problem solving activities (mostly through spreadsheet use) and information processing activities (mostly through Internet use) more than in the previous two cohorts, and curriculum practice activities, such as Drill and Practice, less.
- The 2003 cohort also seems to have focused rather more than the previous cohort on generic cognitive skills as identified learning outcomes of students' ICT use, and rather less on the achievement of specific curriculum objectives. However, we still note that the general trend for student use of ICTs to be much greater for some categories of learning outcomes than others, itself remains fairly constant across the cohorts.
- The 2003 cohort had a much higher proportion of secondary teachers and schools in it than earlier clusters. However, the only effect difference this seems to have generated was an increased use of spreadsheets for problem solving activities by comparison with other clusters. The numerous sector-based differences reported for other cohorts remain as sector differences in the 2003 cohort results as well.
Where to find out more
Education Data Requests
If you have any questions about education data then please contact us at:
Email: Requests EDK
Phone: +64 4 463 8065