Outcomes for teachers and students in the ICTPD School Clusters Programme 2006-2008: A national overview
This report focuses on the effectiveness of the 2006-2008 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first six ICT PD programmes. It is the last report of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.
Author(s): Selver Sahin and Vince Ham, CORE Education. Report prepared for the Ministry of Education.
Date Published: May 2010
This report is available as a download (please refer to the 'Downloads' inset box). To view the individual chapters please refer to the 'Sections' inset box. For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
The focus of this particular report is on the effectiveness of the 2006-8 cohort of ICT PD School Clusters programmes and, as such, supplements previous evaluations of the first six ICT PD School Clusters Programmes submitted to the Ministry in 2002, 2004, 2005, 2006, 2007 and 2008.
As outlined in the performance agreements between the Ministry and the ICT PD clusters, the ICT PD School Cluster programmes in New Zealand are aimed at increasing teachers' ICT confidence, skills and pedagogical understandings of ICTs, fostering quality learning communities, and increasing the frequency and quality of the integration of ICTs to support effective classroom teaching and learning. The research reported here takes these performance criteria as its starting point to provide a national overview of the programme's impact on teachers, teaching and classroom learning in New Zealand, by means of an analysis of 2,674 responses from participating teachers to the End of Project survey.
The overall finding of the study is that the 2006-8 ICT PD programme had a marked effect on the teachers and students in cluster schools with respect to its key goals. There were high levels of goal achievement reported by participants, and marked increases or changes with respect to the relevant Ministry objectives as outlined in cluster performance agreements. At the national level, the programme achieved its overall goals of: significantly increasing teachers' skills and confidence with ICTs, improving teachers' understandings of the role of ICTs in teaching and learning, and providing quality ICT-mediated learning experiences for students. The 2006-8 programme achieved levels of participant satisfaction that were somewhat lower than the preceding two (2004-6 and 2005-7) cohorts but achieved overall increases in skills, confidence, understanding and regular or routine classroom/student usage of ICTs for learning in orders of magnitude at least similar to those of earlier cohorts.
We note also that, in regard to a range of outcomes, the ICT PD programme continues to have a greater impact among primary teachers and schools than among their secondary counterparts.
Specifically, we found that:
- The majority of teachers expressed moderate to high levels of satisfaction in terms of goal achievement at the end of the programme. Participant satisfaction varied somewhat by sector and length of time in the programme, with primary teachers and those who had been in the PD programme for most of the 3 years stating higher levels of satisfaction and goal achievement than secondary teachers and those who were in the programme for only a few months or up to a year.
- The programmes were seen by the majority of participants as having been a 'significant' event in their overall development as teachers, contributing well beyond any ICT-specific issues of increased technical skill, to encompass improved understandings in relation to teaching and learning more generally. Over two-thirds (68%) of teachers indicated that the programme had contributed new ideas about teaching and learning, including 10% who felt that the programme had provided them with a whole new approach to teaching and learning. Primary teachers were rather more likely to see the programme as contributing new insights and ideas than secondary teachers.
- Teachers generally appreciated all of the ways the programmes were delivered and the knowledge and expertise of particular facilitators. The content of the programmes as well as the availability of opportunities to share ideas and problems and reflect and produce solutions together on their use of ICTs for teaching and learning purposes were among the most appreciated aspects of the programme that teachers reported.
- At the end of the programme, the greatest persisting concerns for teachers with regard to their professional use of ICTs were a lack of student access to equipment, a perceived lack of time to keep up to date with the range of ICTs available, technical reliability, and some concern about their continuing need for PD after the programme's formal end point.
- There was a marked increase in teachers' reported ICT skills over the period of the programme. Teachers' reported skill levels on entry to the programme were generally moderate to high already, though somewhat variable across different ICTs. By the end of the programme the great majority of teachers reported moderate or high skill levels across the whole range of educationally useful ICTs measured.
- There was also a marked and significant increase in teachers' confidence about their professional use of ICTs over the period of the programme, both in terms of their confidence as personal users and in relation to students using ICTs in their classes. Teachers reported that on entry the great majority of them had been either 'anxious', 'not confident' or 'neutral' about their professional use of ICTs; with female and primary teachers being less confident than male and secondary teachers. By the end of the programme over three quarters of all the demographic groups studied were reporting moderate or even high levels of such confidence. Moreover, the longer teachers were in the programme the greater was the extent of their gain in confidence. Throughout the programme levels of confidence as personal users remained higher than those related to classroom use of ICTs.
- The reported effect of the ICT PD programme on teachers' classroom practices was variable from teacher to teacher and from school to school, but substantial overall. Over a third of teachers acknowledged that over the programme period their classroom practices had changed to 'a large extent' or 'completely', while 81% felt their practices had changed at least to 'some extent'. Female and primary teachers reported greater change in this regard than male or secondary teachers.
- Participants reported that, on entry to the programme, they were already generally positive about the value of using ICTs for teaching and learning. At the end of the programme, they showed a similarly positive disposition towards the value of ICTs in the teaching and learning process, but many stated that they now had a clearer conception of how its educational value might be judged. The most prominent effects of the programme in terms of developing teacher understandings about learning were expressed as: a better understanding of student-centred teaching and learning, increased knowledge of teaching and learning theories, styles, and approaches; and challenging their pedagogical perspectives through sharing and discussion.
- There was a marked and significant increase in teachers' use of ICTs with their classes as a result of the programme. Approximately two thirds (62%) of teachers had either never used ICTs with classes prior to the programme, or had only used them once or twice a year. By the final year of the programme, over three quarters (77%) of participating teachers reported that their students were using ICTs on a routine or regular basis.
- Primary teachers reported using ICTs in a greater proportion of their units of work than secondary teachers. The other significant predictors of increased classroom usage of ICTs by teachers in the clusters were their rising levels of confidence with and about the technology, and the length of time they were actively engaged in the PD programme.
- The surveys provide some proxy evidence of the conscious alignment of ICT-based classroom activities to a wide range of student learning outcomes. Teachers tended to report the learning outcomes from student use of ICTs in terms of: increased student-centredness in lessons, increased student motivation, coverage of a wide range of curriculum topics and objectives, student acquisition of ICT skills, and increased opportunity for learning activities which promoted communications skills, inquiry skills, high order thinking, creativity and a range of social skills. There were no statistically significant sector differences in relation to increased student use of ICTs for higher order thinking and critical thinking skills, but primary students were more likely to engage in frequent use of ICTs with classes than secondary students for all of: creative activity, curriculum content acquisition, information gathering or processing, collaborative learning, motivation and technical skill acquisition.
- The largest proportion of ICT-based student activities reported by teachers related to the English Learning Area (20%), followed by Mathematics (17%), Social Science (13%) and Science (11%).
- In most respects, the effects and trends listed above for the 2006-8 ICT PD national cohort are similar in both nature and magnitude to those reported for the earlier cluster cohorts. In general, the same effects, of a similar size, have been identified in all cohorts. The only notable exceptions or differences between the 2006-8 cohort and earlier cohorts in terms of the programme effect or size were:
- The skill levels of teachers in the 2006-8 cohort on entering the programme, and also on exiting it, were generally higher than those of the earliest cohorts but similar to those reported for the 2004-6 and 2005-7 cohorts that immediately preceded it. Skill levels, both on entry and exit, as well as programme impacts on skills, are 'flattening off' as successive cohorts begin and finish their programme with higher skill levels across the range of ICTs used in education.
- There were some cohort differences in terms of the learning outcomes of ICT use by students in both the 2005-7 and 2006-8 cohorts compared to earlier cohorts. There is a clear tendency for students in these two cohorts to show higher frequencies of use of ICTs than previous cohorts with respect to all of: static presentation (mostly word processing and slide shows), problem solving activities (mostly through spreadsheet use), information processing activities (mostly through Internet use), online communication (email, social software), and curriculum practice activities (mostly games, Drill and Practice or interactive/multimedia tutorials).
- Finally, we note that levels of goal achievement and meeting of expectations were lower than in the 2004-6 and 2005-7 cohorts, but still higher than in the 2003-5 cohort, across all of the groups of goals identified.