Outcomes for teachers and students in the ICTPD School Clusters Programme 2005-2007: A national overview
This report focuses on the effectiveness of the 2005-2007 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first five ICT PD programmes. It is part of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.
Author(s): Vince Ham, CORE Education. Report prepared for the Ministry of Education.
Date Published: July 2009
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Section 3: Effects of the 2005–2007 ICTPD Programme on teachers
The main programme-level goals of the ICT PD school clusters initiative with regard to the effects on teachers themselves were:
Increased skills among teachers across a range of educationally useful ICTs. Increased teacher confidence about their personal use of ICTs and about the use of ICTs with and by students in classes. Improved understandings of the roles that ICTs can play in improving classroom teaching and learning. Engagement of teachers in critically reflective communities of practice through and about ICTs.
Nationally, there was a clear and significant increase/improvement in relation to all of these indicators over the period of the programme.
Teachers' ICT skills
Over the period of the programme teachers' skills in using ICTs increased significantly, especially, but not exclusively, for those who at the beginning of the programme had rated their skill levels as either very low or non-existent. As can be seen in Table 5, below, there were significant reductions across the board in the proportions of teachers who rated their skills as low or non-existent, and significant increases in the proportion who rated their skill level as high or very high. This was the case even with regard to ICTs such as word processing where the great majority of teachers entered the programme already with reasonable pre-existing levels of competence. By the end of the programme very solid majorities of teachers felt they had moderate or high skills with regard to file management (94%), basic computer operation (94%), word processing (98%), Internet (94%) and telecommunications (96%). Lower but still relatively high levels of end of programme competence were reported with regard to graphics (81%), spreadsheets (67%), databases (57%) and multimedia packages (72%), though these still show significant increases compared to entry point proportions. The increases in teachers' skill levels during the programme was considerable across all ICTs, but it was most notable in relation to graphics and multimedia applications.
|File Management: Before||29%||33%||37%|
|File Management: After||60%||34%||6%|
|Basic Operation: Before||21%||41%||39%|
|Basic Operation: After||49%||45%||6%|
|Word Processing: Before||43%||36%||21%|
|Word Processing: After||72%||26%||2%|
|Multimedia Presentation: Before||9%||19%||71%|
|Multimedia Presentation: After||35%||37%||28%|
Demographic analysis of these results show some continued relationship between gender and teachers skill levels in favour of male and secondary teachers at the start and the end of the programme. The gender differences reduced over the period of the programme but remained significant across virtually all ICTs even at the end. The sector differences all but disappeared the end of the three years, the only notable remaining differences being that secondary teachers reported higher skill levels with basic operations and spreadsheets, and primary teachers reported higher skill levels in the use of graphics.
The impact of the programme on teachers' skill levels was clearly significant across the full range of educationally useful skills measured, though skill with certain ICTs such as spreadsheets among secondary teachers and multimedia and graphics packages among primary teachers seem to have been emphasized in the different sectors.
Teacher confidence about the use of ICTs
Changes in the confidence of teachers about using ICTs were investigated with regard to two elements: their confidence as personal users of ICTs, and their confidence about using ICTs with classes of learners. Both of these increased significantly over the period of the programme from what were moderate and low levels of confidence respectively on entry.
At the end of the project, many teachers reported that at the beginning of the project they had been less than confident as personal users of ICTs. Seven percent of teachers classified themselves as having been 'anxious' and 24% of them were 'not confident' in this regard. By the end of the programme less than 1% of the teachers said they were 'anxious' and only 1% were still 'not confident' about the personal use of ICTs. By the end of the programme over four-fifths (84%) of teachers stated that they had become either 'confident' or 'very confident' about their personal use of ICTs.
Teachers' confidence about ICT use with classes also increased significantly during the programme. Teachers reported that on entry, they had been even less confident about using ICTs with classes than they had been about their personal use. At that point 14% of them self-identified as 'anxious' about this, and 30% identified themselves as 'not confident'. By the end of the programme, however, the percentage of 'anxious' or 'not confident' teachers had dropped from 44% to 3%. Correspondingly, the percentage of 'confident' or 'very confident' teachers had increased from 28% to 77%. Figure 1 illustrates the change in teachers' confidence about the use of ICTs with classes.
Figure 1: Teachers' confidence about using ICTs with classes before and after the ICTPD programme
As was also the case for confidence gains in relation to personal use, these confidence gains in relation to classroom use of ICTs were related to all of gender, sector and length of involvement demographics. Female teachers made significantly more confidence gains than male teachers (X2>35; df=3; p<.001), primary teachers reported significantly more gains in confidence about classroom use than secondary teachers (X219.28; df=3; p<.001), and confidence gains were also positively correlated with the length of time teachers had actively participated in the programme (X2=22.89; df=8; p<.001) (See Table 6, & Figure 2). The longer teachers took active part in ICT PD the greater the increase in their confidence about using ICTs with classes.
|Change in Confidence||Female||Male||Primary||Secondary|
Figure 2: Increases in individual teachers' confidence about use of ICTs with classes, by length of active involvement in programme
Teachers' understanding of the role of ICTs in teaching and learning
Teachers in this cohort generally had positive views on the role of ICTs in education. At the end of the programme many of the teachers' still felt there were a number of constraints on their effective implementation of ICTs into teaching and learning, but their views had, if anything, become even more positive about their potential in other respects, both in improving their teaching and in fostering effective learning.
To address this indicator of the impact of the ICT PD programmes on teachers, we asked teachers what they saw as the benefits of ICTs in teaching and learning on the basis of their experience with ICTs during the programme; what concerns they had about the incorporation of ICTs into their teaching programmes; how, if at all, incorporating ICTs had changed their ways of teaching, and the contribution the programme had made to their understanding of teaching and learning in general. Their responses to these questions give an overview of what the teachers' own broad understandings were at the end of the programme in relation to the role of ICTs in education and how widely spread such views were among them.
Benefits of integrating ICTs into teaching and learning
The qualitative analyses of previous cohorts reported benefits of integrating ICTs into their teaching and learning programmes, show that these may be grouped into about ten key categories (Ham et al 2005, 2006, Ham 2007). These are a mixture of teacher or teaching-oriented benefits and learner or learning-oriented benefits, with, from the teachers in this cohort, a heavy emphasis on the latter. The categories of benefit and their distribution for this cohort are:
|Teaching with more confidence and enthusiasm|
|Enhancing/expanding their own teaching skill set and pedagogical knowledge|
|Using a wider range and variety of teaching-learning activities in class|
|Allowng more authenticity, real-worldness and relevance to children's lives in teaching and learning tasks,|
|Making learning for students more engaging and interesting|
|Facilitating more efficient learning of specific curriculum content and objectives, easier access to information improved presentation etc.|
|Enabling a focus on thinking skills (especially higher order thinking and metacognition)|
|Encouraging collaborative of cooporative enquiry, contributing to social skill development,|
|Making learning more personalised or individualised.|
|Fostering more independence and agency in student learning.|
Technical issues, equipment reliability and inadequate access to ICTs for students seemed to account for most of the perceived constraints on ICT use, these being much more significant in peoples' minds than pedagogical issues. Over a third of the teachers, for example, identified significant concerns about access to equipment for students (38%) and reliability, (36%) alongside lack of time (37%) for themselves to become familiar with the range of ICTs available as their main persisting concerns around the use of ICTs with students at the end of the programme (Figure 3). As has also been the case for previous cohorts, many of the teachers at the end of the project felt there was still a need for continuing PD around ICT use, even though the programme had significantly increased and improved their effective practices in this regard, though in comparison with earlier cohorts the proportions of teachers expressing significant concerns at the end of the programme does seem to have reduced across the board.
Figure 3: Teachers' continuing concerns about the use of ICTs with classes
Contribution to wider understandings of teaching and learning more generally
When teachers were asked to describe the ways in which the ICT PD programme had contributed to their understanding of teaching and learning generally, responses ranged from 'no effect' to significantly deeper understandings of learning and pedagogy in general, well beyond the connection with ICTs.
Of all the comments made about the ways in which ICT PD programmes had contributed to teachers' 'understandings' of teaching and learning in general, about a third related to the specific connections they had been able to make about the educational value of ICTs in teaching and learning, rather than what they had learned about teaching and learning 'more generally'. Of those that did refer specifically to the role of the programmes in developing their 'more general' understandings of teaching and learning, about 18% said that there had been little or no contribution to their general pedagogical understandings, beyond the acquisition of practical ideas for integrating ICTs into their classroom programmes.
- "By being able to have students use particular software I appreciate that they learn at a far faster rate than I am able to provide material. It has speeded up the learning process."
- "Students are more involved in goal setting learning intentions assessment."
- "Made me think more about what the students need in their future."
- "More flexibility & student centred research."
- "[It has] extended my knowledge of systems of delivery and understanding and application of student centred learning systems."
- "Much more child centred and cooperative learning approach."
- "Students are more engaged in the learning as a result of the ICTs. Students are now creating and expressing themselves in a wider range of ways."
About half the comments of teachers, however, did identify such deeper or wider understandings and identified a number of key ways in which the programme had supplemented, challenged and even changed those understandings.
Many of these (c. 11% of total comments) involved teachers reporting that they now had a better understanding of student-centred teaching and learning, or that in some way their teaching had become more student-focused or more relevant to students as a result of the programme.
An even larger proportion (c. 30%) stated that they had significantly increased their knowledge of different teaching/learning styles and theories, or were enabled to make clearer connections between their day to day practice and the various learning and teaching theories and models outlined in the programmes. Most prominent among the particular theories and models identified were 'enquiry learning', 'learning styles', 'collaborative and/or cooperative learning', and various taxonomies of 'thinking skills' (Bloom's, SOLO etc.).
- "awareness of use of pedagogies eg Brain based learning/inquiry/ability to link chn to world outside classroom with greater relevance and immediacy."
- "Provides increased opportunities for co-operative learningSupports a range of learning styles."
- "[I'm] making use of learning processes multiple intelligence theory."
- "It was good to read some up to date articles on learning theory."
- "I have added to existing knowledge on Inquiry learning, have a better understanding of our research can impact teaching and learning."
- "Being introduced to a variety of thinking and learning strategies such as Habits of Mind, SOLO, and the Fish Philosophy has encouraged me to become more reflective about what is happening in class and it has rubbed off on the children as well."
Thirdly, there was also a substantial proportion who commented on the programme as challenging and changing their pedagogical perspectives and understandings, either through the content of the PD programme, or, more often, through the opportunities it provided for sharing and critical discussion with colleagues, outside 'experts', and so on. Many of these comments spoke of the teachers adopting a more 'critical' or more 'reflective' approach to their teaching as a result of their new awareness of pedagogical and learning theories and research.
Finally, we note that there were two elements of significant sector difference in the nature of the comments on wider deeper understandings gained from the programmes. The first is that primary teachers were twice as likely as secondary to comment on increased understanding of specific learning theories. The second is that secondary teachers were twice as likely as primary to state that the programme had had little or no effect on their pedagogical understandings in the broader sense.
- "It has helped me become a more multi media teacher."
- "Evaluating and reflecting on why we teach the way we do has really developed my understanding of teaching and learning."
- "Over the last 3 years I have been challenged to think about; childrens learning, the inquiry process, thinking skills, how the brain works.... I wonder who will challenge my thinking about my teaching and classroom programme now the contract has finished?"
- "My teaching and planning has changed to automatically incorporate ICT where appropriate."
- "Reading the thoughts and views of global web2.0 experts and communicating directly with them has given me a better perspective on the learning required for children of the 21st century."
- "The course/conference PD provided ideas for reflecting on teaching and learning."
Teachers' classroom practices
When teachers were asked about the extent to which their classroom practices had changed as a result of participation in the ICT PD programme, over a third of them indicated their classroom practices had changed to 'a large extent' or 'completely', and just under half of them (48%) said their classroom practices had changed 'to some extent'. Only a very small group (3%) said no change had occurred in their classroom practices at all (Table 8).
|Extent of change in classroom practices||%|
|Not at all||3%|
|To some extent||48%|
|To a large extent||33%|
There were sector (but not as in previous cohorts, gender) related differences in teachers' responses to this question. Primary teachers stated higher levels of change in their classroom practice than secondary teachers (X2>35, df=4, p<0.001). Whereas 43% of primary teachers indicated a complete or large change happened to their classroom practices, 25% of secondary teachers respectively indicated the same levels of change (Table 9). While the great majority (86%) of teachers reported at least some element of change in their classroom practices, secondary teachers were almost twice as likely as primary teachers to be in the group of teachers reporting little or no change in classroom practice as a result of the programme.
|Levels of Change in Classroom Practice||Female||Male||Primary||Secondary|
|Not at all||3%||5%||3%||4%|
|To some extent||48%||48%||44%||57%|
|To a large extent||34%||31%||37%||23%|