Outcomes for teachers and students in the ICTPD School Clusters Programme 2006-2008: A national overview
This report focuses on the effectiveness of the 2006-2008 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first six ICT PD programmes. It is the last report of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.
Author(s): Selver Sahin and Vince Ham, CORE Education. Report prepared for the Ministry of Education.
Date Published: May 2010
This report is available as a download (please refer to the 'Downloads' inset box). To view the individual chapters please refer to the 'Sections' inset box. For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Section 3: Effects of the 2006-08 ICTPD Programme on Teachers
The main programme-level goals of the ICT PD school clusters initiative with regard to the effects on teachers themselves were:
- Increased skills among teachers across a range of educationally useful ICTs.
- Increased teacher confidence about their personal use of ICTs and about the use of ICTs with and by students in classes.
- Improved understandings of the roles that ICTs can play in improving classroom teaching and learning.
- Engagement of teachers in critically reflective communities of practice through and about ICTs.
Nationally, there was a clear and significant increase/improvement in relation to all of these indicators over the period of the programme.
Teachers' ICT Skills
Over the period of the programme teachers' skills in using ICTs increased significantly, especially, but not exclusively, for those who at the beginning of the programme had rated their skill levels as either very low or non-existent. As can be seen in Table 5, below, there were significant reductions across the board in the proportions of teachers who rated their skills as low or non-existent, and significant increases in the proportion who rated their skill level as high or very high. This was the case even with regard to ICTs such as word processing where the great majority of teachers entered the programme already with reasonable pre-existing levels of competence. By the end of the programme very solid majorities of teachers felt they had moderate or high skills with regard to file management (94%), basic computer operation (93%), word processing (97%), Internet (87%) and telecommunications (85%). Lower but still relatively high levels of end of programme competence were reported with regard to graphics (77%), spreadsheets (69%), databases (49%) and multimedia packages (71%), all of these still show significant increases compared to entry point proportions. The increases in teachers' skill levels during the programme were considerable across all ICTs, but were most notable in relation to graphics and multimedia applications.
|File Management: Before||37%||33%||30%|
|File Management: After||60%||34%||6%|
|Basic Operation: Before||27%||41%||32%|
|Basic Operation: After||50%||43%||7%|
|Word Processing: Before||48%||36%||16%|
|Word Processing: After||71%||26%||2%|
|Multimedia Presentation: Before||23%||23%||65%|
|Multimedia Presentation: After||33%||38%||29%|
Demographic analysis of these results shows that although gender differences decreased over the period of the programme, they still remained significant in favour of male teachers across virtually all ICTs, even at its end. The exceptions were word processing, for which female and male respondents indicated the same high skill levels, and graphics, for which female teachers reported higher skill levels.
There were some sector differences both at the beginning and end of the programme. Secondary teachers tended to report higher skill levels on entry to the programme, but at the end of the programme, this trend reversed. While at the end secondary teachers still reported higher skill levels with spreadsheets, primary teachers at the end reported higher skill levels with respect to file management, word processing, graphics, Internet, multimedia and telecommunications.
Teachers' Confidence About the Use of ICTs
Changes in the confidence of teachers about using ICTs were investigated with regard to two elements: their confidence as personal users of ICTs, and their confidence about using ICTs with classes of learners. Both of these increased significantly over the period of the programme from what were moderate and low levels of confidence respectively on entry.
Figure 1: Teachers' confidence about using ICTs with classes before and after the programme (n=2594 Before, 2560 After)
At the end of the project, many teachers reported that at the beginning of the project they had been less than confident as personal users of ICTs. About 4% of teachers classified themselves as 'anxious' and 20% of them were 'not confident' in this regard. By the end of the programme, less than 1% said they were 'anxious', while only 2% reported that they were still 'not confident' about the personal use of ICTs. By the end of the programme, over four fifths (84%) of teachers stated that they had become either 'confident' or 'very confident' about their personal use of ICTs.
Teachers' confidence about using ICTs with classes also increased significantly during the programme. Teachers reported that on entry, they had been even less confident about using ICTs with classes than they had been about their personal use. At that point, 9% of them self-identified as 'anxious' about this, and 28% identified themselves as 'not confident'. By the end of the programme, however, the percentage of 'anxious' or 'not confident' teachers had dropped from 37% to 3%. Correspondingly, the percentage of 'confident' or 'very confident' teachers had increased from 34% to 77%. Figure 1 illustrates the change in teachers' confidence about the use of ICTs with classes
As was also the case for confidence gains in relation to personal use, these confidence gains in relation to classroom use of ICTs were related to all of gender, sector and length of involvement demographics. Female teachers made significantly more confidence gains than male teachers (X2>35; df=3; p<.001), primary teachers reported significantly more gains in confidence about classroom use than secondary teachers (X2>35; df=3; p<.001), and confidence gains were also positively correlated with the length of time teachers had actively participated in the programme (X2>35; df=15; p<.001) (See Table 6, & Figure 2). The longer teachers took active part in ICT PD the greater the increase in their confidence about using ICTs with classes.
|Change in Confidence||Female||Male||Primary||Secondary|
Figure 2: Increases in individual teachers' confidence about use of ICTs with classes, by length of active involvement in programme (n= 2527)
Teachers' Understanding of the Role of ICTs in Teaching and Learning
To address this indicator of the impact of the ICT PD programmes on teachers, we asked teachers what they saw as the benefits of ICTs in teaching and learning on the basis of their experience with ICTs during the programme; what concerns they had about the incorporation of ICTs into their teaching programmes; how, if at all, incorporating ICTs had changed their ways of teaching, and the contribution the programme had made to their understanding of teaching and learning in general. Their responses to these questions give an overview of what the teachers' own broad understandings were at the end of the programme in relation to the role of ICTs in education and how widely spread such views were among them.
Teachers in this cohort generally had positive views on the role of ICTs in education. By the end of the programme many of the teachers' still felt there were a number of constraints on the effective implementation of ICTs into teaching and learning, but their views had, if anything, become even more positive about their potential in other respects, both in improving their teaching and in fostering effective learning.
Benefits of integrating ICTs into teaching and learning
The qualitative analyses of previous cohorts reported benefits of integrating ICTs into their teaching and learning programmes, show that these benefits may be grouped into ten key categories (Ham, 2008; Ham, Toubat & Williamson-Leadley, 2006; Ham, Graham, Toubat & Williamson-Leadley, 2005). They are a mixture of teacher or teaching-oriented benefits and learner or learning-oriented benefits, with, from the teachers in this cohort, a heavy emphasis on the latter. The categories of benefit and their distribution for this cohort are outlined in Table 7.Percentages represent approximate proportions of statements relevant to a given category. (n=2225 respondents provided c.5063 statements. A random sample of 1156 statements were coded)
|Facilitating more efficient learning of specific curriculum content and curriculum objectives, easier access to information, improved presentation etc.|
|Making learning for students more engaging and interesting|
|Fostering more independence and agency in student learning|
|Encouraging collaborative or cooperative inquiry, contributing to social skill development|
|Enabling a focus on thinking skills (especially higher order thinking and metacognition)|
|Allowing more authenticity, real-worldness and relevance to children's lives in teaching and learning tasks|
|Using a wider range and variety of teaching-learning activities in class|
|Enhancing/expanding their own teaching skill set and pedagogical knowledge|
|Teaching with more confidence and enthusiasm|
|Making learning more personalised or individualised|
Technical issues, such as equipment reliability, and inadequate access to ICTs for students were among the most-selected concerns about ICT use, these being much more significant in peoples' minds than pedagogical issues. Over a third of the teachers, for example, identified significant concerns about access to equipment for students (39%), technical reliability, (33%) and lack of time (33%) for themselves to become familiar with the range of ICTs available as their main persisting concerns around the use of ICTs with students at the end of the programme (Figure 3). As has also been the case for previous cohorts, many of the teachers at the end of the project felt there was still a need for continuing PD around ICT use, even though the programme had significantly increased and improved their effective practices in this regard.
Figure 3: Teachers' continuing concerns about the use of ICTs with classes (n = 2521~2566)
Contribution to wider understandings of teaching and learning
When asked whether or not the ICT PD programme had contributed to their understanding of teaching and learning in general terms, over two-thirds (68%) of respondents indicated that the programme contributed new ideas about teaching and learning, including 10% who stated that the programme provided them with a 'whole new approach'. It would also seem that primary teachers were rather more likely to see the programme as contributing new insights and ideas in this regard than secondary teachers (Table 8).
|Provided a new whole approach to teaching and learning||10%||13%||5%|
|Contributed some new ideas about teaching and learning||58%||63%||51%|
|Confirmed current ideas/understandings about teaching & learning||25%||20%||31%|
|Not at all||7%||4%||13%|
When they were asked to describe the ways in which the ICT PD programme made a contribution to their 'understandings', responses1 ranged from the educational value of ICTs to significantly deeper understandings of learning and pedagogy in general, well beyond the connection with ICTs.
Of all the comments made about the ways in which ICT PD programme had contributed to teachers' 'understandings' of teaching and learning in general, about a quarter (23%) related to the specific connections they had been able to make about the educational value of ICTs in teaching and learning, rather than what they had learned about teaching and learning 'more generally'. Almost three-quarters (70%) of the comments that teachers made, however, did identify deeper or wider understandings and identified a number of key ways in which the programme had supplemented, challenged and even changed those understandings.
Thirdly, there was also a substantial proportion (c. 28%) of comments on the programme as challenging and changing their pedagogical perspectives and understandings, either through the content of the PD programme, or, more often, through the opportunities it provided for sharing and critical discussion with colleagues, outside 'experts', and so on. Many of these comments spoke of the teachers adopting a more 'critical' or more 'reflective' approach to their teaching as a result of their new awareness of pedagogical and learning theories and research.
Examples of statements about challenging their pedagogical approach and understandings
- "It has exposed me to a whole new learning environment. I have become a facilitator of learning instead of the one holding all of the knowledge. I would see something in PD sessions and just have a go...initiate stage of an ICT environment."
- "My pedagogy has changed to understanding the different ways to formatively co-construct and get feed forward feedback from children using different ICT skills and hardware eg: Interactive whiteboard."
- "It has made me more capable of teaching to the needs of all students more fully and gaining a wider range of strategies to use to deliver curriculum in different areas."
- "[It] has made me more aware of what is out there and can be done. Always looking of better ways to teach and certainly feel that, through moodle, out of class work and assignments are more educational and meaningful for the students."
- "It has provided an opportunity for our school to go right back to the basics and explore our core values in learning and teaching. We have reviewed our beliefs on how children learn and how we can best prepare them for their futures."
- "[It has] widened my horizons and given me a whole new pedagogy."
A slightly larger proportion (c. 23%) of responses stated that the teacher had increased their knowledge of different teaching / learning styles and theories, or were enabled to make clearer connections between their day to day practice and the various learning and teaching theories and models outlined in the programmes. Most prominent among the particular theories and models identified were 'inquiry learning', 'learning styles', 'collaborative and/or cooperative learning', and various taxonomies of 'thinking skills' (Bloom's, SOLO etc.).
Examples of statements about linking practice with learning theories, models and research
- "Some PD has been around thinking skills, inquiry learning and the new curriculum, so, relevant to develop my understanding of teaching and learning through the use of ICT."
- "I have learnt more about different inquiry approaches and have clarified my thoughts on some models. I have extended my knowledge of thinking tools."
- "In terms of our inquiry learning, this has greatly changed the way I view the role of the teacher/student. Letting go of all of the control has been great."
- "Conferences gave exposure to new approaches such as inquiry learning which we adopted as a school. Interactive whiteboard training and daily use and trouble shooting has lead [sic] to changes in teaching styles."
- "Understanding of inquiry learning and its place with a constructivist approach to situated learning."
Many of these (c. 20% of total comments) involved teachers reporting that they now had a better understanding of student-centred teaching and learning, or that in some way their teaching had become more student-focused or more relevant to students as a result of the programme.
Examples of statements on student-centredness
- "Giving children many avenues to explore, practise and present information has added a new dimension to my teaching and it really enthuses and turns some children onto learning, particularly boys and those who find pen and paper work difficult."
- "Teacher = facilitator of learning opportunities. The importance of students being actively engaged in questioning and seeking information."
- "[I have] realised the value of making learning child centered and learning directed by the child."
- "The use of ICT engages students quickly - they are able to problem solve and choose suitable applications for a variety of different tasks. Independence is key and the teacher is not the first point of reference."
- "[ICT] allows the teacher to enter the world of the student and is less teacher directive-the students take control of their own learning to learn."
Finally, we note that there were two elements of significant sector difference in the nature of the comments on wider deeper understandings gained from the programmes. The first is that primary teachers were twice as likely as secondary to comment on increased understanding of specific learning theories. The second is that secondary teachers were twice as likely as primary to state that the programme had had little or no effect on their pedagogical understandings in the broader sense.
Teachers' Classroom Practices
When teachers were asked about the extent to which their classroom practices had changed as a result of their participation in the ICT PD programme, over a third (35%) of them indicated that their classroom practices had changed to 'a large extent' or 'completely', and just under half of them (46%) said their classroom practices had changed 'to some extent'. A smaller group (18%) indicated that little or no change had occurred in their classroom practices at all (Table 9).
|Extent of Change in Classroom Practices||Percentage|
|Not at all||5%|
|To some extent||46%|
|To a large extent||32%|
There were significant gender and sector related differences in teachers' responses to this question. Female teachers indicated higher levels of change in their classroom practices than male teachers (X2>35, df=4, p<.001). Some 39% of female teachers reported that their classroom practices changed 'completely' or 'to a large extent' as a result of their participation in the programme, while only a quarter (25%) of male teachers stated this (Table 10). Male teachers were almost twice as likely as female teachers to state that the programme made 'no' or 'little' change in their classroom practices.
Regarding sector differences, primary teachers stated higher levels of change in their classroom practice than secondary teachers (X2>35, df=4, p<.001). Whereas 46% of primary teachers indicated a 'complete' or 'large' change happened to their classroom practices, 19% of secondary teachers indicated the same levels of change (Table 10). While the great majority (81%) of all teachers reported at least some element of change in their classroom practices, secondary teachers were almost four times as likely as primary teachers to report little or no change in classroom practice as a result of the programme.
|Levels of change in classroom practice||Female||Male||Primary||Secondary|
|Not at all||4%||8%||2%||9%|
|To some extent||45%||48%||45%||48%|
|To a large extent||35%||23%||41%||18%|
In terms of the types of changes teachers reported making in their classroom practices, these were more or less equally divided among:
- Respondents identifying increased use of ICTs in classes per se, as a 'changed practice' (e.g.: "I use ICT on a more regular basis as I have a better understanding of what programmes and how to use them effectively with the children."; "More use of ICT by students"; "Implementing ICT across all curriculum areas more."; "I now experiment with different forms of ICT with the hope of enhancing learning"; "I'm now Using Mimio in classrooms").
- Respondents identifying positive changes in the students' learning attitudes and behaviours or in the general 'student-centredness' of their lessons and classroom activities. (e.g.: "More pupil centred applets help with hands on learning"; "My confidence with computers/ICT has allowed me to let the children have more freedom - and to show me what they know"; "It has got me away from teacher centred learning more and more. Students interacting more and taking more responsibility for their own learning and the learning of their peers"; "There's now a greater focus on student responsibility for acquiring knowledge and the value of inquiry based learning")
- Respondents identifying changes in their own pedagogies, especially their trying out new and different teaching approaches in classes, based on either their new understandings of teaching and learning or their increased familiarity and confidence with certain ICTs. Just over 50% of respondents who identified 'changed practices' did so in these terms. (E.g.: "I have incorporated more thinking skills in my teaching"; "Conferences gave exposure to new approaches such as Inquiry learning which we adopted as a school"; "I am now thinking much more about incorporating different ways to learn, to cater for different learning styles and strengths of individuals"; "I am now thinking much more about challenging the ideas, beliefs and values of my students and how to stretch them"; "Combined with thinking skills and higher learning initiatives development my in-class questioning and planning content have changed for the better"; "I can make lessons more visually interesting.".)
No teachers said that their classroom practices had changed for the worse as a result of their participation in the programme. By far the most prominent 'negative' change identified was the notion that teaching with ICTs had made teaching more difficult or frustrating in terms of time and resource management. This was largely seen to be the result of insufficient time, poor access, or lack of sufficient ICT resources to allow them to put into practice the ideas gleaned during PD events. The programme had clearly put significant strain on the technical infrastructures of many schools and their ICT resource base, and for many teachers the use of ICTs by students had logistically complicated their teaching at the same time as it had made it more learner and learning-centred.
- "We now want even more ICT equipment. :)"
- "Since I've realised what I can do in ICT and how useful it is, I've been frustrated with lack of resources and time in the classroom with the demands of our crowded days"
- "Not enough time or computers to be able to keep all children up to speed with what we are doing or what they want to do"
- "Computers [are] not working well in class or in library. No IWB - and to be able to use one involves major reorganisation of [the] class to go to another class to use them"
- "Greater demands on the teacher. Extra time is needed by teachers to learn new ideas and software and also to create new things for teaching because greater expectations were being placed on them."
- 1754 teachers commented on the impact of their involvement in the programme on their 'understanding' of teaching and learning.