Publications

Outcomes for Teachers and Students in the ICT PD School Clusters Programme 2006-2008 - A National Overview

Publication Details

This report focuses on the effectiveness of the 2006-2008 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first six ICT PD programmes. It is the last report of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.

Author(s): Selver Sahin & Vince Ham - CORE Education [Report to the Ministry of Education]

Date Published: May 2010

Effects of the 2006-8 ICT PD Programme on Teachers

The main programme-level goals of the ICT PD school clusters initiative with regard to the effects on teachers themselves were:

  • Increased skills among teachers across a range of educationally useful ICTs.
  • Increased teacher confidence about their personal use of ICTs and about the use of ICTs with and by students in classes.
  • Improved understandings of the roles that ICTs can play in improving classroom teaching and learning.
  • Engagement of teachers in critically reflective communities of practice through and about ICTs.

Nationally, there was a clear and significant increase/improvement in relation to all of these indicators over the period of the programme.

Teachers’ ICT Skills

Over the period of the programme teachers’ skills in using ICTs increased significantly, especially, but not exclusively, for those who at the beginning of the programme had rated their skill levels as either very low or non-existent. As can be seen in Table 5, below, there were significant reductions across the board in the proportions of teachers who rated their skills as low or non-existent, and significant increases in the proportion who rated their skill level as high or very high. This was the case even with regard to ICTs such as word processing where the great majority of teachers entered the programme already with reasonable pre-existing levels of competence. By the end of the programme very solid majorities of teachers felt they had moderate or high skills with regard to file management (94%), basic computer operation (93%), word processing (97%), Internet (87%) and telecommunications (85%). Lower but still relatively high levels of end of programme competence were reported with regard to graphics (77%), spreadsheets (69%), databases (49%) and multimedia packages (71%), all of these still show significant increases compared to entry point proportions. The increases in teachers’ skill levels during the programme were considerable across all ICTs, but were most notable in relation to graphics and multimedia applications.

Table 5: Teachers’ skill levels with various ICTs before and after the programme, as reported in the end of project surveys. (n = 2554 ~ 2648)

Skill levelFile ManagementBasic OperationWord ProcessingSpreadsheetsDatabase
BeforeAfterBeforeAfterBeforeAfterBeforeAfterBeforeAfter
High/Very high37%60%27% 50% 48% 71% 16% 28% 7% 13%
Moderate33%34%41% 43% 36% 26% 27% 41% 22% 36%
Low/Nil30%6%32% 7% 16% 2% 56% 31% 70% 50%


Skill levelGraphicsInternetTelecommunicationsMultimedia presentation
BeforeAfterBeforeAfterBeforeAfterBeforeAfter
High/Very high12% 35% 21% 46% 22% 43% 12% 33%
Moderate28% 42% 36% 41% 33% 42% 23% 38%
Low/Nil60% 23% 43% 13% 45% 15% 65% 29%

Demographic analysis of these results shows that although gender differences decreased over the period of the programme, they still remained significant in favour of male teachers across virtually all ICTs, even at its end. The exceptions were word processing, for which female and male respondents indicated the same high skill levels, and graphics, for which female teachers reported higher skill levels.

There were some sector differences both at the beginning and end of the programme. Secondary teachers tended to report higher skill levels on entry to the programme, but at the end of the programme, this trend reversed. While at the end secondary teachers still reported higher skill levels with spreadsheets, primary teachers at the end reported higher skill levels with respect to file management, word processing, graphics, Internet, multimedia and telecommunications.

Teachers’ Confidence About the Use of ICTs

Changes in the confidence of teachers about using ICTs were investigated with regard to two elements: their confidence as personal users of ICTs, and their confidence about using ICTs with classes of learners. Both of these increased significantly over the period of the programme from what were moderate and low levels of confidence respectively on entry.

Figure 1: Teachers’ confidence about using ICTs with classes before and after the programme (n=2594 Before, 2560 After)

Image of Figure 1: Teachers’ confidence about using ICTs with classes before and after the programme (n=2594 Before, 2560 After).

At the end of the project, many teachers reported that at the beginning of the project they had been less than confident as personal users of ICTs. About 4% of teachers classified themselves as ‘anxious’ and 20% of them were ‘not confident’ in this regard. By the end of the programme, less than 1% said they were ‘anxious’, while only 2% reported that they were still ‘not confident’ about the personal use of ICTs. By the end of the programme, over four fifths (84%) of teachers stated that they had become either ‘confident’ or ‘very confident’ about their personal use of ICTs.

Teachers’ confidence about using ICTs with classes also increased significantly during the programme. Teachers reported that on entry, they had been even less confident about using ICTs with classes than they had been about their personal use. At that point, 9% of them self-identified as ‘anxious’ about this, and 28% identified themselves as ‘not confident’. By the end of the programme, however, the percentage of ‘anxious’ or ‘not confident’ teachers had dropped from 37% to 3%. Correspondingly, the percentage of ‘confident’ or ‘very confident’ teachers had increased from 34% to 77%. Figure 1 illustrates the change in teachers’ confidence about the use of ICTs with classes

As was also the case for confidence gains in relation to personal use, these confidence gains in relation to classroom use of ICTs were related to all of gender, sector and length of involvement demographics. Female teachers made significantly more confidence gains than male teachers (X2>35; df=3; p<.001), primary teachers reported significantly more gains in confidence about classroom use than secondary teachers (X2>35; df=3; p<.001), and confidence gains were also positively correlated with the length of time teachers had actively participated in the programme (X2>35; df=15; p<.001) (See Table 6, & Figure 2). The longer teachers took active part in ICT PD the greater the increase in their confidence about using ICTs with classes.  


Table 6: Increases in teachers’ confidence about use of ICTs with classes during the programme, by gender and sector (n=2565 gender; n= 2560 sector)
Change in confidenceFemaleMale PrimarySecondary
Decrease2%3% 2%3%
No change29%48% 24%50%
Slight increase36%34% 40%31%
Significant increase33%16% 34%17%

Figure 2: Increases in individual teachers’ confidence about use of ICTs with classes, by length of active involvement in programme (n= 2527)

Image of Figure 2: Increases in individual teachers’ confidence about use of ICTs with classes, by length of active involvement in programme (n= 2527).

Teachers’ Understanding of the Role of ICTs in Teaching and Learning

To address this indicator of the impact of the ICT PD programmes on teachers, we asked teachers what they saw as the benefits of ICTs in teaching and learning on the basis of their experience with ICTs during the programme; what concerns they had about the incorporation of ICTs into their teaching programmes; how, if at all, incorporating ICTs had changed their ways of teaching, and the contribution the programme had made to their understanding of teaching and learning in general. Their responses to these questions give an overview of what the teachers’ own broad understandings were at the end of the programme in relation to the role of ICTs in education and how widely spread such views were among them.

Teachers in this cohort generally had positive views on the role of ICTs in education. By the end of the programme many of the teachers’ still felt there were a number of constraints on the effective implementation of ICTs into teaching and learning, but their views had, if anything, become even more positive about their potential in other respects, both in improving their teaching and in fostering effective learning.

Benefits of integrating ICTs into teaching and learning

The qualitative analyses of previous cohorts reported benefits of integrating ICTs into their teaching and learning programmes, show that these benefits may be grouped into ten key categories (Ham, 2008; Ham, Toubat & Williamson-Leadley, 2006; Ham, Graham, Toubat & Williamson-Leadley, 2005). They are a mixture of teacher or teaching-oriented benefits and learner or learning-oriented benefits, with, from the teachers in this cohort, a heavy emphasis on the latter. The categories of benefit and their distribution for this cohort are outlined in Table 7.


Table 7: Examples and distribution of teacher-identified ‘benefits’ of using ICTs with classes. Percentages represent approximate proportions of statements relevant to a given category. (n=2225 respondents provided c.5063 statements. A random sample of 1156 statements were coded)
Facilitating more efficient learning of specific curriculum content and curriculum objectives, easier access to information, improved presentation etc (28%)
  • “Becoming familiar with programmes e.g. PowerPoint, developing research skills rather than just 'Wikipedia' and 'Google', [and] enhancement of understanding/enrichment.”
  • “Literacy skills improve. Oral language developed when using video on the computer”
  • “improvement in finding, evaluating and using information and justifying their choices , editing and reworking of written both fiction and information sharing material to improve content and form”
  • “Giving students a range of options to choose from when presenting data.”
  • “Cementing of information using a variety of programmes / games, enjoyment.”
  • “[Students are] more confident in their writing, knowing and locating information, [and] being able to discuss and analyse information.”
  • “[Students] can quickly and easily find information looked for.”
  • “[ICTs] broaden their [students’] research knowledge and skills. [There is an] increase in presentation skills and creativity.”
Making learning for students more engaging and interesting (25%)
  • “Motivation/fun, accessing information that they [students] particularly wanted to know, learning without realising they are.”
  • “[Students are] excited about learning [and] engaged in their learning.”
  • “They [students] enjoy using computers and programmes so it enhances their learning.”
  • “Increased motivation [and] increased level of communication and cooperative work.”
  • “Motivation to learn is greater.”
Fostering more independence and agency in student learning(14%)
  • “[Students are] becoming confident in what they once believed were unachievable skills.”
  • “[Students] had control of their own learning and a sense of independence.”
  • “The children were quick to learn and go further – they could identify possibilities and were keen to take risks and explore.”
  • “[ICT] reinforces learning we have been covering in class. They [students] are not afraid to take a risk and are willing to problem solve for themselves when it comes to new ICT.”
Encouraging collaborative or cooperative inquiry, contributing to social skill development(10%)
  • “They [students] were able to help each other when working on a laptop.”
  • “Willingness to discuss, willingness to try new things [and] wanting to share with others.”
  • “Children helping other children, more collaborative learning.”
  • “They [students] are more supportive of each other, sharing knowledge and ideas.”
  • “[Students are] sharing ideas and working collaboratively.”
  • “Seeing their confidence in using a new tool and being able to teach someone else what they have learned.”
Enabling a focus on thinking skills (esp. higher order thinking and metacognition) (7%)
  • “[Students] discovered that not all that was on the web was true [and] became more critical thinkers when they could not find same answer in more than one place.”
  • “They think critically about using graphics to make their work visually attractive”.
  • “Thinking skills developed.”
  • “[ICTs] encouraged higher order of thinking, more refined searches and detail in answers of research and contacts to get further info from.”
  • “Developing the use of thinking and questioning skills.”
Allowing more authenticity, real-worldness and relevance to children’s lives in teaching and learning tasks(4%)
  • “Different opportunities such as creating music, making movies and creating blogs have been available to children.”
  • “Learning of new skills and programmes to enhance current learning - authentic and purposeful.”
  • “Authentic reflections / assessments and learning.”
  • “We made connections with the wider world. We have increased our interaction with other classes in the school and community”
  • “Giving students an audience [and] preparing them for today and tomorrow's society.”
  • “They know which programme to use for different purposes. Children got a lot of satisfaction out of producing something using an ICT. [It] enhanced children's authentic learning.”
Using a wider range and variety of teaching-learning activities in class (4%)
  • “More access to ICT, increased teacher knowledge, wider variety of activities.”
  • “[It] opened the door to a number of new resources that could be directly used in the classroom.”
  • “Greater variety of options for sharing knowledge. Increased creativity and interest because of this. Wider variety of sources for research.”
  • “Increased awareness and exposure to varied ICT [and] experience with different programmes e.g. kidpix, photostory etc.”
  • “[It] allows for multiple opportunities to demonstrate learning.”
  • “[ICT] gives a wide range of learning methods to meet learning styles.”
Enhancing/expanding their own teaching skill set and pedagogical knowledge (3%)
  • “[Students have] access to alternative instruction”.
  • “They [students] drive their own learning to a greater extent. Less pressure on teacher to be a fount of all knowledge and always to be right. It’s stimulating.”
  • “Formative practices are reinforced.”
  • “Better able to support students struggling with content. [ICTs] helped with long-term planning of course work [and] encouraged a greater variety of approaches to classwork.”
  • “Students were able to access more tools because I knew how to use them ,”
  • “School becomes relevant again. Teachers become learners too.  Answers come from the shared learning experience.”
Teaching with more confidence and enthusiasm(3%)
  • “Increased application, increased confidence, increased knowledge.”
  • “My gain in confidence benefited them [students] so that I could support them.”
  • “They are much more confident and so am I.”
  • “The ongoing PD has given me confidence to provide many more ICT opportunities for my students.”
  • “What I have learnt thus far has contributed to the enthusiasm for me as a teacher and also the students I teach within the classroom”
Making learning more personalised or individualised, (2%)
  • “[ICT] personalises their learning.” 
  • “Work pitched closer to individual needs.”
  • “Meeting individual learning needs”.
  • “[Being] able to use an ICT that suits their learning style.”
  • “[ICT] caters for variety of styles, needs.”

Concerns

Technical issues, such as equipment reliability, and inadequate access to ICTs for students were among the most-selected concerns about ICT use, these being much more significant in peoples’ minds than pedagogical issues. Over a third of the teachers, for example, identified significant concerns about access to equipment for students (39%), technical reliability, (33%) and lack of time (33%) for themselves to become familiar with the range of ICTs available as their main persisting concerns around the use of ICTs with students at the end of the programme (Figure 3). As has also been the case for previous cohorts, many of the teachers at the end of the project felt there was still a need for continuing PD around ICT use, even though the programme had significantly increased and improved their effective practices in this regard.

Figure 3: Teachers’ continuing concerns about the use of ICTs with classes (n = 2521~2566)

Image of Figure 3: Teachers’ continuing concerns about the use of ICTs with classes (n = 2521~2566).

Contribution to wider understandings of teaching and learning

When asked whether or not the ICT PD programme had contributed to their understanding of teaching and learning in general terms, over two-thirds (68%) of respondents indicated that the programme contributed new ideas about teaching and learning, including 10% who stated that the programme provided them with a ‘whole new approach’. It would also seem that primary teachers were rather more likely to see the programme as contributing new insights and ideas in this regard than secondary teachers (Table 8).


Table 8: Contribution of ICT PD to general understandings of teaching and learning (n=1583 (P), 995 (S))
 TotalPrimarySecondary
Provided a new whole approach to teaching and learning10%13%5%
Contributed some new ideas about teaching and learning 58%63%51%
Confirmed current ideas/understandings about teaching & learning25%20%31%
Not at all 7%4%13%

When they were asked to describe the ways in which the ICT PD programme made a contribution to their ‘understandings’, responses1 ranged from the educational value of ICTs to significantly deeper understandings of learning and pedagogy in general, well beyond the connection with ICTs.

Of all the comments made about the ways in which ICT PD programme had contributed to teachers’ ‘understandings’ of teaching and learning in general, about a quarter (23%) related to the specific connections they had been able to make about the educational value of ICTs in teaching and learning, rather than what they had learned about teaching and learning ‘more generally’. Almost three-quarters (70%) of the comments that teachers made, however, did identify deeper or wider understandings and identified a number of key ways in which the programme had supplemented, challenged and even changed those understandings.

Thirdly, there was also a substantial proportion (c. 28%) of comments on the programme as challenging and changing their pedagogical perspectives and understandings, either through the content of the PD programme, or, more often, through the opportunities it provided for sharing and critical discussion with colleagues, outside ‘experts’, and so on. Many of these comments spoke of the teachers adopting a more ‘critical’ or more ‘reflective’ approach to their teaching as a result of their new awareness of pedagogical and learning theories and research.

 

Examples of statements about challenging their pedagogical approach and understandings

  • “It has exposed me to a whole new learning environment. I have become a facilitator of learning instead of the one holding all of the knowledge. I would see something in PD sessions and just have a go...initiate stage of an ICT environment.”
  • “My pedagogy has changed to understanding the different ways to formatively co-construct and get feed forward feedback from children using different ICT skills and hardware eg: Interactive whiteboard.”
  • “It has made me more capable of teaching to the needs of all students more fully and gaining a wider range of strategies to use to deliver curriculum in different areas.”
  • “[It] has made me more aware of what is out there and can be done. Always looking of better ways to teach and certainly feel that, through moodle, out of class work and assignments are more educational and meaningful for the students.”
  • “It has provided an opportunity for our school to go right back to the basics and explore our core values in learning and teaching. We have reviewed our beliefs on how children learn and how we can best prepare them for their futures.”
  • “[It has] widened my horizons and given me a whole new pedagogy."

A slightly larger proportion (c. 23%) of responses stated that the teacher had increased their knowledge of different teaching / learning styles and theories, or were enabled to make clearer connections between their day to day practice and the various learning and teaching theories and models outlined in the programmes. Most prominent among the particular theories and models identified were ‘inquiry learning’, ‘learning styles’, ‘collaborative and/or cooperative learning’, and various taxonomies of ‘thinking skills’ (Bloom’s, SOLO etc.).

 

Examples of statements about linking practice with learning theories, models and research

  • “Some PD has been around thinking skills, inquiry learning and the new curriculum, so, relevant to develop my understanding of teaching and learning through the use of ICT.”
  • “I have learnt more about different inquiry approaches and have clarified my thoughts on some models. I have extended my knowledge of thinking tools.”
  • “In terms of our inquiry learning, this has greatly changed the way I view the role of the teacher/student. Letting go of all of the control has been great.”
  • “Conferences gave exposure to new approaches such as inquiry learning which we adopted as a school. Interactive whiteboard training and daily use and trouble shooting has lead [sic] to changes in teaching styles.”
  • “Understanding of inquiry learning and its place with a constructivist approach to situated learning.”
  •  

Many of these (c. 20% of total comments) involved teachers reporting that they now had a better understanding of student-centred teaching and learning, or that in some way their teaching had become more student-focused or more relevant to students as a result of the programme.

Examples of statements on student-centredness

  • “Giving children many avenues to explore, practise and present information has added a new dimension to my teaching and it really enthuses and turns some children onto learning, particularly boys and those who find pen and paper work difficult.”
  • “Teacher = facilitator of learning opportunities. The importance of students being actively engaged in questioning and seeking information.”
  • “[I have] realised the value of making learning child centered and learning directed by the child.”
  • “The use of ICT engages students quickly - they are able to problem solve and choose suitable applications for a variety of different tasks.  Independence is key and the teacher is not the first point of reference.”
  • “[ICT] allows the teacher to enter the world of the student and is less teacher directive-the students take control of their own learning to learn.”


Finally, we note that there were two elements of significant sector difference in the nature of the comments on wider deeper understandings gained from the programmes. The first is that primary teachers were twice as likely as secondary to comment on increased understanding of specific learning theories. The second is that secondary teachers were twice as likely as primary to state that the programme had had little or no effect on their pedagogical understandings in the broader sense.

Teachers’ Classroom Practices

When teachers were asked about the extent to which their classroom practices had changed as a result of their participation in the ICT PD programme, over a third (35%) of them indicated that their classroom practices had changed to ‘a large extent’ or ‘completely’, and just under half of them (46%) said their classroom practices had changed ‘to some extent’. A smaller group (18%) indicated that little or no change had occurred in their classroom practices at all (Table 9).


Table 9: Changes in teachers’ classroom practices (n=2594)
Extent of change in classroom practicesPercentage
Not at all5%
Very little13%
To some extent46%
To a large extent32%
Completely changed3%

There were significant gender and sector related differences in teachers’ responses to this question. Female teachers indicated higher levels of change in their classroom practices than male teachers (X2>35, df=4, p<.001). Some 39% of female teachers reported that their classroom practices changed ‘completely’ or ‘to a large extent’ as a result of their participation in the programme, while only a quarter (25%) of male teachers stated this (Table 10). Male teachers were almost twice as likely as female teachers to state that the programme made ‘no’ or ‘little’ change in their classroom practices.

Regarding sector differences, primary teachers stated higher levels of change in their classroom practice than secondary teachers (X2>35, df=4, p<.001). Whereas 46% of primary teachers indicated a ‘complete’ or ‘large’ change happened to their classroom practices, 19% of secondary teachers indicated the same levels of change (Table 10). While the great majority (81%) of all teachers reported at least some element of change in their classroom practices, secondary teachers were almost four times as likely as primary teachers to report little or no change in classroom practice as a result of the programme.


Table 10: Changes in teachers’ classroom practices by gender and sector (n= 1835 (F), 758 (M), 1561 (P), 992 (S))
Levels of change in classroom practiceFemaleMale PrimarySecondary
Not at all4%8% 2%9%
Very little11%18% 7%23%
To some extent45%48% 45%48%
To a large extent35%23% 41%18%
Completely changed4%2% 5%1%

In terms of the types of changes teachers reported making in their classroom practices, these were more or less equally divided among:

  • Respondents identifying increased use of ICTs in classes per se, as a ‘changed practice’ (e.g.: “I use ICT on a more regular basis as I have a better understanding of what programmes and how to use them effectively with the children.”; “More use of ICT by students”; “Implementing ICT across all curriculum areas more.”; “I now experiment with different forms of ICT with the hope of enhancing learning”; “I’m now Using Mimio in classrooms”).
  • Respondents identifying positive changes in the students’ learning attitudes and behaviours or in the general ‘student-centredness’ of their lessons and classroom activities. (e.g.: “More pupil centred applets help with hands on learning”; “My confidence with computers/ICT has allowed me to let the children have more freedom - and to show me what they know”; “It has got me away from teacher centred learning more and more. Students interacting more and taking more responsibility for their own learning and the learning of their peers”; “There’s now a greater focus on student responsibility for acquiring knowledge and the value of inquiry based learning”)
  • Respondents identifying changes in their own pedagogies, especially their trying out new and different teaching approaches in classes, based on either their new understandings of teaching and learning or their increased familiarity and confidence with certain ICTs. Just over 50% of respondents who identified ‘changed practices’ did so in these terms. (E.g.: “I have incorporated more thinking skills in my teaching”; “Conferences gave exposure to new approaches such as Inquiry learning which we adopted as a school”; “I am now thinking much more about incorporating different ways to learn, to cater for different learning styles and strengths of individuals”;  “I am now thinking much more about challenging the ideas, beliefs and values of my students and how to stretch them”; “Combined with thinking skills and higher learning initiatives development my in-class questioning and planning content have changed for the better”; “I can make lessons more visually interesting.”.)
No teachers said that their classroom practices had changed for the worse as a result of their participation in the programme. By far the most prominent ‘negative’ change identified was the notion that teaching with ICTs had made teaching more difficult or frustrating in terms of time and resource management. This was largely seen to be the result of insufficient time, poor access, or lack of sufficient ICT resources to allow them to put into practice the ideas gleaned during PD events. The programme had clearly put significant strain on the technical infrastructures of many schools and their ICT resource base, and for many teachers the use of ICTs by students had logistically complicated their teaching at the same time as it had made it more learner and learning-centred.

 

  • “We now want even more ICT equipment. :)”
  • “Since I've realised what I can do in ICT and how useful it is, I've been frustrated with lack of resources and time in the classroom with the demands of our crowded days”
  • “Not enough time or computers to be able to keep all children up to speed with what we are doing or what they want to do”
  • “Computers [are] not working well in class or in library. No IWB - and to be able to use one involves major reorganisation of [the] class to go to another class to use them”
  • “Greater demands on the teacher. Extra time is needed by teachers to learn new ideas and software and also to create new things for teaching because greater expectations were being placed on them.”
  •  


Footnote

     
  1. 1754 teachers commented on the impact of their involvement in the programme on their ‘understanding’ of teaching and learning.

 

 


 Copyright © Education Counts 2014   |   Contact information.officer@minedu.govt.nz for enquiries.