Outcomes for teachers and students in the ICTPD School Clusters Programme 2005-2007: A national overview

Publication Details

This report focuses on the effectiveness of the 2005-2007 Information and Communication Technologies Professional Development (ICT PD) School Clusters programmes and supplements previous evaluations of the first five ICT PD programmes. It is part of an ongoing evaluation of the ICT PD teacher professional development initiative, which has been implemented in New Zealand since 1999.

Author(s): Vince Ham, CORE Education. Report prepared for the Ministry of Education.

Date Published: July 2009

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.

Section 3: Effects of the 2005–2007 ICTPD Programme on teachers

The main programme-level goals of the ICT PD school clusters initiative with regard to the effects on teachers themselves were:

  • Increased skills among teachers across a range of educationally useful ICTs.
  • Increased teacher confidence about their personal use of ICTs and about the use of ICTs with and by students in classes.
  • Improved understandings of the roles that ICTs can play in improving classroom teaching and learning.
  • Engagement of teachers in critically reflective communities of practice through and about ICTs.

Nationally, there was a clear and significant increase/improvement in relation to all of these indicators over the period of the programme.

Teachers' ICT skills

Over the period of the programme teachers' skills in using ICTs increased significantly, especially, but not exclusively, for those who at the beginning of the programme had rated their skill levels as either very low or non-existent. As can be seen in Table 5, below, there were significant reductions across the board in the proportions of teachers who rated their skills as low or non-existent, and significant increases in the proportion who rated their skill level as high or very high. This was the case even with regard to ICTs such as word processing where the great majority of teachers entered the programme already with reasonable pre-existing levels of competence. By the end of the programme very solid majorities of teachers felt they had moderate or high skills with regard to file management (94%), basic computer operation (94%), word processing (98%), Internet (94%) and telecommunications (96%). Lower but still relatively high levels of end of programme competence were reported with regard to graphics (81%), spreadsheets (67%), databases (57%) and multimedia packages (72%), though these still show significant increases compared to entry point proportions. The increases in teachers' skill levels during the programme was considerable across all ICTs, but it was most notable in relation to graphics and multimedia applications. 

Table 5: Teachers' skill levels with various ICTs before and after the programme (as reported in the end of project surveys)
ICTs Skill Level
High/
Very High
Moderate Low
/Nil
File Management:  Before 29% 33% 37%
File Management:  After 60% 34% 6%
Basic Operation:  Before 21% 41% 39%
Basic Operation:  After 49% 45% 6%
Word Processing:  Before 43% 36% 21%
Word Processing:  After 72% 26% 2%
Spreadsheets:  Before 13% 23% 64%
Spreadsheets:  After 25% 42% 33%
Database:  Before 9% 24% 67%
Database:  After 18% 39% 44%
Graphics:  Before 11% 27% 62%
Graphics:  After 37%  44% 18%
Internet:  Before 23%  40% 37%
Internet:  After 57%  37% 6%
Telecommunications:  Before 31%  39% 30%
Telecommunications:  After  58%  38% 4%
Multimedia Presentation:  Before 9%  19% 71%
Multimedia Presentation:  After 35%  37% 28%


Demographic analysis of these results show some continued relationship between gender and teachers skill levels in favour of male and secondary teachers at the start and the end of the programme. The gender differences reduced over the period of the programme but remained significant across virtually all ICTs even at the end. The sector differences all but disappeared the end of the three years, the only notable remaining differences being that secondary teachers reported higher skill levels with basic operations and spreadsheets, and primary teachers reported higher skill levels in the use of graphics.

The impact of the programme on teachers' skill levels was clearly significant across the full range of educationally useful skills measured, though skill with certain ICTs such as spreadsheets among secondary teachers and multimedia and graphics packages among primary teachers seem to have been emphasized in the different sectors.

Teacher confidence about the use of ICTs

Changes in the confidence of teachers about using ICTs were investigated with regard to two elements: their confidence as personal users of ICTs, and their confidence about using ICTs with classes of learners. Both of these increased significantly over the period of the programme from what were moderate and low levels of confidence respectively on entry.

At the end of the project, many teachers reported that at the beginning of the project they had been less than confident as personal users of ICTs. Seven percent of teachers classified themselves as having been 'anxious' and 24% of them were 'not confident' in this regard. By the end of the programme less than 1% of the teachers said they were 'anxious' and only 1% were still 'not confident' about the personal use of ICTs. By the end of the programme over four-fifths (84%) of teachers stated that they had become either 'confident' or 'very confident' about their personal use of ICTs.

Teachers' confidence about ICT use with classes also increased significantly during the programme. Teachers reported that on entry, they had been even less confident about using ICTs with classes than they had been about their personal use. At that point 14% of them self-identified as 'anxious' about this, and 30% identified themselves as 'not confident'. By the end of the programme, however, the percentage of 'anxious' or 'not confident' teachers had dropped from 44% to 3%. Correspondingly, the percentage of 'confident' or 'very confident' teachers had increased from 28% to 77%. Figure 1 illustrates the change in teachers' confidence about the use of ICTs with classes.

Figure 1: Teachers' confidence about using ICTs with classes before and after the ICTPD programme

Figure 1: Teachers' confidence about using ICTs with classes before and after the ICT PD programme

As was also the case for confidence gains in relation to personal use, these confidence gains in relation to classroom use of ICTs were related to all of gender, sector and length of involvement demographics. Female teachers made significantly more confidence gains than male teachers (X2>35; df=3; p<.001), primary teachers reported significantly more gains in confidence about classroom use than secondary teachers (X219.28; df=3; p<.001), and confidence gains were also positively correlated with the length of time teachers had actively participated in the programme (X2=22.89; df=8; p<.001) (See Table 6, & Figure 2). The longer teachers took active part in ICT PD the greater the increase in their confidence about using ICTs with classes.

Table 6: Increases in teachers' confidence about use of ICTs with classes during the programme, by gender and sector
Change in Confidence Female Male Primary Secondary
Decrease 2% 3% 2% 2%
No Change 27% 37% 23% 35%
Slight Increase 35% 38% 38% 30%
Significant Increase 41% 22% 38% 34%
Figure 2: Increases in individual teachers' confidence about use of ICTs with classes, by length of active involvement in programme

Figure 2: Increases in individual teachers' confidence about use of ICTs with classes, by length of active involvement in programme

Teachers' understanding of the role of ICTs in teaching and learning

Teachers in this cohort generally had positive views on the role of ICTs in education. At the end of the programme many of the teachers' still felt there were a number of constraints on their effective implementation of ICTs into teaching and learning, but their views had, if anything, become even more positive about their potential in other respects, both in improving their teaching and in fostering effective learning.

To address this indicator of the impact of the ICT PD programmes on teachers, we asked teachers what they saw as the benefits of ICTs in teaching and learning on the basis of their experience with ICTs during the programme; what concerns they had about the incorporation of ICTs into their teaching programmes; how, if at all, incorporating ICTs had changed their ways of teaching, and the contribution the programme had made to their understanding of teaching and learning in general. Their responses to these questions give an overview of what the teachers' own broad understandings were at the end of the programme in relation to the role of ICTs in education and how widely spread such views were among them.


Benefits of integrating ICTs into teaching and learning

The qualitative analyses of previous cohorts reported benefits of integrating ICTs into their teaching and learning programmes, show that these may be grouped into about ten key categories (Ham et al 2005, 2006, Ham 2007). These are a mixture of teacher or teaching-oriented benefits and learner or learning-oriented benefits, with, from the teachers in this cohort, a heavy emphasis on the latter. The categories of benefit and their distribution for this cohort are:

Table 7: Examples and distribution of teacher-identified 'benefits' of using ICTs with classes (Percentages represent proportions of statements of benefit (n=c.2707) relevant to a given category)
Teaching with more confidence and enthusiasm
(1% )
  • "…being able to advise students on areas of presentation and use of ICT that I previously knew nothing about."
  • "Students being more confident that I could help them."
  • "I am now more confident teaching skills"
  • "[I'm] now enthusiastic and excited about using the computer."
  • "As I became a more confident ICT user, I Integrated it into my class programme, so everyone benefits."
  • "It means that I am using methods they are familiar with and the connect more to the work because of this."
Enhancing/expanding their own teaching skill set and pedagogical knowledge
(1%)
  • "The teacher becomes less visible in the learning process."
  • "Different style of teaching - a break from the usual routine."
  • "I mainly teach topic work so it added a new dimension to our teaching."
  • "There's more diversity in teaching styles."
  • "Teaching and learning have become student centred … Learning has to become co-constructed."
Using a wider range and variety of teaching-learning activities in class
(8%)
  • [There is]..an extended choice of research material / a variety of presentation ideas.
  • "I am able to use a variety of ICT and therefore able to pick out the best one for a particular job."
  • "Variety of approaches to projects"
  • "Variety of Learning Experiences."
  • "Wider variety of resources."
  • "Gathering information from a wider range of sources and can now manipulate this information in various ways."
Allowng more authenticity, real-worldness and relevance to children's lives in teaching and learning tasks, 
(4%)
  • "Interest in computers as their writing was for an authentic audience, makes them want to blog more and more to communicate."
  • "The exposure of [the] world beyond classroom adds to what we are doing."
  • "Using  it as an authentic learning tool which the children can relate to.This  can sometimes lead to increased participation and interest."
  • "It's related to the real world …[and] Engaging in a way they relate to."
  • "[I see] Engagement, Relevance (real time information available) [and] Transfer (see skills used in a number of subjects)."
Making learning for students more engaging and interesting
(28%)
  • "High level of motivation. High level of success."
  • "Students are more involved in their learning."
  • "Increased enthusiasm and confidence in their abilities."
  • "Highly motivating and exciting way of presenting information and finding information."
  • "Children enjoyed the more professional look to their completed work."
  • "The children can become more engaged in there [sic.] learning."
Facilitating more efficient learning of specific curriculum content and objectives, easier access to information improved presentation etc.
(25%)
  • "Using  the computer as a tool for presenting their inquiry learning as well as  gaining information through the internet has been a huge benefit."
  • "Improved quality of ICT presentations."
  • "A sense of achievement when they were doing maths activities (moved up a level)."
  • "It has allowed children with lower level Literacy skills to get involved at a deeper level than previously."
  • "[It's] easy to access information."
  • "Being able to widen their scope of information. / Use skills to present their findings in an improved manner."
Enabling a focus on thinking skills (especially higher order thinking and metacognition)
(12%)
  • "Developing thinking skills."
  • "Thinking skills have helped children to more effectively sort their learning and to think further than they would have."
  • "Their thinking skills have been extended"
  • "higher  order thinking - creative answers to questions, extended their thinking  / Reflecting - children can now reflect on their own learning."
Encouraging collaborative of cooporative enquiry, contributing to social skill development,
(9%)
  • "More interactive group work is happening."
  • "Learning to work together and share when a group is working together on the    computer."
  • "Students share their skills with others. Students teaching students."
  • "Students tend to work more collaboratively sharing and comparing sites and    discussing information."
  • "Collaboration work, questioning skills very beneficial."
Making learning more personalised or individualised.
(7%)
  • "[ICT] switches on the non reader with reinforcement activities."
  • "It was another way of learning for those who have difficulty."
  • "Greater ease of writing and presentation for slow writers."
  • "Use of ICT caters for a variety of different learning styles."
  • "Digital images are very powerful prompts for facilitating communication and    independence with special needs students."
Fostering more independence and agency in student learning.
(5%)
  • "Students are more empowered to solve problems themselves."
  • "Independence in utilising ICT technologies and approaches."
  • "Students are able complete independent inquiries and present these to a high    standard."
  • "Gives independence to learning."
  • "Independence / Increased personal responsibility for learning / Knowledge of    own learning habits."
  • "The development of some independence in their own learning."


Concerns

Technical issues, equipment reliability and inadequate access to ICTs for students seemed to account for most of the perceived constraints on ICT use, these being much more significant in peoples' minds than pedagogical issues. Over a third of the teachers, for example, identified significant concerns about access to equipment for students (38%) and reliability, (36%) alongside lack of time (37%) for themselves to become familiar with the range of ICTs available as their main persisting concerns around the use of ICTs with students at the end of the programme (Figure 3). As has also been the case for previous cohorts, many of the teachers at the end of the project felt there was still a need for continuing PD around ICT use, even though the programme had significantly increased and improved their effective practices in this regard, though in comparison with earlier cohorts the proportions of teachers expressing significant concerns at the end of the programme does seem to have reduced across the board.

Figure 3: Teachers' continuing concerns about the use of ICTs with classes

Figure 3: Teachers' continuing concerns about the use of ICTs with classes


Contribution to wider understandings of teaching and learning more generally

When teachers were asked to describe the ways in which the ICT PD programme had contributed to their understanding of teaching and learning generally, responses ranged from 'no effect' to significantly deeper understandings of learning and pedagogy in general, well beyond the connection with ICTs.

Of all the comments made about the ways in which ICT PD programmes had contributed to teachers' 'understandings' of teaching and learning in general, about a third related to the specific connections they had been able to make about the educational value of ICTs in teaching and learning, rather than what they had learned about teaching and learning 'more generally'. Of those that did refer specifically to the role of the programmes in developing their 'more general' understandings of teaching and learning, about 18% said that there had been little or no contribution to their general pedagogical understandings, beyond the acquisition of practical ideas for integrating ICTs into their classroom programmes.

Examples of statements on student-centredness
  • "By being able to have students use particular software I appreciate that they learn at a far faster rate than I am able to provide material. It has speeded up the learning process."
  • "Students are more involved in goal setting learning intentions assessment."
  • "Made me think more about what the students need in their future."
  • "More flexibility & student centred research."
  • "[It has] extended my knowledge of systems of delivery and understanding and application of student centred learning systems."
  • "Much more child centred and cooperative learning approach."
  • "Students are more engaged in the learning as a result of the ICTs. Students are now creating and expressing themselves in a wider range of ways."


About half the comments of teachers, however, did identify such deeper or wider understandings and identified a number of key ways in which the programme had supplemented, challenged and even changed those understandings.

Many of these (c. 11% of total comments) involved teachers reporting that they now had a better understanding of student-centred teaching and learning, or that in some way their teaching had become more student-focused or more relevant to students as a result of the programme.

An even larger proportion (c. 30%) stated that they had significantly increased their knowledge of different teaching/learning styles and theories, or were enabled to make clearer connections between their day to day practice and the various learning and teaching theories and models outlined in the programmes. Most prominent among the particular theories and models identified were 'enquiry learning', 'learning styles', 'collaborative and/or cooperative learning', and various taxonomies of 'thinking skills' (Bloom's, SOLO etc.).

Examples of statements about linking practice with learning theories, models and research
  • "awareness of use of pedagogies eg Brain based learning/inquiry/ability to link chn to world outside classroom with greater relevance and immediacy."
  • "Provides increased opportunities for co-operative learningSupports a range of learning styles."
  • "[I'm] making use of learning processes multiple intelligence theory."
  • "It was good to read some up to date articles on learning theory."
  • "I have added to existing knowledge on Inquiry learning, have a better understanding of our research can impact teaching and learning."
  • "Being introduced to a variety of thinking and learning strategies such as Habits of Mind, SOLO, and the Fish Philosophy has encouraged me to become more reflective about what is happening in class and it has rubbed off on the children as well."


Thirdly, there was also a substantial proportion who commented on the programme as challenging and changing their pedagogical perspectives and understandings, either through the content of the PD programme, or, more often, through the opportunities it provided for sharing and critical discussion with colleagues, outside 'experts', and so on. Many of these comments spoke of the teachers adopting a more 'critical' or more 'reflective' approach to their teaching as a result of their new awareness of pedagogical and learning theories and research.

Finally, we note that there were two elements of significant sector difference in the nature of the comments on wider deeper understandings gained from the programmes. The first is that primary teachers were twice as likely as secondary to comment on increased understanding of specific learning theories. The second is that secondary teachers were twice as likely as primary to state that the programme had had little or no effect on their pedagogical understandings in the broader sense.

Examples of statements about challenging their pedagogical approach and understandings
  • "It has helped me become a more multi media teacher."
  • "Evaluating and reflecting on why we teach the way we do has really developed my understanding of teaching and learning."
  • "Over the last 3 years I have been challenged to think about; childrens learning, the inquiry process, thinking skills, how the brain works.... I wonder who will challenge my thinking about my teaching and classroom programme now the contract has finished?"
  • "My teaching and planning has changed to automatically incorporate ICT where appropriate."
  • "Reading the thoughts and views of global web2.0 experts and communicating directly with them has given me a better perspective on the learning required for children of the 21st century."
  • "The course/conference PD provided ideas for reflecting on teaching and learning."

Teachers' classroom practices

When teachers were asked about the extent to which their classroom practices had changed as a result of participation in the ICT PD programme, over a third of them indicated their classroom practices had changed to 'a large extent' or 'completely', and just under half of them (48%) said their classroom practices had changed 'to some extent'. Only a very small group (3%) said no change had occurred in their classroom practices at all (Table 8).

Table 8: Changes in teachers' classroom practices
Extent of change in classroom practices  %
Not at all 3%
Very little 11%
To some extent 48%
To a large extent 33%
Completely changed 5%
Grand Total 1,215


There were sector (but not as in previous cohorts, gender) related differences in teachers' responses to this question. Primary teachers stated higher levels of change in their classroom practice than secondary teachers (X2>35, df=4, p<0.001). Whereas 43% of primary teachers indicated a complete or large change happened to their classroom practices, 25% of secondary teachers respectively indicated the same levels of change (Table 9). While the great majority (86%) of teachers reported at least some element of change in their classroom practices, secondary teachers were almost twice as likely as primary teachers to be in the group of teachers reporting little or no change in classroom practice as a result of the programme.

Table 9: Changes in teachers' classroom practices by gender and sector
Levels of Change in Classroom Practice Female Male Primary Secondary
Not at all 3% 5% 3% 4%
Very little 10% 13% 9% 15%
To some extent 48% 48% 44% 57%
To a large extent 34% 31% 37% 23%
Completely changed 6% 3% 6% 2%
N 907 308 867 332

Where to find out more

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065