Executive Summary

This research report is the third of three submitted to the Ministry of Education as part of an evaluation of the so-called 'New Initiatives' announced in the 1998 document Interactive Education: A National Strategy for Information and Communication Technologies in Schools (Ministry of Education 1998). The 'New Initiatives' were:

1. The Principals First: First Principles programme of professional development in ICT for school principals.
2. The Online Resource Centre, subsequently entitled Te Kete Ipurangi (TKI).
3. The 23 ICTPD School Clusters programme of professional development for teachers in ICT.

The purpose of the study reported here was to evaluate the 23 ICTPD School Clusters programme of professional development (PD) for teachers in ICT, as implemented in the period from mid 1999 to the end of 2001. More specifically, the research questions guiding the evaluation were:

1. What were the most effective operational characteristics of the various 'school cluster' PD models in meeting stakeholder and participant goals?
2. How effective were the ICTPD cluster programmes in terms of promoting administrative efficiency, policy development and strategic planning for ICT in participating schools?
3. How effective were the ICTPD cluster programmes in increasing teachers' skills and knowledge related to the educational applications of ICTs, and in increasing classroom usage of ICTs?
4. What has been the educational worth or value to students of the ICT-based learning activities implemented by participating teachers in their classrooms during the course of the ICTPD programme?

The main findings of the study with regard to these questions are:

1.  Effective school cluster models for professional development in ICT

• Within broadly similar parameters, the goals and objectives pursued by particular clusters, and by particular participants groups within the clusters, were quite varied, as were the modes of operation by which particular clusters and cluster facilitators implemented their professional development programmes.
• Principals and teachers both tended to concentrate on goals related to increasing teachers' ICT skills and increasing their use of ICTs with students in classrooms. Facilitators were as a group more focussed on objectives related to increasing teachers' pedagogical understandings and the effective practice of classroom teaching and learning.
• Overall, participants reported very high levels of goal achievement as a result of the ICTPD programmes, although there were some role and sector based differences in the extent to which participant groups felt they had achieved particular goals or groups of goals. The particular benefits of the professional development programmes highlighted by teachers were the sharing of professional expertise, increased confidence in relation to ICTs, and developing understandings about both the practice of, and the professional rationale for, teaching and learning with ICTs.
• The basic concept of school clustering for the provision of professional development through devolved funding was widely seen as having been very successful. Where difficulties were experienced they were seen more as the result of practical implementation difficulties, often related to the organisational or political dynamics of a cluster than to the concept of clustering in itself.
• While there was no single, replicable recipe for teacher professional development that showed itself to be more effective in meeting stakeholder goals than others, there was a fair consensus as to what the ingredients in such a recipe might be. The main factors affecting the extent to which clusters' PD programmes met stakeholder goals and objectives were:
• • The organisational form of the cluster. The sector composition of clusters, the geographical spread of the cluster, the number of schools, or more importantly the number of teachers, involved in the programme, the frequency and timing of professional development events, and the length of time an individual teacher was actively involved in a programme were the most important organisational factors affecting cluster performance.
  • Programme content. Programmes which focussed on all three of personal skill development, practical classroom ideas for the use of ICTs, and the development of sound pedagogical or theoretical rationales for the use of ICTs in teaching and learning had more wide ranging, and possibly more long-term, effects than those which focussed more narrowly.
  • Professional development strategies. Teachers valued most those professional development strategies which maximised the time available to them to come to grips with ICT skills and uses, and which combined substantial `time out' on the one hand with ongoing access to collegial support on the other. 'Trickle down' models appear to have been more successful in relation to the development of skills in schools than in the development of ICT-related pedagogical understandings and professional rationales.
  • Social, interpersonal and political dynamics. The extent to which programmes acknowledged and addressed the affective domain needs of teachers in relation to ICT, the terms of employment of the facilitators, and even more importantly, the professional abilities of the facilitators in a teacher-educator role and the extent of commitment, collaboration and understanding shown by senior management in participating schools, all had a significant impact on the operation and effectiveness of the programmes
  • Sponsorship and national coordination. Several clusters made effective use of supplementary external sponsorships of various kinds to the benefit of their programmes. The regional and national meetings, workshops and conferences organised and run by the National Coordinator were particularly influential in building the expertise, competence and collegial networks of the cluster facilitators.

2.  ICT uptake in cluster schools: policy, planning and administration

• There was a marked increase in most participants' use of ICTs for certain aspects of administration and most aspects of lesson preparation over the period of the ICTPD programme, although this was in many cases only the indirect or partial result of the operation of the ICTPD programmes themselves.
• The extent of increased ICT usage for administrative and lesson preparation purposes tended to be exaggerated in the minds of principals and facilitators, compared to the perception of the teachers themselves.
• Increases in teachers' use of ICTs for administrative purposes across the clusters were greatest with regard to writing reports for parents, using spreadsheets or databases for assessment and assessment records, and emailing colleagues on administrative matters.
• Increases in teachers' use of ICTs for lesson preparation across the clusters were greatest with regard to using DTP or word processing packages for the preparation of lesson resources and using the internet to search for lesson ideas and resources.
• While the principals of the participating schools and the programme facilitators both hoped to increase the incidence of policy and planning development on ICT in schools through the ICTPD programmes, these goals were seldom shared by participating teachers, whose focus was clearly on technical upskilling and techniques for the use of ICTs for teaching and learning in classrooms.
• By the end of the project all participating cluster schools had developed strategic plans or policies for ICT, though in relatively few clusters was this the direct or exclusive result of the ICTPD programme operating in the cluster. Primary school principals appear to have been more satisfied with the policy and planning outcomes of the ICTPD programmes than secondary principals.

3.  ICT uptake in cluster schools: teachers' skills, knowledge, attitudes and classroom usage

• As a result of their participation in the ICTPD programme, teachers made significant gains with regard to their understandings of the roles of ICTs in teaching and learning, their personal competence with ICTs, and their confidence about using ICTs with students.
• The ICTPD School Clusters programme directly and positively impacted on the general teaching practices of most of the participating teachers. The great majority of participating teachers reported that the ICTPD programme had increased their effectiveness as teachers, had increased their enthusiasm for teaching, both generally and for using ICTs in particular, and had helped them to offer more varied, motivating and creative teaching/learning activities in their classrooms.
• The main perceived effects on teaching of introducing ICT-based activities into classroom learning programmes were to make teaching more learner centred and more interesting, and to increase teacher stress and workload. Teachers were divided on whether or not incorporating ICT activities made it easier or more difficult to teach their class as a whole, or easier or more difficult to individualise lessons.
• The main concerns participants had at the end of the project about ICTs in teaching and learning were: a continuing frustration with equipment failure and, especially in isolated areas, an unreliable infrastructure, a lack of access to ICTs on the part of their students, a heightened awareness that the need for professional development in the area was likely to continue after the end of the project, and a lack of time in which to undertake such development.
• Significant gains in participants' ICT competence were found across a range of ICT skills and software applications. The greatest gains occurred in relation to the basic operations of computers, graphics applications and the use of the internet. The relatively high rates of no change in relation to teachers' skills with spreadsheets and databases are probably attributable to the fact that these applications were not emphasised in the content of many PD programmes.
• Teachers gained in competence through the ICTPD programmes, but it was probably associated affective domain gains in confidence and the connection of ICT use with their understandings about teaching and learning that primarily determined whether or not such personal competence translated into increased use with students in classes.
• Both the confidence and competence gains made as a result of the ICTPD programme were more marked among primary teachers than among secondary teachers, and more marked among female teachers than male teachers.
• There was a substantial increase over the period of the contract in the frequency of usage of ICTs in the classrooms of teachers in the ICTPD programmes, but rather less growth in the range of ICTs individual teachers used with classes.
• At the beginning of the ICTPD programme almost a third of participants had not used ICTs with classes at all, and just under half had used ICTs with classes 'only once or twice'. By the end of the PD programme approximately 70% of participating teachers were using ICTs routinely with their classes.
• The increase in the frequency of teachers' use of ICTs for teaching and learning in classrooms was significantly greater for participating primary teachers than for participating secondary teachers. Primary teachers were also more likely than secondary teachers to incorporate a broad range of ICTs in their classroom teaching.
• In terms of the Essential Learning Areas, ICTs were used most often in cluster classrooms to support objectives in the Language Learning Area, and least often to support The Arts, Technology and PE/Health. Moreover, in all Learning Areas a core group of generic ICTs were used more often than subject specific or content based ICTs.
• By far the most frequently reported and observed usage of ICTs in participating teachers' classrooms was as a tool or medium for the presentation of work or for information gathering through the Internet. Presentations were sometimes in the form of authored multimedia productions but were mostly in static print form. Other curriculum uses made of ICTs, in order of frequency, were:
• • using CD-ROM based references or the Web for information gathering.
  • using software such as simulations, spreadsheets, or databases as tools for curriculum related problem solving.
  • using content specific programmes for subject specific skill practice or knowledge recall.
• As a general rule, the length of time teachers were involved in an ICTPD cluster programme did not significantly affect their growth in terms of acquiring personal ICT skills and competencies. However, the length of time teachers were actively participating in the ICTPD programmes did significantly affect their level of confidence about using ICTs with classes, and the frequency with which they actually used ICTs with classes.
• A minimum of six months active involvement in an ICTPD programme was apparently necessary in order for an individual teacher to significantly increase the frequency of their classroom use of particular ICTs.
• Teachers who were involved in the ICTPD programmes for the full three years were significantly more confident about classroom use of ICTs, and significantly more likely to increase the frequency of their use of ICTs with classes, compared with those who participated for shorter periods.

4.  The educational value of students' use of ICTs in cluster schools

• For teachers, the main perceived effects on student learning of introducing ICT-based activities into classroom learning programmes were to make learning activities more varied, to increase the range of skills and abilities demonstrated by students, and to increase students' motivation.
• The great majority of teachers did not believe that ICTs in themselves had led to increased student achievement as measured by formal testing, although almost half of the primary teachers did express the view that incorporating ICT-based activities had increased the amount of higher order thinking demonstrated by children in their classes.
• Teachers tended to judge the value of ICT-based classroom activities in terms of curriculum relevance, cognitive gain, inter-student collaboration, student-centredness, and, especially, student interest/motivation, derived either inherently from the use of ICTs as such, or from an increased variety and range in the skills and abilities exercised. Many also justified such use wholly or partly in its own terms. That is, it was seen as important for students to learn ICT skills for their own sake as well as to use them to support their cognitive and social learning in specific curriculum areas.
• The research team assessed the quality of ICT-based teaching and student learning in the participant teachers' classrooms in terms of the following indicators:
• • The range of ICTs students used and the extent to which ICT-based learning activities related to particular Curriculum Objectives.
  • The levels of technical (ICT) capability required of, and demonstrated by, students in their use of ICTs.
  • The range of Essential Skills intended and applied in the students' use of ICTs.
  • The levels of cognitive or creative ability required of, and demonstrated by, students in their use of ICTs.
  • The effectiveness of student collaborations when using ICTs.
• The great majority of ICT-based classroom activities clearly related to curriculum objectives. Most observed and reported activities related the Language Essential Learning Area, followed by Technology, Science, Social Sciences. Maths, Health & PE, and the Arts.
• The technical ICT abilities and operational knowledge of the students observed were generally very high, except in terms of on-screen multi-tasking and more complex transformations of data. However, there was a tendency for students either to know how to operate aspects of a piece of software, or not to know. Few chose to problem solve their way out of technical or operational difficulties, preferring to seek help from the teacher or another, 'expert', student.
• In terms of the Essential Skills there was an emphasis on the middle and lower order elements of Communication Skills, Information Skills and Problem Solving Skills in students' classroom use of ICTs.
• The thinking levels required by, and demonstrated during, ICT-based activities were mostly at the Knowledge, Comprehension and Application levels on Blooms' Taxonomy of Thinking Skills. Activities at the Application level were as often related to the application of ICT knowledge and skills as to the application of curriculum based principles or skills.
• Generally, student collaborations around the classroom computer were effective, both in terms of task completion and learning benefit. Teachers often had pragmatic (access) reasons for setting up collaborations as well as curriculum reasons. There was also a high proportion of unplanned and informal student collaborations around computers, often focussed on aspects of the technical operation of the software.
• There was a significant amount of 'sharing of ignorance' among students with regard to technical skills, especially in lower primary classes (NE-J2), where student 'experts' were often not as 'expert' as the teacher expected them to be.
• Most of the teachers on the ICTPD programmes had used ICTs little with classes before the programme. Taken as a group, by the end of it they had achieved relatively high levels of 'incorporation' of some ICTs into their teaching programmes, but only moderate levels of 'integration', if the latter is taken to mean the routine, ubiquitous, authentic, transparent and monitored use of a wide variety of ICTs in contexts which optimise the educative 'value' and 'quality' of the experience for students.

Downloads