Evaluation of Kaupapa Ara Whakawhiti Mātauranga (KAWM)
Kaupapa Ara Whakawhiti Mātauranga (KAWM) encompassed a number of school improvement initiatives and aimed to:
- improve student achievement;
- improve school performance;
- strengthen school and community relationships;
- upgrade school ICT infrastructure; and
- improve teachers' professional capability through ICT.
Author(s): Pauline Waiti, New Zealand Council for Educational Research.
Date Published: 2005
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
The above list of initiatives were five school clusters within the umbrella of KAWM, and the strategies used to achieve the aims of KAWM were specific to the respective clusters involved. This evaluation focused on four of the five clusters.
- The wharekura cluster participated in all four aspects of KAWM. They received video conference kits with technical support, training for the wharekura e-teachers, laptops and professional development for their teachers, and school-based ICT infrastructure.
- The Paerangi schools' cluster participated in three aspects of KAWM and received the video conference kits with technical support, laptops and professional development for teachers, and school-based ICT infrastructure.
- The Wairoa schools' cluster and Kiwa (Gisborne) cluster participated in one aspect of KAWM only and received thin client networks with high-speed network connections and technical and learning facilitation support. This aspect, school-based ICT infrastructure, is called Project Rorohiko for the Wairoa and Kiwa clusters.
NZCER was contracted by the Ministry of Education to undertake an evaluation of the KAWM initiatives during 2002–03.
The key questions guiding this research were:
- What use is actually being made of each of these four aspects of KAWM? What factors lie behind any differences in use?
- What are the relationships between the uses being made of these three ICT tools, and the Wharekura Expert Teachers' initiative, and wharekura and school curriculum organisation and content, teaching approaches, learning, student interest, teacher interest, motivation, retention, professional support and development, inter-school collaboration, school management, and community involvement and support for wharekura and schools?
- What are the conditions, skills, learning, and resources which allow the best use of KAWM, and, conversely, what are the conditions, skills, type and amount of learning, and resources, which inhibit its use, or which raise questions about its sustainability?
- What are the implications or lessons to be drawn from the initial implementation and use of KAWM for other policies and initiatives?
In 2002 and again in 2003 interviews with principals and teachers were undertaken in a total of six wharekura, three Paerangi schools, seven schools in the Kiwa cluster, and six in the Wairoa cluster. Twelve video conferencing classes and 59 ICT-related classes were observed. The findings from the first round of fieldwork in 2002 were reported in June 2003. This report draws on the findings of the fieldwork from both 2002 and 2003.
Summary of main findings in relation to the research questions
Use of the four aspects of KAWM
The online classrooms and Wharekura Expert Teachers' initiative
The wharekura and Paerangi Mäori boarding schools decided to participate in the KAWM project to provide their senior students with a wider curriculum and to facilitate building and extending relationships between schools. The video conferencing classes were referred to respectively as Te Kura Ataata and Te Kura Hiko in each cluster, and both encountered a number of issues in implementing video conferencing classes. Developing a shared timetable for video conference lessons proved difficult until the appointment of a part-time KAWM co-ordinator in mid 2002. Initially the technology proved temperamental and the workload of the e-teachers, particularly within the wharekura, was very high.
Over time the quality and reliability of the technology improved, and the new bridge due for installation from the beginning of 2004 was expected to provide additional quality. The funding provided by the Ministry of Education to support online teaching enabled the wharekura to give the e-teachers more release time for preparation and greater administrative support. It was also found that all classes required a supervising teacher and this placed an additional burden on kura that were already facing staffing shortages. The demanding workload of e-teachers was partially addressed by strategies such as appointing a person to manage liaison with offsite students and providing support for resource development and production. There was a growing trend towards electronic resource sharing between e-teachers. There was the potential to exploit this further, for example by establishing a database of electronic resources suitable to be used in the online classroom, but at the time of the evaluation there was not the critical mass of available electronic resources to make this worthwhile. The e-teachers themselves recognised the skills they developed through experience and felt they had much to offer new e-teachers.
Despite the challenges in implementing the online classroom, principals, teachers, and students were supportive of the video conferencing initiative and believed it was the best option in providing senior students with greater subject choice in te reo.
The uptake of online classes offered by Paerangi teachers to Paerangi students was more limited. However, because they did not require lessons to be in te reo, Paerangi schools also had the option of online Correspondence School courses through video conferencing, and the numbers of students participating in these classes have increased. Student participation in Correspondence School courses was viewed as a pragmatic decision because it placed fewer demands on the school involved.
In both the wharekura and Paerangi Mäori boarding schools, the effective implementation of the online classroom required e-teachers to think about their teaching in new ways. There was a need to plan ahead so that the online students had the required resources, and teachers needed to seek new ways to interact with their students to provide quality feedback. Hui were held at the beginning of the year for the online students and their teacher and at times during the year. These hui were seen as critical in developing the relationships needed for teachers to effectively support their students' learning.
The video conferencing equipment was used for online classes for students, online social hui for students, online meetings for principals, and in Te Kura Hiko, for online meetings of hostel staff. These activities were dependent on careful planning by a co-ordinator and in the case of the online classes also relied on the support of a supervising teacher. It was found that the online lessons needed to be supported by opportunities to build a relationship between the teacher and online students and this was achieved through face-to-face hui. These hui, and other inter-school events, helped to build relationships between the students in the respective clusters, an opportunity valued by the principals, teachers, and students.
The laptops and professional development
There was general satisfaction with the ICT Laptop professional development programme although it was thought it would have been more valuable to spread the programme over 2 years rather than having it condensed to 13 weeks. Access to a laptop and the skills to use it enabled teachers to use computers on a regular basis to plan, gather and prepare resources, complete administrative tasks, and communicate with other teachers. Further professional development and onsite technical support were seen as key to improving classroom-based ICT use.
The school-based ICT infrastructure
The school-based ICT infrastructure made a difference in schools, improving processes and opportunities for students to interact with ICT, despite a number of technological problems identified by most schools. By 2003 most schools were working to develop the "personnel infrastructure" that was needed for the effective use of the network as well as to support its ongoing technical development.
Teachers generally enjoyed their own and their students' better access to ICT, and wanted to use it. As the teachers became more experienced they were extending their ICT use from administration and lesson planning tasks to activities that supported their classroom programmes. Many teachers believed that the use of ICT within classroom programmes served to motivate and engage students by providing a greater variety of experiences. There was evidence of change in ICT use in all clusters over the 2-year evaluation period. While learning to use the tools of ICT appeared to be the primary purpose of many classroom ICT-related lessons, students also began employing a greater range of software programs and utilising multimedia as well. In 2003, ICT activities directly related to learning within other curriculum areas, such as literacy and numeracy, were more evident in the classrooms observed within Project Rorohiko. In all the clusters, however, many principals and teachers expressed concern about their ability to access ongoing professional development that would enable the more effective integration of ICT in all curriculum areas.
The principals in all clusters believed that the ongoing maintenance and development of their ICT infrastructure will require continued additional resourcing, particularly in the form of expert technical assistance. Most principals also identified the need for assistance with professional development to support teachers' use of ICT within the context of all curriculum areas.
The relationships between the uses being made of KAWM and characteristics of schools, students, teachers, professional development, and community
There did not appear to be any relationship between the uses of KAWM and the size or type of school. Over the 2 years there was little community use of ICT in any of the schools, although some intended to encourage this development. There were two key factors in the variable use made of KAWM in the schools within this evaluation. First, the "readiness" of the school to utilise the support and resources provided. "Readiness" was influenced by factors such as the ability to get the required infrastructure installed and operating effectively, having the necessary administrative and technical support systems in place, and having sufficient professional development to enable everyday use of ICT. Second, the degree of alignment between the initial reasons the school joined KAWM, and the aims of the KAWM initiative itself. Most of the wharekura, for example, were highly motivated to get the video conferencing infrastructure in place, as they saw it as the best option in providing their senior students with greater curriculum choice.
The factors that allowed the best use of KAWM
Successful implementation of initiatives such as KAWM are dependent on the principal for leadership, guidance, organisation, and motivation—an issue raised by a number of teachers interviewed. Most principals were positive about KAWM although initially some had reservations because "it came like a bolt out of the blue" and they did not have enough time to consider the long-term practicalities of the project. Many principals were happy with the educational promise of the equipment and technical support, and committed to the project on this basis. Generally, the principals were unclear about what was expected to be achieved through their involvement in KAWM and it took time to determine their roles and responsibilities. As principals determined their school's own priorities around ICT, they focused on those most relevant to them at the time, and so there was not necessarily a direct link between the school-based goals and those of the KAWM project. It was not just the principal's leadership that was important as there was evidence of distributed leadership for ICT development, especially in the larger schools. The role of the enthusiastic and committed teacher was another key factor that facilitated good use of KAWM.
The cluster structure provided support networks for many schools, including technical, professional, and collegial support. It was somewhat easier for the wharekura and Paerangi schools to network effectively on a number of levels as the schools within these clusters had a long history of networking prior to KAWM and much more in common than just belonging to this initiative. The cluster arrangement provided an opportunity for inter-school sharing of resources and expertise and there have been some initial developments, such as sharing of resources between e-teachers, but time and other resource constraints have limited such sharing. There is considerable scope for building on the existing cluster arrangements, and expanding these. This would provide the opportunity for more interactive relationships between teachers who are seeking to develop their knowledge of teaching and learning using ICT.
Equipment and technology
Reliable equipment was key to making the best use of KAWM initiatives. This depended on not only the quality of the equipment provided, but also on the quality of the ICT and physical environment it went into, the compatibility of the new and exciting systems, the availability of adequate technical support, and on what people expected the equipment to be able to do. Initially, high expectations that could not be met within the expected time frame dampened some people's enthusiasm. Some schools that already had an established ICT infrastructure were already in a state of "readiness" to implement another initiative such as KAWM, while others felt they were always "trying to catch up".
The major issue at the end of the KAWM evaluation is the ability of the schools to maintain the current equipment and to make informed decisions about updating hardware and software. These are complex decisions as they need to take account of a variety of factors such as: the values and priorities of the school; purposes for which the school is using ICT, for example online classes; financial constraints; the expertise of teachers; and the availability of technical and professional support. In developing a long-term strategy to address issues of maintenance and development, principals and boards need a clear indication of the nature of ongoing targeted support from the Ministry of Education so they can factor this alongside their own resource allocation.
Technical support was available to all schools, but immediacy of access to it became an issue for schools, even the schools that are not remote. Those schools that hosted the cluster ICT technician were definitely at an advantage, and after 3 years most schools thought all schools should have access to technical support on their own site at all times, as the on-call support they experienced was not satisfactory.
Professional support and professional development
Professional support provided by the KAWM co-ordinator or the learning technologies facilitator/ICT lead teacher was another pivotal factor in the success of KAWM in schools. Such support is an ongoing issue because if progress to date is to be built upon, there is a need for professional development to move beyond a skills focus and into teaching and learning. Training for the online teachers, for example, provided them with the technical skills in using the video conferencing equipment. It did not provide the opportunity for the teachers to develop their pedagogical content knowledge in the context of ICT. Similarly, the majority of teachers using laptops and those who had the thin client networks in their classrooms identified the need for a concerted professional development programme to support their attempts to integrate ICT into their teaching.
Associated with this recognised need for knowledge about how to use ICT to support learning is a concern about having the time and expertise to select new software and other resources that best suit the needs of the teachers and their students.
Lessons from KAWM for future e-learning development in New Zealand schools
The KAWM evaluation provides useful messages for teachers, principals, professional development providers, and policy makers who seek to support similar kinds of e-learning developments in schools. The KAWM evaluation also provides an opportunity to examine some of the "big picture" issues for e-learning development in New Zealand. Three areas have been highlighted as important for the consideration of e-learning development in New Zealand schools (Wenmoth, no date). The first area relates to the changes in thinking about curriculum and pedagogy that are often linked to the idea of learning in a digital age. In particular, the idea that the digital age will require new ways of thinking about "knowledge" (and hence, curriculum), and a shift towards "learner-centred" rather than "teacher-centred" pedagogies. This requires a fundamental shift in thinking from seeing knowledge as static, an artefact that can be shared and exchanged, to regarding knowledge as dynamic and evolving. For schools, this will require an increased emphasis on the processes of constructing knowledge and the skills required to do this. The KAWM evaluation highlights the need to provide clearer transitions or links between these future-focused ideas about teaching and learning in "digital age" schools, and the current curriculum, and teachers' current pedagogical practices.
The second area of concern pertains to issues of ICT infrastructure in a "networked" learning environment. The learning experiences of a networked learner depend heavily on the quality, accessibility, and reliability of the network to which they are linked, and this network comprises everything from computers and peripherals to the wires and infrastructure that link them together. Because a networked learning environment can transcend the immediate environs of the individual school, issues of standards and interoperability become relevant. The KAWM initiative gave the schools involved sufficient support to provide a "critical mass" of hardware, software, technical, and professional support for each one of the initiatives to make a difference to the schools involved. However, different standards of technology within and between clusters created some problems for the KAWM schools. Principals and teachers commented that the current ICT infrastructure was failing to keep up with their expectations for technological integration into teaching and learning. It was suggested that there needed to be a national infrastructure in place, centrally controlled and allowing all schools participating in a networked learning environment to have the same standard of technological infrastructure. The KAWM evaluation supports the need to define a networked learner/an e-learner and proceed to furnish this kind of learner with the appropriate standard and types of technology. With the limited resourcing schools have available, the technology issues will need to be addressed in a way which involves strategic and coordinated planning, development, and implementation that will see ICT effectively support learners.
The third area for consideration is the way that funding and support for schools, and the school system, is administered within the government's budget for education. The KAWM evaluation included discussion about appropriate structural arrangements needed to achieve excellent outcomes in e-learning including the roles of key players such as the Ministry of Education, the elearning clusters, and individual schools. In every KAWM school there was comment about the role of the Ministry of Education in the project, from "joy" that it was actually happening, to "despair" about the extra workload participation entailed. The schools were grateful for the support they were receiving, however some were concerned about the ad hoc nature of the development, and schools were hoping for a co-ordinated approach to planning in the future.
Schools had concerns that KAWM was "always a project" and so they found it difficult to think of KAWM as part of their "normal ongoing business".
Schools thought the following issues could be addressed centrally rather than have individual schools "problem solve" their own way around these.
- Funding – the continuation/sustainability and level of funding and the criteria used for this.
- Technology – the standards currently in schools were not consistent enough to truly support the New Zealand networked learner.
- The learning environment – schools wanted furniture and teaching spaces that would enable ICT to be accommodated in every classroom.
School staff also commented on the need for further resource development and teacher professional development to support e-learning development in schools.
A co-ordinated approach to the development of a framework for e-learning in New Zealand, informed by research and best practice, would help to guide schools and policy makers. However, such a framework would need to achieve a balance between centralised decision making to support e-learning development in all schools, and schools' roles and responsibilities to selfmanage in order to:
- meet their own educational goals and priorities; and
- develop approaches to curriculum and pedagogy that are appropriate to the schools' underlying educational philosophies and/or the specific needs of their learners.
Where to find out more
For more information about this publication please email the: Research Mailbox