Executive Summary

The Ministry of Education contracted with tertiary institutions to provide graduate and/or post-graduate courses and on-site support for teachers, in order to enhance the quality of teaching for Māori and Pasifika students.  The foundational tenets of QTR&D were partnerships, quality teaching, collaborative inquiry and cultural responsiveness, with each provider determining the interrelationship of these elements in their Hub.

Careful documentation and ongoing reflection within QTR&D has provided tremendous insights into the complexity and the importance of mounting large-scale professional learning programmes. This summary details the key learnings from the external evaluation of QTR&D, based on the analysis of documents submitted to the Ministry in relation to the programme, supplemented by attendance at Research Facilitators and Research Coordinator meetings, visits to several Hub presentations and interviews with key informants from within the programme.

When the Ministry of Education designed the QTR&D Programme, it was a strong statement that learning and teaching of Māori and Pasifika students is a high priority.  In its conception, QTR&D was a bold and ambitious initiative that melded many elements (partnerships, high quality teaching, inquiry, responsiveness to culture), in order to support teachers' professional learning for working in Māori and Pasifika contexts. QTR&D was built on solid foundations and knowledge about the elements of professional learning that might enhance learning for Māori and Pasifika students and the pilot process produced many examples of changes in teachers' beliefs and practices, some of which have promise for influencing student learning. Unfortunately, like many professional learning programmes before it, many of the findings were relatively limited or hard to measure and there was huge variability in the changes that were implemented and in the impact of these changes on teachers' thinking and practice and student outcomes. This is consistent with other school reform research. In the words of Richard Elmore (1996):

We can produce many examples of how educational practice could look different, but we can produce few, if any, examples of large numbers of teachers engaging in these practices in large-scale institutions designed to deliver education to most children.

Although many innovations have made their way through various educational systems, they often failed to have a fundamental impact on schooling.  QTR&D was designed in the hope of identifying promising avenues for professional learning to stimulate widespread change, with the promise of system application to address the seemingly intractable issue of Māori and Pasifika underachievement. It has succeeded in establishing a range of examples and possibilities that hold promise for future directions. In this summary, we have tried to highlight the successes and to identify the challenges that are inherent in such a multifaceted and profound undertaking.

Implementation in 9 Hubs

The QTR&D programme was successfully implemented in 9 Hubs across three language settings (English, Samoan bilingual, Māori), with over 100 teachers enrolled in courses at one of 5 different universities.  These teachers not only participated in graduate or post-graduate study but were supported by Research Facilitators to examine their own teaching practice with Māori and Pasifika students and produced a research/inquiry report of their work.

Extensive Collection of Resources and Readings

Each of the Hubs has developed and used a wide range of useful resources and readings that can form the basis of a compendium of materials for use in similar programmes. The course outlines are replete with reference lists and the Research Coordinators', Research Facilitators' and teachers' reports contain many additional references. Across the Hubs there is an extensive list of national and international resources and articles related to cultural responsiveness, practitioner research and quality teaching for diverse learners.  This material could be collated and made available to others addressing the same issues.

Identifying and Recruiting Participants

The original intent in QTR&D was to locate teachers who had been involved with previous Ministry professional development initiatives and had developed the necessary knowledge, skills and dispositions within their designated curriculum area, to conduct inquiry into their own practice with a view to improving learning outcomes for Māori and Pasifika students. Finding these people proved to be a difficult task and many of the participating teachers, although eager, required more support than was anticipated in foundational areas of teaching.  It is not clear what criteria might be used to identify ideal candidates for the kind of professional learning that is envisioned within QTR&D.  Nevertheless, deciding who should participate in intensive professional learning is likely to be a complex task that requires careful consideration.

Because teachers in New Zealand are very much connected to particular school cultures and priorities, the school setting and goals are as important as the readiness of the candidates. Recruiting candidates requires a combination of locating promising contexts where the programme is consistent with school priorities and supportive leadership and familiarising individuals within these settings with the goals of such a programme, in order to garner interest and establish the conditions in the schools to support and foster the learning programme.

Situating Intensive Professional Learning Programmes

QTR&D was intended to provide the best of academic courses and ongoing support to the participants through a focused professional learning experience that also resulted in credits towards a graduate or post graduate degree. This structure was both an advantage and a disadvantage. Being part of a tertiary course meant that the teachers involved were earning credits towards an advanced degree and they had access to high quality lecturers and university resources. They were also bound by university timelines and rules that made enrolment and some elements in the programme difficult. Although some of the Hubs experienced frustration with the bureaucracy of tertiary institutions, other universities were able to streamline the process and these are likely issues that could be addressed if there was a long-term agreement with tertiary institutions.

Of more importance is the fact that QTR&D comprises much more than one or two university papers.  It is an integrated package that requires the engagement of schools, communities, lecturers, and on-site Research Facilitators, working together over an extended period of time to provide a wide range of learning opportunities. This ongoing individual support provided to participants was an essential part of the process as experts and allies in the teachers' change process who guide and lead the teachers towards routine collaborative inquiry that happens every day, not just at the university in a paper. This raises the question of whether tertiary institutions are the ideal setting for an extended professional learning programme.  It is even uncertain whether tertiary institutions would or could offer a professional learning programme like QTR&D if it were not funded by the Ministry of Education.

The Importance of Support

Changing beliefs and practices is hard work.  Most often, even if people realise that something needs to change, they "don't know what they don't know" and may be reluctant to pursue new learning alone.  In fact, the most comfortable position is generally to preserve and conserve the status quo.  However, dissonance and disequilibrium are critical prerequisites to new learning and it is hard to get there without help to push beyond the comfort zones. QTR&D was designed and delivered in a way that was very supportive of the participating teachers, while also challenging them to change their beliefs and their practices.

Throughout the programme teachers were supported by the Research Facilitators most of whom provided regular and personalised support to help the teachers integrate their learning in the tertiary course with their classroom practice and to undertake their research/inquiry project.  In some cases they served as "critical friends" – trusted people who ask provocative questions, provides data to be examined through another lens, and offer critique of a person's work, as friends (Costa & Kallick, 1995, p. 154). In this role they could challenge assumptions, beliefs or simplistic interpretations in non-judgmental and helpful ways, remind the participants of what they had accomplished and facilitate their movement towards the next goals. This support was an essential element in QTR&D that created an environment for experimentation and reflection, in which teachers felt safe engaging in new learning and practices.

Partnerships and Relationships

QTR&D focused on partnerships and relationships across various groups to create the social capital to move the principles of QTR&D forward within and beyond the programme.  The programme facilitated the development of some relationships with the potential to have lasting influences for professional learning for teachers of Māori and Pasifika students and it was less successful in establishing models of other relationships that could be productive.

The partnership between the Ministry and universities has shown that it is possible, although sometimes logistically problematic, to combine professional development for teachers with graduate or post-graduate courses.  These partnerships have also helped to highlight Ministry documents and policies in the tertiary sector, albeit with a limited number of the faculty.  Of more significance is the number of tertiary personnel who worked together within and across the universities in QTR&D.

Relationships have been established that were not only very productive in sharing and extending expertise but are also likely to continue in other initiatives related to and investigating professional learning for teachers. Relationships among the participating teachers within QTR&D were less evident and, in most cases, they were unlikely to become lasting contacts that would move professional learning forward over time.

New relationships were evident between students and students and teachers and students in a small number of classrooms.  These changes had a powerful effect on individual teachers and will probably affect their practice in the future. However, this kind of activity was limited across the programme as a whole and is unlikely to have widespread influence on other teachers or schools.

Relationships between teachers and communities, especially with parents and whanau, did not figure in most of the participating teachers' projects, except as resources to support the content of the curriculum. They were even sometimes considered to be difficult to establish and to manage.

Impact on Teachers and Students

Many of the participating teachers reported new learning and insights that caused them to rethink their assumptions and their practices.  In many cases, the teachers experimented with new approaches and activities as part of their inquiry project. However, there was great variability across Hubs and within each Hub in the depth and potential impact of the changes that teachers undertook in their classrooms.

There is some limited evidence that the changes made by teachers' influenced the Māori and Pasifika students in their classrooms but it is not clear whether these changes were sustainable for these students or if continuing with these practices would be sufficient to contribute to the achievement and success of other Māori and Pasifika students over time.

Teacher Learning Through Inquiry

Teacher inquiry (or action research) is focused on the role that teachers play as knowledge-generators (Robinson, 2003).  This process provides a vehicle for teachers to untangle some of the complexities that occur in the profession and is a critical contributing factor to improving the quality of teaching and learning outcomes for all learners.

The courses within QTR&D were action research courses that introduced teachers to a process of examining an existing problem; intervening with actions to improve practice; and monitoring the effects of the action through data collection and analysis. The courses and research/inquiry assignments in all of the Hubs focused on teachers studying about teaching for diverse learners and using this new knowledge to change and investigate their own practices.  This approach certainly focused teacher attention on their work with students.  However, the inquiry cycle that has emerged from the Professional Learning and Professional Development Best Evidence Synthesis moves the process of teachers investigating their practice even closer to students (Timperley et al., 2008). This inquiry cycle begins with a consideration of student learning needs, moves to an explicit articulation of the relationship between teacher practice and student learning in relation to the student learning requirements, and charts a course for professional learning that will deepen professional knowledge and translate into focused changes in practice for those students.

Action research and focused inquiry both contribute to professional learning for teachers.  However, if the goal is enhanced learning for students, focused inquiry begins with the students and focuses the teachers' learning and investigation of practices to students' needs directly.

We raise the question, therefore, of whether the inquiry projects undertaken were focused enough or were sufficiently presented as the core business of a professional teacher, as opposed to being considered as "research" projects that are required in tertiary study. Although the label "research" gives recognition to the effort involved, when participating teachers perceive the inquiry process as something uniquely associated with a university course it may have an unanticipated consequence of reducing teachers' realisation that inquiry, particularly collaborative inquiry is an important new skill that should be embedded in their professional practice if outcomes for students are to improve.

Collaborative inquiry is a systematic process for learning in which a group works together in repeated episodes of reflection and action to examine and learn about an issue that is of importance to them. Engaging in collaborative inquiry allows educators to work together searching for and considering various sources of knowledge (both explicit and tacit) in order to investigate practices and ideas through a number of lenses, to put forward hypotheses, to challenge beliefs, and to pose more questions. It is the foundation of conceptual change as individuals come across new ideas or discover that ideas that they believe to be true don't hold up when under scrutiny. Although many different forms of collaboration were evident in the QTR&D Hubs, only a few groups appear to have moved from what Warren Little (1990) calls "sporadic contacts and idiosyncratic affiliations among peers" to "joint work of a more rigorous and enduring sort" that is likely to produce genuine new knowledge for the teachers involved.  By recognizing the natural human propensity for assimilation, it is possible to understand just how difficult "inquiry", as a need for deep understanding, truly is.  Deep understanding very often means much more than confirming what is already known as tacit knowledge (or what people think they know).  It means changing what people think and know.  Inquiry involves changing the filter in a way that fits the evidence, not just engaging with the evidence in a way that fits the filter.  This is the hard work of conceptual change.  It means learning to live with the ambiguity and the feeling of dissonance when tacit knowledge and evidence are incompatible and recognizing that this kind of psychological discomfort is necessary to new understanding.

Having teachers engage in a single action research or inquiry project is unlikely to result in widespread change. The long-term strength of action research and the focused inquiry cycle appears when they become a way of doing business, a way of thinking, a habit of mind, rather than discrete events. For collaboration to enable routine knowledge creation and sharing, practitioners need forums to make their knowledge accessible and explicit, and then to subject it to scrutiny and challenge from evidence and argument and external expert facilitation in the particular area of focus.

Cultural Responsiveness

Responding to the unique cultural reality of individual students is a complicated and sometimes problematic undertaking that is not well-established in education. Teaching generally occurs with a class of students and teachers use their own mix of activities and materials, adjusted to suit their understanding of what their students need.  Cultural responsiveness is much more than introducing myths or metaphors into classes. It means interacting with the students and their families to truly understand their reality; it means understanding the socio-political history and how it impacts on classroom life; it means challenging personal beliefs and actions; and, it means changing practices to engage all students in their learning and make the classroom a positive learning place for all students.

QTR&D did not impose a specific definition of cultural responsiveness beyond the principles that 'culture counts' and "culturally inclusive and responsive learning communities". The differences among the QTR&D Hubs in their methods for and attention to cultural responsiveness demonstrate that being responsive to culture as a mechanism for enhancing students' learning is complex and challenging.  The approach, and even the orientation to responding to culture, was grounded in each Hub by the literature and current theories espoused in the curriculum area and demonstrated the lack of a shared understanding of this complex notion, even among providers.

Given the variability in interpretation and approach to cultural responsiveness across the QTR&D Hubs, this appears to be an area that requires considerable further attention and study to clarify the concept and engage the teaching profession in ongoing dialogue about what it means to be responsive to culture, what it might look like in practice, what pitfalls exist for teachers as they pursue this direction and what kinds of supports are necessary to facilitate cultural responsiveness. The QTR&D programme was able to make an initial foray into this area that provides a basis to be built on in future.

Scope of the Programme

The QTR&D programme was very ambitious and the challenge of the endeavour may have been underestimated in a number of areas. All of the Hubs found the QTR&D programme ambitious and challenging to deliver in a bounded time frame. The major concepts within QTR&D (partnerships, quality teaching, responding to culture and collaborative inquiry) are all major areas for learning and for change. Several of these concepts, cultural responsiveness and collaborative inquiry, are relatively rare in education and there is considerable evidence that they require teachers to develop quite important but challenging new skills that require a good deal of new learning, reflection and practice to become automatic ways of working.  The participating teachers undertook a single research/inquiry project that introduced them to a process and some new skills, but this was not sufficient for them to become routine inquirers, always interrogating their own practice in relation to their students and the curriculum.  Cultural responsiveness is much more than sharing traditions; it is also a way of being and of thinking that requires teachers to confront their own personal beliefs and their relationships with students and with communities, as well as learn new customs and new language.

The other QTR&D concepts, partnerships and quality teaching, also proved to be more difficult that might have been anticipated.  Many of the teachers found themselves lacking content and pedagogical knowledge in the curriculum areas, and some were even unsure of the curriculum itself. They were constantly building their teaching and assessment repertoire, as well as working on responding to the cultural realities of their Māori and Pasifika students.  QTR&D did create some important partnerships within the programme, especially across tertiary personnel in Hubs. The intent was also to engage the community, as a mechanism for enhancing relationships and connecting student's disparate worlds.  In this realm, there was little evidence that the QTR&D programme inspired many connections or links with communities that were likely to extend beyond a short contact for a particular purpose.

Given the complexity and challenge of embedding these important ideas into a single programme, the QTR&D programme provided the participants with a "taster" of ideas and processes that positioned them to begin the journey of collaborative inquiry and cultural responsiveness.  However, it is clear that none of the concepts could be explored and experienced in depth in a short time frame and can only be addressed in a systemic programme over an extended period of time.

Developing a Knowledge Base

One of the key goals of QTR&D was learning from the pilot study and sharing the learning with others beyond the participants.  Although QTR&D was a worthwhile professional learning project that provided over 100 teachers with exposure to new ideas and created the experiences for them to develop professionally,  this was an individual process, rather than a collective knowledge-building one. Within several of the Hubs, the teachers' stories have been collected, edited and collated for sharing with other groups and there are some plans for broader sharing but, at the time of writing, it is unclear how much additional dissemination there will be.

We hope that the dissemination of learning from this evaluation and from the reports produced by the Research Coordinators and Research Facilitators can capture the big ideas in ways that can be shared more broadly to stimulate discussion and provide the foundations for future programmes.

Summary

QTR&D was initiated with the realisation that large-scale educational change is challenging and multifaceted, especially given the importance and range of dimensions that were included in the programme.   Each Hub entered into the programme wholeheartedly and mounted a valuable learning experience for the participating teachers. They also found that deep and widespread change is hard to achieve. It is clear that without thoughtful attention to the underlying theoretical and logistical issues, it is very difficult to create the kinds of conditions and experiences for teachers to genuinely learn together and change their practices, individually and collectively to support learning for Māori and Pasifika students.  At its foundation, QTR&D was intended to change the hearts, minds and behaviours of the participating teachers.  There is certainly evidence of some of these changes, with some of the participants, but there was tremendous variability and it is not clear how much change will be sustained over time.  Nevertheless, QTR&D has provided incredible insight into the complexity of changing practices in ways that will have a significant and sustained influence on Māori and Pasifika students.