Evaluation of Te Kotahitanga: 2004-2008
In 2007, Victoria University was contracted by the Ministry to produce an external evaluation of the effectiveness of Te Kotahitanga. It is the first external evaluation of Te Kotahitanga.
This is the full technical report of the evaluation of Te Kotahitanga in 22 schools from phase three and four of the programme, from 2004-2008. Substantive findings from the evaluation report concluded that Te Kotahitanga is a sound and effective process for improving classroom teaching and learning for Māori students.
Also available on Education Counts is the Te Kotahitanga Summary Report, which outlines the key findings.
Author(s): Luanna Meyer, Wally Penetito, Anne Hynds, Catherine Savage, Rawiri Hindle, and Christine Sleeter. Report for the Ministry of Education.
Date Published: August 2010
This report is available as a download (please refer to the 'Downloads' inset box). To view the individual chapters please refer to the 'Sections' inset box. For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Chapter 3: The Te Kotahitanga Professional Development Model
In this chapter, we synthesise what we learned about a) how the Te Kotahitanga professional development model actually works in the 22 schools we visited, b) how it reflects needed pedagogical leadership for reforming schools for Māori student achievement, and c) the kind of support required to make the professional development model work well.
How the Te Kotahitanga professional development model actually works in practice
This section describes perceptions that school leaders, teachers, and facilitators articulated about the professional development model. First we will describe how they viewed the programme as a whole, then how they saw individual parts of it.
Professional development model as a whole
All of the principals, as well as most other leaders we interviewed (deputy principals, heads of department) enthusiastically emphasised value of the Te Kotahitanga professional development model, using phrases such as "by far the most effective professional development I've ever seen," "an outstanding process," and "it should be in every school throughout the country." They saw it as very well conceptualised, and as offering far more substance than what is usually the case in professional development; several expressed appreciation for its focus on improving what happens in the classroom. One principal commented that, "It's made me think along different lines in terms of the way in which we do professional development. [It's] made me observe teaching practice and learning in a slightly different way than I would have done before I became involved with Te Kotahitanga."
School leaders also saw the model as changing how teachers talk about students. As one put it:
To be brutally honest, what I don't hear anymore—whether it was here or at other schools—I do not hear "those Māori kids" I do not hear that conversation. And for too long, staffrooms have been full of dissenters. "Oh, I can't teach them." You know what I'm saying? So those conversations have almost dried up. And that may be due to a number of factors, maybe because there was nobody else listening anymore. That number of dissenters has almost disappeared. So has it changed staff behaviour? Absolutely!
Teachers' comments about the model overall were also overwhelmingly positive. We repeatedly heard comments such as, "It's so good you want more," and "It's been the best professional development for me, really helped me to reflect on my teaching and on what I can do to help the kids learning… so it's all good." One theme in their comments was their valuing the opportunity to reflect on their practice that the programme prompted. The phrase "reflection on your own practice" was heard several times, and was supported by how Facilitators discussed their work with teachers:
We question them, get them to self assess. [With] self reflecting, it's not that we're telling them you have to do this, or you should be doing this. They're reflecting on their own practice and they have to see I could do this. That doesn't take that much more to do, I could do that—and I could see how this could then fit the kids and make my life easier.
Several teachers appreciated learning a different way of teaching. As one point out, she had "been trained as a teacher who teaches with desks placed in rows, and very much transmissive;" it had not occurred to her that there was a different approach to teaching. Several teachers said that they valued the programme's focus on Māori culture, its ability to help them understand their Māori students better, and the opportunity to learn cultural concepts such as mana, wairua, and manākitanga. For some international teachers, this was their first sustained opportunity to learn about Māori culture.
The few negative teacher comments about the programme focused on four concerns. First, several teachers were put off by what they saw as its focus on Māori students to exclusion of other groups; as one put it, "If all kids are important, why do we just pick out the Māori kids?" Second, a few teachers disliked the idea that deficit theorising blamed teachers for low Māori student achievement, pointing out that there are other influences on student achievement besides the teacher. Third, a few teachers and facilitators commented that when participation in the programme was compulsory, teachers resisted. A facilitator commented, "Because it was compulsory, I felt it was obvious the teachers who didn't really want to be on there, they became the more challenging feedback sessions, in terms of their thinking." Fourth, there were a few comments that the programme needs to be more responsive to the different needs of different subject areas, such as math, a theme that recurs throughout this chapter.
The facilitators, as one might expect, were very positive about the model. They believed that the structure of the Te Kotahitanga professional development cycle—in-class observation and feedback, goal setting, shadow-coaching and co-construction meetings—enabled teachers to make positive changes. They talked about it from the vantage point of being both facilitators and also teachers. For example, one stated: "[It's] the best professional programme I've ever been involved in, the most effective one. For me as a teacher because it forces me to look at what I'm doing and to make changes." The facilitators emphasised that the structure Te Kotahitanga provided helped teachers to set goals and reflect on their work, giving them the basis for making needed changes to their classroom programmes. This in turn supports the notion of teachers being responsible for their class, which benefits all students. For example,
Across the school one of our goals, and our strategic plans is to improve formative assessment. The professional development structure is very much based on Te Kotahitanga, where we have a number of facilitators, we've had ten facilitators across department groups. This involved some professional development, followed by setting some goals in our groups, group goals and individual goals and then you come back and, and share evidence of what we've been doing.
Some facilitators also emphasised the centrality to the programme of embedding Māori and non-Māori partnerships within the school. As one commented, "It was Ako, right from the beginning it has been a two way thing, partnerships with all our Māori staff in the school. There has been good support with everyone down to the Māori lady who works in the library."
Chairs of the Boards of Trustees viewed the programme positively, although as one would expect, most were not familiar with the intricacies of the programme. Based on their interactions with school leadership, facilitators, and occasionally teachers, they said things like, "Everyone who goes on the hui really comes back fired up. It's great, I mean from a professional development aspect," and "It seems extremely good. It seems like a lot of work to me but it's got results."
We asked the teachers and the facilitators to comment on specific parts of the professional development process they found most or least useful.
Perceptions of the initial hui
A few teachers talked specifically about the initial hui, although for many we interviewed, this was probably an experience distant enough that they did not talk directly about it. Most comments about the hui were positive, such as "awesome," "brilliant," and "lovely." Teachers who commented on its impact mainly focused on the power of the narratives in Culture Speaks to help them understand the point of view of the students and whānau, and to put words to the idea of deficit theorising. A couple of teachers also commented that the hui was a great way to begin the year, and integrate new teachers with the rest of the staff.
There were isolated criticisms of it. A couple of teachers thought the presentation of statistics was too abstract, and a couple were critical of the time the hui took when they needed the time for planning. A math teacher wished more linkage were made between the ideas presented and her discipline.
Classroom observations with feedback
As described in chapter 1, teachers are observed in the classroom by a facilitator using an observation tool that is designed to capture various aspects of the effective teaching profile, then conferring with the teacher afterward. These observations occur once per term, and are focused on dimensions of the Effective Teaching Profile as well as targets that might be set for students in the classroom who are Māori. In the post-observation conferences, teachers learn to analyse the observation data for themselves, and to reflect on it with the facilitator, then the teacher and facilitator together establish goals for future growth that will form the basis for subsequent instructional planning, observation, and feedback. There are also shadow-coaching sessions which are similar to observations but are sessions in which the facilitator provides input to the teacher during the lesson; shadow coaching typically focuses on something that the teacher would like assistance with.
Facilitator observations with feedback
Of the various dimensions to the professional development process, teachers were most enthusiastic about this process of classroom observations followed by feedback sessions, and this was the element that they commented on the most, For example, of the teachers interviewed for Phase 4, half of them talked about it. We heard very few negative comments about this part of the programme, and many very positive comments such as "awesome," "brilliant," "fantastic," and "great."
Teachers appreciated the observations for various reasons. Several teachers discussed valuing the observation data about what they were doing when teaching, such as which students they interacted with and which they missed. They commented that having another set of eyes in the classroom was incredibly valuable; as one teacher put it, "you can only look in one direction at a time." They appreciated being able to see their progress with students over time. Several noted that once teachers have been hired and completed the first couple of years of teaching, they may never again get feedback on their practice. They valued the practical suggestions and ideas they were offered. Several mentioned that this professional development process up-skills teachers, building on what they learned in their teacher training programme and adding to their teaching repertoire. One commented that over time if no adult holds them accountable, they just "switch off."
Teachers valued the support given after the observations. Several described the process of being observed, then discussing the observation as promoting their reflection on their practice. (The few feedback sessions we observed confirmed that this is a reflective process in which the teacher usually does at least half of the talking, and all of it is directly related to what the data in the observation mean for the teacher's practice.) One teacher commented, for example, that the process "forces me outside my comfort zone and it was quite scary at first… but now I really look forward to the observation and feedback sessions." While teachers often felt nervous at the beginning of the process, less than a half-dozen teachers described the process of being observed as threatening and persistently uncomfortable. The great majority embraced it as a form of professional development with enthusiasm.
Similarly, the Lead Facilitators believed that teachers found the classroom observations and reflections useful. They pointed out that the cycle of in-class observation, feedback and analysis had enabled teachers to work more professionally. They saw the observation tool and feedback loop as being critical; as one put it, "It's amazing, I look what a feedback [session] does to kind of help someone look forward. A thing that happens with kids, and [teachers] get feedback. The observation tool for teachers is a huge thing."
This process of a facilitator observing a teacher and then providing feedback enabled particular barriers to be overcome, including changing the belief that teachers own their classrooms and need to defend their territories; as a facilitator put it, "People used to own their territory, don't you dare come in here and tell me to observe what I'm doing. But now [being observed] is an expectation, that is a big change and it is working as a professional with other professionals."
Teachers' experiences with shadow-coaching were more varied than they were with the observation and feedback based on the effective teaching profile. In several interviews, teachers were not sure what the difference was between formal observations and shadow coaching; teachers' comments about the observation-feedback sessions likely include a few reflections on shadow-coaching for this reason. The effectiveness of shadow-coaching from the teacher's perspective appeared to be dependant on the facilitator's knowledge of the process and /or skill in being able to align the process to teacher's individual needs.
I like the observations, and again, I think it's some useful data on what I'm doing. Shadow coaching has been variable, as it depends on who's been observing and what they've had to offer. Sometimes it's been fantastic and they have actually sat down with me and said "here's a different way you could've done it, and here's some ideas" and I've found that really [useful], and other times they haven't really offered anything at all, and so that has varied from person to person and observation to observation. (Teacher)
A few facilitators acknowledged that they lacked expertise and knowledge of the process, acknowledging that they themselves had not actually done shadow coaching yet. These comments seemed to reflect a lack of confidence in providing advice around different curricular areas for which they felt less prepared. Other facilitators explained that their role was very new and 'foreign' and that as teachers they had never experienced a similar professional development programme. This lack of prior experience with the approach to shadow-coaching could impact on a facilitator's sense of confidence within this new and challenging role:
It's quite a foreign process to me. I've never had a shadow coaching or that particular observation tool, or co-construction meetings. I've never done any of that in any school I've been at.
Individual Facilitators and Lead Facilitators acknowledged that shadow-coaching was not working as effectively as other components of the professional development model. These facilitators believe it was important to clarify the purpose and structure of shadow-coaching:
Shadow coaching wasn't as strong as in-class observation or co-construction. We came home from the hui in term 2 deciding that we didn't know what they were talking about and we had better find out. Everybody said 'shadow coaching? I don't think we're doing any.' And we weren't. We have been trying to work on strengthening our shadow coaching. (Lead Facilitator)
I started to question the model of shadow coaching. I'd been doing a bit of reading about it and didn't actually know where we fitted in the reading that I had been doing. Was it a business model, where external goals were set, where there was help for people to meet the external goal, or was it a counselling model or was it a bit of both or, like, what was it? (Facilitator)
In some schools, the process of shadow-coaching had not been put into place mainly because the facilitators did not have time for the additional observations. As one facilitator explained, "We make it through to observations [and] feedback about individual goals, but [we're] not getting to the follow-up for everybody. The hardest part is following those teachers up as well … there're not enough hours in the day. I mean, I'm busy, they're busy."
The co-construction meetings normally involve small numbers of teachers who teach the same students but in different subject areas, meeting once every month or two to share concerns and strategies for improving Māori student achievement. The meeting is led by a facilitator who has observed the teachers in the classroom. Co-construction groups are designed to analyse a teaching-learning problem that the teachers share, using evidence of learning as the basis for developing a group goal. At a subsequent meeting, they analyse what they have been doing to improve their practice relative to that goal. A few facilitators described how the use of data improved the quality of discussion in the group; for example:
We could go there when we had our co-construction meeting on Monday, 100% of the staff would give evidence to support data to show what they are doing to lift it in the school. And when I first started, when I first had a co-construction meeting it was a disaster and everybody couldn't stand them and so that shift from how the co-construction meetings have changed and no one was bringing data at all, so on Monday everyone was bringing something, I think just signifies a quiet shift in people's thinking: 'I'm going to this meeting, what am I…" You know? That puts an onus on them, they have to stand up and front up to what they are doing for Māori students in their classroom and I thinks there is evidence there for the success of the meeting, would you say?
The teachers generally found co-construction meetings valuable. About one-third of the teachers commented on co-construction meetings, and about three-fourths of these comments were positive. We were not sure, however, based on teachers' descriptions of co-construction meetings, how many groups were actually using student data meaningfully. Teachers said that they valued the time to reflect and to solve problems with other teachers who are working with the same students, and that usually teachers do not get time to do this. For example, one teacher explained that, "We had one co-construction meeting where it became fairly obvious that we were all struggling with the same people and the same issues;" teachers then brainstormed strategies that might engage these students better. Another teacher explained that the team developed consistency in their approach to working with students, which stopped students from trying to get away with things. Teachers in co-construction groups that worked well described the group as a "professional learning community" they found very supportive.
According to the facilitators, the quality of evidence used generally in co-construction meetings or within the school to assess Māori students' progress varied, and teachers often shared anecdotal evidence and impressions of progress, seeming reluctant to share data. Gathering the right kind of data to monitor shifts in Māori students' achievement was considered challenging in some cases; for example, "So what was the evidence that our teachers were bringing to co-construction, because we thought that that was pretty useless, some of the evidence, and we wanted more, meatier evidence to help our kids."
A Regional Facilitator acknowledged the challenges associated with the use of data on Māori student achievement within co-construction meetings and within schools. Emphasising that the Te Kotahitanga programme challenged established patterns of behaviours within the secondary school context, this key person reflected that a greater focus on teacher practice and school systems associated with data on Māori student achievement and learning was needed.
It's such a shift from established patterns of behaviour in a secondary context. You know? Why would I bother to talk to an art teacher about my English results? That for a start is a huge break down. There is a journey to travel in it I guess I'd say. I do think that some teams have had less of a handle on that than others. The systems within [some] schools have restricted the configuration of co-construction meetings, so they haven't been able to meet around a core group of students. The conversation becomes much less relevant if I'm just talking to you about what I'm doing, but we don't share students. And so there are challenges in there, some of which are about teacher behaviour and teacher practice, some of which about school systems and structures.
In summary, there was broad-based validation for the Te Kotahitanga professional development model. With the possible exception of shadow-coaching, the great majority of participants found the various elements to the model, and the way they interconnected, to be quite effective and helpful. Also, with few exceptions, the teachers found the facilitators they worked with to be helpful, resourceful, and approachable. Although below we address concerns about deepening the knowledge base of the facilitation team, the large number of interviews we gathered substantiated the soundness of the model itself, as a vehicle for improving classroom practice.
The few teachers who did not find the co-construction meetings beneficial cited logistical or planning problems: a few were assigned to a group based on one of their classes that was not actually the main class in which they needed help; a couple of teachers experienced inconsistency between the co-construction group facilitator and the facilitator who was observing them in the classroom; and a couple of teachers commented that use of a non-contact period for co-construction meetings took away from time to plan for classes.
Pedagogical and cultural leadership for enabling change in the classroom
Reforming classroom practice to support Māori student achievement, using the Te Kotahitanga professional development model, requires expertise in several related areas. In the Te Kotahitanga professional development model, that expertise is concentrated in the facilitation team; it is their responsibility to use that expertise to teach teachers and other members of school. Below, we discuss various role groups' perceptions of the facilitators and their expertise, three specific (and overlapping) areas of expertise that facilitators need, and facilitator professional development.
Principals' perceptions of facilitators
Principals and other senior/middle managers overwhelmingly agreed that the role of the facilitators and particularly that of the lead facilitator was crucial to the success of Te Kotahitanga. They were also clear that this was a challenging role that didn't suit everyone, and that finding and keeping the right person was affected by staffing changes over time. The dispositions, skills, understandings, and even the credibility of the facilitator as someone in the role of supporting other teachers can be an issue. As one put it,
If you get the wrong people, it is a disaster so you've got to get people who have got respect from the rest of the teaching community. If you get someone who is seen as an idealist or, you know, right out there by the staff, the staff are pretty critical, they're pretty astute, our profession is to critique, so we'll find a hole straightaway. One of our younger staff and one of our young facilitators is a young buck, so hasn't got credibility with the staff so he's got to work through that.
Teachers' perceptions of facilitators
In various contexts, the teachers talked about the facilitators, mainly expressing appreciation for their knowledge, support, and help. We heard words like "fantastic," "incredible," "awesome," "really, really good," "great" and "supportive" used to describe the facilitators. Teachers commented mainly on their expertise and support in the classroom. As one put it, "The Facilitator is kind of like your on-site expert that teachers never had." Teachers particularly appreciated the facilitators' expertise in teaching and in Māori culture, as well as their resourcefulness. One teacher remarked, "The beauty with the Facilitator was like they're trained. They know how to have a professional conversation with you." Another said, " In terms of personal support and so on, ... and partly in terms of shadow coaching and then also in regular feeding of information "have you tried this? Have you read this article? Here is a copy of it on the e-mail" or something like that. And those things have been quite useful." For the most part, teachers also valued the flexibility and approachability of the facilitators.
A few teachers said they would have liked more balance between Māori and non-Māori facilitators, noting that "there should be more representation of Māori on the facilitation team" because of the importance of expertise in Māori culture. A few also said there was a bit of disconnect when facilitators only worked in their school one or two days per week.
When teachers critiqued facilitators, their main concern was wanting someone with more subject-specific knowledge, who could make a more direct connection to their subject matter, such as graphics, maths, or French. There were also about a half dozen teachers who felt that a facilitator they had worked with was too forceful with his or her ideas, "imposing how they think things should be on me." As one teacher put it, "Sometimes I think people get a little bit, doctrinaire about things. And lose sight of what I would call common sense." Overall, however, teachers valued the expertise of the facilitators, and found their role and their knowledge central to improving classroom teaching for Māori students.
Facilitators' perceptions of the importance of facilitator expertise
To be most effective working with teachers, facilitators need expertise in several areas: Māori culture and its relationship to culturally responsive classroom pedagogy, subject matter expertise that they can connect with culturally responsive pedagogy, and the process of working with teachers and other adult learners. Facilitators were quite candid in discussing both their knowledge as well as their challenges and limitations in these areas.
Culturally responsive pedagogy and culture. Facilitators we interviewed were aware of their need for strong expertise in Māori culture, and the relationship between culture and culturally responsive pedagogies. An issue that is implicit throughout Te Kotahitanga but not explicit in the model or in most discussions is the cultural identity (Māori vs. non-Māori) and Māori knowledge base of the lead facilitator and the Facilitation team. Some facilitators stressed that having Māori strongly visible within the facilitation team lent credibility to the school's commitment to Te Kotahitanga:
I wanted to make sure we had a Māori led facilitation team—it didn't have to have only Māori Facilitators but we think we have broken a bit of a barrier this year as we've got a Māori Facilitator, which was really a gap in our team.
An area of particular challenge for teachers working towards the Effective Teaching Profile involves the extent to which facilitators are, as one put it, "steeped in Māori tikanga" and can demonstrate culturally responsive practices. The lead facilitators are not necessarily Māori nor are they necessarily fluent in Māori, but the extent to which all are bicultural could be crucial to the programme's ability to provide teachers who are not bicultural with guidance on things Māori and how to reference culture across the curriculum.
Overwhelmingly, facilitators expressed that the most difficult part of the Te Kotahitanga professional development for teachers and for them as facilitators was the explicit focus on culture and culturally responsive pedagogies, because, as one put it, "it is an enormous amount to come to terms with. You've always got to question, you've always got to unpack." Another commented:
It's [culturally responsive teaching] also taken a lot of people to quite difficult places with themselves and their practice. So there's been some really challenging, um, interactions, not horrible ones, but, you know, really difficult conversations. Um, for me personally I felt resistance from people that before I was a Facilitator I would never have had. You know there's a, an uncomfortableness. So I guess it's that dissonance stuff. Um, and, and it's good at the end but those are real obvious things, because now it's being talked about and the word Māori is being struggled with and, and BOT at that level as well. It's like … we're not used to that in New Zealand, we're not used to describing a group as Māori. All of those dissonance type interactions are happening.
Facilitators encountered groups of teachers who struggled to understand the importance and impact of relationship-based pedagogies for Māori students. As one facilitator commented, "There are some people who still can't get their head around this whakawhanaungatanga." Although the focus on Māori students was particularly challenging for some staff members, facilitators had to work to maintain that focus rather than allowing shifts to discussions about all students, and they worked to ensure that schools and teachers took ownership of the issue of underachievement. One facilitator explained,
The fact that it focuses on Māori students [is] challenging for a lot of our staff, to focus on Māori students. Someone might feel like the others are missing out, so, I think it's challenging to actually sell that to the staff really. The data's showing that [focussing on Māori] is important. National data's saying that this is where the need is.
Another commented, "If it wasn't here, I do wonder whether we would lose that focus on Māori student achievement."
Some facilitators emphasised a need for their own on-going development of expertise in Māori culture. Because the position of Māori culture within Te Kotahitanga is central to the programme, lack of expertise, or discomfort with directly addressing Māori culture sometimes led to friction within the facilitation team. One facilitator explained, for example,
I don't know if it was so clear, and it wasn't a priority that the Māori side of the training. [It] should have been the priority. But some [facilitators] were fairly young. They thought, well, I've got a handle on that but in fact the Māori side of it comes from that Hui structure. The Facilitators didn't go on training, didn't necessarily have a shared experience and particularly staying on the Marae and staying at the Hui, not everybody chose to do that.
Culturally responsive pedagogy and curriculum. As many as half of the facilitation teams saw a need for greater focus on the links between curriculum and culturally responsive pedagogies within the professional development programme to enable teachers, facilitators and principals to work together to improve classroom practice and learning outcomes for Māori students. The facilitation teams generally had a strong repertoire of teaching skills, and although the great majority of teacher comments about the helpfulness of facilitators underscored their pedagogical knowledge. Making these links requires that facilitators have both pedagogical and subject matter expertise. Teachers working in certain subject areas—maths was named most often—struggled to develop culturally responsive pedagogies aligned with their chosen curriculum area. One noted, "I've worked in two schools, and the maths department found it the most difficult to work in a different way, start to begin to work in different ways." It is evident in classroom observation data that many teachers in maths and science managed to make these connections despite the perceived challenges of doing so.
But facilitators occasionally commented on lacking the subject matter background to help them. For example:
It could also just be my lack of experience [working with different subject teachers] because I'm an English teacher, and I haven't been thinking, widely in those other cultural contexts for those other subjects. Next year, [I would like to] bring in a teacher with more subject expertise in the science and maths area.
We heard similar concerns from some teachers, who wondered if there might be "culturally neutral" subject areas because they were having difficulty envisioning what culturally responsive pedagogy would look like. For example, a science teacher discussed how difficult it was to conceptualise "bringing the cultural perspective into science. It's quite difficult, depending on what topic you are teaching and how are you going to get the cultural perspective into science." A maths teacher spoke to the need to address culturally responsive pedagogy in a subject-specific way:
It's a blanket programme and it doesn't identify enough, in my opinion, that different subjects have different needs and different approaches. Mathematics, for all you may want to do group work and differentiated learning, which we will come back to if you are interested. There is a sense in mathematics where you do have to be the teacher and impart knowledge. And again if you push and say 'all your work has to be in group work' and 'Māori kids will only work if they are in a group with Māori kids' you sort of put that as an absolute. Which is one of the things that we've informally heard, you create barriers.
Some facilitators indicated that subject matter expertise was not the only pedagogical area in which many facilitator teams felt stretched; they sometimes felt inadequate with respect to their own abilities to support and challenge teachers about their classroom practice for other reasons. Facilitators felt particularly stretched to figure out how to work with teachers who already had a high level of skill in linking culturally responsive pedagogy with curriculum, teachers who could be readily identified as 'high' implementers of the programme compared to others. At present, the professional development programme does not differentiate a role for them that would make use of and further develop their expertise. For example, one facilitator explained:
We have teachers who we call high implementers of Te Kotahitanga. I think the high implementers would possibly like more opportunities to have co-construction meetings, opportunities to bounce ideas and talk about pedagogy. I mean I have been part of co-construction meetings where there have been high implementers who have looked stunned because of comments from other people who aren't high implementers.
At the other end of the spectrum, a related challenge confronting facilitators was working with teachers who had poor teaching and/or classroom management strategies and skills. Sometimes this was viewed as an issue of teacher competence within the school that was not being addressed by the senior management:
We were never, ever designed to be a project that picked up poor [teachers] who weren't managing their classes, that was the end of it. So last year I pulled back a bit and said 'In the end, we cannot do constructive Te Kotahitanga feedback in classrooms that are not managed [and are] simply out of control.' So I wrote to the two people and said 'Look, you need to get some help from specialist teachers, you have got to go to that, [so] you feel like you are able to manage your class better.'
Teacher professional development. Teacher professional development and working with adult learners was a third important area of expertise that facilitators needed, and for which their expertise was uneven. They needed to know how to establish and maintain a supportive, trusting working relationship with teachers while also providing enough challenge to drive teaching practice improvements, which was a delicate balancing act. Although most teacher descriptions of facilitators underscored their ability to do this well, the facilitators themselves were aware the need to pay attention to building relationships while at the same time offering critical feedback; this is a theme that came up frequently in interviews. For example:
It's quite a different job to teaching. It's a hard job, handling difficult situations. I made [that teacher] cry. I think we are going into teachers' classrooms and challenging them, and that is a big ask.
Just recently when I was in a challenging situation, it was actually starting to unfold as we were doing the feedback. So it's like, how do I address this, and it got to a place where there was not actually a happy medium. As a result of that, you actually have to step back and I felt that the best thing to do was to step back and have someone else come in and then regroup to get it to that place which is where we have got to now. I don't actually think you can be prepared for it.
The facilitators need to learn to work diplomatically with teachers who are resistant to change, and particularly when confronting deficit views towards Māori students. A couple of them pointed out that Te Kotahitanga is both a philosophy that conflicts with that of some teachers, as well as a set of practices that teachers who are uncomfortable with relationship-based teaching resist. Learning to navigate, confront, and attempt to change resistant attitudes requires skill that the facilitators were in the process of learning.
A challenge that facilitators often mentioned was learning to work with and offer constructive criticism to teachers who were identified as traditional, older, more experienced and/or had higher status (for example, Heads of Departments) than facilitators. Facilitators—even lead facilitators—come to their roles as colleagues of teacher peers, and yet they are cast in a position of providing technical advice and support to others who may consider they are equally qualified and expert in dimensions of the ETP. Facilitators spoke openly and often about the challenges of managing the tension between providing encouragement that is culturally safe for teachers while also providing constructive criticism where this is needed. For example:
For me as a new Facilitator, the biggest challenge is challenging someone's positioning especially if I'm dealing with a teacher that is a lot more experienced than I am as a teacher. So that's quite scary, we've got someone who's been teaching for 25 years, to say 'well … have you thought about doing something differently?'
A challenge I had when I first started was giving advice to people who I considered tuakana, older than myself, or more experienced than myself as well. I didn't like to tell people where I think they might need to improve. I'm basically at this stage having to explain to more experienced teachers than myself. I've only been a teacher for the last five years or so, [giving advice to] teachers that have been here for 20 odd years.
Developing facilitator expertise
The areas of expertise we discussed above—deep knowledge of Māori culture and its relationship to culturally responsive pedagogy, connecting culturally responsive pedagogy with curriculum, and knowledge of and skill in facilitating teacher professional development—are central to school reform for Māori students, and require facilitator professional development. Facilitators had been trained by the Waikato team, and they clearly gave these training sessions high marks. Phase 4 facilitators were better prepared than Phase 3 facilitators had been; experience helped the Waikato team identify more clearly the training that facilitators would need.
The professional development hui for facilitators at the beginning of the year was also well-regarded as a form of facilitator professional development, credited with clarifying how the programme worked. As one explained:
[Facilitators] need to do the hui. A new Facilitator would need to do the hui, the Te Kotahitanga facilitating hui, because once I did that this year, it just made things clearer and sort of put in place. [At the end of last year] because I knew I was coming on[as a Facilitator] this year, I did a little bit of shadow coaching with [the lead facilitator] and observing and stuff, and it just made a huge difference when I went to this hui because I could understand actually what was going on, what was being said in the observation tools, and if you do that then it makes you more confident in being able to go into a classroom. That was one of my fears too coming out of the hui, oh my gosh, I'm going to have to now go in and do this observation, but it's really not as bad as that.
Some facilitators noted that their role was very new and 'foreign' and that as teachers they had never experienced a similar professional development programme. Lacking prior experience with this approach to professional development could impact on a facilitator's sense of confidence within this new and challenging role. As one noted, " It's quite a foreign process to me. I've never had a shadow coaching or that particular observation tool, or co-construction meetings. I've never done any of that in any school I've been at." All of the facilitators pointed out that they came into the role without the range of expertise they would need, and benefited greatly from professional development. The following comment was typical:
I started at the school solely as the head of [a particular department] and they said to me: Would you like to be part of Te Kotahitanga? At that stage I had no idea what it was but I always say yes and then think about what I have let myself in for afterwards.
Our evaluation research team felt that more emphasis on and preparation for the role of being a "critical friend" could assist with these challenges. Facilitators advocated for more ongoing training coupled with face-to-face support to enable them to cope with the new and challenging demands of their roles, for example:
I'd really like more face to face support, probably with other schools would be really good [to share] what's working well in your school. The Te Kotahitanga website is really useful as well, but it's not face to face. For me personally, I'd rather have someone to sit down and talk to, someone who can challenge me in my practice, and if I say something [who] can say 'Let's think about that again.' I guess what I'm looking for is sort of feedback and ideas on how to improve.
Training and ongoing professional learning and development for facilitators will require attention in the future. There hasn't been a specific training programme that could, for example, serve as a pre-requisite to being employed as a Te Kotahitanga Facilitator. A programme along the lines of the RTLB professional, postgraduate programme leading to a formal qualification might be useful both for current facilitators and those seeking to become facilitators. There should be encouragement and support for such initiatives as well as exploration of alternative strategies for ongoing skill training. The hui and associated support from University of Waikato has sufficed for these first few years of Te Kotahitanga, but the staffing difficulties reported here suggest that this will become increasingly difficult to sustain as additional schools adopt the model. The online network has met some needs for ongoing support, but opportunities to undertake short courses leading to credits towards a qualification would no doubt be interesting to these professionals and teachers as well. The facilitators themselves indicated that they would appreciate and benefit from opportunities to gain further professional development.
We also suggest that opportunities be expanded to enable experienced facilitators to mentor less experienced facilitators across schools as another strategy to enhance further the quality of training and support provided to this role. At present, there is little formal contact across schools or persons in parallel roles across schools, something that could add further value to the project.
In this section, we have examined the areas of expertise that are needed to enable the Te Kotahitanga professional development programme to work, reflected in the work of the facilitators. Facilitators expressed concern that teachers and HoDs, as well as some facilitators themselves, lack the necessary knowledge, experience and confidence to assist teachers across different curriculum areas to improve practice and learning outcomes for Māori students. If schools are to continue to work on improving classroom practice and learning outcomes for Māori students, we believe it is critical that expertise for doing this effectively be recognised and developed. Schools need at least one trained facilitator as well as a facilitation team that has expertise in the areas we discussed above.
Supporting the professional development model
Our interviews probed what kind of support within the schools might strengthen the Te Kotahitanga professional development model. We discuss two main areas: support by the school's leadership, and staffing concerns.
All lead facilitators who were interviewed believed that it essential that the school have strong leadership supporting the vision and philosophy of Te Kotahitanga if change is to occur and be sustained within the school. Strong leadership entails vision, ownership and commitment to making a difference to Māori student achievement. They spoke about the need for supportive relationships between the lead facilitator, the principal, the Senior Management Team (SMT) and the Board of Trustees (BOT).
It was important that the Principal was seen to be leading changes within the school. It was also necessary that the Facilitation team and SMT work in partnership to achieve desired goals, requiring open, regular communication. One facilitator explained, based on experience, the link between strong Principal leadership, active support, and impact on Māori students:
The original principal had the vision and the strength to say that this is what [he/she] wanted for the school, talked to the board and people like that, advocated strongly for Te Kotahitanga all around the country. In the school, [he/she] put all the structures in place and the funding and made sure everything was here for the programme. The [first principal] had such strong beliefs that it was going to make a difference and when the times got really tough could have pulled out or the school could have backed away from the hard decisions, but they made the hard decisions and I believe it paid off in the end. Our results show Māori students are doing better than they were 4 years ago.
Strong support from senior management and heads of department was also considered necessary. Some lead facilitators were adamant that what was needed to improve practice and outcomes for Māori students required at the school level wider issues of school reform along with ongoing training and upskilling for all school personnel, not just classroom teachers.
Evidence of leadership support takes a variety of forms, such as seeing that the facilitation team is adequately staffed, verbally supporting the reform process to the staff, working with and sharing school-based data to guide efforts, and resourcing the programme. Lack of ongoing funding commitments for the programme was a major concern to facilitators, as inadequate funding risks job security, and thereby, continuity in the professional development programme. One of the facilitators, for example, commented, "They [other facilitators] would come up every so often, would say 'How do we plan, what do we do, what about our careers? We like what we are doing but what if it all suddenly changes, what happens to us?'" Some facilitators explained that they had tried to split the role in the past with part-time teaching in order to continue being credible as a teacher. This, however, had meant juggling competing demands which they felt impacted on the effectiveness of programme implementation. For example:
Funding for Facilitators to enable Facilitators to do the job they do, space for Facilitators to work, all those things that are just money issues. I do my facilitation 4 days a week because the position is .8, and I made that decision. I wanted to focus solely on Te Kotahitanga this year when I became the Facilitator, as previous to that I had been [a head of department]. Trying to juggle two jobs was too hard, I didn't feel as though I was giving anybody a good service and so that meant that I am now part-time.
Facilitators advocated more focus on leadership at different levels of the school if schools were to improve classroom practice and learning outcomes for Māori students. Some facilitators were particularly concerned about the lack of leadership at middle management level. They considered that Heads of Departments were key players who could either support or disrupt change from occurring within the school.
Facilitators believed that in the unique hierarchy of secondary schools, Heads of Department are responsible for curriculum leadership. This meant that the support teachers received would vary depending on whether HoDs supported the professional development initiative. Again, some participants believed that HoDs knowledge, confidence and experience of culturally responsive pedagogies aligned to their curriculum area was a key factor in supporting or disrupting reform within schools:
This is the principal's vision, but actually the DPs don't want to be part of this, or even HODs. Often that middle management level is where some of the greatest resistance [occurs], and why would we be surprised? Why would we be surprised that some of these people would start to be a little bit twitchy around something that actually challenges status quo?
Facilitators seemed poignantly aware that there were different layers of responsibility in secondary schools that all have an impact on school change and the experiences of students. They felt a disconnect between Te Kotahitanga activities driven by the principal's commitment and teacher participation, on the one hand, and the layers of other school leaders with major roles to play in the process and the life of the school.
Support of the Board of Trustees is also very important. Some chairs of the Boards of Trustees spoke to their commitment to the professional development programme, including ensuring that it is funded:
Our lead Facilitator needs to have the people, the budget, the time basically. If she doesn't have the people, she's going to be doing two people's jobs. So she needs people time to do it properly.
We need to make sure that whether it is Ministry funded or we try and fund it, that she has what she needs otherwise it's going to be a struggle. Money's the main enabler, isn't it?
Earlier we noted that Board of Trustees chairs had only a general idea of the nature of the professional development programme. As pressure to fund this programme increasingly may compete with other things, we suggest that the Board will need to know more about how the programme operates, why it operates as it does, and cost implications of its specific features.
Team composition, staffing
Our interviews yielded insights regarding staffing and building facilitation teams that have implications for the future. One issue is developing stability of the team, and particularly of who is in the position of lead facilitator. A number of lead facilitators explained that they were very new in their role, and in some schools there had been several people in the lead facilitator role. Turnover in this role creates the problem of stability within a school, as well as the need for the school to expend time and energy repeatedly identifying new lead facilitators and making sure they have time to be trained for their role. Facilitators—particularly lead facilitators—would appear to be greatly in demand, a situation that has escalated as Te Kotahitanga has been extended to additional schools. Hence, frequent staffing changes occurred requiring constant readjustment of existing relationships and making new ones with teaching staff and the senior management team at the school. Stability of staffing in the facilitation team was considered a major issue, since lack of job security and high turnover within the school facilitation team were also regarded as barriers to effective implementation.
A second issue is the fact that most facilitators were part-time, and the remainder of their full time employment represented commitments to other duties such as teaching, advising, and providing technical support to teachers in other roles. Principals shared that while it was helpful to have professionals from the regional advisory team or an RTLB from their cluster assigned part-time to the facilitator role, this was also a challenge. Many of these part-time facilitators are at the school only 1-2 days weekly, which introduces scheduling and other problems, such as continuity in their work with teachers through the observations, feedback sessions, and co-construction team meetings. Other facilitators are at the school in teaching or middle management roles, which can represent a 'conflict of commitment' for these professionals even on the days when they are carrying out their Te Kotahitanga roles. The position staffing commitments are temporary, so the uncertainties associated with non-permanent positions may contribute further to staff attrition as highly qualified professionals may be attracted away from the project on a regular basis to assume other positions in their schools and elsewhere.
Facilitators believed that logistical constraints such as lack of time, workload issues, and shortages of qualified staff representing ongoing challenges having impact on implementation of the programme at their schools. We heard comments such as the following:
Yeah, it is not having enough time dedicated to Te Kotahitanga because of my workload with my teaching, and you haven't got enough time to do a lot of researching or a lot of follow ups because you've really got constraints on you in that area because of your other duties you have to do as a teacher as well, so probably not enough time to give it the all that it deserves.
A major dilemma for facilitators whose non-facilitator FTE involved being a teacher at the school was finding the right balance between teaching and facilitator duties. Facilitators emphasised how important it was for the school's management team to understand this and allocate appropriate time for them to carry out their facilitator responsibilities. When high staff turnover occurred within the Facilitation team, this was considered a major barrier said to have a severe impact on programme implementation:
It doesn't help that we've had a different facilitation team every year for the last five years. There's been a high turnover in our Facilitator team, so we've had a new team and had to re-train the team every year.
A third issue is ensuring that facilitation teams have breadth of expertise across all subjects. As noted earlier, the facilitators were aware that their skills do not necessarily extend across all subjects taught in Years 9-10, and teacher interviews support the need for subject-focused as well as general pedagogical advice and support to demonstrate growth in the Effective Teaching Profile. The facilitation teams also acknowledged that some teachers at their schools were themselves "high implementers" and had developed a high level of mastery of the ETP.
A possible solution to some of these issues, which one of the teachers spoke to, is to rotate "high-implementers" onto teams for specific periods of time, particularly those in key subject areas for which the rest of the team lacks subject-matter expertise:
You can't develop a (inaudible) without having more people on board taking particular Facilitator positions, and so I guess it's kind of an informal approach to try and bring in people- like some of us for instance, you maybe able to take on a more of a (inaudible) kind of role, and so I think that fits as a possibility for it. Yeah I mean it depends on the funding and those kinds of things, doesn't it? I think it would be really useful, if you want to keep an eye on this project to make sure it gets embedded over time, I think that's something that you've got to look at doing. Not just crying literacy and some of these things, but something concrete, in writing to set it up. Because over time you are going to get people moving on, for instance, from one school to another. In my experience PD initiatives in the past, sometimes you lose momentum with things like language across the curriculum and things like that, there are a lot of people getting professional development and then moving on to somewhere else.
The evaluation team suggests that facilitation teams move away from hiring part-time members that include people who work in more than one school, and toward a system of incorporating high implementer teachers into the team, especially to provide subject-specific advice and support. We see rich possibilities for developing teacher leadership by reconfiguring facilitation teams in this way.
Key findings: The Te Kotahitanga professional development model
- Across the board and with very few exceptions, teachers, principals, Boards of Trustees chairs, and facilitators were most enthusiastic about the Te Kotahitanga professional development model, viewing it as a sound and effective process for improving classroom teaching for Māori students.
- Teachers valued the interconnected parts of the model, voicing most enthusiasm for the classroom observations with feedback which they saw as not only improving their teaching but also improving their ability to reflect on their teaching. Co-construction meetings appeared most effective and useful when all teachers in the group were trained in Te Kotahitanga and problem-solving was based on student evidence provided to the group on a regular basis. The implementation of shadow coaching across Phase 3 and Phase 4 schools appeared variable. Data analysis indicated that some lead facilitators and facilitators were less knowledgeable and/or confident about the implementation of shadow-coaching in their school. Some teachers also indicated that they were unclear about the process of shadow-coaching, how it differed from other components of the PD model and/or its contribution to their ongoing professional development. More attention needs to be placed on facilitators' knowledge and use of shadow-coaching and the quality of its implementation within participating schools.
- Trained facilitators are critical to the success of this professional development model. Facilitators as well as teachers affirmed that the facilitator role required expertise in Māori culture and its relationship to culturally responsive classroom pedagogy; subject matter expertise that they can connect with culturally responsive pedagogy; and the process of working with teachers and other adult learners. This is not a role that can be shifted to other personnel in the school who have not developed this expertise.
- The Te Kotahitanga professional development model works best when: it has active support from the school's leadership team, particularly the principal and the other senior managers; the leadership team views it as an essential vehicle to improve academic achievement of Māori students; and there are effective communications between the school's senior management team and the lead facilitator. It also appears to function best when there is stability in the facilitation team; when most facilitators are based within the school thus connected to its school community; and either full-time or, if part-time, have a sufficiently flexible schedule for project responsibilities.
- The challenges for facilitators in providing effective professional development support for teachers included uneven availability of: curriculum expertise for ETP exemplars across different subject areas; timely student outcome data for feedback to teachers; and differentiated PD activities to accommodate teachers at different stages of implementation, expertise and cultural knowledge.