Evaluation of the Inservice Teacher Education Practice Project (INSTEP)

Publication Details

INSTEP was a research and development project aimed at improving the quality of inservice teacher education. The project, carried out by the Ministry of Education, set out to improve knowledge and understanding about effective inservice teacher education, develop greater consistency and coherence in the practice of inservice teacher educators (ISTEs) and trial approaches that would lead to improvements in their practice. This evaluation report offers insights into the way in which participating in INSTEP has contributed to bringing about shifts in knowledge, skills and expertise of ISTEs and identifies early indicators of change resulting from the project.

Author(s): Meenakshi Sankar, MartinJenkins. Report prepared for the Ministry of Education.

Date Published: August 2009

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  To view the individual chapters please refer to the 'Sections' inset box.

Part 2: Demonstrating the value of INSTEP

A key project outcome for INSTEP was the development of a model for the ongoing professional development of professional development facilitators owned and operated in a self-sustaining way of teacher education professionals across the country. Therefore, determining the extent to which this outcome has been achieved is an important step for this evaluation. In this section we discuss the value and benefits of INSTEP for participants in bringing about the shifts desired in knowledge, skills and attributes regarding ISTE practice. The discussion is presented around the theory of a change diagram developed to represent and convey the essence of INSTEP and its contribution to participants. The diagram illustrates how INSTEP has contributed to bringing about the desired shifts in ISTE knowledge and practice and the nature of this contribution to improved teaching quality, which in turn contributes to improved student outcomes. As illustrated in the diagram, the short to medium-term impact of INSTEP is on ISTE practice. While it can be argued that impact on teachers can be achieved directly through impact on ISTEs, in the context of INSTEP building a shared understanding of role and purpose of ISTEs across the sector was a critical step, a bridge to link ISTEs' practice to teachers' practice.

This diagram has been generated from the evidence presented by participants and illustrates how INSTEP has contributed to the outcomes in the real world. The analysis of the findings from INSTEP presented in this section is wrapped around this analytical model.

Figure 7: How INSTEP works to bring about desired shifts - an analytical model 

Image of Figure 7: How INSTEP works to bring about desired shifts - an analytical model.

The rest of the discussion in this section is focused on each component and step in the diagram to help illustrate the nature and value of the contribution made by INSTEP at each level.

Implementation of INSTEP

In order to understand the value and contribution of INSTEP, we have posited INSTEP as the 'intervention' in this above diagram. Fundamentally INSTEP was an intervention designed to build a deeper understanding of ISTE practice and to promote change in ISTE practice as a result of this understanding. Therefore, understanding the implementation features of the intervention and identifying aspects of the implementation that worked well or didn't work well were critical first steps to understanding the nature of its impact on ISTEs, their provider organisations, teachers and students. The phase one report (summarised on pages 16 and 17 of this report) discussed these features in great detail and we urge readers of this report to refer to it. This will help build a more holistic picture of the ways in which INSTEP has operated to generate benefits for ISTEs and the sector.

Overall, the INSTEP project was described as an "invaluable investment in building capability of ISTEs" by all participants. Despite the challenges posed by the research and development approach, the project's goals of bringing together practitioners from across the sector to work collaboratively to examine, inquire and build knowledge about the practice and learning of ISTEs were highly commended by all participants. This investment was an affirmation of the importance and value of ISTE as a lever for effecting change in the teaching and learning area and served to bring about greater consistency and coherence in practice. While participants recognised that there were inbuilt tensions across providers given the current contestable environment (which runs counter to the collaborative inquiry promoted within INSTEP), there was some degree of openness to engage and share different interpretations and approaches to inservice teacher education.

The key elements in the design of INSTEP 'intervention' that enabled the project to achieve positive outcomes were:

  • the research and development approach
  • leadership by a core group of national facilitators
  • inquiry/action research within communities of practice
  • management by a project team located within the Ministry
  • concurrent development of learning materials
  • additional support through research mentors, online communities, international speakers, and so on.

These features collectively contributed to building trust and confidence of ISTEs to engage in investigating gaps in knowledge about effective ISTE practice and to collaborate with other practitioners to inquire into their practice as they implemented the research and development activities of the project. Despite some initial challenges in implementation of the INSTEP project, the above-mentioned features played an important role in enabling ISTEs to stay engaged with the project and the process.

Participants undertake inquiry into their practice and trial new approaches

In keeping with the R&D nature of the project, participants were given a fair amount of flexibility to frame their inquiry into their own practice around what matters most to them as ISTEs. The INSTEP project structures provided a broad framework/infrastructure within which this inquiry into practice took place (for example, national facilitators supporting regional facilitators who in turn supported inquiry into practice by ISTEs; national facilitators inquiring into ISTE practice and identifying principles underpinning their work). It also offered ISTEs the opportunity to identify problems of practice or practice puzzles and to pursue this inquiry to achieve better outcomes for teachers and students.

By inquiring into the identified problems of practice collaboratively with a group of peers/colleagues and systematically gathering data about progress or shifts, ISTEs involved in INSTEP generated new knowledge and insights about their work and used this to inform their planning and action. Facilitating the process of collective reflection and inquiry was a critical part of INSTEP project design. This was due to the fact that much of the practitioner knowledge was tacit and in order for ISTEs to integrate current research knowledge into their professional knowledge, they needed to go through a systematic process of making their existing practices explicit. The INSTEP project allowed participants to undertake this inquiry in an open, inclusive and non-threatening way. This in turn provided an intrinsic motivation for improvement.

Over the course of the INSTEP, project ISTEs developed a range of different approaches to inquiry, each of which was designed to facilitate critical reflection on a problem of practice identified by them. The research reports provided to the Ministry by the NFs at the conclusion of INSTEP documents the nature of inquiry undertaken by each of the 12 pods within the project. Our analysis of these reports showed that there was significant variability across the pods and that across the 12 pods, changes were occurring at two broad levels: both at an individual ISTE level and at the provider organisational/institutional level. The longitudinal case studies were selected to offer insights about how these changes played out over time and the factors that helped or hindered the sustainability of these shifts. In this section we discuss examples of the broad approaches used within INSTEP to inquire into different dimensions of ISTE practice. This is not an exhaustive list of the inquiries undertaken by all participants and is intended to illustrate the scope and breadth of the inquiry.

Professional learning progressions

The professional learning progressions were developed and trialled in INSTEP to support the professional practice of ISTEs and to guide an individual's professional learning. The progressions were aligned to the dimensions of practice identified in the early stages of the INSTEP project (knowledge and theory, change for improvement, inquiry and evidence-based practice and communications and relationships). Within each of these dimensions, a series of steps were described which then formed a learning progression for use by ISTEs involved in INSTEP, in this case study site. The progressions were designed to allow the ISTEs to identify where they were situated within their practice at a particular point in time and within a specific context or problem of practice. The steps on the progressions provided direction for the next professional learning of the ISTE. Participants were encouraged to use feedback from teachers, observations by other regional facilitators, ISTEs' own reflection to verify and validate their placement on the progressions. This then allowed them to use the next step on the progressions as the basis for guiding their professional learning. There were four steps to the progressions:

  • Step 1: Placement on a continuum
  • Step 2: So what now? What does this mean for my professional learning? What?; how?; with whom?; and by when?
  • Step 3: How will I know I have been successful?
  • Step 4: Reflect and record in journal.

For majority of the ISTEs involved in trialling this approach, the progressions effectively captured the essence of their work and helped them map the next step in their individual journey towards improvement. The progressions were described as pathways, frame of reference and tool or development in order to reach a desired level of practice.

Professional learning groups - trialling a pod structure

In one case, the organisation trialled a pod structure for supporting and fostering the professional learning of ISTEs to examine in-depth the role and contribution of the pod structure and the experiences it generated to enhance learning for all participants. Care was taken to ensure that the pod was heterogeneous and had a mix of gender, work streams, geographic locations and part-time and full-time ISTEs.

In this organisation each inservice teacher educator was allocated a learning group or 'pod' which was led by a regional facilitator. Each group met twice a month to share, debate and focus on practice-based issues. Each pod was asked to identify a range of issues and to focus on what matters most to the individuals in the pod. Within each pod, ISTEs focused on individual problems of practice. Each pod was conceived as a supportive learning environment where trust was a critical element. Trust was created through selected interactive activities that encouraged pod members to feel 'safe' in sharing opinions, ideas and experience.

Before our professional development was content driven and disconnected from practice. We had PD days but we did not ask ISTEs to interpret or make sense of the new knowledge in terms of their day to day practice. The PLGs or pods were set up to provide a structure for these discussions and allowed for mentoring, coaching and collectively reflection on aspects of ISTE practice. Formalising this is critical as we have seen the benefits and value for our people and looking to ways in which we can embed this into our organisational systems. (Regional facilitator)

In another case, each RF and their ISTEs created their own professional learning community within INSTEP and focused on issues relating to effective facilitation practice. The rationale underpinning the establishment of a professional learning community in this case was based on facilitator views about their own job: 'facilitating is a lonely job'; 'sometimes I am not in contact with another facilitator for three-four weeks and so if I face a problem, I have to find a way to resolve it on my own'; 'we are expected to just go and do the work'.

The professional learning communities provided a forum for public sharing of problems of practice within the group which led to dual outcomes: first, it gave the ISTE the confidence to engage with a problem of practice in a meaningful way with their peers; and secondly, there was a sense of accountability to the group as one had to come back and talk about actions they had taken that transformed or changed their practice, as is evident in the following quotation:

The fact that we can face up to our colleagues and say that I realised that I was not as sharp as I ought to have been in that instance, but guess what I have now realised it and am doing something about it. Here is something I tried and it worked. That is very rewarding. (ISTEs)

The professional learning communities were described by participants in this instance as being very successful, as they promoted collaborative learning and open and honest discussions on practice, both successful practice, as well as unsuccessful practice or problems of practice. The use of evidence to frame these discussions also meant that there were opportunities to explore dissonance or gaps in ISTE analysis and reflection.

In a third case, a pod was set up within a secondary school and the professional development coordinators assumed the role of an inservice teacher educator within this project. The focus was on developing the coaching and mentoring skills to facilitate literacy learning amongst teachers and to increase the achievement of students. This approach was based on the belief that for professional development to be effective it needed to be thoroughly embedded in the context of the school in which it was to take place. This allowed planning to be informed by the culture of the school and the systems of support available in that school. Through INSTEP, the regional facilitators and in-school ISTEs in this case explored ways in which they could develop their own knowledge and skills base to become coaches and mentors to other teachers in their school.

Applying theories of Model II learning in action

Getting buy-in from teachers and school leaders was seen as a critical step to achieving positive outcomes from any professional development and learning intervention. Most ISTEs understood the value of getting buy-in at a rational level. However when faced with teachers who were rigid in their attitude or put up barriers to learning, the ISTEs' usual response was to withdraw and try again later. The impact of this response on facilitator practice is illustrated aptly in the following quotation:

As an ISTE the most challenging aspect of our work is getting into a school and negotiating that first interaction and meeting. When we enter a school for the first time, we need to get buy in, get people on our side, come across well, come across as an interesting and intelligent person who has stuff to share. A lot of the times, principals sign teachers up for things and when we go in, the teacher has no interest in learning of cannot understand why she needs to listen to any of this stuff. In such a situation, I will leave and try again, come back another time. I have schools that don't want to see me but I still go and try every now and then. Nothing comes of it. I only worry as the school really needs help but what can I do? I just go through the motions and I will go away again and come back the following year (ISTE).

In one case study, the national facilitator and the regional facilitators examined theories and concepts underpinning communications and relationships in establishing effective learning relationships with teachers. In trialling application of Model II learning theory and approach in INSTEP, participants (as in the above case) examined their behaviour and practice more explicitly with their peers and realised how their own beliefs and assumptions impacted on creating and perpetuating this dynamic, creating barriers to learning. These beliefs and assumptions described and reflected Model I learning approach as presented by Argyris and Schön (1974) where the focus was on "winning (proving oneself right), and suppression of any data that does not fit the actor's assumptions". Model II by contrast represented a more consultative approach in which the participants in learning had 'bilateral' control of the process where winning was not being 'right' but rather making sense of the evidence, and where no dialogue was suppressed, even if it was painful. It required people to pay close attention to their own behaviour and to the way they interacted with others. Argyris and Schön suggest that 'in general, Model II learning tends to facilitate others' learning which in turn facilitates one's own learning.

The more I examined my own beliefs and assumptions about how I engaged with teachers, I realised my part in creating these barriers to learning. I videoed my conversations with teachers and realised what mind set I was operating from. This was huge for me personally. I am much more conscious about my assumptions and values now. So now when I first go to a school, I don't jump into planning how we are going to deliver the PD plan. I now go in with a more open approach and talk through their needs are, what are the issues they are grappling with, what would they like to do and use that to help co-construct the plan. I don't make any judgement about what they need or go into an engagement with any pre-conceived idea about what will happen. I am also much more conscious about my body language and reading the teacher's body language. The reason is that I am now not focused on delivering my plan, I am actually focused on ways in which I can work with the teacher on things that matter to her/him. (National Facilitator)

De-privatising practice

Many ISTEs involved in INSTEP pointed out that their practice had evolved over the years on the basis of their experience and additional study they had undertaken. However the notion of taking a more systematic approach to collecting, analysing and interpreting data in order to gain deeper, more meaningful insights about practice was investigated in INSTEP by most participants. The tools and approaches used to gather data and evidence included audio and video recordings, peer observations, student voice, teacher feedback, role plays, field notes, and case notes.

INSTEP was a trigger, it really was, for us to actually start talking about our practice, you know. Actually what are we doing as facilitators? What is facilitation? So, it's that de- privatising or whatever you want to call it really. I don't like that term. But, it' got us to open up our practice. One of the other things was that we used audio and video - I was videoed and audioed in those six months and now as team leaders, we are extending that practice across all of our teams and expect that each member of the team will bring at least one video or one audio to analyse. We then analyse it as a group together. (ISTE)

In this way, ISTEs created their own cultures of evidence-based inquiry and sought support and challenge from trusted colleagues, interrogated a range of data selected to address specific problems of practice, and drew on external feedback and research to help make sense of the findings. By inviting colleagues to comment on and discuss individual recordings of practice, ISTEs were confronted by the reality of what they were actually doing and provided them with valued feedback that helped them improve their practice. The following quotations illustrate the role of these artefacts in enabling inquiry into practice.

Listening to my recording of the conversation with a teacher and reading the transcript with my INSTEP group really showed how much I was trying to dominate the conversation. I was jumping in with a solution without hearing them out fully, like as if I wanted to end the interaction. (ISTE)

Videos were powerful as they offered a number of benefits - a tool for reflection, a record of the event, training of new staff to demonstrate how the conversations can unfold and how they can strategise, and most importantly it is the first step towards using other tools for learning. (Regional facilitator)

The critical insight we got from videoing our practice in our project was the number of us that thought we were great facilitators but when we looked at ourselves we were shocked. It challenged us, and it was so visible that it shook us - we realised that we were actually not listening, we were sometimes quite manipulative, we were there to have our say and move on. It was very uncomfortable to be in the spotlight, and many of us were quite threatened initially. But we ensured that the climate was right and that everyone felt they could trust the other person to put learning at the centre of the discussion and not make it personal. That was the key. But there is no going back for us, we will now incorporate this in some way in or work for the future. It is powerful. (National facilitator)


As part of INSTEP we had the opportunity to have the Regional Facilitator come and observe our practice and give us feedback. Talking through my work and how I work with another person was very helpful as it brought an external perspective to my work. Initially I was uncomfortable but then I thought that this is what we ask of our teachers, to let us come in and observe them, it made me realise that we need to demonstrate our own comfort levels with this process. We set the rules of engagement early. (ISTE)

All the above examples point to the need to have an agreement and willingness of participants to the observation and a shared understanding of the purpose of the observation. In each instance, care was also taken to ensure that feedback was not personalised and was relevant to the purpose of the observation as this was felt to promote learning. It also highlighted that analysing and critiquing practice on the basis of these tools needed a level of understanding and skill which took time to develop for some facilitators in INSTEP.

Taking a school-based approach to inservice teacher education

One of the case studies took an explicit school-based approach to facilitating professional development and professional learning owing to their belief that PD/PL was most relevant when it responded to the context of the school and learners in the school. This approach was consistent with the definition of ISTEs in INSTEP which included school leaders of professional development. Consequently, in this case a learning pod was created within the identified INSTEP school and lead teachers or syndicate leaders played the role of a Regional Facilitator who facilitated professional learning for the teachers in their school. This allowed the ISTEs to recognise and respond to the diverse and unique context of the schools in which they worked. Two projects led by the National facilitators in this case were run entirely in schools, thus allowing the NFs, RFs and others involved to examine the conditions that helped/hindered engagement in PD/PL in sustainable ways. This approach also allowed for a whole-school approach where the NFs and RFs worked in the classroom alongside teachers and students more closely and invited teachers to be part of the research process.

Our project was based on our belief that learning occurs best when it is situated within authentic contexts and problems and that students need to learn the skills to be independent problem-solvers. Such a position requires then that students receive high quality instruction that reflects their learning needs, which in turn means that teachers receive high quality professional development and learning that meets their needs. By working with a group of teachers in a school and engaging leadership in the PD/PL, we were aiming to develop critically reflective skills necessary for ISTEs or those responsible for PD/PL support to identify, analyse and problem solve their way through the issues faced by teachers. (National Facilitator)

By engaging us as teachers in the design of the professional learning package, we feel like it is not something that is being shoved on us. We are all involved, we all need to upskill to ensure that we are able to provide ongoing support, develop learning communities in our school and establish systems in the school to record and monitor student's progress. Whatever shifts we make we have enough support within the school to make it happen and follow it through (Teacher as ISTE)

Impact on ISTEs

Participating in INSTEP has had the following impact on ISTEs and their provider organisations:


 
Through the trialling of the various approaches to inquire into ISTE practice facilitated through INSTEP, ISTEs gained significant insights about what constitutes effective ISTE practice and the knowledge and theoretical base that influences, supports and shapes their practice. In this section of the report, we discuss the short to medium term impacts of participating in INSTEP for those involved. The purpose of the longitudinal case study research was to explore the extent to which the early effects and perceived value of INSTEP have been sustained and embedded into the everyday practice of ISTEs. Consequently, the discussion in this section combines a case study story built over two data points - immediately on completion of the INSTEP project (October 2007) and 12 months after the project drew to a close (October 2008).

Overall, INSTEP participants unanimously stated that they found the INSTEP project to be invaluable. Adopting a R&D approach over three years and investing in understanding ISTE practice in great depth has contributed significantly to the knowledge base around this area. This investment was seen as an acknowledgement by the Ministry of the importance of inservice teacher education as a lever for change and enhanced connections and collaboration between ISTE organisations within New Zealand. Prior to INSTEP, the sector was seen as a collection of individuals and providers with different views and concepts about what works in inservice teacher education resulting in variable and inconsistent practice. The investment in developing the Teacher Professional Learning and Development: Best Evidence Synthesis Iteration (BES) was seen as the first step towards building a knowledge base around what we know about learning for teachers. The INSTEP project was felt to have extended these understandings and heightened awareness and understanding of what constitutes effective practice and also extended the knowledge and skills of ISTEs more generally. These benefits and impact of INSTEP on ISTE practice are discussed below.

Let me put it this way. There is no going back. Some may think it is a good idea because we were able to hide behind our busyness and workload but personally it means you are going back to a system where nobody knew what we were doing including ourselves. There was no quality assurance about what went on. Now we know what we want, what to expect and how we can get there. (Regional Facilitator)

Heightened awareness of what constitutes effective ISTE practice

Through implementing INSTEP in their particular contexts, engaging in problems of practice and working as a community of practitioners to research and further their understandings, INSTEP participants have developed a shared understanding and awareness of what constitutes effective ISTE practice. This is a significant achievement as, prior to INSTEP, these understandings were mainly personal and not widely shared across the sector. In particular, INSTEP had sensitised inservice teacher educators to their own practice and demonstrated the value and need for an anchor to ensure that professional development and professional learning was relevant and responsive to teachers' needs. Prior to INSTEP, ISTEs viewed their role as one that was limited to facilitating professional development and professional learning and believed that they had a limited role in promoting successful student learning since their practice was a step or two removed from direct teaching of students. However involvement in INSTEP highlighted the ways in which ISTEs' practice could impact on student outcomes, particularly through ongoing analysis of student and teacher needs. Asking questions of themselves and linking their learning needs to learning needs of teachers and school leaders, which in turn leads back into student learning needs, is the essence of the inquiry and knowledge-building cycle that anchors ISTEs' practice. This grounding of ISTE practice provided the context for their work and ISTEs believed this was achieved through the INSTEP inquiry cycle as illustrated in the learning materials published from INSTEP:

Figure 8: An Inquiry and Knowledge-building Cycle for Inservice Teacher Educators

Image of Figure 8: An Inquiry and Knowledge-building Cycle for Inservice Teacher Educators.

The findings of the survey corroborate these findings. When INSTEP participants were asked to identify factors that drive and influence their practice post INSTEP, needs of teachers and students emerged as being the most important with over 54% stating their practice is influenced by student needs and 36% stating teacher needs. A small minority of respondents stated that their practice was influenced by School Support Contract deliverables (2%).

Figure 9: Factors that influence ISTE practice

Image of Figure 9: Factors that influence ISTE practice.

Yet another aspect that has been brought to their attention through INSTEP is an awareness of ISTEs' own learning needs and experiences, particularly relating to content and activities. In Ki te Aotūroa - Improving Inservice Teacher Education Learning and Practice "'content' relates to what is being learned (conceptual understandings and theoretical principles, and their relationship to practice) and 'activities' relates to how it is being learned ie ways in which people try to learn, extend and apply new understandings and skills." Applying these learnings in the context of INSTEP has led to heightened consciousness of the need to examine the evidence of their changed practice so that they can identify the impact of their learning on that practice. Therefore, asking questions that surface what ISTEs have learned and evaluating the impact of their own practice in promoting teacher and student learning have served to lend a level of rigour and robustness to ISTE practice.

Inquiry into practice through INSTEP has also led to increased consciousness of the different theories and frameworks that inform ISTE practice. When asked in the survey to identify the theories and frameworks that inform their work, ISTEs said that their practice was informed by theories of learning, adult learning, leadership pedagogy, and action learning/action research. These theories helped inservice teacher educators to understand how people learn and to use this understanding to make decisions about how best to support their own and others' learning.

Figure 10: Theories that inform ISTE practice

Image of Figure 10: Theories that inform ISTE practice.

ISTEs acquired new knowledge and skills

A key focus of INSTEP was strengthening the capability of teacher educators through an R&D approach framed around reflection and inquiry into their own practice. INSTEP was set up to allow ISTEs in the project to play dual roles - engage in INSTEP as 'learners' (reflecting on their own practice) and also as 'teachers' (sharing their learnings with ISTEs more generally). They were supported in this endeavour through a range of resources (such as materials, regular meetings, time, access to research mentors) that encouraged them to integrate and embed new knowledge into their practice. Therefore, examining the extent to which the ISTEs involved did acquire new knowledge and skills was a critical question for the evaluation. A key focus for the evaluation was to identify:

  • What new knowledge and skills have ISTEs gained?
  • What shifts have they made in their practice as a consequence?
  • What is the evidence of these shifts?

Understanding these shifts offers insights into 'provider pedagogical content knowledge', a term introduced by Timperley et al. (2007) to refer to the knowledge and skills of ISTEs. The term is defined in the glossary as follows:

Provider pedagogical content knowledge: The knowledge and skills that providers of teacher education need if they are to assist teachers to make a difference to students. This includes knowledge of the pedagogical changes teachers need to make in order to improve their practice, as well as knowledge of how to make the content meaningful to teachers and manageable within the context of teaching practice.

The findings of the evaluation showed that participants involved in INSTEP have gained significant new knowledge and skills through investigating their problems of practice and engaging with their peers through the professional learning communities set up within INSTEP. These shifts have been well documented in the research reports written by participants. The sector survey further illustrates these shifts. When asked to identify aspects of ISTE practice that had changed as a result of INSTEP, respondents identified the following:

  • better evaluation of ISTE practice
  • greater understanding of ISTE practice
  • increased reflection of personal practice
  • more aware of their needs and where to get support for improving their practice
  • greater focus on ISTE practice development.

Through the case study component of the evaluation, we were able to compare and contrast these shifts vis-à-vis current practice so as to discern the extent of the shift and how they do things now as opposed to how they used to do things in the past. This would provide important contextual information to help understand the true value of these shifts in light of current practice.

Knowledge areas

The following table summarises the most significant learnings in the knowledge area of ISTE practice. The understandings of the shifts in the knowledge and skill areas were generated and validated in our interviews with NFs, RFs, ISTEs, teachers and provider organisations over a 12-month time frame through the longitudinal case study research.

Table 1: Shifts in knowledge as a result of INSTEP
Knowledge Area: Data (Gathering, Analysis and Use)
Conventional/
Current Understandings
Emerging,
New Understandings
Evidence
of Shifts
Focused purely on standardised student achievement data as outcomes for students defined in terms of student achievement Focus on student achievement as the ultimate goal but recognise the need to focus on development of a professional learning approach that could change teacher practice and understandings Source of Evidence
- ISTE self reports
- Teacher feedback
- Analysis of problems of
  practice

Actual Evidence
- Enhanced capability to
  analyse data
- Use of data to inform
  teaching practices
- Data from surveys or
  interviews from
  students
- Student achievement
  data
- ISTEs investigating data
  more deeply and asking
  questions
- Team based analysis of
  data within professional
  learning groups in
  schools
Greater reliance on assessment data to tell the story Appreciation of the need to integrate wider sources of evidence to build the story including surveys of students and teachers; observation guides; student voice
Data analysis limited to building a statistical picture of achievement for reporting purposed Asking deeper and more complex questions of the data to challenge existing beliefs and assumptions or investigating hypothesis Data as input into "collegial problem solving"
Knowledge Area: Ways of Working
Conventional/
Current Understandings
Emerging,
New Understandings
Evidence
of Shifts

Supporting teachers to improve their practice by modelling ways of working

Supporting teachers to improve their practice by modelling ways of working

Source of Evidence
- ISTE self reports
- Teacher feedback
- Analysis of problems of
  practice

Actual Evidence
- Practice informed by
  research
- Increased use of in class
  modelling
- Increased advisor
  confidenceto work with
  leadership and
  management
- evidence of mentoring
  relationships
Operating on gut instinct, assumptions; habit-driven practice; knowledge is tacit Recognition and valuing the types of theories that educators use to shape their practice Knowledge about how to make the content meaningful to teachers
Professional development focused on teachers; tended to be in isolation; minimal involvement from school leadership Taking a wider systems view and involving leadership and management across the school
Knowledge Area: Nature of the Job
Conventional/
Current Understandings
Emerging,
New Understandings
Evidence
of Shifts
Focused on delivering a 'package', recipe book approach to professional development Thoughtful, more considered, being guided by teacher learning needs and structuring professional development and learning around this Source of Evidence
- STE self reports
- Teacher feedback
- Analysis of problems of
  practice

Actual Evidence
- Stronger, more
  evaluative relationships
  with schools increased
  evidence of professional
  learning groups of pods
  within provider
  organisations
- Teacher feedback on
  quality & effectiveness
  of professional
  development
Out and about all the time; writing milestone reports; working alone Engaging in professional learning conversations
Working alone and in particular curricular areas Professional learning communities to share and grow practice knowledge and expertise; stronger mentoring and coaching role particularly for new ISTEs
Isolated and siloed approach to professional development; no real sense of a community 'Brokers' who foster connections between research and practice; ISTE community and the classroom; within the different levels in the school

Skills

An explicit expectation of the INSTEP project was that through inquiring into their practice systematically and reflecting on the implications of this on teacher and student learning, ISTEs would acquire new skills and grow as reflective practitioners. The findings from the evaluation indicate that the project has been successful in building skills of ISTEs in a number of critical areas - planning and designing a professional development intervention; facilitation skills and critical reflection skills. Combined with new knowledge, the acquisition of these skills has led to significant gains for inservice teacher educators.

Table 2: Skills gained as a result of INSTEP
Skills Area: Planning and Designing PD/PL
Conventional/
Current Understandings
Emerging,
New Understandings
Evidence
of Shifts
Directed learning; solution focused; looking for quick fixes Leading learning - co-constructing learning; seeking engagement from teachers in designing PD/PL programme Source of Evidence
- Teacher engagement
  and feedback
Leading the agenda The importance of mutual agenda setting; checking in
Skill Area: Facilitation 
Conventional/
Current Understandings
Emerging,
New Understandings
Evidence
of Shifts
Death by questioning - 'kept asking why till I get the answer I want' Artful questioning Source of Evidence
- Review of video data
  from the learning
  materials
- Audio transcripts
- Teacher feedback
- Self-report by ISTEs

Facilitator of professional development

Facilitator of learning

Delivery-oriented - "here are some resources for you to read and review that you may find useful" Using more "we" language rather than "I" or "you"
Skill Area: Critical Reflection 
Conventional/
Current Understandings
Emerging,
New Understandings
Evidence
of Shifts
Reflection on the run Critical reflection; honest reflection Source of Evidence
- Evidence of co-facilitation
- Timetabling time for
  reflection

Actual Evidence
- Professional learning group
  conversations
- Changing language of ISTEs
Too busy to reflect or tended to locate the problem with teachers or principals Developing an 'inquiry habit of mind'
Personalised reflection; not explicit Collaborative, shared reflection; inviting alternative perspectives on practice by involving others
Navel-gazing; reflecting for the sake of it Reflection that reflects the ISTE inquiry and knowledge building cycle


We all thought we did reflect on our work but did it differently. I went for a walk, others did it while they were driving on the way home and all sorts of things. We now realise that those sorts of reflection was not as effective and they were not timely and planned for. That actually the value of reflection is when we did it as a group and on all of focusing on the same thing, trialling something and then coming back to the group and talking about it and reflecting on the experience. It allowed a more thorough reflection. So there was a lot of discussion about our journey as facilitators and how at the beginning we were very around delivery and information and we had to be the fountain of all knowledge almost and how we developed over that time, moving away from that to be much more interactive and recognising where our teachers are at, and moving them on from there sort of stuff.

I worked in this school for two years and pretty much use the model of facilitation I had always been using - I stand in the front and deliver a package and they pick it and run with it in their schools. When I looked for signs of change in the classrooms, as a result of my classroom observations, I didn't find any charge. Teachers were doing all the talking and the children sat back and listened. The kids were passive. I blamed the teacher initially and through my inquiry in INSTEP I now realise that it is my professional development approach that is not working. I was putting teachers in a passive role of learning.

ISTEs gained more confidence; 'voice' to describe practice

Inservice teacher educators commented that INSTEP had given a 'voice' to their work and practice. Prior to INSTEP, inservice teacher education were unclear about the 'professional' status of their practice and felt that they were often not given the respect or the mana that reflected their value and status to educational outcomes. With INSTEP and the knowledge gained as a consequence of the research and development project, there is a growing sense of a professional identity for ISTEs which was deeply comforting and rewarding for inservice teacher educators. Surfacing the underpinning theory around teacher education practice and highlighting the nature of its contribution to outcomes for teachers has imbued the practice of ISTEs with a level of respectability.

Table 3: Attributes acquired as a result of INSTEP
Personal Attribute: Confidence 
Conventional Current Understandings Emerging, New Understandings
Hiding behind structures, content and paper Flexible, open and encouraging
'I really want to be liked" Challenging teachers respectfully
Personal Attribute: Inclusive
Conventional/Current Understandings Emerging, New Understandings

Owning the initiative or the professional development

Collaborating and sharing the learning

Personal Attribute: Listening
Conventional/Current Understandings: Emerging, New Understandings
'Sharing my war stories with them and telling them what the problem really is based on my years of experience" "Focusing on what they really are saying and listening"
Following the programme set by me Checking in and making sure that the PD meets their learning needs


Findings relating to the confidence and growing understandings of the ways in which ISTEs can work effectively with teachers through INSTEP were further validated by the survey results. When asked to describe how they work with teachers, over 50% said that their work now includes modelling behaviours, working with teachers to define their learning objectives and regular checking in to ensure shared understanding of the professional development purpose. Interestingly, ISTEs are still coming to grips with some aspects of their role, particularly with respect to challenging teachers' and leaders' assumptions and beliefs about their practice. These findings point to the fact that the ways in which ISTEs work with teachers are undergoing gradual change and ISTEs are aware of the need to engage in behaviours that challenge current practice.

I do struggle with the whole idea of challenging beliefs and creating dissonance. I know that it is important as it is the beginning of self reflection. But it is a process that we still have to work through as ISTEs. When I have tried to engage in challenging conversations, I have come away feeling hollow as people are not grateful and are not nice to deal with.

Figure 11: How ISTEs work with teachers

Image of Figure 11: How ISTEs work with teachers.

Institutions acquire new knowledge and understandings about how to support ISTE practice

Achieving sustainable shifts in ISTE practice depends on the extent to which education provider organisations introduce and formalise changes to their structures, systems and processes to embed new learning. In the absence of such institutional adjustments, learnings and insights gained from INSTEP could easily be limited to the life of the project and learnings not transferred to ISTEs' everyday context. For instance, in implementing INSTEP, participants developed a deep understanding of the role and contribution of critical reflection to their practice and trialled ways in which such reflective practices could be integrated into their everyday work to enhance the impact of their work with teachers. This experience highlighted the power of reflection and inquiry for ISTE practice and participants were keen to build on this knowledge and experience post INSTEP. However sustaining these practices in an ongoing way requires provider organisations to structure their work differently and to provide appropriate mechanisms and support to allow meaningful reflection on practice. The purpose of the longitudinal case study work was to explore these issues in some depth, identify strategies used by organisations involved in INSTEP to address issues around sustainability and gain some insights into factors that helped or hindered institutions to embed changes in the wider system.

A key expectation was that as National Facilitators gained new knowledge, skills and understandings, they would use these to influence their organisational contexts and to lift the discourse on quality and effectiveness of ISTE provision. The evaluation found that the National Facilitators have had a strong impact on their own institutions and have worked relentlessly with the leadership and management in their respective organisations to discuss and debate the implications of INSTEP experiences. In a number of cases, it would be fair to say that INSTEP has triggered changes, including reviewing management structures, performance appraisal processes, structure and focus of ISTE professional development days, induction processes and creation of professional learning communities.

We created a new position in our organisation of a Professional Development Coordinator to provide leadership and structure to how we support the professional development and learning of our people. (Provider organisation)

In the past our professional development days were really business sessions really. Once in a while someone from the sector was brought in and they taking about something but there was no follow up, no discussions about how we could or would apply it in our work. We now realise that ultimately we need to work out what does this mean for us, what are we doing currently and how we can change that? (Provider organisation)

We as an organisation are now alerted to the fact that we need to look at how we work and how we support our people to work effectively. Now we now that you cant come here as an advisor and go out and do what you like. We now are clearer about what we expect from our advisors and are willing to support them to achieve optimum results for teachers and students. (Provider organisation)

Creating professional learning communities

In one case, the provider organisation has set up Professional Learning Groups (PLG) as a structure for facilitating inquiry into practice. Each and every ISTE in the organisation is attached to a PLG which is run by a PLG coordinator. An ISTE engages with a problem of practice within a PLG and can choose to use the PL progressions to frame their inquiry. Their inquiry journey is mapped and documented in a Professional Learning Portfolio and they systematically inquire into their practice and gather evidence of the impact of the shifts they make using a combination of tools such as teacher feedback, observations, videoing, and narrative stories. These portfolios are then used as critical evidence of shifts and progress by the ISTE in performance appraisal discussions and meetings. In this way, the organisation is attempting to bring about greater synergies between the inquiry into problems of practice and its contribution to lifting the overall quality of ISTE provision. Key challenges faced in this organisation include growing variability between PLGs with some performing well and others tending to take a softer approach. There was also a reluctance amongst ISTEs to video their practice or to allow peer observations. Consequently, the management in this organisation is considering ways in which these issues can be resolved to be truly effective learning organisation.

Refocusing professional development days

In another case, the organisation has reshaped their professional development structure and processes for ISTEs. In their team days, where the focus is on the professional development for all staff, there are sessions around the INSTEP project and sharing of the growing body of knowledge in this area. The management has also initiated their own research to ascertain what ISTEs consider to be the most effective facilitator practices and tapping into the tacit knowledge of their own ISTEs to prioritise and come with a list. In this way, organisationally, there is a shared understanding and view about what constitutes effective ISTE practice and the characteristics of an effective facilitator. The aim is to use these understandings to inform induction processes and programmes, as a performance monitoring tool, input into the recruitment strategy or to focus and anchor inquiry into ISTE practice in more systematic ways.

One of the most important things post INSTEP for us as a provider of PD/PL services was to look for ways in which we could embed what we had learnt into our management structures and practices. We created a new role and a new position in our management structure for a Professional Development Coordinator. This person will help review our professional development days, programme content and develop tools and processes to help get maximum value of this investment and time. They will also provide advice in how new advisers are inducted and trained in our institution. (Provider organisation)

In the past our professional development sessions were really business sessions. Once in a while we invited someone to come and talk about something but there never was any follow up or discussion. We now realise that ultimately we need to work out what does this presentation or research or new framework mean for us. What are we doing currently and how can we change that to reflect what is best practice (RF)

Induction

Induction is another area that has received significant attention in INSTEP. The lessons learnt from INSTEP have heightened awareness and understanding of the qualities and competencies of an effective facilitator and this has led providers to question and review their current recruitment and induction programmes. As a result, organisations were looking to creative ways in which they could integrate these understandings to set advisor expectations from the start and promote a culture of inquiry from the first day of advisory work.

We have really focused on induction of new advisers. We didn't really know what was needed prior to INSTEP. So we threw them the car keys and said here is a car, a map and told them to buzz off to schools and get started. When you come across a problem come and talk to us. Often we didn't see them for a while. But now, we know so much more and we are ensuring that induction is strong and sets up expectations that this is a professional learning environment. We talk, we discuss and solve problems collegially. This is a huge step. (Provider organisation)
Reviewing management structures

Growing awareness of the need to provide ongoing support for facilitators had led some large provider groups to rethink their current management structures. This was in keeping with the emerging understanding that if ISTEs were to grow and develop professionally, then organisations needed to provide the professional support needed to encourage this growth. In one case, the provider organisation has introduced a new team management structure in order to provide greater level of leadership and support to ISTEs. The support is wrapped around an ISTE's problem of practice or challenge of practice where they are supported by the team leader to undertake inquiry into the problem in a systematic way as illustrated in the following quotation:

We have a new management structure now and that is really a huge shift and I think probably directly related to INSTEP. We now have team leaders who support facilitators to inquire into a problem of practice and they map this inquiry in what we call professional learning portfolios, where a facilitator is expected to pinpoint, I guess is the word, a challenge of practice and then talk about it, and then have a learning journey for a year and have a look at how they're going to meet that challenge. So there's, and part of that process is what I described to you around the audio videoing, so it's within the professional learning portfolio that the audio video.

These shifts in organisations' understanding and support are further validated in the survey. When asked about the level of support received from the organisation for professional development, the response was overwhelmingly positive with over 90% (n=161) stating that the level of support was either 'very good' or 'good'. Interestingly, those that work for private providers appeared to rate the level of support higher than those that work for a School Support Service organisation. Size could be a factor in this respect as SSS organisations are usually larger and support from management would be more formalised and limited by availability of managers. Some of the SSS organisations have introduced some changes to their structures to offer higher levels of support but these have yet to take shape. For instance, TEAM solutions have recently introduced a new team management structure to provide a greater level of support to advisors. However, at the time this survey was undertaken, these changes were yet to be fully implemented resulting in current perceptions.

Figure 12: Level of support from management for professional development

Image of Figure 12: Level of support from management for professional development.

Nature of support

When asked about the various forms of support offered to ISTEs, a combination of informal and formal support mechanisms appeared to be in place in provider organisations. The following table outlines the different forms of support available.

Table 4: Forms of support offered by providers by provider type
Forms of support Private
Providers
School Support
Services
Formal induction programme 50% 81%
Regular team meetings 82% 75%
Structured feedback 59% 35%
Professional development plans 50% 56%
Formal performance reviews 41% 58%
Time allocated to share best practice with colleagues 77% 79%
Time allocated to reflect on and review practice 68% 65%
Documented ISTE guidelines and procedures 45% 40%
Access to up to date learning materials eg INSTEP 91% 69%


From this table it appears that there is greater similarity than difference between the two provider groups, with the exceptions of formal induction programmes where School Support Services rated higher (81%) and formal performance review processes (58%) where again they rated higher than their counterparts in the private sector. Across both groups, team meetings and professional learning groups were the most common ways in which organisations promoted sharing of ISTE practice.

Shared understanding and clarity across the sector regarding role and purpose of ISTEs

Quality teaching and learning in any community requires a shared vision and understanding of what is to be achieved in practice. An underpinning rationale for INSTEP was to address a gap in educational research and literature about the practice and learning of inservice teacher educators. By adopting an inquiry approach for the development of practice, it was felt that ISTEs across the sector would build a shared understanding of what constitutes effective ISTE practice. Therefore, ascertaining the extent to which INSTEP has been successful in building this shared understanding was critical to the evaluation. The evaluation findings suggest that there is a more sophisticated understanding of the role of ISTEs. Whereas in the past ISTEs were seen purely as facilitators of professional development, they are increasingly seen as agents of change and as facilitators of learning. These roles described above are believed to work collectively to enhance their impact on teachers.

The case study research and the sector survey showed that there was a clearer and sharper understanding of the role and purpose of ISTEs across the sector as a result of INSTEP. In the sector survey when asked whether they have a clear understanding of their role as ISTEs, 95% of those who responded said that they 'strongly agree' or 'agree' with the statement; 3% said that they were 'neutral' and 2 % said they 'disagree' with this statement.

Through INSTEP they also appeared to have developed a clearer sense of their place and contribution to teacher professional development and learning, particularly in relation to achieving improved student outcomes. In this section, we discuss participants' understandings of their role post INSTEP and how they believed playing these roles contributed to bringing about changes in teaching practice.

ISTEs as pedagogical leaders

Through INSTEP there is a growing awareness and understanding of the emerging role of ISTEs as pedagogical leaders. As leaders of teaching and learning, increasingly ISTEs are expected to have access to and know about what is best practice, provide leadership in teaching and provide a range of teaching opportunities for teachers that could upskill them in development of their own pedagogical practice. However, the findings from the evaluation suggest that the notion of ISTEs as pedagogical leaders is still in its infancy and while INSTEP has identified this as an important dimension of an ISTE's role, it would be fair to say that ISTEs have yet to come to terms with what this means for their day-to-day practice.

There are a few ISTEs that understand the implications of this role for their practice. In their view, as a pedagogical leader, they would be expected to possess both curriculum and learning knowledge and skill and would systematically assess and evaluate the effort of their demonstrated pedagogical practice. This would require ISTEs to think beyond inquiry and conversations and take a stronger research orientation to their work - increase focus on data and design interventions on the basis of the data and analyse and evaluate efforts vis-à-vis expected outcomes, regularly inquire into the effectiveness of their work and interrogate research literature to inform their own evaluations and progress.

As pedagogical leaders we will be developing learning approaches to assist teacher practice and development and ensure that these approaches are anchored in student learning. (ISTEs)

ISTEs as change agents

ISTEs have traditionally seen their role as facilitators of professional development and over the years they have come to view their role as facilitators of learning. With INSTEP there has a further shift and there is a growing understanding of their role as agents of change. The current environment for schools is changing rapidly and teachers and school leaders need to learn how to change and adapt their practice to achieve better outcomes for students. This requires ISTEs to conceptualise their role differently and some ideas that are discussed in Ki te Aotūroa include ISTEs as 'brokers' (Wenger, 1998) or people who bridge and foster connections at the boundaries of two communities of practice: the ISTE community and that of the teachers and school leaders with whom they work as change agents.

Through engaging in a joint inquiry into real problems with teachers ISTEs are essentially attempting to (a) extend teachers' understandings of their context and (b) helping teachers to work towards a coherence model of teaching and learning to transform the culture of a school. This happens over time as teachers and their leaders see the impact of the new learning on student learning and become more committed to ongoing cycles of improvement.

Sometimes the change is at the individual teacher level (assisting teachers to routinely reflect on their practice and understand the theories that inform their practice) and other times it is directed at the whole school as the school system needs to adapt to be more learning-focused. Therefore, ensuring that an inquiry results in change and improvement that is enduring and sustained requires a balancing act from ISTEs. Understanding their role as change agents has been an important shift for ISTEs in INSTEP. It has led to a stronger focus on supporting teachers to take ownership of their learning through continual goal-setting and monitoring their own progress towards these goals. It has also led to a focus on the design of an intervention rather than delivery of the intervention and a wider systems focus so as to bring about cultural shifts in the school that supports professional learning.

While sustainable improvements require a focus on long-term outcomes based on a vision of what is possible, ongoing feedback against smaller indicators of success helps provide both pressure and support for change (Guskey, 1995). Guskey (2006) also adds that it is 'particularly important for participants in professional development to see some indication of success early in a change effort. This can help generate commitment to the change from key actors in the system and build strong learning relationships between members of the different communities of practice that are collaborating on the change process.

ISTEs as inquirers

Modelling new learning appeared to be a central component of effective practice and an important step in scaffolding support to teachers and facilitating teacher learning. The inquiry approach embedded in the design of INSTEP was a critical factor in bringing about significant shifts in ISTEs' understandings about their role as inquirers. By trialling different approaches and undertaking inquiry into their own practice, ISTEs critically reflected on various aspects of their role and developed their ability to identify problems of practice to interrogate data to analyse and make sense of these problems; and developed strategies to enhance the learning environment for teachers and students.

Through inquiry, ISTEs in INSTEP critically reflected on what was happening in a given situation, with the ultimate goal of achieving improved learning outcomes for teachers and students. This helped ISTEs to create new knowledge which they then used to inform their subsequent planning and actions. However, pre INSTEP, most of this knowledge was tacit and through INSTEP participants have gone through a process of critically reflecting on their current practices, beliefs and mental models to make these understandings more explicit. Specifically, as a result of INSTEP, ISTEs have begun to understand the need to:

  • continually reflect on their ability to support teacher practice in relation to student learning;
  • ensure their own practice is based on sound theories of teaching and learning for adults;
  • continually inquire into their own effectiveness in a systematic way; and
  • adapt their institutional systems and structures to support ongoing inquiry.

These benefits were consistent with the goals of INSTEP as the project was designed to give ISTEs the opportunity and the mandate to engage with inquiry into their own practice with a view to understanding it and transforming it.

In INSTEP, this was facilitated in a number of ways, including monthly meetings of national facilitators and ISTEs. In the meetings of national facilitators, participants could collectively inquire into their own beliefs and assumptions, ask hard questions about the lens through which they select and interpret evidence or observations and the impact that their decisions and actions have on the people they work with - teachers and school leaders. In the meetings of ISTEs, participants engaged with a problem of practice within their own learning groups. They acknowledged that inquiry was not a comfortable process but it certainly resulted in a willingness to explore different approaches and make use of tools to achieve better outcomes for students.

As inservice teacher educators and leaders of professional learning, ISTEs are well placed to model inquiry and evidence-based practice in their work. As they gained confidence ISTEs supported the use of inquiry in schools they worked with, at two levels: with teachers in classrooms through gathering, analysing and interpreting a range of information to help understand what was going on and to develop strategies to help the student achieve their full potential; and at whole-school level to create cultures of inquiry at all levels of the school.

In the long run it does help us with accountability, because we're really clear about what's going on. If you're looking at people that closely, you've got a really good idea about what they're doing, what they're not doing. We get accused of micro managing at the moment, because everybody is, you know the team leaders are in observing, they're doing their video audio, so people are saying, oh we're getting looked at much more closely. So some people are seeing it as a positive, and some people are saying, they don't trust us any more. So there's that kind of issue that we, you know, you hear them say, well yes we are looking at it more closely because we want to find out what you're doing.  (Provider organisation)

Understanding of these roles combined with pedagogical content knowledge was felt to be critical in affecting change in teaching and achieving positive student outcomes. None of this work in isolation and in order to maximise gains from these understandings, ISTEs need to ensure they work in tandem and balance the emphasis placed on certain aspects of their role in relation to the needs of the teachers and schools they work with at any given time.

Impact on teachers

ISTEs note that as a result of examining and inquiring into their practice, they are able to work with teachers in ways that ensures that teachers are more engaged, they undertake inquiry into their own practice and work towards developing communities of learning within their own schools.

Teachers are more engaged

ISTEs commented that engagement from teachers was a vital clue that their efforts and focus on their practice was working. Videoing of practice in particular was felt to be most useful as it highlighted the ISTE's tendency to dominate the conversation or bring premature closure to the discussion resulting in teachers disengaging from the learning process. The term interactive professionalism (Fullan and Hargreaves, 1996) offers a useful way of describing the ideas and principles that support relationships in which the knowledge and skills of all are shared and valued. The participants in these collaborative relationships are interdependent and there is an explicit recognition that each person brings their own contribution to the common endeavour. ISTEs bring their own knowledge base from their own community of practice and teachers bring their own experience and expertise to the relationship. Valuing these contributions allows the relationship to be built on mutual respect, which in turn creates a positive learning environment.

Our conversations with teachers involved in INSTEP supported these ideas. Teachers commented that the openness and willingness of the ISTE to listen and focus on their needs determined their level of engagement with the professional development interaction.

Teachers as inquirers

Inquiry is a way of reflecting on professional practice so as to help make decisions about practice that will help promote students' learning and well-being. Inquiry can take many forms and in the context of INSTEP, teachers engaged in inquiry by examining their practice with ISTEs using in-class observations, undertaking critical data analysis to unravel interesting patterns in the data and participating in critical dialogue with their peers by interrogating the assumptions and beliefs on which their practice is based. Our conversations with teachers showed that ISTEs were beginning to make forays into these areas and engaging in different types of conversations with their teachers, as is evident in the following quotations from teachers:

For instance, in the past I used to give the same text to all kids in the class. Since the kids are at varying levels by giving them the same text, I realised that some were struggling while others were flying. But I just continued with this approach as I didn't know what else to do. When X (ISTE) came into my classroom she observed and asked me what I was experiencing. I talked to her about my observations about how kids were doing and who was engaged or disengaged from the reading. She then asked me about whether I had tried other approaches and I said no, as I didn't know what to do. She then showed me how I could run a guided reading session by giving different texts to each child and using more visual cues to aid reading. I really feel that my teaching is so much more effective. I know that because the ones that don't usually contribute in my class are now contributing and engaged in the class. (Teacher)

She (RF) made me think about my practice and asked me whether I believed what I was doing was working. I knew it was not working but I didn't know what to do. For example, I was running a guided reading group and tended to give the same text to the group. I would ask the children to read it one by one and it was clearly boring and not motivating much interest. She (RF) discussed this with me and we figured out together that we needn't have all kids reading the same text. (Teacher)

At the beginning of the year, my ISTE videoed the two teachers' lessons in our school and after school one day we all sat around and watched this video clips. She asked us to identify all the teaching decisions that we had made in that 10 minute video clip and our reasons for that decision. It was soon apparent that in many cases, our decisions were based on what we always did! We then realised how our classroom was set up to suit them as teachers - it was really about our managerial issues rather than focused on facilitating children's learning. This was huge for me and the other teacher in my school. (Teacher)

Teachers create communities of learning

Creating communities of learning that promote sharing of practice in schools was felt to be an important outcome sought by ISTEs as it fosters a culture of inquiry that will sustain the improvement. Teachers felt that INSTEP offered the opportunity to engage in ongoing inquiry about issues that matter most to them, exploring data from different sources and making evaluative interpretations on which to base their decisions on practice. Access to mentors, a critical friend, and external expert in the form of an ISTE helped scaffold the learning. Consequently, as INSTEP drew to a close, these teachers sought alternative ways to create and sustain their own learning community in their region. In their view 'there was no going back' and the progress they had made was to be maintained. In one case, the ISTE engaged with a problem of practice relating to transference of literacy skills across curricular areas in the school with a view to improving student achievement across transitions and year levels. This involved creating a learning group of teachers drawn from different curricular areas from within the school and the ISTE working closely with the group to strengthen content area and pedagogical knowledge base. Tools such as observations, concept mapping exercises, literacy diagnostics assessment, student voice, and teacher journals were used to engage in collegial problem solving resulting in deepened understandings and targeted literacy instruction within the school.

This is best illustrated in one case where the RTLit involved in INSTEP as a regional facilitator was approached by teachers to set up a forum where they could discuss and critically reflect on their practice. This has led to the formation of a voluntary professional learning community of teachers in this region who share a common vision and purpose and have negotiated a way of working that reflects this vision. The group meets weekly and the agenda is set by the members. As members have built trust and respect for each other, they have begun to bring student work to the forum to inquire into the data with a view to developing different strategies to improve student learning. In this way, this group has created a professional learning community that is focused on making ongoing improvements to student learning.

I do think it has been incredibly beneficial and as I say it was a slow process it took its time and now I'm thinking some of those things that I learnt in these discussions are fabulous why did I not do them earlier! I have one of the youngest teacher in this group and I used to sit back and listen. But now I actually ask questions to help me plan and examine my work. For example, I have learnt so much about the need to focus on the learning intention. In the past I would have pulled the book out from the shelf, and given it to the kids and asked them to read but not told them why they were reading that or what I'm looking for when they are reading it. But through examining the value of learning intentions in this group, I now realise that I need to be more clear and explicit about this. I stop and think why am I giving the kid this book? Based on that I say to them what I want them to think about when reading the book. For example I may say today we are going to read smoothly and fast. This provides the child with the focus too. The biggest learning for me in these learning groups for me are learning to choose the book according to what that child needs and keeping a record and telling the child exactly why they are reading that book and what you expect them to do. (Teacher)

Challenges in sustaining shifts

While teachers acknowledged that they acquired new knowledge and skills through their involvement in INSTEP, the longitudinal case study research findings indicated that sustaining these shifts posed some challenges to teachers. Lack of support from school leadership and management and a culture of resistance in the school were identified as two critical impediments to sustainability. This is consistent with the work of a number of authors (such as Guskey, 2000; Earl and Katz 2002; Timperley 2003) who have identified a range of issues, conditions and systemic supports that are central to effective professional development. These include:

  • building a shared vision for the work among stakeholders - raised expectations and a focus on student's learning;
  • increasing participation of teachers in professional development through effective delivery, reflection on and application of research and theoretical information;
  • measuring effectiveness of any intervention using a range of methods;
  • focusing at a school level on the development of strong learning communities of teachers where practice is 'de-privatised' and in which there is a focus on collaboration; and
  • ensuring that professional development becomes an everyday part of a teacher's working life that is sustainable, school-based, site-specific and relevant to all members of the school community.

In the context of school-based approaches trialled in INSTEP, the evaluation found that while there were some instances where there were systems in place to support professional learning, there were others where the actions of the school leaders did not match their espoused theory, resulting inpocketsof good practice in the school.

Impact on the students

Unlike other PD interventions where impact on students is paramount, in the case of INSTEP it was apparent early on that impact on students was likely to be achieved indirectly through impact on teachers. Essentially the project was aimed at developing and establishing effective evidence-base approaches focused on the learning and practice of inservice teacher educators and was seen as a capability building project. Therefore, ISTEs viewed and tracked impact on students through their ability to ensure that teaching practice was informed by student achievement and outcomes. The inquiry cycle in the INSTEP materials clearly illustrates the linkages between ISTE inquiry and its impact on student outcomes and the evaluation suggests that ISTEs are consciously anchoring their inquiry in teacher needs which in turn is anchored in student needs.

The Controller and Auditor -General's report on Ministry of Education's suite of professional development support for teachers notes that 'although an analysis of student achievement information can identify areas in need of improvement, relationships between the professional development received by teachers and student achievement are complex. The performance of students can be influenced by a range of factors and circumstances". This was even more challenging in INSTEP which was essentially a capability building project and aimed at strengthening inservice teacher education practice through inquiry approaches. By building the capability of ISTEs INSTEP aimed to enhance the relevance and appropriateness of the professional development and leaning opportunities for teachers which in turn would create a positive learning environment for students.

The inquiry cycle in the INSTEP materials clearly illustrates the linkages between ISTE inquiry and its impact on student outcomes and there is evidence from the evaluation to suggest that ISTEs are anchoring their inquiry in teacher needs which in turn are anchored in student needs.

Impact on sector

A key objective for INSTEP was to support professional leadership and ongoing improvement within the inservice teacher education sector. Consequently evidence of the sector taking ownership and leading the discourse on quality of inservice teacher education can be seen as an important indicator of success in achieving objective 3. Survey data gathered from Sector Reference Group members indicated that INSTEP has had a reasonable impact on building a sense of ownership or community across the ISTE sector given the timeframe for the project. When asked about the extent to which INSTEP had built a sense of community across the ISTE sector, 61% felt that it was 'significant' or 'growing' while another 39% felt that it was 'minimal' or had 'no impact'.

Figure 13: Impact of INSTEP on the sector

Image of Figure 13: Impact of INSTEP on the sector.

The survey results also showed that there was a greater level of awareness and understanding of the competencies of a good inservice teacher educator. Sector Reference Group members were asked to select the four most important competencies of a good ISTE. However, the following four competencies rated significantly higher than others (for example, cultural competence; knowledge of research; teaching experience):

  • ISTE pedagogical knowledge (81%)
  • ability to work in a research/inquiry frame (81%)
  • pedagogical content knowledge (77%)
  • knowledge of the curriculum/content knowledge (45%).

Respondent description of the essential attributes of an effective ISTE practice included:

  • the ability to engage in ongoing inquiry and knowledge building
  • the ability to tailor delivery to the clients' needs
  • the ability to gather, interpret and incorporate evidence
  • the ability to reflect on and modify practice
  • a continual focus on improved student outcomes
  • communication skills and confidence
  • in-depth content knowledge.

Footnote

  1. Ki te Aotūroa: Improving Inservice Teacher Education Learning and Practice, p 44 published by the Ministry of Education, Wellington.

Contact Us

Research
For more information about this publication please email the:  Research Mailbox