Monitoring and evaluating curriculum implementation:

Final evaluation report on the implementation of the New Zealand Curriculum 2008-2009

Publication Details

This report presents findings from a national evaluation of the implementation of The New Zealand Curriculum. The project sought to establish a national picture of implementation progress in English-medium schools in the first two years following the curriculum's launch in November 2007.

Author(s): Dr Claire Sinnema, The University of Auckland. Report prepared for the Ministry of Education.

Date Published: March 2011

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  To view the individual chapters please refer to the 'Sections' inset box.

Section 6: Findings: Support for Implementing The New Zealand Curriculum

What help have educators had to learn about The New Zealand Curriculum?

Findings about the support that educators have had to implement The New Zealand Curriculum are reported, mainly, against three aspects (or factors) of support – internal support encounters, external support encounters and support quality. Internal support encounters are those that are available (typically) at any time within each school. External support encounters are those that require special arrangements to be in order to access the support. These aspects are organised in this way since a factor analysis of both the 2008 and 2009 paper survey data revealed these items to hang together in a common factor structure when all of the individual items were analysed.

Table 10: Items included in support factors
Support Factors Items included in the Factor
Support Encounters: Internal Colleagues from own school
Ministry of Education publications
NZC website
NZC document
Support Encounters: External Advisors
Private consultants
Facilitators from other initiatives
Colleagues from other schools
Support Quality Waste of time - Productive
Irrelevant - Relevant
Tedious - Stimulating
Flawed - Sound


Responses to the support encounter items were on a 4-point scale (0–3) to indicate the number of times each support type had been encountered since the launch of the NZC:

0 – Not at all
1 – 1–2 times
2 – 3–5 times
3 – 6+times

Responses to the support quality items were on a 6-point Semantic Differential scale, with the negative scale point scoring 0 and the most positive scale point scoring 5.

In both 2008 and 2009, there was a trend for far greater engagement with supports that are available internally (both colleagues and resources) than with external support (advisors, consultants, facilitators or colleagues from other schools). The mean for external support encounters was close to the "1–2 times" scale point while the mean for internal support encounters was close to the "3–5 times" scale point.

Figure 28: Support encounters: 2008–2009

Image of Figure 28: Support encounters: 2008–2009.

The emphasis on internally available support is highlighted in the following scenario described by a Primary school team leader. She describes a whole school teacher-only day approach to dealing with the "front end" of the document, followed by staff meetings focused on other aspects (in this case learning areas), led by school-based staff who have looked to The New Zealand Curriculum website as a resource.

Participant: So we've had two teacher-only days about the curriculum; one last year and one this year, and they've been really focused on unpacking the new curriculum and looking at it from the point of view of our school. So what we wanted to do was set up a document that was the NZC curriculum as [our school] sees it... So we [the curriculum team and the associate principal] set up what we're going to do and what direction we're going to take it in and we've done it over a variety of sessions. So we've done for instance at staff meetings, we've done one learning area a staff meeting. So at the beginning of the staff meeting we set out which learning area we're talking about, how we see that learning area in our classrooms, how we all know that we're teaching it effectively. The values and visions and things we've really left for the teacher-only days because they're a little bit longer to do.

Interviewer: So where has your professional development come from, to enable you to do that?

Participant: Nowhere really. That's been a self-taught scenario. So I tend to read articles on the NZC website.

A typical pattern was for schools to work on the "front-end" of the document initially, and many are now beginning to consider the learning areas:

We've done the values and unpacked the first part, but we're doing the learning areas, the curriculum areas next year. Coming up, that's going to be our big goal. It's been really good, a good discussion and working and getting things together and the values especially, that took quite a bit of community involvement, all those sorts of things. So yes, I think that we're feeling quite supported with the new document.

A secondary teacher gave a similar example of internal/school-based support for implementation:

We have professional development every week on a Wednesday morning before school. So the kids start an hour late, which is pretty common in secondary schools these days and we've had time during that time to look at the new curriculum this year. They've done a lot. (FG Secondary)

The frequency of encounters with the various internal supports is outlined in Figure 29. This shows an increase between 2008 and 2009 in those engaging in a more sustained way (6+ times) with each of these support types.

Figure 29: Frequency of encounters with internal supports

Image of Figure 29: Frequency of encounters with internal supports.

Figure 30: Frequency of encounters with Te Marautanga o Aotearoa and Best Evidence Syntheses

Image of Figure 30: Frequency of encounters with Te Marautanga o Aotearoa and Best Evidence Syntheses.

Focus group participants in the first year of the evaluation also reported two additional types of document that were providing support for curriculum implementation: the Best Evidence Synthesis series, and Te Marautanga o Aotearoa. For this reason, items about these were included in the 2009 paper survey. These items were analysed by role (see Figure 30) to indicate the extent to which teachers, principals and other leadership team members had engaged with these documents.

A large proportion of teachers, principals and leadership team members reported not engaging with Te Marautanga o Aotearoa at all. Teachers in particular had engaged less with Te Marautanga than others. Just over a quarter of principals (27%) and a third of leadership team members (33%) had referred to Te Marautanga more than three times, but only 7% of teachers reported the same level of engagement.

Slightly more than half (51%) of the teacher respondents reported engaging with a Best Evidence Synthesis at least once to support their implementation of The New Zealand Curriculum. Slightly more than half of the principal respondents (51%) and other leadership team members (58%) used a Best Evidence Synthesis three or more times.

Comparison of the frequency of encounters with internal supports (see Figure 29) and external supports (see Figure 31) shows that educators are far more likely to have had more frequent encounters with internal than external supports for curriculum implementation.

Figure 31: Frequency of encounters with external supports

Image of Figure 31: Frequency of encounters with external supports.

Opportunities to access sustained support (6+ times) with advisors was also raised as a point of frustration in participants' comments:

When the government decides to slash Team Solutions people who are the experts, then where does it leave us who are out here in the school with this document in front of us, going, "How do I implement this?" (Secondary teacher)

I think having more support for Gisborne in general would be extremely beneficial to schools. We share ideas across schools but still require training/support/strategies from the 'experts'. We recently had ERO and they questioned some of our decisions and where we were up to and it became apparent that we weren't where we thought we were up to! How do we know if we don't have external input? (Web09)

In 2009 more educators (56%) continued to view the quantity of support provision on the miserly, rather than the generous, end of the continuum (see Figure 32).

Figure 32: Views of the quantity of support provision: 2008–2009

Image of Figure 32:Views of the quantity of support provision: 2008%u20132009.

How do educators describe the quality of the support they have had to learn about The New Zealand Curriculum?

Support quality was rated slightly higher in 2009 than it was in 2008.

Figure 33: Support quality: 2008–2009

Image of Figure 33: Support quality: 2008–2009.

The overall support quality findings (see Figure 33) reflect responses to all support quality items (see Figure 34), including those about how productive, relevant, stimulating and sound support has been.

Figure 34: Support quality items: Mean response

Image of Figure 34: Support quality items: Mean response.

The ratings for support quality items were on 6-point Semantic Differential scales which had a negative anchor at one end, and a positive anchor at the other. Respondents indicated at one of the six scale points to indicate their view. Ratings were collapsed to indicate the percentage of respondents who viewed support quality at the negative end of the scale, or the positive end (productive, relevant, stimulating and challenging) of the scale.

Figure 35: Support quality items: Percentage of respondents at positive and negative end of continua

Image of Figure 35: Support quality items: Percentage of respondents at positive and negative end of continua.

In 2009 the aspect of support quality with the greatest proportion of respondents at the positive end of the continuum was relevance (70%) indicating that support tends to be considered relevant to the task of curriculum implementation. The aspects with the greatest proportion of respondents at the negative end were on the items with tedious (43%) and waste of time (39%) as the negative anchors. These findings indicate that approximately one third of respondents in 2009 (and close to half in the case of tedious-stimulating) continue to view their experience of support as more low than high quality.

Comments about high quality support were often made about individuals within schools and nationally esteemed experts whose expertise has been valued by those working in schools:

I think it's excellent. The people who are leading it at our school, she's an AP, and that's her focus, and um I think she's been tremendous. Also we've been bringing our reporting in with the standards, or the possible standards, prior to the knowing them, so she's been doing a lot of work with that as well... I feel very confident that she knows what she's doing.

Criticism related to the quality of externally available resource material:

I think a website is a good idea, because of course you can access that at any time. Perhaps what they need to do is have something that says – be more discriminatory about what they're putting on it. I was going to say have something that's sort of from a panel, so you have a panel of experts, so you might have, I don't know, [academic researcher named] and she runs all the discussion about the key competencies. So I think it needs to be something that says, 'these are the experts in their field and this is the kind of guidance that you can have.' 'These are [the people], if you want to ask questions, then you ask questions.' I just feel like what happens is they get a potpourri of stuff and they just stick it all on there and some of it seems to be of varying quality.

And also to the theoretical rather than the applied nature of resource material:

There's a lot of documents out there about the key competencies and why they're necessary and what we should have and how they're gonna contribute to our kids' learning, but there's very little advice on how you actually put them into the curriculum yourself.

Many comments referred to individual nationally esteemed experts whose expertise has been valued by those working in schools.

How have the materials, resources and programmes supported schools and teachers to make changes?

Responses to support quality items were used to group respondents as reporting low, mid or high quality support for curriculum implementation. A very clear pattern emerged when the responses by those groups to all other factors were compared. Those who reported higher support quality also reported higher ratings for practice (see Figure 36), change (see Figure 37), regard and confidence (see Figure 38) and those differences are all statistically significant.

There were marked differences in all factors (relating to support, receptivity and practice) between those indicating high and low quality supports. In all cases, the effect size calculations showed the difference between low and high quality support groups to be moderate or large (see Table 11).

The Key competencies (pedagogical) (d = 0.88), Parent involvement (d = 0.80), Teaching as Inquiry (d = 0.79), Student agency (d = 0.77), Key competencies (situated intra-/inter-personal) (d = 0.70), Values (d = 0.69) and Key competencies (disciplinary) (d = 0.40) factors all had large or moderate effect sizes. There was a moderate difference between the groups on extent of change to reporting (d = 0.75). There were large effect sizes for the differences between groups on extent of change to classroom practice (d = 0.87), regard (d = 1.55), and confidence (d = 1.29).

Figure 36: NZC practices 2009: Low support quality/high support quality comparison

Image of Figure 36: NZC practices 2009: Low support quality/high support quality comparison.

Figure 37: NZC changes to practice: Low support quality/high support quality comparison

Image of Figure 37: NZC changes to practice: Low support quality/high support quality comparison.

Of all the factors considered, the differences in ratings between those indicating high and low quality support were greatest for regard (d = 1.55 )and confidence (d = 1.29 ) which both had very large effect sizes.

Figure 38: NZC regard and confidence: Low support quality/high support quality comparison

Image of Figure 38: NZC regard and confidence: Low support quality/high support quality comparison.

Table 11: Practices, regard and confidence 2009: Low quality support/high quality support comparison
Notes:
  1. Situated Intra-/Inter-personal.
  2. M = Mean
  3. N = Number
  4. SD = Standard Deviation
  5. Effect Size (Cohen's d)
Variable   Support Quality Low Support Quality High Effect
Size5
 
M2 N3 SD4 M2 N3 SD4
Key Competencies:
- Intra-/Inter-personal1 1.9459 228 .67723 2.3380 532 .50154 0.70
- Disciplinary 2.0801 229 .53634 2.2757 535 .47498 0.40
- Pedagogical 1.2712 228 .69483 1.8248 530 .59841 0.88
Values 1.6100 228 .68087 2.0238 535 .56801 0.69
Teaching as Inquiry 1.7769 229 .56955 2.1725 533 .47143 0.79
Student agency 1.1004 229 .71540 1.6474 529 .70308 0.77
Parent involvement .9230 223 .71881 1.4696 526 .66985 0.80
Extent of Change to:
- Classroom Practice 2.7607 228 1.13101 3.6067 542 .89833 0.87
- Reporting 1.9842 221 1.33177 3.0038 533 1.38092 0.75
Regard 2.7012 222 1.15146 4.0390 543 .71608 1.55
Confidence 2.1031 223 1.07327 3.2716 541 .82668 1.29

Barriers

Respondents to the web surveys were asked to indicate on a 0–5 scale the extent of potential barriers (0, not a barrier – 5, an extremely serious barrier) to curriculum implementation. Lack of time, expertise availability, and high quality support were rated by teachers as the most serious barriers. For nearly all items (except web-based support) the barrier rating became less serious in 2009 than it was rated in 2008 (the "high quality support for implementation" item was included in 2009, but not in 2008) as shown in Figure 39. Information about the curriculum is considered the least likely to be a barrier, with a notable improvement between 2008 and 2009.

Figure 39: Mean scores for teacher ratings on barriers to NZC implementation: 2008–2009 comparison

Image of Figure 39: Mean scores for teacher ratings on barriers to NZC implementation: 2008–2009 comparison.

While the improvement in the mean scores for barriers between 2008 and 2009 is clearly positive, there remains a perception by a significant proportion, that lack of various supports are preventing curriculum implementation efforts. More than two thirds (67%) of respondents to the 2009 web survey indicated that lack of time for planning and implementation is still a notable barrier (moderate, difficult or extremely serious), 43% for lack of expertise, 44% a lack of face-to-face support and 37% lack of a professional learning community.

Figure 40: Percentage of respondents rating barriers to NZC implementation as 'Moderate', 'Difficult' and 'Extremely serious': 2009

Image of Figure 40: Percentage of respondents rating barriers to NZC implementation as 'Moderate', 'Difficult' and 'Extremely serious': 2009.

Supportive educational contexts

Three other important considerations were raised repeatedly in focus group sessions as participants talked about what had enabled or constrained their capacity to implement the curriculum—leadership, competing demands and school structures and systems.

Leadership

The importance of school leadership, and in particular, leaders' capability to lead others' curriculum learning, was raised by many:

Any change takes time and many people take a lot of time to change. It is frustrating to hear about some of the approaches schools are taking which amount to nothing more than clip-on activities. Sadly a lack of intellectual grunt is missing at the management level of many schools. (W09)

That's my biggest thing with being, leading professional development is trying to inspire teachers to recognise that they are presenting a model of the ultimate learner. You know, they've got their own subject area about which they are incredibly passionate and they're a great learner and that, but being a teacher is about trying to um inspire people to want to learn anything. And in order to do that, you have to be the best learner you can be yourself and model that for the kids, show them how you learn. (FG Secondary AP)

Many leaders acknowledged that their own understandings may not yet be deep enough to support others in the best way:

Well I was thinking, I think if there were examples of relating to others and how it's been explicitly taught, people might actually be able to identify or not identify with it much easier. I mean I'd like to—I was just about to ask you, what is an example then of explicitly using the key competency of relating to others? Because I myself, I'm sitting here going, off the top of my head, I'm not sure if I can think of something. (FG Secondary AP)

It's not just me but it's all of us in a way isn't it? [that need more information]. I mean I would class myself as a member of staff who actually has a pretty good grip of what is actually in the book, in having spent a lot of time dealing with it, but there are a lot of staff who haven't got to that stage and to help them, I need to have the tools to help them as well. (FG Secondary AP)

Competing demands

Throughout this evaluation, competing demands of assessment/qualifications and curriculum have been raised by secondary participants as a barrier to implementation.

I was just going to ignore all that thing [the curriculum] and just go with what's asked for in the achievement standards, then that's what I'm going to teach and so they pass and that's it. (FG Secondary)

We are completely governed—although we shouldn't be but we are, by assessment. (Secondary teacher)

I actually think there is a conflict between the new curriculum and assessment full stop, because one is confining, and that's the assessment, and the curriculum is trying to open new horizons and let us run with it. And as long as you require a national examination system for senior school, you're going to put the brakes on that. It's just the nature of it. (Secondary Deputy Principal)

The introduction of National Standards to the primary sector led many participants to express serious concerns about what they perceived to be competing demands of The New Zealand Curriculum and National Standards. This was raised in all of the focus groups held immediately prior to and at the time of the announcement of the National Standards policy, and by 10% of the respondents to the 2009 web survey who chose to make a comment in a "general comments" section of the survey. They describe reduced confidence, commitment and momentum in curriculum implementation as consequences of the perceived conflict:

There are mixed messages out there at the moment in relation to policy direction. This is causing unrest and uncertainty! (W09)

Keep the National Standards out of the equation and let the schools develop their curriculum in line with the essence of NZC. It is the best thing that has happened in Education for a long time, don't allow it to be dumbed down by schools worrying about the possibility of league tables and being judged by very narrow criteria. (W09)

It will be interesting to see how the largely unprescriptive curriculum works alongside a largely prescriptive national standards measurement of achievement particularly for students who have Special learning needs at primary school level. (W09)

It's a great curriculum and one to be proud of. I just hope those ill-informed and flawed National Standards don't dilute it and cause our enlightened 21st century education system to revert to a 19th century model!!!!! (W09)

School structures and systems

In several focus groups, discussion turned to the organisational structures and systems that may need re-considering in light of the possibilities presented by The New Zealand Curriculum:

The structures within schools and people who have been doing the same things year on year and you've got all those textbooks and you've got four hours of work or you've got whatever you've got traditionally, that's quite hard to turn all of that on its head.

For many the timetable, in secondary schools in particular, is viewed as a constraint to curriculum implementation:

I also think the timetable structure is a huge constraint. You have a group of students for an hour and you just get started and then they've got to drop and go to something which is completely different. And I think that helps stop the continuity of actually being able to work with them to work out what it is that as you say they can choose to do that. (FG Secondary)

They are always feeling that they are under time pressure at secondary school. Um your students taking eight—eight subjects, perhaps up to ten in the junior school; different hour every day doing something different, I think that that doesn't allow them to develop things either. And doesn't allow them to become really engaged in a lot of it. Some of the classes get to the point where they're lining up to try and get out of the room at ten to ten to go to the next class at ten o'clock and you say to them, "What's so good next period that you want to rush out of here," and they look at you as if to say, "Well that's not the point, we just know we've got to move on." You know they're not actually um going to the next subject because they really want to go there, or they've really hated what they've just had, it's just this treadmill. It's a treadmill day, and I think that's one of the biggest constraints in secondary school, is the fact that they don't get time. (FG Secondary)

Contact Us

Education Data Requests
If you have any questions about education data then please contact us at:
Email:      Requests EDK
Phone:    +64 4 463 8065