Strengthening Education in Mangere and Otara (SEMO)
Strengthening Education in Mangere and Otara (SEMO) is a Ministry of Education intervention designed to increase the capacity of the schools and communities of Mangere and Otara to offer high quality learning environments for children. This is the third and final evaluation report on the SEMO initiative. Three separate studies are considered in this report: School Governance, Reporting to Parents, and Perceptions of Pasifika Student Achievement.
Author(s): Viviane Robinson and Helen Timperley in association with Lorrae Ward, Lili Tuioti, Violet Tu'uga Stevenson, Sue Mitchell. Report prepared for the Ministry of Education.
Date Published: 2004
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Strengthening Education in Mangere and Otara (SEMO) was a Ministry of Education intervention designed to increase the capacity of the schools and communities of Mangere and Otara to offer high quality learning environments for children. It was one of several area-wide initiatives being co-ordinated under the national school support policy of the Ministry of Education. One of the special features of SEMO was the commitment to a three-way partnership between the Ministry, the schools and the communities to strengthen education in the two districts. The evaluation of this initiative was awarded to a team of researchers from the School of Education of the University of Auckland in 1998, and this is their third and final report. The initiative itself has evolved, with new partnerships being developed and old ones re-negotiated. As a result, SEMO has been replaced with a range of initiatives managed by local school clusters and supported by the regional and national office of the Ministry of Education. One of the most important of these initiatives is AUSAD (Analysis and Use of Student Achievement Data) designed to help principals, teachers and board members improve teaching through analysis and discussion of student achievement data.
Three separate studies are considered in this report: School Governance, Reporting to Parents, and Perceptions of Pasifika Student Achievement. The first two studies were undertaken as follow-ups to studies reported in the first two evaluation documents (Timperley, Robinson & Bullard 1999; Robinson, Timplerley & Bullard 2000). The third study addressed a gap in the earlier work.
During the 1989 educational reforms the highly centralised and regulated system for administering New Zealand's state schools was dismantled and replaced with a model of single-school lay governance (Minister of Education, 1988). Under this model, a school's board, comprising a majority of locally elected parent trustees, was given more responsibilities than in any other system of school-based management.
In the 12 years since the introduction of Tomorrows Schools (Minister of Education, 1988), there have been few empirical studies of the performance of boards. Despite that, the general consensus among professional and academic commentators has been that the 15,000 lay volunteers who govern New Zealand schools at any point in time are doing at least as well as the national and regional levels officials who came before them (Fiske & Ladd, 2000; McKenzie, 1999). As a result there are few calls for its overhaul, let alone reversal.
However, this generally satisfactory report is being increasingly qualified, as evidence emerges about variable board performance, with those in small and low decile communities being most at risk. In 1996 the Education Review Office published a highly critical evaluation of schooling in Mangere and Otara, which formed the catalyst for the SEMO initiative. As well as alleging widespread governance failure, the report raised questions about the appropriateness of the lay trustee model in communities where it was difficult to elect or co-opt sufficient financial and professional expertise, and where cultural norms conflicted with the requirement that boards act as the employer and appraiser of the principal (Education Review Office, 1996). The authors concluded that in some schools, boards gave principals both more discretion and less assistance than was intended by the legislative framework under which New Zealand schools are governed.
One strand of the SEMO initiative involved the improvement of governance through both area-wide and school-based initiatives including the appointment of board mentors, the systematic involvement of boards in decisions about expenditure of SEMO monies, and increased school and cluster-based training. One outcome of this work was the formation of the Otara Boards Forum, a collective of the Chairpersons of Otara schools.
The first two chapters of this report detail the findings of a study which was designed to produce both quantitative and qualitative data about how principals and chairs of Otara schools, along with Ministry officials involved in the initiative, understood selected governance tasks. The methods were based on a set of four standardised scenarios, each of which portrayed the work of a hypothetical Board of Trustees. The four scenarios described situations that were similar to those we had encountered during earlier research in the area. Three of the four scenarios focused on the educational, rather than financial or property dimensions of a boards' role. These were monitoring student achievement, responding to a parental complaint and developing a homework policy. The fourth scenario considered the process of appraising the principal.
At the time of this study, there were 15 state and integrated schools in Otara, 14 of which were members of the Otara Boards Forum. Interviews were completed with 12 chairpersons and 11 principals. The remaining chairpersons were either unavailable during the data collection period, or declined to participate. The nine Ministry officials who were interviewed were employed in either the national or Auckland regional office, and had responsibility for policy or operational aspects of school governance. About half of the Ministry sample had direct responsibilities for the SEMO initiative.
The findings of the study are reported in the first two chapters of this final report. In the first Chapter, Lay Governance of New Zealand Schools: An Educational, Democratic or Managerialist Activity, we explore how principals, trustees and Ministry officials think about governance. We are interested in the extent to which the ideas that have shaped governance policy have penetrated the thinking of those to whom governance responsibilities have been delegated. We are also interested in the extent to which the thinking of the three groups is comparable.
In the second chapter, The Difficulties of School Governance: Perceptions of Trustees, Principals and Ministry Officials, we consider participants' perceptions of the extent and nature of the difficulties which boards encounter in completing key governance tasks. This is done through a detailed examination of the match between the demands of selected governance tasks and the difficulties that trustees encounter in completing them.
Lay Governance of New Zealand's Schools: An Educational, Democratic or Managerialist Activity
Participants' understandings of governance were revealed by asking them to rate the performance of the hypothetical board in each scenario and to give reasons for their ratings. On two of the four scenarios, there were significant differences between lay trustees and Ministry officials. The two scenarios which portrayed a Board as not understanding data on its students' achievement, and handing responsibility for the principal's appraisal over to the principal, were rated significantly more positively by lay trustees than by the Ministry sample.
Three major components of participants' conceptions of good governance emerged during analyses of their explanations for their ratings. The first of these was compliance with governance formalities. Such formalities include learning a governance language and way of thinking, and include correctly determining who does what and in what sequence. While this emphasis on formal role was shared across all three groups, close analysis of their views revealed differences in their conceptions of the governance role and of the governance-management distinction in particular. The evaluations of the Ministry sample were guided by their understanding of national policy requirements and guidelines, while the point of reference for principals and board chairs was their own locally developed school policy.
Comments about the effectiveness and efficiency of governance tasks were the second most frequently employed evaluative category for all three respondent groups. Trustees and principals made more comments about the efficiency of task performance than did Ministry officials.
The third category, frequency of reference to communication and relationships suggests that governance is not only about formal roles and task completion. Nearly as important is the quality of relationships through which roles are enacted and tasks pursued. Respondents expected those relationships to allow trustees to gain the information and understanding they need to exercise their responsibilities. They also agreed on the nature of the barriers to gaining such understanding - professionals' use of complex educational language, trustees' reluctance to ask questions and their assumption of professionals' expertise.
The analysis of interviewees' evaluations of the four scenarios also showed that, in justifying their evaluation of the Boards' performance, very few references were made to either educational values or the interests of the community that the Board served. The former finding was somewhat surprising given that three of the four scenarios were written to capture the educational (concerned with aspects of teaching and learning) rather than non-educational aspects of a Board's work. It suggests that educational purposes and values, while strongly espoused by participants in governance, do not yet inform the practical understandings and actions of our sample. The latter finding is also interesting, given that one key purpose of the reforms was to strengthen schools' responsiveness to their communities through a form of local democracy.
The Difficulties of School Governance: Perceptions of Trustees, principals and Ministry Officials. As part of the wider school governance study described above participants were asked whether their own boards, or in the case of the Ministry staff, whether boards similar to those in Otara, found it hard to perform the tasks portrayed in each scenario, and if they did, to describe the nature of any such difficulty. The first section of the report considers which tasks were perceived as difficult by the different participant groups while the second investigates the nature of the perceived difficulties.
Difficulties were more likely to be reported for the two tasks of most relevance to the core business of teaching and learning (achievement and reporting) and for principal appraisal, than for the fourth task of responding to parental complaints. The latter task requires skills of communication and diplomacy, while the other three tasks also require considerable educational and human resource expertise - qualities, which are less likely to be present in lay governors. In other words, dealing with parental complaints may be seen as posing fewer difficulties because it is better matched to the skills and experience of lay governors than the other three governance tasks.
Principals, Ministry officials and chairs attributed much of the difficulty of governance tasks to limited understanding and expertise. Respondents varied in their accounts of the difficulties, with some seeing them as arising out of communication problems and the use of professional jargon. Others saw them as going far deeper to the question of whether laypersons have the conceptual frameworks needed to make sense of the language, even when it is suitably simplified. While most respondents saw it as the professionals' responsibility to help trustees to understand, a few principals raised serious doubts about the appropriateness of this strategy.
The data presented suggest that when trustees have limited understanding of what to do and how to do it, they become very reliant on the advice and guidance of professional staff. The latter, in turn, may unwittingly contribute to this dependence through an inability or unwillingness to coach their trustees, who are also their employers, into their governance role. The dependence is further sustained in the sample of trustees we spoke to, by a cultural tradition of deference to high status professionals. The end result may be a self-sustaining cycle of trustee delegation of governance duties, which overburdens principals and produces incorrect assumptions on the part of both principal and trustees of their respective roles.
The consequence of this dynamic is that the quality of governance depends crucially on the willingness of the principal to adopt the role of adviser and coach of their trustees. Depending on the particular approach taken, such coaching can result in a one-sided relationship which reinforces the dependence of the board, or over time it can lead to the empowerment of trustees. Our data suggested considerable variation in principals' willingness to adopt this role and uncertainty about their obligation to do so.
Reporting to Parents
The development of a partnership between schools and their communities formed a central tenet of the reforms of education administration in New Zealand more than a decade ago (New Zealand Government, 1989; New Zealand Public Service Association, 1988). This partnership was to be based on the free flow of information between schools and their communities. While the sentiments of openness and shared information formed the basis of the policy, for the next 10 years the content of the information to be shared between the partners was not specified clearly.
In chapter three we report on a series of three studies which describe and explain the reporting practices of schools in Mangere and Otara. Over the period of the three studies, the schools' reporting practices became increasingly focused on student achievement in response to the SEMO initiative and the changed reporting regulations. We sought to answer the research questions, "To what extent did the reporting practices involve parents /whanau as educational partners with the school?" and "What difficulties were experienced by school personnel and parents / whanau when changing their reporting practices in the directions required by the new regulations?"
The first study involved an interview survey of school personnel and a document analysis of reports to parents from thirteen randomly selected schools. Parent reaction to the reports was sought by interviewing parents who had attended report evenings in two of the thirteen schools. The first part of this study was reported in the first SEMO Evaluation report (Timperley, Robinson, & Bullard, 1999). The findings showed that, consistent with national practices in primary schools at the time, student achievement was likely to be evaluated against implicit individual or class-based referenced standards. As a consequence, parents of under-achieving children were frequently misled about their children's achievement levels. The major themes that emerged from this study were communicated to professional and community groups through a story about a hypothetical student called Lita who received excellent reports throughout her primary schooling, and was then shocked to learn at high school that her achievement levels were several years behind the national average. The story named "O le Tala ia Lita" or "Lita's Story" described the reactions of the parents to learning that their daughter was not doing nearly as well at school as they had been led to believe. The discussion of Lita's story resulted in a deeper understanding of both the causes and consequences of schools' current reporting practices, and a commitment on the part of some school leaders to changing the way they reported students' achievement to their parents.
Two years after the first study, a second was carried out to assess the impact of these discussions and of the changes in the legislative requirements for reporting student achievement. This second study involved a questionnaire, interview survey and document analysis of fifteen schools in the two communities. All schools were planning changes to their reporting practices. The survey indicated that teachers believed student achievement information should be reported to parents / whanau more accurately and with reference to national standards. Analysis of the actual school reports showed that more schools were reporting student achievement against explicit standards or were planning to do so. For most respondents, however, this focus on becoming more accurate did not appear to be driven by a desire to involve parents more closely in their children's education. The idea of a partnership where accurate information is shared for the purpose of working together to improve student achievement did not appear to be part of the thinking underpinning either the current or planned reporting practices.
The third study explored the long journey undertaken by two schools as they introduced more accurate and transparent reporting of student achievement. Data collection methods for these case studies included interviews of school personnel and parents, and analyses of selected school reports. Both schools in this study believed that it was important to report achievement against the national curriculum, to benchmark achievement against national rather than local standards, and to communicate to parents in ways that were readily understood. Our data suggest that a number of technical, ethical and professional dilemmas need to be resolved before schools can meet their goals. The technical dilemmas arise because teachers want to reference achievement to national curriculum levels yet there are few tools available to help them to do this and the curriculum levels themselves are so broad that they provide little guidance as to a student's progress. The ethical dilemmas arise around how to report low achievement honestly yet constructively and safely. The professional dilemmas arise for teachers who want to report in ways that convey their professionalism and yet can be understood by parents with limited knowledge of the New Zealand curriculum and, in some cases, of the English language.
These three New Zealand studies raise issues consistent with those identified in the international literature on reporting student achievement to parents. Schools believe that parents have a right to know, and the parents indicate that they want to know, about their child's achievement. Meeting this apparently simple goal requires resolution of complex technical, ethical and professional dilemmas.
Pasifika Student Achievement
The Ministry of Education, researchers, educators and the Pasifika communities in New Zealand have expressed concern with the educational achievement of their students. A recently released Pasifika Education Plan (Ministry of Education, 2001) describes the Governments' commitment "to reducing disparities and improving the well being of Pacific peoples in the New Zealand education system." Various strategies have been proposed as a means to strengthening Pasifika students' cultural identities and creating a better educational match.
In chapter four we report on a study in which we seek the views of participants on what they believe is needed to raise the achievement of Pasifika students. We then examine the congruence between these views and the strategies proposed in the Ministry's Pasifika Education Plan (2001) and what is proposed in the research literature. Pasifika teachers from four schools in Mangere, with Pacific bilingual units, were interviewed along with the principal and a second member of the Senior Management team from each school. The 24 teachers who took part in the interview study were a mix of bilingual and mainstream teachers. Six Ministry of Education officials, who had direct responsibility for Pasifika education issues and policies or their implementation in Mangere, were also interviewed.
The interview schedule included both open-ended questions that were designed to capture the participants' views about how to raise Pasifika students' achievement and more closed questions that probed specific issues, such as employing more Pasifika teachers and creating additional bilingual units. We also specifically asked what respondents believed the Ministry of Education should do to raise student achievement. Finally participants' were asked to rank the importance of particular school tasks such as preparing children for the next level of schooling or nurturing children's ethnic identity. The issues relating to the various strategies for raising achievement are reported in part one while the rankings of school tasks are reported in part two of this chapter.
The strategies suggested by the respondents for raising achievement fell into eight broad categories. In summary, all respondent groups considered teacher quality and home-school liaison issues as most important although there were considerable differences between the groups. Another area considered to be in need of a change by at least some respondents in each group was the provision of bilingual education. Other responses included increasing the number of Pasifika teachers, improving assessment practices, increasing teaching resources, being more positive about the achievement of Pasifika students and changing school management practices. Each of these response categories is elaborated below.
Issues of teacher quality included concerns about teacher attitude and the need to change teaching practices in ways that reduce the mismatch between the Pasifika students' cultural capital and the expectations of the school as the key strategy to improving achievement.
One way to facilitate a teaching process that utilises Pasifika children's cultural capital is to increase the number of Pasifika teachers. However, there was a greater endorsement from all groups for bilingual education than for increasing the number of Pasifika teachers. Although the introduction of bilingual programmes is dependent on employing fluent native speakers who can also speak English, it appeared that this more focused strategy was preferred over the more general one of increasing the proportion of Pasifika teachers. In fact the only group for whom more than 10% of responses endorsed an increase in Pasifika teachers in order to raise student achievement was bilingual teachers.
In the interviews, the frequency with which parent and school-community liaison issues were mentioned was second only to the teaching process. While the research literature focuses on appreciating what parents bring to the home-school partnership most of our interviewees framed their responses in terms of parents learning from the school so they can support their children at home.
Analysis of the other categories showed that although there were some specific concerns with the experiential basis and language of current assessment, changing assessment practice was not seen as a way to raise student achievement by most of our respondents. It also appeared that more resources were not on the minds of these respondents when thinking about raising student achievement in general, but became so when asked specifically about the role of the Ministry of Education. Being more positive about Pasifika achievement was also important for some of our respondents.
A consistent theme in the responses to the question of what the Ministry needed to do was for the Ministry to be a more active partner with schools. Overall, the findings suggested that more discussion needs to occur to establish greater understanding of what each group believes to be important in the schooling of Pasifika students.
In part two of the study, the analysis of the rankings given to the importance of various school tasks revealed a lack of agreement amongst participant groups as to which educational tasks were held to be important in the education of Pasifika students. Tasks rated highly by two of the groups were not so highly rated by the third. For example, "Preparing children for the next level of schooling" received the highest ranking by teachers and the second highest by Ministry respondents but was ranked sixth by management. Items ranked relatively highly by all groups included "Teach reading, writing and maths in English" and "Make children feel good about themselves".
Although there were differences in the details of the responses between the different groups, there was an overall consistency between researchers, Pasifika Education Plan (Ministry of Education, 2001) and Pasifika teachers. Reports identified from the research literature on promoting Pasifika children's identity and reducing the mismatch between the cultural capital students bring to a Palagi-dominated system and what is expected of them were evident in the Ministry's plan and our interviewees' responses. The underlying assumption appears to be that improving achievement is likely to occur through teachers giving greater recognition to Pasifika children's culture, improving the interface and understandings between home and school, increasing bilingual provision and resources and giving more positive publicity to the achievement of those who have succeeded.
The concern we have raised in concluding this chapter is that most of these strategies have been tried in some form, but few have been evaluated systematically in terms of their success in raising student achievement.
Where to find out more
For more information about this publication please email the: Research Mailbox