Summary

This research was undertaken in order to evaluate the Literacy Leadership initiative, one of a number of initiatives that built on recommendations from the Literacy Taskforce. With the ultimate aim of raising student achievement in literacy, principals and literacy leaders in schools were up-skilled in literacy matters through workshops and site visits. The model involved establishing multi-level professional communities. A community of expert literacy facilitators was established whose task was to run workshops, then work with school leaders to establish literacy goals for individual schools; to mentor leaders in carrying out classroom initiatives in literacy, and to promote the development of self-supporting learning communities within schools. It was anticipated that schools would draw on further specific expertise to assist them in meeting their literacy goals.

Data for the evaluation were obtained from structured interviews and from documentary sources, namely the resource materials supplied by Learning Media and achievement data from their classroom initiative supplied by individual schools. Facilitators at interview nominated schools that they perceived to have been differentially successful in terms of the Literacy Leadership initiative. Personnel interviewed from the purposively selected sample of 29 schools were the principal, literacy leader and up to two teachers who had been involved in the classroom initiative. These schools were asked to provide achievement data related to their classroom initiative and also standardised testing data.

This evaluation attempted, at the Ministry's request, to find evidence of improved student achievement as a result of the Literacy Leadership initiative. Two thirds of the schools were able to provide the evaluators with some student achievement data regarding their initiative. The data that were provided, however, did not enable an independent judgment about improved student achievement to be made. This does not mean there were no improvements; it is just that we were unable independently to verify them. Principals and literacy leaders reported that they believed their class-based projects were successful because they focused on teacher practice and beliefs. Further, many believed the projects had worked because they perceived that reading levels had improved or that instruction was more focused. Over half the teachers reported that they had learned something that resulted in changed teacher practices. Most others indicated that it had reinforced or affirmed what they already knew.

Some possible explanations for the limited availability of student achievement data that would have enabled us to verify independently the claimed success were investigated. Teacher feelings of self-efficacy and expectations of students were examined and dismissed; teachers did not report low expectations of their students nor did they perceive that they had little influence on outcomes in literacy. Similarly, other explanations for lack of evidence, like the existence of a high degree of satisfaction with student achievement reported by teachers that may have obviated the need for data, were considered. This notion of perception of success rendering the collection of evidence less likely is a possible explanation. More likely, is that schools lacked the capacity to engage in such data based review. They may have lacked knowledge and skills to the level required to obtain data useful for evidence-based decision-making in their class-based projects. This finding is based on the responses to the scenario that examined knowledge of principles of evidence-based decision-making.

The Literacy Leadership model was partly premised on sufficient capacity already existing in schools that would allow them to benefit from the mentoring of the facilitator and, partly, on the development of a literacy community of support and feedback systems for each school so that it could be largely independent in working to raise literacy achievement. The evaluation demonstrated that the assumption there is capacity to make evidence-based decisions about literacy initiatives may be unfounded. Similarly, the professional communities of support and feedback within the schools tended to focus more on the support aspects than providing critical feedback. The literacy community extended also to the facilitators and to Learning Media itself where an advisory board was set up for consultation purposes. The establishment of a community of professionals appeared to be more successful at the level of the facilitator than the schools.

The evaluation concluded that there was room for strengthening the model, for example, through testing assumptions about capacity and knowledge and providing professional development to ensure appropriate levels of such and matching the support given to leaders to analysed needs.