Resource teachers literacy: Operational and policy review Publications
This report has been prepared by the Ministry of education to conduct an evaluation of the Resource Teachers Literacy (RTLit Service. The RTLit Service is designed to provide short, intensive support to students with high literacy needs, and to identify students with persistent learning needs who will require ongoing specialist support. RTLts are specialist teachers whose role is to support and assist staff in schools to meet the needs of Years 1 to 8 students experiencing difficulties with literacy learning. This report synthesises the work across all stages of the evaluation to make evidence based conclusioins about the operation of the service and its effectiveness.
Author(s): Martin Jenkins & Associates Limited for the Ministry of Education.
Date Published: September 2014
The overall objective of the evaluation was to investigate the operation of the RTLit model, structures and outcomes through assessment of:
- the operation of the Cluster Management Committees
- the effectiveness of RTLits’ processes and practices
- students on the RTLit roll (including who they are and their outcomes)
- the relationship between the Service and the wider literacy support system
- how the Ministry supports the Service.
The three key evaluation inputs were:
- initial scoping: interviews with key stakeholders and review of key documents
- online surveys of RTLits (95% response rate) and Host Principals (94% response rate)
- case studies of six clusters, chosen to represent diverse characteristics (interviews with the Host Principal, the RTLit(s), Cluster Management Committee members, Service users, and other local stakeholders; data review; brief telephone interviews with non-using schools; interviews with regional Ministry staff).
The evaluation identified three overarching factors that significantly affect many aspects of the Service’s implementation.
A lack of clarity in the Service’s purpose and goals, including its target group
- There is wide variation and inconsistency in RTLits’ and Cluster Management Committees’ understanding of:
- which students the Service is intended for
- what the specific criteria for enrolment should be
- what adequate (or inadequate) progress looks like
- when a student should be successfully discontinued or referred elsewhere
Cluster-specific characteristics, especially geography and scale of need, shape every aspect of the operation of the Service at the local level – ie what support is offered, to whom and how.
- Geography acts as a constraint on the ability of an RTLit to serve a given area; rural areas face particular difficulties to efficient provision given the large distance and travel times involved and because of low population density.
- Variations in the scale of need represent a different type of challenge; clusters fall into two broad categories – low need clusters made up of rural and/or high decile schools, and high need clusters made up of urban and/or low decile schools; dense urban clusters with high needs report significant excess demand resulting in the rationing of service provision.
(See the section on ‘Service provision and outcomes are driven by key factors’ on page 25 of the report).
Resource allocation has not responded to changes in enrolments or patterns of demand across clusters.
- Since the establishment of the Service in 2001, the level of RTLit resource has been held constant (at 109 FTEs), and the allocation of RTLit resource across the 86 clusters has been static, despite significant changes in school rolls and a doubling of the number of students enrolled in the Service.
- Reviewing the current allocation against the Ministry’s original formula shows a number of clusters that appear to be significantly under-resourced relative to others. These under-resourced clusters tend to be located in dense urban areas with concentrations of low decile schools.
(See the section on ‘Distribution and allocation of RTLits’ on page 81 of the report).
Other important issues
The evaluation identified a series of significant, cascading effects arising from these three overarching factors.
Access to the Service is inconsistent and levels of unmet need are unknown.
- Demand for the Service currently outstrips the available resource. Rationing processes (including referral and acceptance processes and school practices) are devised and implemented locally, leading to inconsistent approaches and poor matching of available resources to those with the very highest level of need across the country as a whole. Some rationing practices are driven by practical considerations (eg rotating access to different schools in different years), rather than student need.
- Lack of systematic referral processes mean it is not possible to accurately quantify the level of unmet need, or the characteristics of those who are missing out on support. However, unmet need is likely to be highest in dense urban clusters with low decile schools.
(See the section on ‘Accessing the Service’ on page 31 of the report).
RTLit practice and processes are variable.
RTLits typically practice in isolation, and there are no formal mechanisms for sharing best practice amongst RTLits. Key practice issues identified by the evaluation were:
- the way RTLits assess students is not consistent, and there are indications that some may do this key task better than others
- there is no agreement amongst stakeholders (including RTLits) as to whether support should be aimed directly at students or indirectly through their teachers
- the reasons students are discontinued from the Service are inconsistent – students may be discontinued because they achieve a ‘successful’ outcome (though rarely does success mean reaching national standards), because of resource restrictions (ie to give somebody else a turn), or because they aren’t progressing
- RTLits (and other stakeholders including schools) have significant concerns about the quality and appropriateness of existing referral options
- (See sections on ‘The intervention: assessing need and RTLit practice’ on page 42 and ‘Practice outcomes: discontinuation and referrals to other services’ on page 58).
It is not clear that the current RTLit workforce has the right level of capability, and specific training gaps were identified.
- RTLits (and some literacy stakeholders) identified a need for better initial practical training and ongoing support to work with ‘well below’ students.
- The capability and willingness of RTLits to work successfully with teachers (using an indirect model) is a concern for RTLits and schools.
(See the section on ‘RTLit Workforce’ on page 88 of the report).
It is challenging to assess the overall effectiveness of the Service because intended outcomes for students are not clearly defined.
- Students enrolled in the Service are thought to make better literacy progress than they would have without RTLit support.
- Whether the amount of literacy progress is optimal, or whether the support is being provided to the right students is unclear. The lack of clear Service goals means there is no clear basis to judge the value and effectiveness of the Service.
(See the section on ‘Effectiveness and efficiency of the Service’ on page 62 of the report).
The cluster management model is not operating effectively or efficiently.
- There are only a small number of RTLits in each cluster (most clusters have one RTLit, a small number have two or three) and there is inconsistency and wide variation in the way the clusters operate.
- There are concerns about the lack of guidance for Cluster Management Committees. Individual cluster operation is primarily driven by RTLits as Cluster Management Committees do not provide strong governance, direction or guidance to RTLits.
- Cluster Management Committees and Principals responsible for clusters do not feel adequately supported to manage RTLits.
(See the section on ‘Operational management of the Service’ on page 71 of the report).
The Ministry’s current influence on the operation and effectiveness of the Service is limited.
- The Ministry has not provided strong support or leadership to the Service. This lack of leadership in combination with low RTLit buy-in and understanding of the Ministry’s Professional Practice Manual (the PPM), and the Service’s devolved and itinerant implementation model means that the Service lacks strong direction. An absence of ongoing professional learning and development for RTLits from the Ministry is seen as a problematic gap.
(See the section on ‘Need for Ministry support and overall leadership’ on page 76 of the report).
The fit of the RTLit Service within the wider system of literacy infrastructure is unclear and problematic.
- The devolved nature of the Service and other key services (eg Reading Recovery and the RTLB Service) means that the way the Service fits within the wider infrastructure varies from cluster to cluster. It is variously perceived as: following classroom teaching and intensive support if progress is not being made; intensive support only for those with high learning needs; or specialist support, only available as a last resort for those with the very highest learning needs.
(See the section on ‘Relationships to other services and literacy interventions’ on page 93 of the report).
Implications for future operation of the Service
The evaluation findings have a number of potential implications – specifically for the operation and implementation of the Service, and more broadly for the wider literacy system.
The findings suggest that any changes to the Service are unlikely to be effective unless key policy issues are addressed first. These include:
- clarification of the purpose and goals of the Service, and its target group – including the need to review expectations for student outcomes to ensure they are realistic and aligned with the target group for the Service
- investigation of whether service provision could be improved by taking a segmented approach to delivery: the appropriate operational model for delivery in a dense urban cluster is likely to be different to the appropriate model for delivery in a rural cluster
- review of the level and the allocation of resource for the Service across the country and relative to the target group
- clarification of the fit of the Service with other literacy services.
Following from this, the evaluation suggests:
- a need to strengthen the referral process to ensure that the available resource is used to support an agreed target group
- a Ministry-led workforce strategy is needed with a clear focus on ensuring ongoing workforce capability, providing ongoing support and ensuring future workforce needs are met
- the appropriateness of the current cluster model needs to be reviewed
- if the model is retained, there needs to be better support for committees and Host Principals to perform their roles (including performance management) and better operational guidance and tools (eg referral forms)
- if the model is retained, a mechanism for sharing best practice between clusters should also be considered
- improved monitoring of student outcomes is needed
- a need for clear and visible leadership for the Service from the Ministry, or some other leadership function
- linkages and referral processes between the Service, schools and other literacy services may need to be strengthened.
Education Data Requests
If you have any questions about education data please contact us:
Email: Requests EDK
Phone: +64 4 463 8065