The Alert Program® Pilot Evaluations Publications
Publication Details
The Alert Program® was developed in the 1990s in the United States by two occupational therapists, Williams & Shellenberger. The Alert Program® fosters self-regulation through teaching children how to change how alert they feel and teaching adults how to support learning and positive behaviours.
The Alert Program® pilot tested a model for integrating social-emotional learning into the curriculum while adapting the programme to the New Zealand school context using the Ministry of Education’s layered learning support model.
The Alert program® pilot in two schools had two phases (Phase 1 and Phase 2).
Author(s): Standard of Proof, Dr Sira Engelbertz
Date Published: December 2024 (Phase 1) and April 2023 (Phase 2)
Phase 1 Evaluation - Summary
The Alert Program® promotes a shared language using an engine analogy, and provides tools and strategies to strengthen self-regulation skills. The Ministries of Health and Education jointly piloted the first phase of The Alert Program® at two New Zealand primary schools from July to December 2019.
The first phase was embedded as a school-wide approach, providing school staff with training and support. As staff learn and embed the skills, it's expected that students will improve their ability to self-regulate and both teachers and students will improve their overall wellbeing.
The first phase of The Alert Program® resulted in significant positive effects for both school staff and children.
School staff developed a consistent understanding of key elements of the Alert Program® Although the co-design approach resulted in different implementation timelines, staff across both schools were either developing or embedding tools and strategies into their classroom routines by the end of the term.
Students similarly demonstrated significant and large behavioural shifts, applying the Alert Program® strategies more frequently in the classroom by the end of the term at one school. Although only over one term, there was also a small but significant reduction in parent-reported child difficulties and teacher¬-reported classroom disruptions.
Although no change was detected in teachers' self-reported confidence and satisfaction levels, notably at one school, these were already high at baseline.
Several factors appeared to influence the effectiveness of Phase 1 of The Alert Program® Because The Alert Program® and the schools shared similar values and principles, the staff willingly engaged in the programme. The engine analogy was also simple, which made it easy to adopt, and empowered both teachers and children with a shared language to communicate more effectively with one another.
By building an understanding of sensory principles, The Alert Program® supported a change in teachers' perceptions of classroom behaviours and how they responded to those behaviours.
Phase 2 Evaluation - Summary
This report synthesises emerging findings from the evaluation of the Alert Program® pilot’s second phase (Phase 2), which focused on selected learners with additional needs, their teachers, family and whānau. Data was captured using a mixed-method approach, and included measures of:
- A questionnaire examining child-behaviour (Strengths and Difficulties Questionnaire - SDQ) and checklists measuring implementation fidelity (Alert Program® implementation) and learners' skills (Alert Skill Development) completed at two time-points – before and after implementation of Phase 2 – by teacher (n=16) and family and whānau (n=12) to detect change.
- Perceptions of the programme and its implementation by a sample of teachers (n=8) and family and whānau (n=4) before and after implementation of Phase 2.
- Perceptions of the programme implementation by stakeholders (n=4), including school leaders and programme facilitators.
- Feedback by teachers (n=16) after implementation of Phase 2.
The Phase 2 evaluation is to be read in conjunction to the first phase evaluation of the Alert Program® pilot (Phase 1).
Phase 2 and the evaluation experienced a series of delays due largely to longer-than-anticipated approval process and COVID-19 lockdowns. The latter impacted the data collection and potentially the evaluation results, notably the SDQ scores and Alert Skill Development checklist. Although the evaluation had initially anticipated a bigger sample, pilot schools were able to identify a total of 28 learners between them to be included in Phase 2 while also considering feasibility criteria. However, the small size of the Phase 2 cohort presents limitations for the conclusiveness of the evaluation evidence (i.e. there are limitations in finding statistically significant differences or effects). Further, the focus of the programme was expanded to individualised/tailored (Tier 3) support in response to family and whānau needs and preferences. This put pressure on the time required to engage family and whānau and limited the capacity allocated for delivering Phase 2.
Despite the limitations due to the sample size mentioned above, the evidence were informative and adequate to answer the key evaluation questions. Overall, the quality of evidence used was deemed appropriate for the learning purpose of this evaluation.
Rubrics were used to guide judgements about the programme’s effectiveness regarding participant-level and child-level outcomes. Rubrics had been developed during the planning phase for the evaluation and agreed by the Ministries and pilot schools.
Navigation
Contact us
Education data requests
If you have any questions about education data please email us at Requests Data and Insights