Alert Program® in schools – Phase 2 evaluation Publications
Publication Details
This report synthesises emerging findings from the evaluation of the Alert Program® pilot’s second phase (Phase 2), which focused on selected learners with additional needs, their teachers, family and whānau.
Author(s): Dr Sira Engelbertz
Date Published: April 2023
Summary
The Alert Program® was developed in the 1990s in the United States to foster self-regulation through teaching children how to change how alert they feel and teaching adults how to support learning and positive behaviours. The Alert Program® pilot seeks to test a model for integrating social-emotional learning into the curriculum while adapting the programme to the New Zealand school context using the Ministry of Education’s tiered learning support model.
The present report synthesises emerging findings from the evaluation of the pilot’s second phase (Phase 2), which focused on selected learners with additional needs, their teachers, family and whānau. Data was captured using a mixed-method approach, and included measures of:
- A questionnaire examining child-behaviour (Strengths and Difficulties Questionnaire - SDQ) and checklists measuring implementation fidelity (Alert Program® implementation) and learners' skills (Alert Skill Development) completed at two time-points – before and after implementation of Phase 2 – by teacher (n=16) and family and whānau (n=12) to detect change.
- Perceptions of the programme and its implementation by a sample of teachers (n=8) and family and whānau (n=4) before and after implementation of Phase 2.
- Perceptions of the programme implementation by stakeholders (n=4), including school leaders and programme facilitators.
- Feedback by teachers (n=16) after implementation of Phase 2.
The Phase 2 evaluation is to be read in conjunction to the first phase evaluation of the Alert Program® pilot (Phase 1).
Phase 2 and the evaluation experienced a series of delays due largely to longer-than-anticipated approval process and COVID-19 lockdowns. The latter impacted the data collection and potentially the evaluation results, notably the SDQ scores and Alert Skill Development checklist. Although the evaluation had initially anticipated a bigger sample, pilot schools were able to identify a total of 28 learners between them to be included in Phase 2 while also considering feasibility criteria. However, the small size of the Phase 2 cohort presents limitations for the conclusiveness of the evaluation evidence (i.e. there are limitations in finding statistically significant differences or effects). Further, the focus of the programme was expanded to individualised/tailored (Tier 3) support in response to family and whānau needs and preferences. This put pressure on the time required to engage family and whānau and limited the capacity allocated for delivering Phase 2.
Despite the limitations due to the sample size mentioned above, the evidence were informative and adequate to answer the key evaluation questions. Overall, the quality of evidence used was deemed appropriate for the learning purpose of this evaluation.
Rubrics were used to guide judgements about the programme’s effectiveness regarding participant-level and child-level outcomes. Rubrics had been developed during the planning phase for the evaluation and agreed by the Ministries and pilot schools.
Navigation
Contact Us
Education Data Requests
If you have any questions about education data please contact us:
Email: Requests Data and Insights
Phone: +64 4 463 8065