Executive Summary

The Early Childhood Education (ECE) centre-based Parent Support and Development (PSD) programme was one of a number of Government initiatives aimed at improving outcomes for vulnerable children through early intervention. The programme differed from other initiatives in that it offered parent support and development through a universal service that was accessible to all parents, while simultaneously attempting to target vulnerable parents.

A total of 18 pilot ECE sites were selected and funded to provide parent support and development for a three-year period starting from 2006.  Pilot sites were selected, in part, on the basis of the high concentration of vulnerable families living in the surrounding area. The sites offered a wide range of activities, including educational, social support and outreach activities.

The programme’s policymakers identified four specific objectives for the programme:

• to improve effective parenting by vulnerable parents; building on their skills and knowledge
• to increase participation and engagement in ECE by vulnerable children and their families
• to improve the consistency between what children learn at home and in the ECE environment
• to lead vulnerable parents to be better connected to broader social supports and informal networks.

The programme’s ultimate objective, that is, improvement of family and child well-being, was dependent on the achievement of these objectives.

Methodology

Evaluation purpose

The planning phase of the evaluation identified two overarching purposes of the evaluation:

• to identify areas of improvement to ECE centre-based PSD – this led to the design and implementation of a process evaluation phase intended to provide the Ministry of Education (the Ministry) with real time information about how well pilot sites were delivering against programme objectives
• to determine the value of ECE centre-based PSD – this led to the design and implementation of an outcomes¹ evaluation phase intended to provide the Ministry with robust information about the extent to which pilot sites had contributed to achieving the intended outcomes of the programme.

Evaluation approach

The evaluation was conducted over a three-year period in four phases. Key activities undertaken in each phase is described below:

• Phase one involved review of relevant project documents, interviews with policy and operational policy staff and two visits to pilot sites – this led to the development of an Evaluation Plan
• Phase two involved two rounds of telephone interviews, analysis of monitoring data for the period 2007/08 and case-study research with eight pilot sites
• Phase three involved case-study research with five pilot sites and analysis of monitoring data for the period 2008/09
• Phase four involved synthesising the data and findings and developing this final report.

A more detailed description of the methodology can be found in Part 2 of this report.

Key findings

Programme implementation

The Ministry consciously adopted a flexible approach to programme implementation and allowed pilot sites to deliver services to meet the needs of their communities. However, not all pilot sites were well placed to take advantage of this opportunity and some sites needed considerable support and input to further develop their ideas. The Ministry responded by guiding and working with pilot sites, particularly during the application phase. While all pilot sites were required to base their application and subsequent programme design on sound needs analysis and community consultation, the reality was that most pilot sites found it difficult to do this due to the limited amount of time and resources available. Instead, they relied on prior knowledge and experience to underpin programme design and implementation.

The evaluation revealed significant variability in programme implementation across pilot sites and some apparent departures from the original intent as outlined in sites’ proposals. This was in part due to the iterative approach to implementation adopted by the Ministry and in part due to the fact that, in some sites, the person responsible for writing the proposal was different to the person handed the responsibility for implementation. This suggests there is a strong need for consistent management guidance at a service level, to ensure better alignment between programme intent and programme reality.

The variability in implementation revolved around three key focus areas of the programme and attempts were made by the Ministry to help sites reshape and refocus their programmes during the three-year period. The areas addressed by the Ministry covered:

• Definition of the target group – the findings from Phase 2 of the evaluation identified pilot sites had different views about who the target group for the programme was. Some sites defined the target group as all parents, with an enhanced focus on vulnerable parents, whereas other sites defined the target group as parents of children between 0-5 years old. Pilot sites’ implementation of PSD was influenced by these definitions.
• Emerging operating models – the operating models developed to deliver the programme were influenced by pilot sites’ understanding of the intent and purpose of the programme. Some sites viewed the programme as an opportunity to offer value-added services to the parents of enrolled children (described as ‘closed’ sites) whereas others viewed the programme as a service available to all parents in their community (described as ‘open’ sites).
• Type of PSD activities offered by pilot sites – the evaluation findings indicated that there were two approaches taken by sites – those that took a structured, intentional approach to delivering PSD to their community, while others took a more developmental approach (whereby sites focused primarily on creating increased opportunities for networking and support with the underlying hope that it would translate into parents’ acquiring knowledge and skills about parenting).

Parents’ participation in PSD

A total of 2,246 parents participated in PSD over the two-year monitoring period.  The monitoring data showed that a wide cross section of parents participated in the programme:

• the majority of participants were female (88%)
• over half of all participants identified as European, and over one-third as Māori
• over a third were aged 25 years or younger (this figure includes 13% who were under 20 years old)
• more than one-third had no qualifications, and almost a third held a tertiary qualification
• almost 60% were not in paid employment
• two-thirds had only one child aged under five years.

Although the programme was intended to focus on vulnerable parents, the overall proportion of participating parents identified as vulnerable by PSD workers was relatively low (17%).  This is probably because all pilot sites provided services to all parents regardless of their vulnerability, with many sites philosophically of the view that seeking out vulnerable parents would stigmatise them.

The data also showed more than one-third of parents had an ongoing relationship with PSD – attending 11 times or more over the two-year period.  Over 80% of parents participated in social support activities, which were expected to result in parents’ confidence and social support networks increasing.  Activities that were primarily expected to increase parents’ knowledge about parenting were attended by almost half of all parents, and a third participated in outreach² activities. Most parents (57%) took part in more than one type of activity, and most activities were undertaken in a group setting.  Approximately 12% of parents engaged in one-on-one interactions with a PSD worker.

Key achievements and challenges

The overarching objective of the programme was to improve family and child well-being, with a special emphasis on vulnerable children. The evaluation findings highlighted significant achievements of the programme – for parents and families, for the pilot sites, and for the wider community. There were also some significant challenges in achieving the programme’s objectives, particularly in terms of reaching the target group – vulnerable parents. The key achievements and challenges are briefly summarised below.

For parents and children

Interviews with parents provided strong evidence that positive outcomes were achieved for participating parents. Parents learnt about a wide range of things from specific topics relating to parenting (such as dealing with challenging behaviours, toilet training, seat belt safety, sleeping) to general life skills (such as financial literacy, dealing with domestic violence). Parents reported that participating in these activities made an important contribution to improving their overall quality of family life. In particular, parents reported increased confidence in their parenting abilities, reduced social isolation, enhanced sense of belonging to the community and increased access to other services which resulted in them participating more meaningfully in their community. Children benefited from the programme through parents improving their parenting style and, in some instances, through increased access to ECE (which gave them access to quality learning opportunities and the chance to form relationships with a wider group of peers). However increased participation in ECE was limited by full rolls in many pilot sites.

For pilot sites

Pilot sites significantly changed how they interacted with parents as a result of their participation in this programme. Sites reported that the programme had encouraged them to develop new networks, furthered their reach into the community and expanded their focus beyond children. A key factor that contributed to sites’ success in this regard was the level of support they received from their umbrella organisation.³

A number of key achievements were identified for pilot sites:

• Improved service capacity – the additional funding provided by the programme allowed sites to expand, develop and improve what they previously offered to parents. The funding was used in a variety of ways, from recruiting new staff and purchasing equipment, to creating a dedicated space within the ECE centre (thus allowing parents to meet on a regular basis and allowing other services to visit and offer their services in an accessible venue).
• Heightened awareness of their aims and goals – most sites acknowledged that participating in the programme had given them the opportunity to expand their focus to include parents and the wider whānau.  It also allowed them to become more purposeful in their efforts to meet the needs of their wider community.
• Increased capability and skill of site staff – staff involved in ECE centre-based PSD increased their skills and knowledge about a wider set of issues (such as facilitation skills, problem solving skills, networking skills including social, health, legal and education services).

The key challenges for pilot sites were in attracting and engaging vulnerable parents in PSD. Even though sites were located in areas where high numbers of vulnerable parents lived, this was not sufficient for vulnerable parents to attend the programme. Sites needed to make concerted efforts to engage vulnerable parents in the programme and they struggled with this and often felt they did not have the tools and strategies to reach vulnerable parents. Other challenges identified by the evaluation included: difficulties in finding and securing suitably skilled PSD workers for the duration of the pilot; lack of available ECE places, limiting the number of additional children who could participate in ECE; and a lack of clarity as to how sites could strengthen the connection between home and the ECE environment.

For the community

One of the expectations of the programme was that the pilot sites would evolve and grow into a community hub. This would allow other agencies and services to use the ECE centre, thereby increasing the community’s access to a range of services. Pilot sites that most successfully achieved this tended to be sites that focused strongly on parents’ needs and offered a separate designated parent space to promote parental engagement and contact.

Communities also benefited as the programme facilitated greater interagency collaboration including joint activities and sharing infrastructure or costs for services. The result was increased coordination leading to better service quality and accessibility for the community. The most common examples of increased coordination were when PSD workers invited a number of agencies to speak about their services at the site, or when the ECE pilot site hosted cross-agency meetings at the centre.

Perhaps the most significant benefit was the growing sense of ‘community’ and belongingness fostered amongst participants. In addition, the programme provided opportunities to learn and grow parents’ skills not merely in their role as parents, but also in their roles as members of the wider community. In one instance, a parent who participated in a PSD activity built her confidence to such a level that she went on to volunteer for another social service agency that promoted important educational messages to parents. In another instance a group of parents valued the support they received so much that they created a hub of their own, providing support to new members in their community. These initiatives suggest there is enduring value created for the wider community beyond parents who participated in the programme.

Lessons learnt from implementing ECE centre-based PSD

Identifying ‘what works’ in PSD is challenging, owing to the highly diverse and iterative approach taken by pilot sites. From an evaluation perspective, the diversity of sites’ approaches makes it difficult to compare across pilot sites using a common standard or criterion. In light of this, it is useful to reflect on the evaluation findings with a view to raising questions for consideration by policymakers in the future.

• Targeting provision of PSD through universal services provided a ‘soft’ entry point for parents to access services and an opportunity to interact at a level of intensity determined by them. However, most pilot sites experienced difficulties with specifically engaging vulnerable parents (there were some exceptions), with some sites not making any distinction between parents with different levels of vulnerability, while others focused solely on parents of enrolled children (regardless of whether they were vulnerable or not). This suggests that sites would have benefited from a sharper focus on targeting of their programmes and assistance to develop strategies to identify and engage vulnerable parents (such as access to risk-profiling tools, training).
• Flexibility in implementation (that is, inviting pilot sites to shape content and focus depending on their local context) was appropriate as it allowed pilot sites to design and deliver services and activities that were relevant to their communities. However, in a number of instances sites moved away from core objectives of the programme, by delivering activities that became an end in themselves rather than a means to achieving the intended objectives of the programme. Balancing flexibility with focus on overarching programme goals is challenging and the Ministry could have played a stronger role in this regard.
• The skills of parenting support and development workers have implications for pilot sites’ ability to attract and retain vulnerable parents in the programme. These skills may not always be available within ECE settings, with ECE teachers trained to work with children, not vulnerable parents. This points to the need for higher levels of training provided to ECE centres and their workers to effectively engage parents in PSD activities.
• The development of pilot sites as community hubs was an important outcome for this programme. It allowed parents to access a number of services in familiar, local settings and built stronger relationships with teachers (as there were increased opportunities to interact with teachers in non-learning situations). Sites that had a designated parent space were able to achieve this more successfully as its use was unconstrained and parents could drop by any time they chose. This has implications for sustained delivery of PSD through universal services as not all ECE centres have access to additional space.

Footnotes

1. A key question for the outcome evaluation was to explore the added value to the programme, as perceived by vulnerable parents and providers.
2. Outreach activities are defined as referrals to and/or working collaboratively with other services, undertaking marketing activities, participation in community events and provision of off-sites support for parents, including home visits.
3. The term umbrella organisation refers to the organisation that has management and governance responsibility for the ECE and within which the ECE is often located (for example, a church or a social service organisation).