Summary

The Secondary-Tertiary Alignment Resource (STAR) has been available to secondary schools since 1996, enabling them to purchase non-conventional tertiary level courses leading to credits on the National Qualifications Framework. Unlike some other forms of funding for schools, STAR is not targeted at any particular group of students (e.g., "at risk" students) but is instead available to all students at Year 11, 12, and 13 (for full courses) and Year 9 and 10 (for short taster courses). Secondary schools apply for STAR funding annually on an EFTS-basis. The funding allocated to the school is managed by a designated STAR coordinator.

STAR's main purpose is to assist senior secondary school students in finding suitable pathways into work or further study at secondary or tertiary level by enabling schools to:

• facilitate smooth transition and access from schooling to employment, including work-based learning; or tertiary type study or training;
• improve retention in senior secondary schooling.

STAR sits within the wider international context that has made the transition points in young people's lives, particularly from school to work and/or further study, a concern and major focus of policy development. In New Zealand, STAR is, in essence, the most widely-used school resource for addressing the issues of engagement and relevance in the senior secondary schooling years. This makes current debate concerning curriculum, qualifications, and school retention/exit particularly pertinent in any decisions regarding the future of STAR.

While the challenges associated with STAR's operation in schools, particularly for STAR coordinators who do not have the "shelter" of a transition department or any recognition of their position through management units or the respect of other staff, should not be minimised, they can be regarded as a sign that schools are having to engage with a number of newer realities about the environment in which they perform.

STAR provides a way for schools to acknowledge (and begin to deal with) the contestability of their school curriculum and pedagogy alongside those of universities, polytechnics, and other private tertiary establishments. In practice this opens up opportunities for school students to experience learning in different settings, trial their career aspirations, experience success in school, and gain credits towards qualifications which they can continue to pursue after leaving school.

Research Objectives and Design

STAR has never before been evaluated. The main research objectives of this project overall were three-fold:

• to provide sound information on the operation of STAR in schools;
• to gather the views of key stakeholders on how successfully STAR assists student transition to further education or to the workforce;
• to identify and collect data on any outcome measures that could assist in an evaluation of how STAR is meeting its objectives.

Meeting these objectives has entailed a three-stage approach to the project:

1. Interviews with STAR coordinators, school principals, and external provider representatives to determine the scope of issues concerning STAR and inform the design of stage two.
2. Questionnaires for STAR coordinators and school principals in all schools receiving STAR funding, and for a sample of external provider representatives.
3. Visits to schools "doing well" with their STAR funding, including an interview with the STAR coordinator and focus group interviews with students participating in STAR courses.

Methodological Concerns

In 2001, $4 million was diverted from the STAR funding pool to the Gateway programme's 2001 and 2002 pilot years. This reduction of the STAR funding pool available to schools, together with an evaluation taking place so many years after STAR was established, led to a perception among many STAR coordinators, principals, and tertiary providers that STAR funding would continue to be reduced or even cut altogether. The entire project has been foreshadowed by that perception despite assurances to the contrary from the Ministry of Education.

At first the NZCER team regarded this state of affairs as an unfortunate challenge methodologically. How could we get around the potential for STAR coordinators to misconstrue NZCER's role-as being involved with a Ministry of Education/Tertiary Education Commission scheme to possibly cut STAR funding? Would schools, providers, and students still talk to us openly about how they operated or participated in STAR in their institutions? The political implications and ethical considerations of research have always been important and sometimes prove thorny. However, we soon realised that what teachers and principals, in particular, were worried about was very much part of what needed to be said about the operation of STAR in schools in this evaluation: that STAR has become integral to secondary schools' functioning, curriculum, and arrangements for meeting student needs. STAR is not an add-on. It is not used as extra funding for a small select group of students within a school. It is not somehow additional to what schools do. It is, in fact, a vital feature of what schools do.

The high regard in which STAR funding is held by schools and coordinators was reflected in the response to the evaluation throughout the course of the research. There was an overwhelming response to the mailout survey (83 percent response rate from coordinators). Moreover, a number of coordinators included personal notes and business cards with their returned surveys volunteering any further participation as necessary or a willingness to be interviewed if need be. Several other people, including tertiary-level provider representatives and STAR/transition coordinators contacted NZCER on their own initiative in order to offer their perspectives on STAR.

Main Findings

Interpretations of Purpose

STAR has two means by which the goal of assisting students to find suitable pathways may be achieved. Both means - facilitation of transition from school to employment or further study and retention at school - may be in tension with one another at times. This tension contributes to some of the STAR planning and operational challenges for schools. It also means that a broad understanding of "transition" and how it works is necessary because STAR may be used to facilitate transition through retention, allowing students a longer or better-directed transition period.

Most STAR coordinators reported understanding STAR's purpose as being related to the provision of links to tertiary courses and tasters, and provision of courses that the school would not otherwise provide. Some schools focus on the facilitation of transition aspect of STAR by having STAR coordinators match individual students to STAR courses which lead in to specific employment or careers. Often in these schools, STAR is run as a separate, alternative programme to a conventional school programme. In other schools, the focus is less structured and more related to the retention aspect. STAR involves guiding students into courses which are not necessarily part of a cohesive approach to beginning a career. There is also considerable utilisation of short taster courses. However, most schools attempt to meet both of STAR's aims and, regardless of any focus on one over the other in the school, have a range of STAR courses to meet different student needs in a number of different ways.

Tertiary providers of STAR courses reported perceiving STAR's purpose as being one of bridging secondary and tertiary education through the provision of non-conventional subjects. They therefore tended to focus on the facilitation of transition aspect of STAR. This fits neatly with providers' endeavours to recruit students for their institutions. Provider descriptions of the most successful features of STAR for them were characterised by references to their own needs (recruitment and satisfaction of customers) and how these are bound up with the needs of students (to study/work in an area of interest). These differing perceptions of STAR's role and purpose suggest that more thought needs to be given to defining student needs before any real conclusions can be made about whether, or how, STAR meets those needs.

STAR Courses

Nearly all schools reported offering industry-related courses to students and over half also offer short taster courses to both junior and senior students. Almost half of the STAR courses running were described as tasters or general skills courses. More than half of the schools reported that they offer courses specifically to meet the academic needs of students.

More than three-quarters of schools timetabled at least some of their STAR courses within their usual school timetable of short periods (e.g., 45-60 minutes). However, almost two-thirds had timetabled STAR in other ways that did not necessarily correspond with the school timetable (e.g., block courses running over several days at a time).

STAR timetabling issues in schools were reported as causing a great deal of friction. Through their efforts to organise STAR courses within the school, the STAR coordinator frequently found themselves isolated and positioned as an impediment to the smooth running of the school. Students frequently faced difficulties getting exemptions from their conventional teachers in order to attend STAR courses or in catching up on conventional classes missed because of attending STAR courses. There were also difficulties for some schools and external providers in matching school and tertiary provider timetables and terms. This posed attendance and student engagement problems for some schools.

STAR Course Delivery

The delivery decisions and practices for STAR courses highlighted a tendency for school needs to dominate and constrain student needs, even though addressing student needs is the preferred aim and focus of schools.

Most STAR coordinators reported using external providers to deliver tasters and senior level STAR courses. There was a fairly even spread of schools in terms of the extent to which they used external providers to deliver senior courses. Just over a third used external providers to deliver more than half of their STAR courses. However, almost a quarter used external providers less than a fifth of the time.

The reasons given by coordinators for the extent of external provider use centred upon reasons for using external providers as much as the school did. Their responses illustrated the pressure schools face to stretch their STAR funding as far as possible. For many schools, this means trying to deliver STAR courses internally and save on the extra costs associated with purchasing courses and delivery from external providers. This is of course extremely frustrating to providers who understand this to be a downgrading of their role and their alignment focus in Secondary-Tertiary Alignment.

More than half of the STAR coordinators reported changing the extent to which they used external providers. Some cited the introduction of new courses, suggesting an increase in external provider use. However, just as many gave reasons related to decreasing the use of external providers, suggesting a trend towards increasing internal delivery of STAR courses.

The competing course delivery demands faced by schools were also highlighted by STAR coordinators perceptions of the benefits and difficulties associated with internal and external delivery of courses. The benefits cited for external delivery emphasised the knowledge and resources held by external providers, over and above what schools could offer, particularly in the area of "real world" experience.

The benefits cited for internal delivery focused on ways in which internal delivery could enhance the school or avoid the pitfalls of external delivery (e.g., transport issues). While the pitfalls did not focus on saving money per se, they did signal the predominance of school needs. However, the responses of the more than half of coordinators who reported experiencing difficulties with external providers showed a concern with providers not meeting student needs.

Two-thirds of the difficulties with external providers cited by coordinators were reported to have been resolved, usually by using different providers or working closely with the current provider. In contrast, only about half of coordinators reported being able to solve difficulties with internal course delivery. Nearly three-quarters of the difficulties were related to a lack of staffing resources or facilities within the school. There were comparatively few responses about the specifics of how these difficulties were resolved, suggesting that they were somewhat more intractable than the difficulties faced with external providers.

Student Participation and Needs

Across all schools, Year 11-13 students had high participation rates in STAR-funded courses, indicating that the funding is spread across many students in each school rather than being concentrated amongst just a few students. More than half of the coordinators reported that at least 40 percent of their school's senior students participated in STAR courses. Just over a quarter of schools had over 60 percent of their senior students involved.

Nearly all schools had a system for assessing student needs and these systems varied widely. Three-quarters of schools used face-to-face contact between the STAR coordinator and the students as at least one of their methods for determining student needs. More than half of schools also carried out surveys of student interests. There was also generally some interaction between students and deans although most student STAR needs assessment activities were generally instigated by the STAR coordinator

The majority of STAR coordinators indicated a perception of STAR being for students with an interest in a particular industry or for students who would otherwise leave school without any qualifications. Like the coordinators, external providers saw STAR as being for students with an interest in a particular industry. However, just as many also thought STAR was for all senior secondary students, which is unsurprising given providers' focus on STAR as a recruiting ground. It is likely that providers are focused on students who are the most likely to succeed in STAR courses and go on to further study (with that provider). Schools, on the other hand, are likely to be more interested in providing an opportunity for a successful experience at school for students first, and concerned with further study, second.

Nearly all coordinators reported using STAR to meet vocational or work experience needs. Two-thirds reported using it to keep students at school. However, over a third of STAR coordinators reported using STAR to meet academic student needs. Nearly two-thirds also reported that STAR met some students' needs for basic life skills. This life skills possibility has been compromised by the redefinition of some alternative subjects and core generic skills as "conventional", making these ineligible for STAR funding.

A number of case study schools required students (or their parents) to pay bonds before they were allowed entry to a STAR course. This may contravene STAR regulations which stipulate that students may not be charged fees for STAR. Considering that many bonds are obviously not intended for recovery of STAR costs, bond schemes may be symptomatic of the lack of status of STAR courses and coordinators generally, and may represent an attempt to raise that status. It may also signal anxiety about managing relationships with external providers and minimising the fallout from students' poor behaviour, or from students withdrawing from, or failing to complete, courses.

Student Perceptions of STAR

Students found out about STAR in a range of ways, including subject choice booklets, the STAR coordinator, school notices. Most students found it difficult to separate how they got their initial information about STAR from making the decision to do STAR courses. This is likely to be because students relied on STAR coordinators to help them make sense of information in ways that would be meaningful to them for career or life plans.

Students' decisions to do STAR courses were heavily influenced by enjoyment of, or interest in, the subject. This is consistent with findings from other research on student subject choice motivation. For many students, interest in STAR was driven by an interest in doing practical, hands-on work, and in looking for new and different challenges in their school work. Many students cited an interest in trying many STAR courses in order to gain experience of careers, tertiary study environments, and workplaces.

Students were generally very excited and enthusiastic about STAR courses and programmes. They particularly enjoyed being "treated as adults" and having an opportunity to build new kinds of relationships - with tertiary level tutors, with adult classmates, and with students from other schools. Many also reported pleasure in the independence and freedom of STAR. They enjoyed being away from the school environment, taking responsibility for choosing new courses, discovering the tertiary environment, and organising their own transport.

Many students also enjoyed the fun and practical work involved in their courses. Some were disappointed in courses that included a high proportion of theoretical work. Others found the practical work in some courses boring and repetitive. They wanted a greater challenge from their course. While course design may have been the issue - it can sometimes be difficult for external providers to "pitch" the course to the right level - students tended to see the lack of challenge as related to the career associated with the course and used STAR as a way to try out and reject certain career options.

The Role of the STAR Coordinator

STAR coordinators work is multi-faceted and varied. They are more than administrators; coordinators have a role in school planning, and spend time building relationships with students, other staff, and external providers. In a similar vein to comments made by Gateway coordinators, STAR coordinators believed the job required passion and a genuine commitment to students. However, a lack of status within the school made the job difficult and exacerbated existing difficulties with timetabling and STAR programme organisation.

STAR coordinators generally spend 5 hours or less working on STAR each week, although a third reported spending between 5 and 9 hours a week. Many coordinators did the job as an add-on to their role as careers adviser or transition teacher. Over two-thirds of coordinators had no management units. Only 8 percent of coordinators believed that other school staff saw their role as important. More than half believed staff saw them as just "fairly important" and a quarter believed staff saw them as "not very important". A number of coordinators commented that it was only the staff who were directly involved with STAR in some way who saw the importance of the coordinator's role. For other staff, the coordinator perceived they were seen as the person who disrupted the school timetable and cut across the other teachers' professional pride.

Professional development was seen by coordinators as one step towards improving their status and getting other staff to think about the importance of STAR for students. Coordinators also wanted more information and assistance from the Ministry of Education, particularly in the area of gathering information for students and administration. Many felt that professional development in the form of STAR cluster meetings would provide a way to easily share information and experiences.

STAR Funding and Use

STAR funding is paid as part of the school Operations Grant, leaving STAR coordinators reliant on their school principals redirecting the allocated funding to them. Nearly three-quarters of STAR coordinators reported that they did receive all the funding from their principals. However, over a quarter do not receive all the funding. Of those coordinators, most reported that their school budgets were decided in advance. This suggests either that the school considers STAR funding to be part of their Operations Grant to be spent as deemed appropriate by the principal, or that coordinators do not actually know how much funding they are entitled to and therefore whether or not they have received it all. Many coordinators throughout this research project suggested somehow ring-fencing the STAR funding.

Eighty-eight percent of coordinators reported that STAR funding was insufficient, either because they had to redirect EFTS-generated funding from courses which more than covered their costs to courses which did not, or because they had to redirect money from the Operations Grant to cover shortfalls. Just over two-thirds of STAR coordinators reported that per-course funding did not cover between 1 and 4 courses in their school. Almost three-quarters of coordinators reported resolving this by topping up their more expensive STAR courses with their less expensive ones. Just over a quarter reported using the Operations Grant as a top-up. A few reported using other top-up means, usually asking students to pay.

More than half of the coordinators thought that the current EFTS system for funding STAR was the most appropriate one. Other suggestions for funding systems, from just under half of the coordinators, included developing a closely-monitored system similar to one used by ESOL funding, tagging a portion of STAR funding for coordinators, basing funding of differentiated course costs, and basing funding on a differential system for decile, size, and rurally. Rural schools were interested in some form of extra transport funding to reduce the extra costs they faced in getting students to courses.

The lack of any substantial audit procedures for STAR is problematic, especially when considered alongside STAR's relative lack of status and the inclusion of STAR funding within the school's Operations Grant. In some schools, there is considerable pressure on coordinators to spend STAR funding on non-STAR activities or resources. There is also pressure in some schools to teach courses internally in order to use the EFTS while spending less money (because internal delivery is cheaper than external delivery). Moreover, there are unresolved issues around the use of EFTS and at what point these actually count as having been used - when the course, or student places on it, are purchased from an external provider, or when students have completed the course? To add to the confusion, schools interpret STAR-related expenses differently and some pass on costs to students that other schools cover with their funding.

Key Recommendations

• Consider "ring-fencing" STAR funding within the Operations Grant.
• Raise principals' awareness of the importance of the STAR coordinator's role in the school.
• Share the Ministry of Education's vision for senior secondary schooling with STAR coordinators; work collaboratively with them through an advisory group of coordinators and external providers.
• Use existing clusters of schools, or create new ones where needed, to share information.
• The relatively higher costs borne by isolated, rural (and some small) schools is an equity issue. There may need to be a transport and accommodation grant made available.
• Clarify the situation on core generic skills for schools. Consider how to avoid disadvantaging small and/or low decile schools through a careful definition and regulation of core generics.
• Taster courses serve an important purpose. That students may not necessarily go on to careers or further study directly related to what they "tasted" through STAR courses is not a failure of STAR. Tasters could even be made more widely available to senior students.
• Keep schools' actual and application data in a Ministry of Education STAR resourcing database.
• Clarify the usage of EFTS. Then audit and monitor usage of STAR funding regularly.
• Give schools clear information on the use of student bonds. Do these automatically contravene the STAR regulations which stipulate that fees may not be charged for students to attend STAR courses?