Enhancing and Igniting Talent Development Initiatives: Research to determine effectiveness

Publication Details

This report presents the findings from research involving five Talent Development Initiatives for improving outcomes for gifted and talented students or their teachers. The purpose of the research was to consider how well the objectives of each participating initiative had been achieved, how the initiative contributed to improved outcomes for gifted and talented learners or their teachers, and how planning to continue to meet the learners’ needs after 2008 had been considered.

Author(s): Tracy Riley and Roger Moltzen, Massey University. Report prepared for the Ministry of Education

Date Published: March 2010

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  To view the individual chapters please refer to the 'Sections' inset box.  For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.

Section 5: Enhance Evaluations

This section of the report consists of the case studies for the professional development provided by GiftNet (The Gifted Kids Programme) and the Gifted Education Centre (formerly the George Parkyn Centre). Each of these Enhance TDIs was evaluated using a case study approach, as described earlier in the report. The research questions for the Enhance Evaluations were:

  1. How were decisions around the programme design arrived at? Who was involved in the decision-making process? How has the process impacted upon the sustainability of the programme?
  2. What changes in the organisation's climate and philosophy have been required for the successful implementation of the professional development programme?
  3. How comprehensive are TDI initiated programme monitoring and evaluations of the professional development? How do the findings inform the professional development programme?
  4. How have resources and personnel impacted the professional development programme?
  5. What role has staff professional development played in achieving the programme goals?
  6. How well has the programme planning occurred in regard to sustainability of the professional development after the three-year funding ceases?
  7. What has the impact been of the professional development for its stakeholders?

For each evaluation, a description of the programme, including purposes, data collection methods, analysis, and results, is provided. A summary of each programme's evaluation results is framed against the research questions. A synthesis across results for the two Enhance programme evaluations shows some common themes, which lead to conclusions and recommendations for both research and practice.

GiftNet

Background

GiftNet is a professional development programme facilitated by the Gifted Kids Programme. The Gifted Kids Programme (GKP) is an out-of-school provider of a one-day-a-week programme for gifted and talented students, with a special focus on children from low-income families, and established in 2000. GKP currently has eight one day units catering for over 550 children from more than 130 contributing schools throughout the North Island (GKP website, 2009). The goals of the Gifted Children's Advancement Charitable Trust are:

  • To identify, assess and nurture gifted children, mainly from lower income communities;
  • To meet the emotional, cognitive and societal 'special needs' of gifted children and to create an environment where they can interact with those of like minds and abilities – thereby boosting their confidence;
  • To make the provision of opportunity for gifted children a priority;
  • To promote a culture where excellence is recognised and celebrated; and
  • To make accessible professional development for all GKP teachers and mainstream teachers.

GKP has been providing a professional development programme for teachers in contributing schools since 2003, when it was awarded a contract by the Ministry of Education as a Talent Development Initiative (TDI). The 2003-2005 professional development programme, Gifted Edge, consisted of a series of mini-conferences, held in Auckland, Wellington, and Rotorua, and complemented by scholarships for teachers to attend national and international conferences, parent workshops, and the development of a professional library. The programme of professional development focused on important understandings that underpin best practice for gifted students (e.g., definitions and characteristics, identification, policy design and implementation), using a broad brush approach (TDI Funding Pool website, 2009). The mini-conferences offered both keynote and concurrent sessions, facilitated by national leaders and experts in the field, as well as GKP staff.

As the contract was drawing to an end, 48 schools were registered participants in Gifted Edge (GiftNet Proposal, 2005). The internal evaluative data collected during the first round of TDI funding showed that the professional development was well-received and had "gone some way to improving gifted education in our contributing schools" (GiftNet Proposal, 2005, p. 6). This data, along with informal communications with contributing schools, also indicated that there was a need to shift to school-based professional development, including regular classroom practices.

In 2005, a proposal was developed for the delivery of GiftNet, a programme of professional development that would build upon and further develop the Gifted Edge programme. In addition to the annual mini-conferences, which would continue to provide opportunities for GKP and contributing school staff to learn from leaders in gifted education, the GiftNet programme would incorporate:

  1. Teacher Workshops : after school workshops held once a term and open to all teaching staff from contributing schools, with a focus upon improving classroom practice.
  2. Resource Kits : development and provision of "take-away" kits (comprised of articles, teaching tools, etc.) to complement workshop content.
  3. Guided Cluster Groups : ongoing professional development for school leaders who coordinated gifted and talented programmes with a school-based, needs-based focus and delivered via cluster meetings, the appointment of a mentor (a different mentor for each of the four clusters), and individual school support provided by the mentor and/or programme director.
  4. Website : development of a site for sharing information within and across clusters, as well as with the wider gifted education community.

The GiftNet programme was a multi-faceted, interrelated programme of professional development, targeting contributing schools in the Auckland, Whangarei, Rotorua, and Wellington regions.

The TDI funding provided by the Ministry of Education was allocated to professional costs and some professional fees with approximately a 50/50 split. The professional costs associated were mostly to cover the expense of travel, accommodation, venue hire, catering, resourcing for facilitators, resources for schools, website development, and resourcing such as photocopying for workshops etc. The allocation for professional fees covered the costs associated with paying for external national service providers in gifted education to present at mini-conferences and after-school workshops, some mentorship fees, and a small amount of administration support. The Gifted Children's Advancement Charitable Trust provided the salaries for the Coordinator and Facilitator from their own fund-raising.

While the entire GiftNet package must be considered in determining the overall effectiveness of the professional development, for the purposes of this research, only the guided cluster group element was evaluated, and this was limited to only one of four clusters. This decision was based on several factors: the clusters were aimed at school leaders, with a school-based, needs-based focus and involving multiple modes of delivery; GKP's internal evaluation processes for the mini-conferences and teacher workshops were established and comprehensive; and there were limited resources allocated for the research. Namely, the constraints of this evaluation related to the use of a case study approach, with no resources to support multiple visits to locations across the country, during which in-depth data from all stakeholder perspectives could be gathered. This also meant that evaluating all of the indicators related to the cluster groups was beyond the scope of this research: it was impossible to measure the achievement outcomes for GKP students in their areas of talent.

The next section of the report provides details of the purposes and structure of GiftNet clusters at the start of the three year period of funding.

GiftNet Cluster

In 2006, the clusters operated in the four main regions of the GKP programme: Wellington (6 schools), Rotorua (5 schools), Auckland (2 schools), and Whangarei (5 schools) (GiftNet Milestone Report 1, 2006). Each cluster school worked with a GiftNet facilitator and a mentor to determine the nature of their in-house professional development and support. Given the unique nature of each school and its needs, this meant that no two schools in the GiftNet clusters had the same professional development goals, delivery, support, participants, and so on. This form of professional development and support was highly personalised, with each school's professional development goals reflecting its needs. The cluster group meetings held each term brought together school leaders for the purpose of more advanced professional development, sharing, and support in their leadership roles.

The criteria for school selection and participation in a cluster group were that it be able to demonstrate:

  • Previous participation in Gifted Edge or other professional development about gifted education, demonstrating some knowledge and experience in the field;
  • School-wide commitment to catering for gifted students and an aim to strengthen this;
  • Supports and systems for gifted education;
  • School leadership committed to cluster involvement and furthering school-based developments; and
  • Commitment of school resources for release time for participation (GiftNet Milestone Report 1, 2006).

The reasoning behind these selective criteria was to ensure an appropriate level of commitment to ongoing, advanced professional development, which moved beyond the workshops offered after school and in the mini-conferences. Each school was also expected to liaise with the GiftNet coordinator to determine a school-based, needs-driven focus for their in-school professional development.

While all GKP contributing schools had access to some aspects of the GiftNet programme, only schools able to make a commitment to ongoing, personalised professional development to support pre-existing school-based initiatives in gifted and talented education were part of a cluster group. The cluster group delivery aimed to meet the intended outcomes through the provision of: a half-day cluster meeting each term for up to two representatives from each participating cluster school; a half-day in-school professional development and support each term; and up to $500 per year per cluster school for professional resources (e.g., books, conference registration, guest speakers). In addition to this, each cluster was appointed a mentor who worked alongside the GiftNet coordinator, providing ongoing professional development, advice, and support.

The intended outcomes of the GiftNet programme were inclusive of both teachers and students, with the proposed objectives being:

  • To bring about positive change in attitudes and commitment to gifted education amongst staff in contributing schools;
  • To improve the quality of teaching and learning for gifted students in contributing schools; and
  • To further bridge the gap between the one-day-a-week programme, contributing schools, and the greater GKP community for better working relationships to support GKP students (adapted from the GiftNet Proposal, 2005).

As the proposed objectives show, the major focus of the GiftNet programme was to increase opportunities and raise achievement for gifted and talented students in their mainstream schools. The negotiated contract with the Ministry of Education (2005) further refined these broad objectives as intended outcomes and indicators, as shown in the table below. The indicators in italics are those that were specifically aimed at cluster group schools, which were the focus of this evaluation. As Table 12 below shows, between the proposal and the final contract for the GiftNet programme, there was a shift from broad to quite specific outcomes, particularly in relation to student achievement, talent development plans, and social and emotional characteristics and needs. The intended outcomes in the final contract also specifically targeted GKP students, as opposed to gifted students in general. These changes will be further discussed later in the report: the interpretation of the outcomes impacted upon the evolving programme development, implementation, and evaluation.

Table 12: Intended Outcomes and Indicators, GiftNet Contract (2005)
Intended Outcomes:
GKP students have increased opportunities and raised achievement in their mainstream schools, in their area/s of talent:
Indicators:
  • Student learning outcomes demonstrate raised achievement in students' area/s of talent.
  • Talent development plans are written and maintained.
  • Differentiation is evident in teaching, planning, learning, and assessment.
Teachers in GKP's contributing schools are more aware of the social and emotional needs of gifted students and cater for these:
Indicators:
  • Behaviours and characteristics are used to identify gifted students (amongst other methods).
  • Teachers can identify the social and emotional characteristics and needs of gifted students.
  • Classroom practice demonstrates an awareness of gifted students' social and emotional needs.
GKP and contributing schools work together to support GKP students:
Indicators:
  • GKP students, GKP teachers and the students' mainstream teachers collaborate to write talent development plans.
  • GKP teaching staff and teachers from contributing schools network at GiftNet mini-conferences and workshops.
  • Sharing of GKP curriculum and teaching practice at mini-conferences and workshops.


This evaluation focused on only one of the four clusters, one which was composed of six contributing schools in 2006, but four schools in 2007 and 2008. For the purposes of the evaluation, only the four schools that were actively involved in the cluster for the three years have been included . These four contributing schools had students attending two GKP units in the region (which served students in 32 schools in 2006, thus, the cluster comprised one-eighth of the units' contributing schools). Table 13 below shows demographic information for each school (2009, tki website), as well as its school-based focus for 2006 (GiftNet Milestone 2, June 2006). Each school had a unique focus for professional development from the outset of the programme, and as this report will show, that remained the case throughout the three years. However, the focal point for each school changed in response to needs and the intended outcomes of GiftNet.

It is important to note that the demographics and focuses of this cluster group are not only dramatically different within the cluster, but also in relation to other schools in other GiftNet clusters. However, the focus on one cluster allows for rich detail of the evolution and impact of these four schools' professional development foci in relation to the intended outcomes of GiftNet. To achieve this, over the three-year period, one researcher worked alongside the GiftNet coordinator, onsite, with this unique cluster of schools.

Table 13: Cluster Group Schools
School Demographics GiftNet Focus
A Contributing State
Decile 8
Roll: 289
Topic Work:
To raise student achievement by 'lifting the ceiling' on learning and assessment.
B Full Primary
State: Integrated
Decile 8
Roll: 442
Numeracy:
To provide increased opportunities and raise achievement for students in years 2 and 5 who show talent in this area
C Contributing
State
Decile 9
Roll: 506
Writing:
To increase opportunities and raise achievement for year 5 students who show talent in this area.
D Full Primary
State
Decile 6
Roll: 271
At the time of the mid-year milestone report the area of focus had not been determined; however, the school was working with an adviser to schools on identification and differentiation (to be expanded by GiftNet).


Despite the limitations of this approach, this evaluation aims to explain the development and implementation of a professional development programme and to determine the outcomes for teachers and schools. From these perspectives, the impact of the programme for participants, their schools, and, to a lesser extent, students is examined, as well as its sustainability. The next section further explains the aims and research questions for this case study.

Research Methodology

The evaluation of the GiftNet programme was a case study of one cluster of schools and based on the following questions as adapted from the Ministry of Education's Request for Proposals (2005) and under the framework of the overall aims of the evaluation:

  • How are professional development programmes designed and implemented for teachers of gifted and talented students?
  • What are the outcomes for teachers in relation to the professional development programme goals?
  • What is the impact of the professional development programme for all stakeholders?
  • Is the professional development programme sustainable?

In order to answer these research questions, a range of data collection methods was employed within a sampling of schools, as the next sections describe.

Data Collection Methods

The on-site evaluation was initially comprised of an annual two-day visit in the final term of the year during a planned half day cluster meeting. However, during the onsite visit in 2006, it was determined that this was an ineffective approach and the evaluation was adapted to allow for two one-day visits (in terms 2 and 4 during the half-day cluster meeting) in 2007. In 2007, a new programme coordinator was appointed, necessitating an informal visit early in the year to familiarise her with the evaluation purposes and plans, followed by one single day visit in terms 2. The reason for a single day visit in 2007 was because the change in coordination led to changes in programme delivery and focus, as well as some internal evaluation and monitoring methods, and the researcher felt a 'settling in' period was needed. This change enabled an additional visit in 2008, during which, three one-day visits were made. One of these visits coincided with the half day cluster meeting, but the other two visits were designed so that the researcher could visit each of the four school sites.

In addition to the planned case study visits, the researcher was also an invited speaker at one of the mini-conferences in the region, attended the annual TDI hui, and had ongoing communication with the coordinator, which provided further opportunities to work together. These ongoing formal and informal interactions enabled the design, implementation, and refinement of a range of data collection tools over the three years. However, the constraints and limitations of the research (as outlined previously), coupled with the evolving and changing nature of the cluster school programme, meant that flexibility was needed as the data collection methods were employed.

The data collection methods used in this evaluation can be organised under several broad methods: document analysis; observation; surveys; and interviews. The table below shows the methods and tools used over the three years. A detailed explanation of each method and the tools utilised follows in the next sections.

Table 14: Data Collection Methods
Data Collection Method: Document Analysis
  • Programme Proposal and
  • Contract
  • Milestone Reports
  • GiftNet www site
  • Cluster School Documentation
Data Collection Method: Observation
  • Field Notes from Cluster Meetings
Data Collection Method: Survey
  • Online Questionnaire: teachers (2006), mentors (2006 and 2008)
Data Collection Method: Interview
  • Focus Groups:
  • school leaders in cluster group, school-based stakeholders
  • Individual: programme coordinator, school leaders in cluster group,
  • school-based stakeholders
Document Analysis

The original programme proposal, Ministry of Education contract, and milestone reports provide both descriptive, and evaluative, information about the GiftNet cluster programme. The GiftNet www site provides further descriptive evidence. The milestone reports, in particular, verify the evolving nature of the programme over the three-year period. These documents also provide insight into how the programme was developed, implemented, and evaluated, and place the cluster group activities in the broader context of GiftNet's multi-faceted professional development programme.

The cluster school documentation (e.g., policies and procedures) shows progress toward meeting school-based goals. This data contribute to understanding the outcomes for teachers and schools, as well as the impact of the professional development programme.

Observation

Observation of the cluster meetings was another part of this programme evaluation, providing an authentic opportunity for determining the quality of the professional development and sharing. Over the three years, the researcher attended one cluster meeting annually. As a participant observer, the researcher took field notes and engaged with cluster leaders during these meetings by asking questions, noting observations, and providing responsive feedback to issues, questions, and so on. This observational data is useful in understanding the programme's delivery, outcomes for teachers and schools, and impact.

Survey

Online surveys for determining the overall effectiveness and impact of the GiftNet programme were administered in 2006 and 2008. In 2006, school leaders in all participating cluster schools across all regions were surveyed. The survey probed participants' perceptions of the GiftNet programme: their levels of participation; professional development goals and progress towards those; the strengths and barriers of the in-school support provided to cluster schools; the ways in which GiftNet had met its goals; and the overall effectiveness of GiftNet. Thirty school leaders were invited to participate in the survey, and 18 of these responded. They represent all regions of the programme, but not all participating schools. Given the low response rate, this survey was not administered in subsequent years; however, data from the survey are useful in understanding and verifying outcomes of teachers and schools, as well as the impact of the cluster work. Additionally, given the limited funding and time for the research, it was deemed necessary to only focus on the one cluster (four schools) in 2007 and 2008.

In 2006 and 2008, the four GiftNet mentors were surveyed. They were asked to describe: their role, its benefits and challenges; progress towards GiftNet goals; and the strengths, challenges, and future directions for GiftNet (all with a focus on the in-school cluster group work). In 2006, three of the four mentors responded, and, in 2008, only two of the four responded. While this response does not represent all mentors' views, the data are helpful in understanding their roles in the development and implementation of the cluster school programme.

Interview

Interviews were also conducted with the GiftNet coordinator, cluster school leaders, and school-based stakeholders. Individual interviews were conducted with the GiftNet coordinator on an annual basis and two individual interviews were conducted with the four school leaders in one cluster. Focus group interviews were conducted with the cluster school leaders in 2006 and 2007. For these planned interviews, schedules were developed and consistently employed throughout the evaluation. The interviews probed participants' perceptions of the programme's effectiveness in meeting its goals. These interviews were recorded and transcribed, as well as supported by the researchers' written notes.

In 2008, cluster schools were invited to nominate stakeholders who could verify their progress towards professional development goals. All four schools nominated GKP students who were interviewed in a group or individually. One school also nominated a teacher and a parent of GKP children, each of whom were interviewed individually. These more informal interviews allowed participants to describe the impact of the GiftNet programme on teachers. Written notes were taken for these interviews. The interview data provide rich evidence of the effectiveness of the cluster grouping, including its development, implementation, and impact, particularly for teachers.

All participants in the research were provided with information sheets describing the study and asked to give informed consent before participation. Following Massey University's code of ethical conduct, students required parental consent to participate in the study.

Employing a variety of both qualitative and quantitative data collection methods, inclusive of the perspectives of the different stakeholders in the programme provided rich insights. The next challenge was to make sense of it all through careful analysis and recording of this TDI's story.

Data Analysis

The data collected for this case study are qualitative, and the analysis of this data, derived from documents, interviews, observations, and surveys, began with the task of sorting and storing records based upon the sample, method, and year of collection into manageable chunks. From there, it was possible to begin an analysis of these according to their speaker and the context, guided by a framework of broad questions. These questions guided a number of pre-ordinate themes, as shown in the table below. The content was analysed and coded based on these themes, in the first instance.

Table 15: Guiding Questions and Themes
Framework: Guiding Questions
How are professional development programmes designed and implemented for teachers of gifted and talented students?
Themes:
  • Programme design
  • Programme delivery
  • Evaluation
  • Resourcing
  • Professional support and development
What are the outcomes for teachers in relation to the professional development programme goals?
Themes:
  • Student achievement
  • Attitudes, beliefs, and teaching practice
  • Shared responsibility
What is the impact of the programme for all stakeholders?
Themes:
  • Evidence of growth, development, and change
Is the programme sustainable?
Themes:
  • Resourcing
  • Future plans and directions


From these pre-ordinate themes, emerging sub-themes or conceptual categories were identified. The conceptual themes which derive from the data were then compared and combined in new ways to create a 'big picture' that includes causal events, descriptive details, and ramifications. The multiple source data is also used for verification and triangulation. The pulling together of all the analysed data purposefully sought to provide a rich, detailed account that approximates the reality of the GiftNet programme.

The aim of the analysis was to present a rich, tightly woven account that made sense, reflected a reality with which others could concur, and was useful. The generalisability of this research is limited in the sense that its findings cannot necessarily be applied to all professional development programmes for all teachers of gifted and talented children in all contexts, nor can it be generalised across the GKP clusters, given the unique demographics, different school approaches, and personalisation of the professional development. It is also important to remember that the cluster groups were only one aspect of a multi-faceted professional development programme and complemented with teacher workshops and mini-conferences, as well as other professional development (by other providers). However, an attempt has been made to provide sufficient information so that others can determine which findings may be applicable to other situations, thus allowing for some transferability. Finally, the analysis aimed to unveil not just what outcomes occurred as a result of the programme, but, equally important, why and how those were achieved. The next section of this report describes the development, implementation, and evolution of the GiftNet cluster programme.

Programme Development and Implementation

This section describes the development and implementation of the cluster programme of GiftNet's professional development as it evolved over the three years. It begins by describing the design of the programme, followed by detailed elements underlying its implementation.

Designing the Programme: Key Players

The GiftNet professional development programme was an extension of the Gifted Edge programme, as has been previously discussed. Based on formal and informal data collected towards the end of the Gifted Edge contract with the Ministry of Education, all indications were that contributing schools needed, and were prepared for, more intensive, personalised, school-based professional development. As the programme coordinator stated, "We were sort of hitch hiking off Gifted Edge … we had ideas about where we wanted to go and what the needs of our schools were so that had guided us in what our intentions were" (Programme Coordinator, interview, 2006). The Gifted Edge programme was enhanced by adding after-school workshops and cluster group approaches, coordinated in four regions by two professional development facilitators. Both coordinators were experienced teachers in the GKP programme and had undertaken extensive professional development and study in gifted and talented education. Their positions as GiftNet coordinators were part-time (one worked 3 days a week, the other worked 2 days), and one maintained a teaching position in GKP. The coordinators were knowledgeable in gifted education, but neither had a background in professional development:

… the greater need for us is actually professional development in professional development … you know gifted is what we do, that's our core business. Yet, professional development has been something that we have been just feeling our way through (Programme Coordinator, interview, 2006).

Despite their lack of training in professional development, their knowledge and skills in gifted education were highly valued by the cluster members. In 2007 a new coordinator was appointed who did not teach in the GKP programme, but had background and experience in professional development (the change of coordinators is discussed later in this section). Thus, the cluster teachers in this evaluation had two different facilitators over the 2006 and 2007 period. In the 2007 interview, the cluster teachers reflected on the attributes of both coordinators. One of the cluster teachers explained that she valued their credibility, particularly as a means of validating her school's approaches (interview, 2007). She also respected the amount of preparation, stating they were able to "deliver at this level – having thought it through". Integrity was another quality the cluster teachers consistently reported: they were able to trust the coordinators' abilities to critique and provide constructive criticism and feedback. As one cluster member stated, "We've had very honest dialogue" (Focus group interview, 2007). Finally, the coordinators were able to link theory to practice, sharing the theoretical underpinnings of best practices in gifted education. The coordinators were able to access theory and research, sharing that with the cluster members who expressed difficulties in having the time or resources to really delve into this important element of gifted education. Although the coordinators had individual points of difference in their personal approaches to professional development, they shared these common characteristics.

The two professional development coordinators worked in complementary roles to one another with each coordinator taking responsibility for the overall organisation of two clusters (one was responsible for Wellington and Rotorua, the other for Whangarei and Auckland). Their responsibilities included planning and organising mini-conferences and workshops in each region, as well as the facilitation of cluster support, both in schools and for the bringing together of school leaders/lead teachers each term. Because the professional development for cluster schools was needs-based, the coordinators also spent a good portion of their time preparing "how we work with schools" (Programme Coordinator, interview, 2006). The pragmatics of the geographical spread of the programme also meant that a good deal of time was spent travelling, organising visits, and communicating with school leaders between the ½ day term spent onsite providing support. A challenge expressed by the coordinator based in Wellington, but facilitating schools in Rotorua, was not being "physically there": she was not able to be as flexible in her time spent with schools nor did she have an ongoing relationship with the schools outside the region in which she taught as a GKP teacher (interview, 2006). To assist in overcoming this challenge, a third programme facilitator was appointed in 2008 in the Auckland region, and it is also important to remember that each regional cluster of schools was appointed a mentor, as part of the original design.

The mentors for each cluster were also working in gifted education, but all four mentors had a background in professional development, and, in 2006, were working as advisers, private professional development providers, or tertiary teachers. The mentors' roles were to further support both the schools and GiftNet coordinators through attendance at cluster meetings, advice and consultation for the coordinators, and, to varying degrees, direct support to schools through communications and meetings (Milestone Report 1, 2006). In the 2006 online survey of mentors, the three who responded indicated they had undertaken this work – all had attended meetings and worked alongside the coordinators and schools in planning professional development. As one mentor stated, "My role was also to plan professional development with the … coordinator … suited to the specific needs of individual schools involved" (Online Survey, 2006). The levels and foci of support provided by mentors varied, and this was a pattern that continued throughout the three-year programme (Final Milestone Report, 2008; Online Surveys, 2006, 2008).

This variance in support may have boiled down to an issue raised by two of the four mentors in 2006: there was a lack of clarity of the roles of the coordinators versus the mentors. One mentor, throughout the three years of involvement, continued to grapple with her role versus that of the coordinator, expressing strong views that the person working with schools needed to be local. She also expressed a desire for mentors and coordinators, across the programme, to meet together on a regular basis (there was no provision for this).

There was also some 'overlap' of roles, as two of the mentors were working as advisers for gifted and talented through school support services with some of the cluster schools in 2006. In the evaluated cluster this meant that the GiftNet professional development for one school simply complemented ongoing work of the adviser, with no school-based goals set for 2006. In another cluster, the adviser was concerned that her relationship with three of the schools, which had included in-depth professional development, would be lost (however, her concern was unfounded – Online Survey, 2006). This confusion and replication of roles can be explained, in part, by a divergence between the initial proposal and the final contract with the Ministry of Education.

As the final milestone report for GiftNet explains,

The objectives in the proposal were mostly associated with general school-wide development in gifted education, but the objectives that were agreed upon were more to do with GKP students, their needs, their achievement and opportunities in their areas of talent. The two behove quite a different way of working (2008, p. 10).

The primary focus of GiftNet shifted from outcomes for teachers to outcomes for students – specifically GKP students as the three years progressed. At the end of 2006, one of the programme coordinators chose to leave that position, and it was not until a new coordinator was appointed that the divergence between the intended outcomes (of the proposal) and the actual outcomes (as stated in the contract) really came to light. The new coordinator had served as a mentor for one cluster in the previous year, and also had experience as an adviser to schools and tertiary educator. She also had an extensive background of both teaching and advanced study in gifted and talented education. In her new role as coordinator, she had to spend time, getting up to speed, and "doing heaps of thinking" (interview, 2007). As she explained in an interview in 2007, it was difficult coming into a project "part way through", trying to work with another coordinator who was geographically on the other end of the country with established "ways of working", and shaping her practices to better fit the contract obligations.

The design of the programme was the work of the 2006 programme coordinators, but in contract negotiations with the Ministry this design changed, seemingly with little input from the coordinators who were not part of the negotiation, but ultimately responsible for the implementation of GiftNet. The coordinators worked in complementary roles, but not without the challenges of time and geographical distance, both of which have resourcing implications. The coordinators also needed support in delivering professional development; it is one thing to be knowledgeable of gifted education, and, yet, another to know how to effectively work with schools as providers of professional development and support. The mentors were important providers of support for the coordinators and 'on the ground' in local schools, but their roles were not always clear or consistent across regions. The change in personnel, part way through the programme, also impacted the design and implementation. GiftNet began its evolution in 2006, and as the next section shows, the structure and delivery of the cluster group work was shaped and re-shaped throughout the three years, with greater and greater alignment to the outcomes in the Ministry contract.

Designing the Programme: Structure and Delivery

The design of the cluster group professional development was part of a three-tiered approach: after school workshops, full day mini-conferences, and in-school support. In a way this was seen as a staircase to professional development, with teachers opting in to workshops or a mini-conference as a first step. Schools also had the option of targeted cluster and in-school support. Cluster school support, as described in the proposal (2006) and final milestone report (2008), consisted of several elements:

  1. One 3-hour cluster meeting per term for lead teachers from each school – Initially the focus of these meetings was on differentiation for gifted students, but as the programme progressed, time was spent on how to address social and emotional needs; developing, implementing, and evaluating school-based initiatives from an evidence-base; and the development of facilitation and leadership skills. The cluster meetings also allowed opportunities for lead teachers to network and share their school-based programmes, resources, issues, and so on.
  2. A half day per term of in-school support from GiftNet coordinators to facilitate the translation of GiftNet goals into school practice – The time spent in schools varied greatly, as a result of the needs-based approach to determining its delivery. For example, the coordinators facilitated staff meetings; worked with and interviewed children from GKP; or provided one-on-one lead teacher support in planning, leadership, and the development of resources. The half-day per term was also interpreted differently for different schools based on need: in one term, more or less time might have been spent than in another term.
  3. The appointment of a mentor – As has been described, the roles and involvement of the mentors were primarily to provide ongoing support to the GiftNet coordinators and cluster schools. The support to cluster schools included in-school advice and guidance on school developments; cluster meeting attendance (and sometimes facilitation); and presentations at after school workshops, mini-conferences, and staff meetings. The roles were determined based upon each school's needs, and often on an 'as-needed' basis. When the mentors presented at after school workshops and mini-conferences, they were paid accordingly, as with other external providers.
  4. Funding – Each participating cluster school was allocated a small portion of funding annually which could be used to support their school-based focus. This could be for student or teacher resources, conference attendance, guest speakers, and so on.

At the start of the GiftNet programme, meetings were held between the school leader and a GiftNet coordinator (in some cases, a mentor also attended these meetings) to determine the in-school professional development focus. Also, in 2006, each school completed a survey to provide baseline data on its systems, leadership, staff development, provisions, and evaluation related to gifted and talented education (Milestone 1, 2006). From these meetings and baseline data, a professional development focus and goals were negotiated. These varied extensively across and within regions, as they were very much school-needs driven. In the first year, many of these were school-wide initiatives, related to the 'basics' of gifted and talented education, and sometimes tied to other school plans (e.g., literacy, numeracy, inquiry learning). Additionally, the target audience for the in-school support ranged from working one-on-one with a lead teacher to the provision of whole staff professional development.

These variances are not surprising and reflect the intentions of GiftNet: to provide personalised, school-driven professional development. It is also important to understand that while the intention of the contract was to enhance opportunities for GKP students, in 2006, few schools, despite their commitment to intensive, advanced professional development were actually ready to pursue such. In conversations with the programme coordinator during 2006, it became evident that some schools were not actually as far down the gifted education track as might be expected, meaning that the in-school support had to "back-track" (interview, 2006).

For example, School C was planned as a pilot school for trialling evaluative tools for this evaluation in 2006. The school's focus was on increasing student achievement in writing for year 5 students. The intention of the pilot was to collect some baseline evidence of student writing; however, the data provided by the school did not necessarily demonstrate the identification of gifted and talented writers. The identification methods were not specific to writing nor were the work samples reflective of outstanding potential or performance in writing (Milestone 2, 2006). It became evident that in order to raise achievement with gifted writers, staff needed a firm grounding in the characteristics and identification of writing ability as a foundation. So mid-year, the focus shifted to mathematically gifted students, who the teachers felt they could more confidently identify:

…led us to basically throwing writing out the window and changing to mathematics which is you know we felt more comfortable because we had more data that we could actually compare to gifted kids in mathematics than we could in writing (Lead Teacher, focus group interview, 2006).

The in-school support for 2006 for School C shifted to differentiated programmes, specifically in mathematics, for one syndicate of teachers. The coordinator provided resources to support this area, as well as ongoing work with the syndicate on identification, pre-assessment, multi-level and open-ended learning experiences, and planning using Bloom's Taxonomy.

What this example demonstrates is the flexibility and responsiveness of the in-school support, a quality valued by lead teachers. In interviews with cluster school leaders, they consistently praised the flexibility of the professional development coordinators, as shown through their availability, responsiveness, and support. The programme coordinator described this as "differentiated professional development" (interview, 2006), and the cluster leaders reinforced this notion in a focus group discussion when they described their "different stages of readiness" (2008). One of the cluster school leaders described a defining feature of GiftNet as being "responsive to needs: administrative, practical, and theoretical". She summed up her impression of the coordinators as, "they walk the walk and talk the talk" (interview, 2007).

However, this example also demonstrates some of the challenges faced by GiftNet throughout the three years: schools' expectations did not always match the intentions of the contracted professional development programme; a general focus on a topic like differentiation cannot be considered advanced professional development; and the professional development focus was driven by teacher needs, as opposed to the needs of GKP students. As the final milestone report (2008) highlights, there were not shared understandings – from the outset or by all stakeholders – of the intended outcomes of GiftNet and those expected by the Ministry of Education.

Beginning from 2007, with the appointment of a new coordinator, efforts were made to better align GiftNet to its proposed outcomes and indicators. While the structure of GiftNet remained intact, after a period of induction, the coordinator changed the delivery approach for the Wellington and Rotorua regions. The first change was a clarification of the roles of the GiftNet coordinator, the mentor, and the school leader as follows:

  • GiftNet coordinator to provide focused support to teachers of GKP students, assisting with collection and analysis of baseline and endline, planning based on that data, and professional development support around the focus area.
  • Mentor to provide in-school support, which might include ongoing development of school-wide procedures.
  • School leaders to organise and facilitate staff meetings to share models of practice being used by teachers of GKP students, and to determine the best ways of transferring this learning to school wide approaches (Milestone 5, 2008).

Each school's action plan for 2007 was amended to include specific ways of tracking student progress: the collection and analysis of baseline and endline data, including interviewing GKP students to determine their learning, social, and emotional needs. This was useful in determining the school-based professional development that would lead to changes in practice beneficial to GKP students.

School A provides an interesting example of these changes in action. At the beginning of GiftNet, the school's focus was on differentiating topic work by using conceptual themes. In 2006 the school was also grouping gifted students for differentiated writing, reading, and mathematics. In 2007, the school planned to embark on inquiry learning, but the new coordinator challenged them to delay starting this journey until some baseline data on the GKP students had been collected. The coordinator interviewed the children and was able to report back to staff that the children did not feel challenged in reading, and more specifically in follow-up tasks. This became their focus for the remainder of 2007, with a strong emphasis on higher order thinking. In 2008, again as a result of the feedback from the targeted GKP students, the school chose to cluster group these students, to progress the 2007 work in the area of reading by identifying accelerated goals such as unpacking concepts in texts and critiquing a self-selected text for an identified purpose, and developing a toolbox of comprehension strategies with the whole school staff.

The delivery changes made in 2007 and 2008 were received with mixed responses by participating teachers. It is important to remember that not all schools in all clusters functioned in the same manner, but the cluster used in this evaluative research was one of those impacted by this shift in determining professional development based on the needs of individual GKP students first and foremost. Moving from a school and teacher-driven plan to a student-driven, student needs-based one was discussed with the cluster teachers in a focus group in 2007. As one cluster member expressed, "It is quite a dramatic change!" One of the cluster leaders felt that by honing in on "in our case three students, two teachers, specific curriculum areas" the whole school was no longer involved, and this conflicted with their philosophy of school-wide approaches to professional learning. However, she did acknowledge that it was important that in her role, she ensured their learning was passed on to other teachers. The other cluster members verified a conflict between this approach and school-wide philosophies, but were also positive that there was potential for a ripple effect, or as one leader expressed, "parallel" opportunities related to other professional development. Gifted education was just one of many professional development foci within the schools, and there was some concern that it was isolated from important, interrelated learning. This is often the case in schools when there are multiple professional development initiatives provided by different providers.

On the other hand, by the end of 2008, the lead teachers were beginning to see the rewards of working from a student-driven focus. Each school in the cluster which was part of this evaluation had taken steps to gathering individual data on their GKP students, developing some sort of individualised profile or plan, and enacting changes within the school to meet those needs. This included clustering students, careful class placements, working more closely with their classroom teachers and parents, increasing opportunities within and out of school, and ensuring that within class teaching was qualitatively differentiated. In the final internal evaluation of the cluster work, the majority of lead teachers, across all regions, indicated increased knowledge or enhanced skills in individual planning (11 of 15 respondents), and much more or some skills in gathering and analysing data related to gifted students (12 of 13 respondents) (Final Milestone Report, 2008). This is evidence that the realignment to the original contract outcomes was having a positive effect upon school practices.

School B provides a good example of how a school leader applied the process of determining student needs as a first step to making changes within the school. In 2008, the lead teacher interviewed identified GKP students to determine their learning, social and emotional needs. As a result, she discovered some social and emotional issues of which she was unaware, particularly in relation to friendships. The interview material was shared with other teachers in the school, and was the basis of decision-making for class placements, ensuring that not only was the teacher well-suited and attuned to the needs of the GKP students, but also to make certain that each GKP student had at least one like-minded friend in his or her class. In an interview with a parent of one of these children, she expressed her delight in the careful class placement of her son, the ongoing dialogue between the class teacher and school leader regarding his needs, the willingness of the school to continue trialling new strategies and approaches, and their "incredible support" (interview, 2008). This was confirmed by the school leader who said, "The difference is just amazing with him!" (interview, 2008).

Another example of this occurred in School C in 2008. The lead teacher's focus for her in-school support was to investigate the development of talent development for GKP students, as well as others identified as gifted. On reflection of the school-wide processes, she noted that although the school had a register of gifted children, which included data on abilities, strengths, and interests, it "just sat in a folder collecting dust" (Final Milestone Report, 2008, p. 36). She worked with the GiftNet coordinator to establish a process of analysis and planning based upon the analysed needs of each student. She was able to successfully apply this process with a young boy in the school, who although not a GKP student (given his age of 5), was clearly demonstrating behaviours and characteristics associated with advanced abilities. The outcome was an accelerated placement into a Year 2 class with a cluster of gifted and talented students. The lead teacher was also working closely with his teacher providing support on differentiation. As the lead teacher explained in an interview, "I can see that it can actually work in a school" (2008).

These are but two examples of how the evolving nature of the delivery of GiftNet impacted upon cluster members. As the programme progressed through its three-year funding cycle, the delivery was adapted, mainly to re-align the programme to its intended outcomes, re-focusing the professional development on a balance of responsiveness to the needs of GKP students and their teachers. The structure of the programme remained intact as originally proposed, but the roles of facilitators, mentors, and GiftNet coordinators were clarified. Some of these changes came about with the change in programme coordinators – not because one coordinator was necessarily better qualified or able to deliver the contract, but as a result of her induction into the role she was able to critically reflect upon the GiftNet cluster work, its purposes, and how to determine its effectiveness. The new coordinator also had a background in professional development, having worked as an adviser to schools, and this gave her insight into the expectations of the Ministry of Education in regards to student outcomes and evidence-based professional development. Stepping back and evaluating the programme, with an eye on the expectations and processes of professional development, was a key factor in its ongoing development. Acting upon the data collected through formal and informal evaluations was a strength of the GiftNet professional development programme, as the next section shows.

Evaluating GiftNet

One of the purposes of the Enhance evaluations was to analyse the programme's internal evaluation methods and data through interviews and document analysis. As the above section shows, the GiftNet programme was developed and evolved over time as a result of informal and formal data collection. All elements of the GiftNet programme were consistently evaluated through questionnaires and surveys, interviews, discussions, and documentation providing qualitative and quantitative evidence of its effectiveness in meeting the intended outcomes and indicators. The programme's self-evaluation was complemented by the Ministry-contracted evaluation, and over time both monitoring processes evolved and changed in response to the findings. This section will focus on GiftNet's evaluation processes over the three year period, highlighting the findings of the evaluative research.

All workshops and mini-conferences were evaluated based on feedback sought from participants specific to the topic(s) explored. Participating workshop teachers were asked to describe their current understandings, practices, knowledge, or skills; their new understandings as a result of participation; and how they intend to apply those to their practice. Mini-conference participants used a "connect, extend, challenge" tool to comment on the content of the mini-conference, again encouraging them to reflect upon how their new learning connects with previous knowledge, extends that knowledge, and challenges their thinking. The responses from these surveys were collated and summarised in milestone reports. From these responses, the coordinators were also able to gain insight into the participants' views on topics, presenters, professional development strategies, and so on, using this information to inform their planning of future workshops and conferences.

Milestone reports also recorded attendance numbers for each workshop and conference by venue and schools represented. Again, this gave an indication of each region's needs, responsiveness to topics, and levels of involvement. In 2008, schools that had not registered for mini-conference and after school workshops were surveyed to determine the reasons for their non-attendance, as well as the topics that would best meet their professional development needs. This exercise showed that the main reason for non-involvement was that gifted education was not a professional development focus for the schools (Final Milestone Report, 2008). Also in 2008, an analysis showed that cluster schools were the most well represented in workshops meaning that in 2009, those will be tailored to their expressed needs (e.g., multi-level planning, Māori concepts of giftedness, personalising planning). These topics will also be addressed in cluster meetings.

The cluster approach was evaluated using several methods. For example, it informally evaluated during the cluster meetings. Each agenda included an opportunity for the participating lead teachers to assist in shaping the topics for future meetings, and in the cluster meetings observed in this evaluation, ongoing feedback and suggestions were sought from the coordinator. Working alongside the individual lead teachers, the coordinators also got a sense of their needs, what was working and what wasn't, future directions, and so on. There was also ongoing communication between the mentors and coordinators. Detailed records of attendance at cluster meetings, school visits, and actions taken were also recorded, analysed, and explained in milestone reporting.

Beginning in 2007, a critical component of the internal evaluation was the collection of samples of teacher planning, school policy documents, and student work samples. These were shared in oral milestone reporting as evidence to demonstrate how GiftNet was impacting student, teacher, and school outcomes related to the project. In addition to this, records of "learning talk" at cluster meetings were reported, as well as interviews with teachers. GKP students were also interviewed by the coordinators to determine their perceptions of the changes within their school. Feedback from the Ministry-appointed TDI Coordinator during an oral milestone reporting this detail indicated that "Outcomes for students have been shown more clearly this year. Teachers are collecting data too" (Minutes of oral reporting, 2007).

Over the course of three years, while the programme was sharpening its focus on GKP students, this was not an easy outcome to measure. The contract included indicators for each planned outcome, the most difficult to measure being "students' learning outcomes demonstrate raised achievement in students' area/s of talent." As the lead teachers explained in a 2007 focus group discussion, there are many variables that affect student achievement gains – it is much more complex than the sort of causal relationship this indicator implies. This was further complicated by the fact that GiftNet was "one step removed from the classroom programme" (Final Milestone Report, 2008) and limited to a half day visit per term. It also became clear that some teachers were not well-equipped to assess differentiated learning, nor collect and analyse relevant student data. While GiftNet coordinators made every effort to determine if student achievement in area/s of talent had been raised, it was a formidable task. However, they were able to demonstrate an increase in opportunities for students.

In 2008, a summative evaluation was conducted with a survey of lead teachers. The survey queried the overall impact of their participation in the clusters, including outcomes related to teaching, students, shared responsibility for GKP students, leadership, and school-wide approaches, as well as topics of need for further professional development. The results of this survey were analysed individually for each school response, as well as collated across all cluster schools. The individual results were used to demonstrate progress made towards meeting positive outcomes in each school's journey of professional development and to set project and school goals for 2009.

In addition to the GiftNet internal evaluations, the external evaluation played a very slight formative role. An issue that was raised throughout the evaluation, by both coordinators in the focus region, was concern that the evaluation was too limited in its resourcing to provide a full, detailed picture of the GiftNet programme across all regions and all tiers of provision. Similarly, the use of a case study undertaken by a single researcher, as opposed to an action research team approach, was viewed as disadvantageous – the levels of input, support, and reflective action were not part of the case study evaluation. Despite these points of difference between the Ignite and Enhance evaluations, all efforts were made to provide some formative results and input to feed back into GiftNet. The coordinator was sometimes included in focus group discussions with lead teachers, and when feedback was sought, particularly in regards to the re-alignment of the programme delivery with its goals and the formulation of internal evaluation methods, it was readily provided. This was an unfortunate limitation of the research, but all efforts were made to be transparent, share information, and be responsive to programme changes.

The external evaluation tools and methods were adapted over the three years, again based upon changes in the programme as it evolved. For example, in 2006, only one of the cluster schools was to be a primary focus of the evaluation (School C) with the collection of student work samples as an indicator of outcomes, but as has been previously discussed, this school changed its focus from writing to mathematics. The decision was made to interview the teachers in the syndicate, but in consultation with the GiftNet coordinator, it was decided that the collection of student samples would not be useful. Similarly, in 2008, school leaders were given an opportunity to self-select interviewees who might be able to discuss the impact of the professional development upon the school (e.g., other teachers, students, parents). This flexibility and responsiveness was a necessary element of the evaluation, ensuring that the data gathered were relevant to the evaluation questions and purposes.

Overall, the internal evaluation of the GiftNet programme was thorough and robust, employing multiple measures of quantitative and qualitative data and inclusive of all stakeholders, including gifted students. More importantly, the information gathered was used to inform programme developments and changes, serving as more than just evidence of how GiftNet was meeting its intended goals. The external evaluation was limited in scope and methodology because of resource constraints, but actively supported by the GiftNet coordinator and lead teachers in the cluster schools in one region.

Resourcing

GiftNet was supported by human and financial resources. The coordinators and mentors provided the overall direction and support for lead teachers and schools, as has been previously explained. Therefore, this section will only discuss the financial resources for GiftNet.

GiftNet was funded by the Ministry of Education as an Enhance TDI. The TDI funding was utilised to cover expenses related to:

  1. Professional fees (for workshop and mini-conference facilitators and mentors)
  2. Professional costs (travel, training, printing, resources, transport, and accommodation)
  3. Operational costs (communication,)

Approximately half of the overall funding was used for professional fees for buying in external experts, and the salaries for the coordinators were raised by GKP. This is because, unlike school advisers, the salaries for personnel are not provided by the Ministry of Education, and unlike private providers, who charge fees for their services, the GiftNet professional development was at minimal costs to schools. Schools were expected to provide release time for lead teachers to attend the once a term cluster meetings and to meet in-school with coordinators and mentors, but apart from those minimal costs, the model was a 'win-win situation' for schools. In addition to receiving professional development at low cost, each school also had access to funding for teaching resources, further professional development, guest speakers, or conference attendance on an annual basis ($500/year). The participating schools also had access to the extensive professional library developed by Gifted Edge and GiftNet, as well as the website resources.

Thus, the programme was almost completely dependent upon TDI funding, with minimal costs to schools – as had been the case since the Gifted Edge programme began in 2003. In 2008, GiftNet applied for an extension of funding, in light of the Ministry's review of gifted and talented education initiatives, and received ongoing funding for 2009. Although issues of sustainability were discussed throughout this evaluation and included in milestone reports, there was little evidence that GiftNet was actively seeking financial sustainability beyond the Ministry of Education support. (Sustainability is further discussed in a following section.)

Concerns regarding the inadequacy of this funding, in relation to the actual work being undertaken, and the time needed to do it well, were raised by the GiftNet coordinator throughout the implementation of the programme. This was a labour-intensive professional development programme, and in order to provide individualised, school-based support, complemented by the planning and delivery of mini-conferences and after school workshops, required a tremendous amount of planning and preparation time. As the final milestone report states, "To date the way GiftNet has run has taken a great deal of facilitator time, often over and above the staffing allocation" (2008, p. 52).

The implications of this resourcing-staffing allocation mismatch relate mainly to the inability of GiftNet to meet some of its indicators of progress, namely the shared development of Talent Development Planning between classroom and GKP teachers (GiftNet contract, 2006; Final Milestone Report, 2008). This impacted the ability of the coordinators to work alongside classroom teachers to collect, analyse, and plan from an evidence-base of student talent. Related to this, there was no resourcing allocation for GKP teachers' workloads, freeing them with time to work with classroom teachers. Finally, the professional development model designed by GiftNet relied on an assumption that lead teachers would have the skills and time allocation to facilitate school-wide development and change.

However, this was not always the case. For example, in the cluster for this evaluation, while three of the lead teachers held management positions in their schools, the teacher in School C did not. Although her personal development and growth was observed throughout the project, she was not in a position to affect school-wide change. As the final milestone report explains, "In schools where the lead teacher was willing to take the lead, already had some confidence and skill required, and time to do this, the model worked well" (2008, p. 9). Initially it was intended that each school would have two lead teachers, one being representative of the senior management team, however, many schools could not, or were not prepared to, sustain the relief costs associated with this level of involvement. The resourcing of the GiftNet programme fell short of meeting their own costs and did not include any allocation of funding for school-related costs for relief.

Another costly factor relates to the delivery of professional development across different regions of New Zealand: travel, accommodation, and so on. As has been discussed previously, one mentor felt strongly that the costs for the GiftNet coordinator to make visits to another region each term were unnecessary, if a local facilitator had been appointed. Ideally, this facilitator would have been a GKP staff member who would bring insight and understandings of the GKP programme. Related to this, the travel time took away from time that could be spent planning and delivering the professional development (GiftNet Coordinator, interview, 2006). The resourcing also did not take into account opportunities for the GiftNet coordinators or mentors to meet together for planning, evaluation, and reflection; however, they were able to meet for limited periods of time and in relation to other activities like oral milestone reporting.

During a 2007 focus group interview, the cluster members expressed a desire for a greater allocation of in-school support time. The half-day each term, in their view, was inadequate given their expectations of GiftNet: they wanted time for the coordinator to work alongside classroom teachers and lead staff meetings, as well as provide the lead teacher with ongoing support. Reference was made to larger professional development contracts, such as the numeracy project, which require greater resourcing and delivery on a larger scale than GiftNet ever intended.

Overall, the resources to support the GiftNet professional development programme, while possibly adequate for the original proposal aims and intentions, were not strong enough to deliver on all outcomes of the final Ministry contract. The financial resources mainly supported the provision of expert personnel to deliver professional development to schools at a minimal cost to them, and complemented by additional 'free' resources to support their developments. The next section details the professional support and development for the GiftNet coordinators.

Professional Support and Development

In order to facilitate a professional development programme like GiftNet, it is critical that the providers are given support and opportunities to enhance their own knowledge and skills. It is also important that providers are able to 'stay on top' of the latest theory, research, and practices within the field, in order to ensure content delivery of the highest standard. However, there was no indication in the GiftNet proposal or contract for the provision of professional development and support for the coordinators of the programme, apart from that offered by the mentors in varying capacities. As was stated earlier, some professional development was desired by the first Wellington-based coordinator, who had a strong background in gifted education theory and practice, but not as a provider of professional development.

Though this was not a planned component of the GiftNet programme, the coordinators were engaged in some forms of professional development: conference attendance; professional reading; and people support. They attended and presented at national and international conferences, and the Ministry of Education sponsored annual hui for TDIs, as well as the GKP professional development for all staff. The coordinators also attended all local workshops and mini-conferences, which were facilitated by national and international speakers, as well as GKP staff. All of these opportunities have led to other consultancy work, sharing, and networking.

In conversations with the Wellington coordinator, as well as from observing the cluster meetings, it was obvious that she was continually engaging with professional readings, www sites, and other documented, relevant resources. Both coordinators had access to the extensive professional library developed by GiftNet, as well as opportunities for selecting resources for it.

Using a team approach to coordination meant there was some support for one another, but this was hindered by their geographical distance and lack of allocated time and resources to support regular opportunities for working together. For the Wellington coordinator, there was also some support provided by the external evaluator and the Ministry's TDI coordinator – but on both accounts this was minimal. An expectation of the TDI contract was liaison with the Ministry's TDI coordinator, and as the final milestone report states, this was "fairly consistent and satisfactory" (2008, p. 3). The oral milestone reporting was also perceived positively, as an opportunity to have face-to-face professional dialogue about the outcomes and matters arising.

Overall, although there were not any planned approaches to professional development and support for the GiftNet coordinators, they were engaged in ongoing dialogue with other professionals. The feedback from participating cluster teachers indicates that the lack of professional development and support did not hinder their ability to provide a satisfactory programme.

Summary: Programme Development and Implementation

GiftNet began in 2006 as an extension of an already programme, Gifted Edge, with the intention of providing advanced, needs-driven existing professional development. Over the three years of funding, the programme structure has remained in tact, but the delivery has been altered to better align the professional development with the intended outcomes (as outlined by the Ministry of Education contract). The overall purposes in GiftNet relate to ensuring the intellectual, social and emotional needs of GKP students are identified and addressed by schools as evidenced in Talent Development Planning; differentiated planning, learning, and assessment; raised achievement in area(s) of talent; and classroom practices. Their internal evaluation processes are strong, and, as a result, the GiftNet coordinators are aware of the areas of strength and weakness, responding with continual change. The cluster group work is just part of the model, but one that the lead teachers valued as an opportunity for ongoing professional development and support. While the major focus of GiftNet was on outcomes for GKP students, these are difficult to accurately measure, given the many variables related to achievement. What was evidenced was an increase in the opportunities for GKP students in their schools. The outcomes for teachers, however, are more determinable, as will be discussed in the next section.

Outcomes for Teachers

The outcomes for the GiftNet programme centred on the needs of GKP students in mainstream schools, but the indicators of those are evidenced by their teachers. It was beyond the scope of this evaluation to determine the effects of GiftNet upon all teachers in all clusters and involved in all aspects of the GiftNet programme. This section describes the teacher outcomes, as reported by the lead teachers in one cluster, one of the GiftNet coordinators, and in milestone reports. The indicators outlined by the Ministry of Education (contract, 2006) and which related specifically to the cluster work are highlighted earlier in this report.

To determine the achievement for GKP students, an expectation of the programme was that talent development plans would be written and maintained by teachers in contributing schools. This objective was not fully met during the three-year contract, as has been explained in the previous sections. At the end of 2008, the lead teachers were developing skills in gathering, analysing, and planning from an evidence-base of individual student needs, but these were not translated into written plans. The final milestone report shows that only two schools had developed plans (one of these was School D); however, the majority were in the process of developing plans. In discussions with the cluster lead teachers, it became clear that they were gaining confidence in the skills necessary for talent development planning, and this is verified in GiftNet's summative evaluation responses.

There was also evidence of differentiation in teaching, planning, learning, and assessment. The summative evaluation results show that teachers had increased opportunities for students to work at their own pace, have more choices in learning, think critically and creatively, and to explore topics of interest. In the four cluster schools, this was evidenced in the changes each lead teacher had implemented either school-wide, or within syndicates. Many schools, including all the cluster schools involved in this evaluation, were also offering students more opportunities to engage with like-minded peers through clustering gifted students in mainstream classrooms, more careful student placement with enthusiastic and knowledgeable teachers, and some withdrawal groups within the school. School C provides a good example of this: the GKP students regularly meet with the lead teacher to share ideas, discuss issues, and pursue their passions.

There was also evidence that teachers were developing more advanced materials for gifted students, with many of these shared in cluster meetings. Some schools were applying the principles of gifted education in their mainstream classes, with many students. This was the case with School D which undertook professional development in inquiry learning embedded in a conceptual curriculum. An interesting development in the cluster was that as the schools developed their own practices, including school-based systems for implementing and monitoring differentiated approaches, there was a shift away from heavy reliance on GiftNet support, and in one school (A) a decrease in the number of student referrals to GKP. This demonstrates positive outcomes for teachers and schools.

As has been discussed, the measurement of students' learning outcomes so as to demonstrate raised achievement in their area(s) of talent is not a straightforward task. It would be naive to assume that student achievement gains could be directly and singularly related to the professional development offered by GiftNet. As the lead teachers reflected on this indicator, they strongly expressed the view that student achievement was much more complex and impacted by a range of different factors (Focus group discussion, 2007). Discussions with the four lead teachers in the cluster do indicate that they were gaining skills in determining students' areas of talent, and making decisions regarding how to further develop these abilities, but they would be reluctant to attribute this growth to GiftNet on its own.

A major focus in 2008 became helping the teachers identify the social and emotional characteristics and needs of their gifted students and from there to enact appropriately responsive classroom practices. The Ministry expected the adoption of behavioural indicators in school-based identification, as an indicator of schools having developed their abilities to identify social and emotional needs. The adoption of these varied across and within clusters: some schools were using these in 2006; other schools have since developed these; and others are yet to have adopted them. All of the cluster schools participating in the evaluation were using behavioural indicators for gifted students, and one lead teacher (School B) was interviewing GKP students to further discuss their social and emotional needs. One reason for the variance in uptake would be the professional development focus chosen by schools, and the final milestone report (2008) indicates that those schools not using behavioural indicators had focused on differentiated teaching.

Teaching strategies need to be well matched to social and emotional needs, and much cluster meeting time in 2008 was devoted to better understanding what these strategies might be. However, it was not until the final year that this objective really came to the forefront of the programme. It is not surprising that in GiftNet's summative evaluation the lead teachers felt more confident than their colleagues in their understandings of social and emotional needs. The GiftNet participants had spent the year with this as a focus, but at the end were just at the stage of being able to start applying the theory to practice. The summative evaluation shows that respondents to the survey felt more confident in identifying needs, than they did in actually meeting those (Final milestone report, 2008). As the milestone highlights, when asked what differentiated strategies they felt confident using, most respondents did not respond to those targeting affective development.

An outcome of GiftNet was the development of shared responsibility for gifted students, and this was to be indicated through collaborative planning between GKP students, GKP teachers, and the students' mainstream teachers. This did not eventuate over the three-year period. Talent development plans and learning journals are designed by GKP teachers for each student and these are shared with mainstream teachers. Also, in 2007, a workshop on talent development planning was held during all mini-conferences. GKP teachers attended and presented at workshops and mini-conferences, allowing for some sharing and networking with mainstream teachers, and GKP classrooms are always open for teacher visits. However, despite these first steps, there were a limited number of mainstream schools adopting talent development plans and no evidence of collaboration between students, GKP teachers, and mainstream teachers sharing in planning.

In the final focus group interview with the cluster teachers, they expressed several key advantages to the GiftNet professional development. All teachers agreed that the opportunity to build and maintain trusting relationships with GKP staff, colleagues in other schools, and experts and advocates from across New Zealand was a strong feature of this programme, and yet, it is difficult to report this sort of collegial relationship building against the indicators set by the Ministry. The lead teachers referred to this as a chance to network, engage, and share with "like-minded" colleagues, and valued this greatly (Focus group discussion, 2008). They saw value in being able to share ideas with other teachers, problem solve together, and support one another through the cluster meetings. Although shared responsibility for GKP students between GKP teachers and mainstream teachers was an intended outcome, the collegiality and relationship building across schools was an unexpected benefit of the programme.

Summary: Outcomes for Teachers

Being involved in ongoing, long-term, needs-driven professional development, like that offered by the GiftNet model, led to positive outcomes for the teachers in the cluster for this evaluation. Of course, a limitation of this case study was the researcher's inability to sufficiently determine the impact for all teachers in all clusters and in regard to all indicators. From the GiftNet milestone reports, it seems there was variability across the programme, both within and across clusters, and in the nature and degree of outcomes met. The reasons for this variability have been previously discussed: different interpretations of the outcomes and indicators of the professional development contract – both within GKP and its contributing schools; and inadequate resourcing to support the intensive level of professional development required to meet these indicators. The indicators set by the Ministry of Education may have always been difficult to achieve given these factors, and equally difficult to measure.

Impact of Programme

The previous sections have provided evidence of the effectiveness of GiftNet. The impact of the programme for all stakeholders is summarised in this section, highlighting the major themes which have arisen from the evaluation. The professional development impacted upon lead teachers and their schools, GKP staff, and GKP students. However, given the limitations of this evaluation, the impact upon students is not directly measurable and is based upon evidence gathered from teacher perspectives.

Impact for Lead Teachers

There was variability in the impact for individual teachers and schools, but some evidence of:

  • Development of knowledge and skills in qualitative differentiation for gifted and talented students, enabling a transferral of theory to practice through the implementation of school-wide and classroom-based strategies.
  • Greater awareness of the social and emotional needs of gifted and talented students, and how to determine these.
  • Enhanced leadership skills to facilitate the development, implementation, and evaluation of gifted and talented education within schools.
  • Greater confidence to advocate, plan, and provide for gifted and talented students within their schools.
  • Stronger relationships with colleagues from other schools, leaders in gifted education, and GKP staff, and, in some cases, gifted students and their parents.
  • Growing awareness of, and skill in, gathering, analysing, and planning from an evidence-base of student abilities, strengths, and needs, including Talent Development Planning.
  • Better understanding of GKP identification practices, goals, curriculum, and strategies for teaching gifted students.
Impact for Schools
  • Development of school-wide policies and procedures to support gifted and talented initiatives.
  • Implementation of school-wide and classroom-based differentiation strategies, appropriate for GKP students, other gifted students in the school, and, in some cases, all students of all abilities.
  • Access to high quality resources and expertise to support gifted and talented education.
  • Stronger leadership teams to drive and support gifted and talented education.
Impact for GKP Staff
  • Greater understandings of effective professional development, including how to create, develop, deliver, and evaluate strategies.
  • Increased opportunities to share GKP practice with mainstream schools.
  • Enhanced skills in the evaluation of professional development outcomes.
  • Stronger relationships with mainstream school leaders and teachers, as well as leaders in gifted education.

Sustainability

Sustainability can be viewed from several angles: financial sustainability; programme sustainability; and the sustainability of outcomes for stakeholders. This section of the report addresses each of these aspects, beginning with the ongoing financial viability of the programme. As the section on resources shows, GiftNet was reliable upon its TDI funding for all aspects of its delivery, and, as an organisation, GKP had been reliant upon this funding since the inception of Gifted Edge in 2003. The Ministry funding was used primarily for professional fees, travel, conference/workshop costs, and to provide resources to schools.

Contributing schools were expected to provide teacher release for cluster meetings, and the commitment to this varied. Although schools were expected to send a member of the senior management team and a lead teacher to each cluster meeting, by 2008 the majority of schools were only sending one representative. Schools were also expected to release teachers for in-school support if this was delivered during the school day. Overtime the reality was that schools' financial contributions to GiftNet decreased, while the input of resources and expertise remained steady and, in 2007 and 2008, perhaps increased with the re-alignment of the professional development with the programmes intended outcomes. The mini-conferences and workshops also came at no cost to schools, apart from teacher release for conference attendance.

The programme has successfully gained ongoing funding for 2009 in order to allow a focus on the outcomes not yet reached, namely collaborative talent development planning. This has required an organisational commitment from GKP for the release of teaching staff to work alongside contributing school teachers in cluster schools who have also shown a commitment and willingness to trial several approaches in 2009. Further to this, the 2009 plans include a continued focus on social and emotional needs with an emphasis on affective teaching strategies. As the final milestone report states, all three main objectives will be pursued in 2009, but with a greater focus on the second and third intentions (social and emotional needs and shared responsibility). In 2009, GiftNet will also work to support a 'new' school in Rotorua, another in South Auckland, and a cluster of rural schools in Dargaville that is establishing a satellite GKP class. After school workshops will continue, but it is proposed that the mini-conferences in Wellington, Whangarei, and Rotorua be abandoned. In those areas, it is proposed that some funding be diverted to subsidise cluster teacher attendance at national conferences planned for 2009.

Without Ministry of Education funding, the financial sustainability of this sort of intensive, personalised professional development is at risk. There was little evidence in milestone reports or interviews with the coordinators of efforts to address the financial viability of the programme. As an out-of-school provider, GKP is completely reliant on the donations and contributions of community members, schools, and the Ministry of Education – and this extends to the professional development programmes offered.

If schools want to continue accessing GiftNet services, it only seems logical that a stronger financial commitment is made to the programme. Interviews with lead teachers in one cluster indicated that this may not be feasible as there were issues around gifted education needing to be a strategic priority for schools, with a commitment from management. The GiftNet focus on some teachers, and, consequently, some students (namely those attending GKP), clashes with schools' philosophies of whole school professional development and programming. In reality, what the lead teachers in the cluster schools evaluated wanted was more in-school support to work with all staff, more one-on-one support for leaders, and more in-school support. When asked if the strong relationships developed amongst cluster lead teachers would enable their continuation of meetings each term, the general feeling was the need for an "external structure" to support these (Focus group interview, 2008), and there was uncertainty as to their schools' commitment to maintaining these. Continuation or further development of the cluster approach will not be viable without greater financial input from schools.

However, programme sustainability, finances aside, is possible. Throughout the three years of the programme there was careful documentation and ongoing evaluation of the GiftNet professional development. As the section on programme development and implementation shows, GiftNet has been documented, evaluated, and refined. Most of this is recorded in the quarterly milestone reports for GiftNet and on the website. Some of this has been disseminated at international conferences. This means that despite any resourcing issues, the professional development model is sustainable and potentially transferable. It is important now that the model be more widely disseminated and critiqued.

The final aspect of sustainability relates to the outcomes for stakeholders, and for the purposes of this evaluation, those are primarily the lead teachers. The changes in school-wide and classroom-based practices were often initiated and driven by the cluster lead teachers, and in the four schools in the evaluation some documentation and systems were being put in place to ensure their sustainability. As the final milestone indicates, the model of professional development targeted these lead teachers as key players in schools, aiming to arm them with greater skills of facilitation and leadership. The danger, however, is that school involvement of more than one teacher decreased, meaning the changes being made were reliant on a single individual in the school. There were varying levels of senior management involvement and support, as well as systems and documentation, in the four schools at the end of 2008. As one of the lead teachers stated, "We are at a vulnerable point" (Interview, 2008). Ongoing professional development and support, whether from GiftNet or other providers, will be critical in sustaining the school-wide and classroom-based practices, and in order to ensure greater teacher involvement, careful documentation, and evaluation within schools.

Summary: GiftNet

The summary of the results of this case study is provided in relation to the research questions.

How were decisions around the programme design arrived at? Who was involved in the decision-making process? How has this process impacted upon the sustainability of the programme?

The design of the programme was the work of the 2006 coordinators, but in contract negotiations with the Ministry of Education this design changed, seemingly with little input from the coordinators who were not part of the negotiation, and yet responsible for implementing the contract. Several factors influenced the decision-making regarding programme design: a team approach; partnerships with contributing schools' lead teachers; previous experience in designing and providing professional development; responsiveness to evaluation results; and emphasis on the translation of theory into practice. Over time the design shifted from professional development focused on teacher needs to a focus on GKP student needs. The design means that sustainability of the programme is heavily reliant upon Ministry funding, unless contributing schools and lead teachers are willing to contribute to its continuation and there is further development and commercialisation of professional development services.

What changes in the organisation's climate and philosophy have been required for the successful implementation of the professional development programme?

The Gifted Kids Programme has provided professional development for contributing schools since 2003 when it was initially funded as a TDI, and therefore had a philosophy which emphasised the need for professional development for contributing schools. The professional development was designed to ensure appropriate learning opportunities for GKP students more than just one day a week. In the 2006-2008 funding round the philosophy shifted from broad brush professional development to an approach tailored toward individual contributing schools and GKP students. For contributing schools this has led to the development of school-wide policies and procedures, stronger leadership teams, and access to high quality resources and expertise.

How comprehensive are provider initiated student and programme monitoring and evaluations? How do the findings of the monitoring or evaluation inform the programme?

The internal evaluation of the GiftNet programme was thorough and robust, employing multiple measures of quantitative and qualitative data and inclusive of all stakeholders, including gifted and talented students. More importantly, the information gathered was used to inform programme developments and changes, serving as more than just evidence of how GiftNet was meeting its goals.

How have resources and personnel impacted on the success or otherwise of the programme?

The physical and human resources of the GiftNet cluster programme was mainly comprised of the coordinators and mentors. The change in personnel after one year of the contract impacted the design and implementation of the programme, shifting it towards greater alignment with the intended outcomes. The programme coordinator's experience in teaching and professional development in gifted and talented education was critical in the success of this programme. However, the resources to support GiftNet, while possibly adequate for the original proposal aims and intentions, were not strong enough to deliver on all outcomes of the final Ministry contract.

What role has staff professional development played in achieving the programme goals?

The first Wellington-based coordinator, who had a strong background in gifted education theory and practice, but not as a provider of professional development, expressed desire for professional development and support. However, there were not any planned approaches to their professional development in the proposal or contract. The coordinators, however, engage in conferences and seminars, professional reading, and ongoing dialogue with other professionals. Using a team approach to coordination also meant there was an opportunity for the coordinators to support one another, but this was hindered by geographical distance and a lack of allocated time and resources. Despite this, the feedback from participating cluster teachers indicates that the lack of professional development and support did not hinder the programme.

How well has the programme planning occurred in regard to sustainability of the programme after the three-year period of funding ceases?

The professional development model is sustainable and possibly transferable because of the careful documentation and ongoing evaluation of the professional development. Yet, there was little evidence of efforts to maintain the financial viability of the programme, which was largely reliant upon the Ministry of Education funding. If schools want to continue accessing GiftNet services, it only seems logical that a stronger financial commitment is made to the programme. The participating schools were expected to provide teacher release for cluster meetings; the commitment to this varied and actually decreased over time. The sustainability of changes in school-based practices was often reliant upon a single individual in the school (the lead teacher). Ongoing professional development and support will be critical in sustaining the school-wide and classroom-based practices, and in order to ensure greater teacher involvement, careful documentation, and evaluation within schools.

What has been the impact been of professional development for its stakeholders?

Lead teachers developed knowledge and skills, greater awareness, enhanced leadership, greater confidence, and a better understanding of the GKP programme. Consequently their schools saw the development of school-wide policies and practices, implementation of differentiated programmes, access to high quality resources, and stronger leadership teams. For GKP students, there were increased school-based differentiated opportunities with like-minded peers and less fragmentation between their GKP and school learning experiences. To varying degrees the students' social and emotional needs were being identified and addressed. For GiftNet staff, the impacts were two-fold: greater understanding of the design, implementation, and evaluation of effective professional development; and stronger relationships with contributing schools which facilitated a sharing of GKP practices.

The Gifted Education Centre Professional Development for Teachers

Background

The Gifted Education Centre (formerly the George Parkyn Centre for Gifted Education) was established in 1995 as a registered non-profit charitable trust to promote professional support services for gifted children, their families, and teachers. The current website: New Zealand Centre for Gifted Education states that the Centre:

  • provides information and guidance for parents and for parent groups.
  • provides high quality professional development for teachers and other education professionals.
  • supports research work in this field.
  • runs the One Day school programme at venues throughout the country and the GO online programme for gifted children who are not close to a One Day school venue.
  • lobbies Government for better support for gifted children.
  • has contributed significantly to change and development at the national level through its lobbying, submissions and membership of the Ministry's Advisory Group (1999-2000) and the Minister's Working Party (2001).

The stated objective of this TDI was to provide effective professional development for teachers and related personnel in the area of gifted and talented education. The contract agreement between the Gifted Education Centre (GEC) and the Ministry of Education notes that the following understandings and principles will form the basis for the programme:

  1. Shared responsibility. The programme allows for a partnership between individual schools and the Contractor to assist schools in developing skills, which will ultimately lead to improved outcomes for gifted students.
  2. A culture of professional development. This programme will be founded on the beliefs that effective professional development:
    • should be centred on enhancing student learning
    • reflects best available research and practice in teaching, learning and leadership
    • is planned collaboratively to ensure ownership by those who will participate in and facilitate the development
    • involves whole-school change
    • requires substantial time and other resources
    • is driven by a coherent long-term plan
    • respects the knowledge and skills of participants and seeks to nurture further development of these
    • provides a range of professional development opportunities to meet different needs
    • focuses on individual, collegial and organisational improvement
    • is realistic in its expectations of participants
    • is ultimately evaluated on the basis of its impact on teacher effectiveness and student learning
  3. Successful professional development requires competent and knowledgeable facilitators. This programme allows for the continued development and coordination of a team of facilitators through twice-yearly meetings.
  4. Data gathering. Programme delivery is based upon a status analysis completed by participating schools, which outlines the needs of staff in the area. Evaluations will be completed by participants following delivery of workshops.
  5. Knowing that changes take time: planning for initiation, implementation, institutionalisation, and reflection.

The contract identified the two personnel who would be responsible for the programme, a Project director (.1) and Programme Development Coordinator (.4). The TDI was to be delivered between July 2006 and June 2009. The evaluation of the GEC TDI was completed six months prior to the completion of the actual programme.

The research to evaluate this TDI was undertaken by a researcher with extensive experience in both gifted and talented education and teacher professional development.

Research Methodology

Data Collection Methods

This research was qualitative and interpretive in nature and drew on a variety of data collection methods including observation, document analysis, milestone reports and interviews. The Enhance programmes were evaluated using a case study approach.

At the first meeting with the new director of the GEC in July 2006 the nature and scope of the evaluation programme was outlined. The collaborative nature of the research was explained and this was particularly salient to this initial discussion as both the TDI project director and the Programme Development Coordinator had resigned. The Programme Development Coordinator was also the Director of the George Parkyn National Centre for Gifted Education and this first meeting was with the recently appointed director. Following this meeting the researcher attended the Centre's Annual General Meeting and briefed its Board on the evaluation programme.

A return visit was made in November 2006 to determine what progress had been made and to negotiate a plan for evaluating the programme. At this stage, although information had been disseminated to schools and some single session workshops had been undertaken, the Centre had not been successful in attracting a school or schools to agree to a 'programme' of professional development. The researcher discussed with the Centre director a range of strategies to achieve 'buy in' to a programme. This informal professional input was provided before data collection had commenced and constituted professional advice more than research activity.

In the first part of 2007 time and effort had been devoted to advertising, exploring different approaches and developing materials. A full day of professional development had been undertaken in the Manawatu, with 15 schools represented. It was hoped that this might lead to a programme of ongoing school-wide professional development in one or some schools. One-off workshops had been presented in a number of other schools, also with the aim of attracting a longer-term commitment. There had been expressions of interest from individual schools and two clusters of schools and it was hoped that this would lead to ongoing programmes of professional development. However, by July there was still no ongoing programme in place to evaluate. In May 2007 the researcher met with the Centre director/Project director and the Programme Development Coordinator and at that meeting considerable time was spent exploring options and approaches to attract participation, and still prior to the formal data gathering process.

The researcher had a short meeting with the Centre director/Project director and the Programme Development Coordinator at the TDI hui in Auckland in November where he was updated on developments. While there was still no programme of the type described in the TDI contract at the end of 2007, the director reported that the GEC had secured contracts to work in three schools in 2008.

During 2008, a number of workshops and staff meetings were undertaken and most of these represented a second visit to schools and a school cluster, and GEC staff were optimistic that these could lead to a longer-term contract. However, GEC now only had agreements with two primary schools for school-wide and ongoing professional development. Although the researcher had been able to view the materials used for the short-term workshops, this presented the first opportunity to evaluate a 'programme' of professional development. In July he attended a staff workshop day at one primary school facilitated by the TDI programme coordinator. The focus for the researcher was both on the programme design and delivery and specifically outcomes for teachers and students. He was able to talk informally with the teachers about the session and the programme, but as it was in its very early stages, the feedback focus was mainly on that day's session. He was subsequently sent the teacher evaluations from that session.

The researcher had hoped to attend a session at the second school, but this was unable to be organised by GEC staff. He received a brief summary of developments at that school from the facilitator.

In summary then, despite the best efforts of GEC staff and the researcher, the actual 'observation' of the GEC TDI programme being delivered to teachers was limited to one day. The effort undertaken by GEC to get to this point was not inconsequential and they worked hard delivering short-term workshops and 'taster' sessions, which have to be recognised as legitimate professional learning activities. Feedback from participants was sought as part of the GEC internal evaluation procedures. However, since these sessions did not comprise the activities specified as part of the TDI, which aimed to deliver ongoing, school-wide programmes of professional development, they also fell outside this external evaluation.

Programme Development and Implementation

Designing the Programme

The design of this professional development programme reflected an appreciation of some accepted principles surrounding effective professional learning. These included school-wide approaches, programmes of professional learning that were ongoing rather than one-off, and the development of content based on the assessed needs of each individual school. The programme was designed to focus on institutional and organisational change but with an emphasis on improved outcomes for students. There was also a focus on inclusion, and on the principle that all teachers are teachers of the gifted and talented.

Professional development content and approaches in gifted and talented education are relatively predictable. Most follow a pattern of exploring a range of conceptual and definitional issues, which is followed by a focus on the characteristics of gifted and talented students and the potential strategies to identify them. This tends to lead on to educational options, programme differentiation, and evaluation. The GEC approach to professional development tended to follow this progression.

Programme Delivery

In hindsight, some of the difficulties experienced by GEC in meeting the terms of the TDI contract were somewhat predictable. Others were less predictable, but should probably be considered and investigated in negotiating a contract agreement. What was predictable was the difficulty that GEC would encounter in trying to obtain schools' agreement to sign up for a long-term school-wide programme of professional development, which would be delivered almost immediately. First, schools tend to plan their professional development programmes well in advance. While they may be attracted to short-term sessions or one-off staff meetings, to obtain a longer term commitment involving all or most staff typically requires a longer lead-in time than this contract allowed. Thus, the notion that schools would respond to these single session workshops with a commitment to ongoing school-wide professional development was very optimistic. Second, the Ministry of Education contracts School Support Services to deliver professional learning in gifted education to teachers in New Zealand schools. This free and nationwide service is delivered by advisors and has existed for more than six years. While it is unlikely that this service could meet the full demand from schools for professional development (especially in the years immediately following the change to the National Administrative Guidelines [NAGs] that mandated provisions for gifted and talented students), the work of the advisors would have considerably reduced the pool of schools that might be interested in professional development from another provider.

The contract itself offers no information about recruitment of schools and teachers. In the absence of any detail here there seems to be an assumption by both parties that recruiting schools was not going to be problematic. Those who have been involved in professional development over many years know that more recently, with so many competing initiatives presented to schools, it can be a challenge to attract their interest and involvement.

A less predictable factor that impacted on this TDI was the unexpected resignation of two members of the GEC TDI staff. What added to the difficulties here was that the person who was primarily responsible for the design of this programme was also the Centre director. The incoming director not only had to oversee the implementation of this contract, she also had to attend to all the other functions and activities of the Centre. Staff change is always a potential issue in contract agreements and while the risks associated with this cannot be eliminated in advance, they can be reduced. One possibility, with contracts such as the TDIs, is to ensure that each contract is formally supported by an advisory committee and this committee's role specified in the contract agreement.

However, while the opportunities to observe this TDI were extremely limited, the researcher is able to make some tentative comments. At the one session observed, the teachers were engaged enthusiastically with the material. This was the first day of the school holidays, but none seemed to begrudge attending. In conversations with the principal and teachers, it was clear that they were looking to make a difference for their gifted and talented students and they believed that the session observed and the overall programme would lead to changes in their practices. As stated previously, the focus of the discussions with teachers was about the relevance of the professional development to their classroom practices and the extent to which they considered these would enhance student learning. The evaluations received by the facilitator were positive and many teachers reported that even at this early stage they could see implications for teaching and learning.

This observed session explored more basic principles and practices in gifted and talented education. The session was informed by a needs analysis and the teachers did not give any impression that this was part of their existing knowledge so on that basis, the researcher concluded that it was pitched at the appropriate level for this audience. The session was interactive and this provided the facilitator with opportunities to respond to specific issues and questions, and to use that input to make adjustments to better match the needs of the participants. As a provider of programmes for gifted learners over many years, the GEC staff has a wealth of expertise and arguably a greater depth of experience working first hand with gifted children than many other providers. This was apparent in the examples and illustrations that the facilitator was able to draw on.

During the observed session teachers were challenged to apply some of the understandings or approaches with their students and share the outcomes from this at the next workshop.

Programme Evaluation

Each professional development session was evaluated by the participant teachers by way of individually completed questionnaires. Programme and workshop feedback was also sought from each school's Gifted and Talented Education (GATE) coordinator. The programme facilitators used this as formative data to inform subsequent workshops. The GEC took a team approach to all aspects of the TDI, including evaluation. There was no opportunity for the researcher to view evaluations of a completed programme of professional development as none had finished at the point that the research period ended.

Programme Resourcing

The costs associated with this TDI would appear to have been met by the budget allocation from the Ministry of Education, with the single largest expense being the employment of facilitators. At each workshop teachers were provided with accompanying resources. Some of these were used in workshop activities and others were of a more practical nature that teachers could use in their classrooms. There was also the provision of material, such as journal articles, that teachers could use outside the face-to-face sessions to further their own knowledge. From what was observed, the workshops would have been enhanced by the greater use of technology, particularly power-point for presentations, and incorporating relevant online resources into workshops.

Professional Support and Development

It appeared that most of the primary professional support for this TDI came from within GEC itself. This is not surprising, as the staff associated with the Centre have expert knowledge and a depth of experience in gifted education. In addition, the Centre has a well-established network of advisory support that it can draw on. The TDI staff, including the director, attended a number conferences and hui during the time the TDI was evaluated and presented on aspects of their professional development programme, and invited feedback from attendees.

As with all aspects of this TDI, the opportunities to observe and appraise how professional support and development impacted on the actual programme were limited. As noted previously, at the time the evaluation of this TDI was completed the Centre had commenced delivery of professional development in its first two schools and much of this work was still to be completed.

Outcomes for Teachers

Given the limited scope of this evaluation, it is difficult to fully ascertain the outcomes for teachers. However, from the session observed, the teachers spoken to, and the documentation viewed, the following positive points were noted that had the 'potential' to change teacher practice:

  1. There was recognition that professional development is most effective when it involves all staff in a school, is offered over a sustained period of time, and the facilitator is involved at the classroom level.
  2. A needs analysis was the starting point for planning each programme of professional development for each school.
  3. A multi-categorical approach to conceptualising giftedness and talent and a multidimensional approach to identification were presented.
  4. The gifted and talented were presented as a heterogeneous not a homogeneous group.
  5. Teachers' experience and knowledge was acknowledged and used as a basis for professional learning.
  6. There was an emphasis on improved outcomes for students.
  7. Differentiation was approached from a learner-centred perspective.
  8. There was some connection to recent research and best practice.
  9. There was a focus on the identification of gifted and talented students who are underachieving, and gifted and talented students with learning difficulties.

Impact of Programme

From what has been tabled above it could be expected that there would be some positive impact for teachers, schools, and students. However, as has been discussed previously, it is impossible to ascertain or even speculate with any degree of confidence the nature or extent of the impact this programme made.

Sustainability

The professional development of teachers is a legitimate and appropriate activity for the GEC to be involved in. The Centre works closely with the teaching profession, particularly with its One Day Schools. The staff is committed to making links between what students do during their day at the Centre and the four days in their schools. The experience Centre staff have in working with gifted children affords them knowledge and perspectives that teachers can benefit from. However, in this instance, sustainability would probably be enhanced if the professional development programme targeted the schools that the Centre's one-day students represented.

Future Directions

The following aspects could be considered for further development, although these are tabled very tentatively in recognition of the limited scope of this evaluation:

  1. The perspectives and approaches of Māori and other cultural groups on giftedness and talent.
  2. Making links between gifted and talented education and the revised New Zealand Curriculum, and Ministry of Education initiatives/developments such as Ka Hikitia, Te Kotahitanga and the Teacher Professional Learning and Development Best Evidence Synthesis report.
  3. Providing teachers with more recent examples of research and practice in gifted education.
  4. Making greater use of technology in professional development workshops.
  5. Evaluating the programme effectiveness against changes in classroom practice.

Summary: George Parkyn National Centre for Gifted Education Professional Development for Teachers Talent Development Initiative

How were decisions around the programme design arrived at? Who was involved in the decision-making process? How has this process impacted upon the sustainability of the programme?

The original proposal and the programme design were the work of the previous Director of the Gifted Education Centre, who had resigned before the TDI was implemented. This probably had quite an impact on the programme's sustainability as the newly appointed Director was required to implement a programme that was the vision of her predecessor, and alongside the numerous other tasks associated with taking up a new role. The programme was modified to reflect the difficulties experienced in obtaining 'buy in' from schools.

What changes in the organisation's climate and philosophy have been required for the successful implementation of the professional development programme?

The GEC's philosophical approach to gifted and talented education offers the basis for effective programme content for professional development in this area. The organisation has a long history of working with gifted and talented children and their families, as well as with teachers and schools. The organisation and the actual professional development facilitators were less experienced working in school-wide, ongoing professional development. If the organisation were to continue in this or a similar role, it would be beneficial for staff to engage with the literature and research around effective teacher professional learning (such as the Ministry of Education's 2007 publication, Teacher Professional Learning and Development Best Evidence Synthesis .)

How comprehensive are provider-initiated student and programme monitoring and evaluations? How do the findings of the monitoring or evaluation inform the programme?

There is limited scope to offer any definitive conclusions here, as the evidence gathered was so limited. The professional development facilitators demonstrated an understanding of the importance of gathering feedback from participants and using this to inform their work. At the stage the evaluation of the TDI concluded, the evaluations focused almost entirely on programme effectiveness rather than on shifts in improved outcomes for students. This is understandable, given that the two schools that GEC staff were working with were still in the early stages of their professional development programmes.

How have resources and personnel impacted on the success or otherwise of the programme?

The resources allocated to this TDI were primarily human, and as has been noted previously, at the point the evaluation was completed there was insufficient evidence available to draw even the most tentative conclusions about its effectiveness.

What role has staff professional development played in achieving the programme goals?

The staff leading the professional development programme both had a background in gifted and talented education. The extent to which this knowledge included a familiarity with more recent research in gifted and talented education, teacher professional development, and relevant trends and initiatives in general education is unable to be definitively ascertained. However, these 'appear' to be areas where staff professional development might be focused.

How well has the programme planning occurred in regard to sustainability of the programme after the three-year period of funding ceases?

This programme continued for six months beyond the period within which it was evaluated. There seemed to be a greater concern with delivering on this contract than focusing on the period beyond it. This is perfectly understandable, given the difficulties GEC experienced in obtaining a commitment from schools. The Centre has the human capacity and the background to legitimately offer programmes of professional learning to schools and teachers.

What has been the impact been of professional development for its stakeholders?

The data that was available came from programmes in two schools that were only partially delivered when the evaluation period ended. This allowed insufficient time to make any judgements on the impact of the TDI on its stakeholders. However, that is certainly not to imply that the programme had not or would not have a positive impact.

Synthesis of Results: Enhance Evaluations

As the case studies of the two Enhance professional development programmes show, the intended outcomes, indicators, and delivery of the two varied greatly. As with the Ignite evaluations, this meant that although a case study approach was used, with some formative elements, each one also differed in data collection methods, and, subsequently, data analysis and results. Across the two Enhance evaluations, there are no common generalisations that can be made with confidence. However, there are some important understandings about professional development which have arisen in response to the research questions.

Research Questions: Enhance Evaluations 

How were decisions around the programme design arrived at? Who was involved in the decision-making process? How has this process impacted upon the sustainability of the programme?

The decisions around programme design for both professional development initiatives were originally those of the programme directors with Ministry of Education input, but changes in personnel meant the implementation was carried out by different, newly appointed staff. Both programmes were modified as a result of these factors, and, to a lesser degree, in responsiveness to stakeholder schools. Changes in personnel can lead to changes in implementation and delivery; similarly, the changes between proposal and contract can lead to changes in actual implementation.

What changes in the organisation's climate and philosophy have been required for the successful implementation of the professional development programme?

Both programmes have an approach to gifted and talented education which lends itself to professional development – the offering of a one day a week programme for gifted and talented students. It is critical that this approach be extended to a philosophy that emphasises the need for professional development for contributing schools with students attending the one day a week programme. This is the case with GiftNet, which designed its professional development to ensure appropriate learning opportunities for GKP students more than just one day a week. This required a shift in their professional development philosophy from a broad brush approach to an approach tailored toward individual contributing schools, and ultimately individual GKP students. Experience in the delivery of professional development is also critical to the successful translation of philosophical ideals to practice. The potential results, as evidenced by GiftNet, are the development of school-wide policies and procedures, stronger leadership teams, and access to high quality resources and expertise.

How comprehensive are provider-initiated student and programme monitoring and evaluations? How do the findings of the monitoring or evaluation inform the programme?

Both programmes measured their effectiveness and used the results to inform and shape their professional development as appropriate. The evaluation methods were influenced by the programme's goals and delivery, which in turn impacted how results were used to inform programme developments and changes.

How have resources and personnel impacted on the success or otherwise of the programme?

The resources and personnel impacted the programmes differently, but in both cases the changes to personnel led to shifts in the design and implementation. Experience in professional development in gifted and talented education, as well as engagement with professional learning theory, also play important roles in implementation, as has been shown with GiftNet. There is a relationship between the intended outcomes of a professional development programme and the resources to enable the implementation of those, meaning that sufficient resources must be carefully matched to intended outcomes.

What role has staff professional development played in achieving the programme goals?

The coordinators delivering professional development in both these programmes needed a strong background not only in gifted education theory and practice, but also teacher professional learning and support. Neither programme showed evidence of planned approaches to the coordinators' ongoing professional development.

How well has the programme planning occurred in regard to sustainability of the programme after the three-year period of funding ceases?

There was little evidence of programme planning for financial sustainability. The sustainability of both programmes is heavily reliant upon Ministry of Education funding, unless participating schools are willing to contribute or there is further development and commercialisation of professional development services. GiftNet's careful documentation and ongoing evaluation mean the model of professional development is sustainable and possibly transferable. The GEC professional development was ongoing at the time this evaluation ended, however, all indicators showed a focus on delivery of the contract rather than the future.

What has been the impact been of professional development for its stakeholders?

In order to have a strong impact, the professional development must be linked directly to the programme and its stakeholders, namely the contributing schools which have students attending the one-day-a-week programmes. As the GiftNet programme demonstrates, this approach impacts upon contributing teachers, their schools, and, ultimately, gifted and talented students.

Summary: Enhance Evaluations

These case studies show that the provision of professional development in gifted and talented education should be underpinned by the following principles and practices:

  • Providers of professional development require their own professional development and support in effective models of delivery and best practices in professional development, complemented by contemporary theory, research, and practice in gifted and talented education.
  • Professional development facilitated by out-of-school providers may serve as a mediating link between schools and out-of-school programmes. In the case of one-day-a-week programmes, this has the potential to strengthen the relationships between and across a continuum of provisions by creating opportunities for shared responsibility for gifted and talented students.
  • Professional development should be based on a needs analysis, inclusive of all programme stakeholders. For professional development to be student-centred, as opposed to teacher-centred, gifted and talented students should have the opportunity to share their experiences, perceptions, needs, and so on.
  • Professional development for gifted and talented education must demonstrate connections with educational principles and practices in both mainstream and specialised education, reflecting national and international theory and research.
  • Gifted and talented education cannot be viewed in isolation, but needs integration with other educational initiatives; this may cause a tension between the provision of specialised and generalised professional development, and should be interpreted as a need for both.
  • Professional development needs to be evaluated rigorously for its effectiveness and the information gained should be used to inform programme changes. Interplay between the development, implementation, and evaluation of professional development should be evidenced.
  • Professional development should not be based on a singular delivery model, but is more effective using multiple strategies (e.g., whole staff, one-one-one, online).
  • Professional development needs to articulate with classroom practices and a continuum of provisions for gifted and talented students, and, as such, should be learner-focused.
  • Teachers value one-on-one, in-school support; however, if professional development is only delivered in this manner, there is potential for conflict with school philosophies and the recommended practice of school-wide professional development in gifted and talented education.
  • The relationship between professional development and outcomes for students is tenuous. Outcomes for students are impacted by teachers and their practices, but also by a number of other variables which must be taken into consideration.
  • Professional development delivered by external providers needs to reflect an understanding of a school's culture, including demands on staff time, competing interests, and other professional development programmes.
  • Professional development in gifted and talented education must be supported by senior management in schools through their direct involvement.

The Enhance TDIs were two very different professional development programmes, as their case studies show, and as such it is difficult to determine if both met the aims of the TDI funding. The first aim of the TDI funding was to develop innovative approaches that result in improved outcomes for students. Innovation was evidenced in the approach used by GiftNet, particularly because the Gifted Kids Programme is a one-day-a-week provider of programmes for gifted and talented students wanting to make stronger connections and share the responsibilities for GKP students with schools. The innovative piece of this approach was the provision of professional development by an out of school private provider, designed to strengthen the relationship between school-based and one-day-a-week programmes. This innovation holds promise in strengthening and enhancing developments for gifted and talented students across a continuum of approaches. Had this approach been adopted by the Gifted Education Centre, some of the problems with recruitment of schools may have been alleviated.

Weaving the core principles of gifted and talented education (Ministry of Education, 2002) into professional development practice, as outlined in the selection criteria for TDIs, is also somewhat difficult, as these relate to programmes and outcomes for gifted and talented students, rather than for their teachers. However, the Enhance programmes were able to infuse these principles in the knowledge and skills shared and developed with teachers. For example, GiftNet had a strong focus on helping teachers identify and understand the social and emotional needs of gifted and talented students. The professional development programme of GEC showed clear understandings of these principles.

Both Enhance programmes were developed by providers with prior experience and commitment to gifted and talented education, as was outlined in the criteria for selection of TDIs. The Enhance TDIs were programmes which had been previously funded, but were entering an innovative phase of an existing programme. This was not the case with both of these programmes: GiftNet was an extension of Gifted Edge, but the professional development offered by the Gifted Education Centre did not directly relate to its development of an online classroom or scholarships for students to attend the One Day School. Another aim of the TDI funding was to conduct research into the impact of innovative approaches on learning and teaching, and that was the overarching purpose of this study. This was achieved with GiftNet, but was not achieved within the existing timeframe in any depth in regards to the professional development of GEC.

Footnotes

  1. Two schools withdrew from the programme at the end of the first year. In 2007 and 2008 data was collected from individual participating schools and it was not deemed important to gather information from non-participating schools. This decision was based on the limitations of time and funding to support the research.