Ua Aoina le Manogi o le Lolo: Pasifika Schooling Improvement Research - Summary Report

4. Achievement Data Patterns

4.1 Achievement in the Focus Cluster

We collected individual student achievement data in reading comprehension (STAR) from schools in the focus cluster over two years (2007-2008). We analysed the achievement results in two ways:

• Rates of gain in achievement over two years
• Average achievement levels over two years

In particular we wanted to know if the rates of gain and achievement levels were influenced by gender, ethnicity (the four main ethnic groups), language spoken at home, first language learnt, school and/or starting achievement levels. The details of the analysis are contained in the Full Technical Report (Amituanai-Toloa, McNaughton, Lai & Airini, 2009).

There were three key findings:

• Significant accelerations in achievement for Pasifika students can be attained through Schooling Improvement. However, achievement is not yet at national norms.
• • Achievement for Pasifika students increased significantly over two years, with the average achievement moving from the “Below Average” band to the “Average” band (see Figure 1 ).
  • The average stanine achievement rose from 3.31 (SD = 1.51) to 4.12 (SD = 1.72) with an average effect size of 0.5. The effect size is comparable to overseas studies (Borman, 2005) for interventions over five years.

Figure 1:  Mean stanine scores for cohorts over two years (four time points).

• Rates of gain are primarily influenced by students’ starting achievement level.
• • Students that started the project with lower achievement levels (stanines 1 - 3) made greater gains than students that started the project with middle to high (stanine 4 - 9) achievement levels.
  • There were no gender or language effects on the achievement gains. In other words, males and females, and students who spoke different languages at home or whose first language was English, made similar gains over the two years. There were some minor effects by ethnicity and school but these appeared to have been influenced by starting achievement levels.
• As distinct from rates of gain in the cluster, four factors were found to influence the average achievement levels - gender, time lived in New Zealand, home language, and school.
• • Students that spoke mainly Pasifika languages and those that spoke two or more languages (Pasifika language as well as English) achieved significantly lower than the mainly English-speaking Pasifika students. However, in the Case Studies we found no evidence of an impediment to achievement for students who speak a language in addition to English. What we report here is a relationship between language to overall levels of achievement when collapsed across year levels and under a variety of classroom and schooling conditions within a cluster. 
  • The mean achievement levels for females were significantly higher than those for males.
  • The mean achievement levels for those that had lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than for those that had lived in New Zealand between one and five years.
  • The mean levels of achievement differed significantly between schools.

There are three important implications from the findings:

• The challenge is how to accelerate the achievement of the middle to high stanine students as well as the students in the lower achievement bands. This requires the ability to differentiate instruction and effectively deliver a differentiated instruction to the class.
• There is a need to examine both rates of gain and levels of achievement when evaluating interventions (McNaughton & Lai, 2009). The tests of effectiveness should be firstly, whether clusters are achieving accelerated rates of gain, and secondly, whether they are shifting distributions of achievement to match national expectations.
• Some groups of students may need to make greater gain if they are to ‘catch up’ with their peers. Male students that speak mainly Pasifika languages and those that speak two or more languages, and new immigrants, may need teaching programmes which accelerate achievement in English at a faster rate.

4.2 Achievement Across Clusters

Overall, it was difficult to determine the achievement of Pasifika students across the clusters because of differences in how data were stored, analysed and aggregated across clusters. The Pasifika Schooling Improvement – Policy Paper examines some of the systemic issues associated with this finding, such as how the self-governing context resulted in responsibilities being devolved to individual clusters/schools without extensive infrastructure support, and outlines policy implications for the Ministry (Lai, McNaughton & Amituanai-Toloa, 2009). The clusters should be commended for their innovation and courage to develop cluster databases and aggregated analyses, in a policy context where they did not have to. These clusters have de-privatised their results and created collaborative communities that critique and support each other to raise achievement. They in turn worked in partnership with the research team; thereby confirming the importance of learning communities when seeking improved student outcomes through schooling improvement.