Ua Aoina le Manogi o le Lolo: Pasifika Schooling Improvement Research - Final Report
Publication Details
The current project focuses on the effectiveness of Schooling Improvement initiatives for Pasifika. The purposes were to identify the practices that work to raise achievement and close the gaps for Pasifika students especially at the classroom, school and cluster levels; to find out how effective existing Schooling Improvement initiatives are in raising achievement for Pasifika students; and to provide information to help existing and new initiatives to improve their effectiveness for Pasifika students.
Author(s): Meaola Amituanai-Toloa, Stuart McNaughton, Mei Kuin Lai, and Airini with Rolf Turner, Deborah Widdowson, Rachel McClue, Selena Hsiao, and Maryanne Pale
Date Published: February 2010
3. Results
3.1 Phase One - Cluster Reports
This section of the report addresses the research question, “What is the achievement of Pasifika students in Schooling Improvement initiatives in New Zealand?” The most comprehensive answer to this question would involve collating student achievement data (at individual student level) from each cluster, creating a database across clusters and analysing the data from that database using a standardised format to enable comparisons across clusters. Financial constraints of this research project only allowed this method of collation and analysis for one of the nine clusters in this study. In that cluster, we collected the raw achievement data from individual students, created cluster databases and used those to analyse the data using a standard format. This cluster’s data (Cluster A) is reported in Section 3.2.
Given that we could not collate the data at individual student level in the other clusters, the research team collated cluster reports containing analyses of achievement data. Nine clusters provided us with information (coded as Cluster A to Cluster I). The clusters ranged from 5 schools to 30 schools, and covered a range of deciles in some clusters (e.g., one cluster consisted of schools from decile 1 to 10). We examined the quality of the evidence of student achievement from each cluster by checking the accuracy of the analyses including checks on a sample of the raw data against the information contained in the reports. Clusters were also asked to report on any standardisation of the test at cluster and school levels, and to report on any cluster or school-wide mechanisms for checking the accuracy of the data. Given the amount of data collected by clusters, we limited the reports to literacy, and in secondary school to Years 9 and 10. This analysis yielded rich data about the analyses and databases in the clusters and we have provided an additional analysis of the state of the evidence about student achievement. A full description of the participants and methodology is contained in the Method section (Section 2).
The results are presented in three main sections. The first section presents the results from the verification of the data reports including the reasons for the current data reports and any changes that have resulted from the feedback with clusters; the second section presents achievement data from clusters with stronger evidence of achievement; and the final section presents achievement data from clusters with weaker evidence that the research team could reanalyse to provide some indication (albeit tentative) of Pasifika achievement. The latter results need to be interpreted with caution.
3.1.1 Verification of the data reports
We first examined the quality of the achievement information in the data analysis reports by analysing the accuracy of the analysis reports. We requested that clusters provide us with a copy of their cluster plans to understand the rationale for the analyses and we further asked clusters to reflect on the quality of their data (Section 2.1.3 and 2.4.1).
Checks of the quality of the achievement information revealed that three of the nine clusters showed stronger evidence of student achievement. By this we mean:
- the data reports contained accurate and appropriate analyses, and the conclusions drawn (if any) were appropriate
- the databases which data reports were drawn from were ‘clean’ (e.g., no columns in the wrong place; data primarily correctly entered)
- there was some form of cluster and/or school-wide standardisation and moderation of the results.
Some minor errors were found, but these did not change the overall patterns of results. The results are described in Section 3.1.2.
Two of the three clusters had previous research publications on the results of their data which were externally reviewed in both international and local publications (Lai & McNaughton, 2008; Lai, McNaughton, Amituanai-Toloa et al., 2009; Lai, McNaughton, Timperley et al., 2009; McNaughton & Lai, 2009; McNaughton, Lai, MacDonald, & Farry, 2004). The research publications contained higher-level analyses such as Hierarchical Linear Modelling (HLM) and included rigorous research designs that were more than just pre-post designs, such as quasi-experimental designs with comparison groups. Note that pre-post designs (e.g., where improvements are reported by examining the same students at different time points) are often associated with higher gains in achievement due to their less rigorous research design (Borman, 2005). More rigorous research designs are quasi- or full-experimental designs.
The other six clusters showed evidence of Pasifika student achievement, but the quality of the evidence was not as strong as the aforementioned three clusters. These six clusters experienced some issues with at least two of the three criteria we used to verify the data (accurate and appropriate analyses, ‘clean’ databases, cluster and/or school-wide standardisation and moderation of the results).
For ethical purposes, and to protect the anonymity of the clusters, we have not provided the detailed issues for each cluster here – each cluster has received feedback on its specific issues and that information will be used formatively. Table 10 provides a summary of the common issues across the six clusters. It is noted that almost all clusters provided very little information about Pasifika students (one to two figures and tables) per report.
| Issue | Examples |
|---|---|
| 1. Incomplete and inconsistent demographic information in the databases | Ethnicity information was collected for only 58.33% of the students, making the data on Pasifika less likely to be representative of the cluster Ethnicity data was collected inconsistently between schools (e.g., two schools did not submit any ethnicity information at all; whilst other schools provided some ethnicity information for lower levels of the schools only) Gender of the students was not recorded Year levels were not recorded Time of tests was not recorded |
| 2. Inconsistent collection of achievement information | Schools in the cluster chose to test at different time points, yet the data were analysed as though the difference in testing times did not matter (e.g., some schools in the cluster tested at the end of the year whilst the rest tested at the beginning of the year. Some schools’ data, therefore, would take into account any drops in achievement over summer, whilst others would not) |
| 3. Incorrect analyses or conclusions | Conversion into stanine scores was incorrectly done for the whole cluster Incorrect conclusions about the data – the research team conducted our own calculation of the data from the data table provided and could not find evidence to support the report’s conclusion Data tables were incorrect (e.g., we discovered that when reporting in bands, percentages were not correctly added) |
| 4. Databases not well constructed | Labels and descriptions of variables were missing or incorrect Data in the databases did not correspond to the file names, thus providing misleading information about the content of the files (e.g., the file indicated that the data were only from schools with all three points of data, but some schools in that file did not have three points of data) Database provided was structured in a form whereby the research team could not reconstruct to even determine the number of students in the cluster Storage of data did not allow for longitudinal tracking Student names were not recorded consistently (e.g., without last names) making longitudinal tracking impossible |
| 5. No ‘official’ cluster data | Data was collected from a non-representative sample of schools (e.g., high and low achieving classrooms) making the data less representative of the cluster Data provided from individual schools had differing formats and labels, making any cluster picture difficult for the research team to reconstruct (e.g., in one cluster the research team received 51 files with different file naming conventions) |
| 6. No cluster or school-wide standardisation for administering the test or checking for accuracy | Self reports from cluster leaders indicated that their cluster did not carry out cluster and/or school-wide standardisation or moderation of the assessments |
| 7. Appropriate statistical analyses not performed | Reporting of percentages without numbers (percentages on small samples are potentially misleading) No statistical testing of data, particularly those where claims about achievement gains are being made Missing standard deviations Mean scores not calculated, even though it would be appropriate to do so |
Reasons for the data reports
The research team investigated why the cluster analysis reports and databases looked the way that they did. There were a set of inter-related reasons summarised in Figure 2 .
Figure 2: Summary of inter-related reasons for the data issues.
Discussions with senior Ministry officials and Ministry Schooling Improvement Coordinators indicate that currently there is no standardised way of storing, checking the accuracy of and analysing data across clusters. Nor is there any Ministry of Education requirement that databases or data reports be standardised across clusters. As such, clusters (or the cluster-appointed analysts) have the autonomy for deciding the shape of the cluster databases and the types of analyses that are conducted. This means that the clusters can choose which subgroups to focus on, if any, and not all clusters had specific goals for Pasifika students. The focus of some clusters appeared to be on all students who needed support regardless of ethnicity.
In addition, there is no consistent system across clusters for the quality assurance of the databases or the data analysis reports. Discussions with two senior Ministry officials indicated that the main quality assurance by the Ministry in Wellington is whether the cluster plan and the cluster activities are based on the achievement data (e.g., cluster activity addresses students’ need identified from achievement data). There is further quality assurance but this is usually focused on specific Ministry priorities, for example, checking that the clusters are tracked against the goals in Ka Hakitia or the Pacific Education Plan. The Ministry generally assumes that the data provided by the clusters is valid, and that the clusters along with local Ministry staff have developed ways of checking the data.
Quality assurance functions are left up to individual clusters and their associated Ministry staff. This is, therefore, highly dependent on the individuals whose responsibility it is to check the analyses and the databases, and it assumes that the individual responsible has the expertise to perform the appropriate quality assurance.
In the six clusters with weaker evidence of achievement for Pasifika students, it may be the case that the role of quality assurance was not clearly delineated. It could also be that there were insufficient resources (including time) allocated for this task and/or possibly a lack of expertise to check the quality of the data and the databases. There is some evidence to support these hypotheses. For example in the six clusters, the individuals responsible for data management and analysis were primarily employed to perform other functions. In one cluster, the individual was hired primarily as a Cluster Coordinator with a data management and analysis component subsequently added to their role. As such, this person may not have had the time or the expertise required to be a data manager and analyst in addition to their primary role as a Cluster Coordinator.
As a consequence, clusters collect, store and analyse their data as they see fit resulting in a proliferation of database types and a variety of analyses that are not standardised across clusters. In addition, in all clusters the focus of the analysis was not specific to Pasifika students. Quality assurance is ad-hoc and dependent on the personnel in charge of data management and analysis, and as such, there is variation in the quality of the databases and data reports. The Ministry and the clusters, however, appear to have accepted all data from databases as valid and accurate. There are some exceptions with three clusters reporting to us that they were aware of the issues with their databases and were in the process of rectifying these issues.
Individual feedback with cluster members responsible for database management and analysis and Ministry staff indicated that the researchers’ analyses of the data reports and the reasons for the data reports reflected the current situation in the cluster. Suggested changes to the constraint set were additions rather than modifications to the original analysis, and these changes are reflected in the analysis presented in this section of the report.
The current state of databases and data analysis systems should not be surprising given the self-governing context, where the responsibilities for developing and creating databases and aggregated analyses are devolved to individual clusters/schools with little guidance from central government on how to do so. In fact, the schools and clusters should be commended for their innovation and courage to develop cluster databases and aggregated analyses when they did not have to, and without extensive infrastructure support. These schools and clusters have de-privatised their results and created collaborative communities that critique and support each other to raise achievement. It may also be that the local innovations have been far more fruitful in developing aggregated data for learning purposes than the top-down models like other countries, which have been often misinterpreted and used for irrelevant compliance or non-productive competition between schools.
Changes to cluster databases
This research project was not specifically set up to monitor changes to the databases or analyses after the feedback, or to work with clusters on how to address cluster issues - although informal support to clusters has been provided (e.g., an extra meeting with one cluster to discuss their database needs). All clusters have since made changes to their databases and analyses following this research. The following summarises some of the key changes across the clusters where known:
- the cluster has standardised testing times to twice a year to allow for comparisons within and between years
- recommended statistical testing (e.g., effect sizes) conducted and shared with principals in a meeting where a member of the research team was present
- the cluster is writing explanations of variables so that new members can understand the databases
- role of Database Manager and Analyst split from Cluster Coordinator role
- errors in database are being corrected
- ethnicity data is now being collected systematically in the cluster.
Further support through the Building Evaluative Capability in Schooling Improvement project should enable clusters to continue improving the quality of their databases.
3.1.2 Clusters with stronger evidence of achievement
Three of the nine clusters that provided achievement data exhibited stronger evidence of student achievement. In this section we summarise the cluster characteristics and results for two of these three clusters, namely Cluster D and E. The third cluster (Cluster A), which is also a Focus Cluster, is reported in Section 3.2.
Cluster D
Cluster D implemented the STAR test to examine its achievement in reading comprehension in Years 4 - 8. The target was for every student to reach the average of stanine 5, but there was no specific target for Pasifika students as the cluster consisted primarily of Pasifika students. The cluster engaged the services of the University of Auckland to support them in their analysis of the cluster and individual school data.
Two types of reports (in the form of PowerPoints) were produced each year. The first report assessed students’ achievement over three time points across two consecutive school years (e.g., beginning and end of 2006 and beginning of 2007) which enabled the cluster to examine any drop in achievement over summer. The second report assessed students’ achievement during an academic year (e.g., beginning and end of 2006) to examine the effects of the cluster intervention. There was also a published article in which higher levels of statistical analyses such as HLM were used to demonstrate growth in achievement in the cluster (McNaughton & Lai, 2009). In addition to the cluster analysis reports, the research team viewed the cluster database which tracked individual students over four years and the cluster plans for each year.
The cluster collected data at the individual student level in a way that allowed for cohorts of matched students to be tracked over time, and for the matched students to be compared to those that did not sit the test. There was no information, however, on why students missed a test (e.g., due to absence or leaving the school). The databases employed consistent ways of recording and storing the achievement information every year and a list of variable names was available. The cluster’s achievement reports were similar in format, with a demographic section followed by the analysis of the results.
The cluster reported both cluster-wide and school systems in standardising the administration of the STAR test. Data accuracy was checked both within each school and across the whole cluster. Data was then rechecked by the researchers and further checked at the time of publication. The publication provided external checks on the method and interpretation of analysis results.
Given that the cluster was not focused specifically on Pasifika students as a collective group, and to more clearly show the pattern of achievement for Pasifika students, the research team reanalysed the cluster’s data. Results from the analysis were similar to those in the analysis reports. Results showed that Pasifika students had lower achievement levels than non-Pasifika students but Pasifika female students achieved similarly to the non-Pasifika students. This can be seen by the overlapping lines of Pasifika females and the non-Pasifika student average in Figure 3 .
Figure 3: Mean stanines by gender and ethnicity over three time points years (Cluster D).
Overall, a gain of 0.37 stanine was made in 2006 (roughly equating to a four month acceleration in addition to nationally expected progress), and a drop of 0.12 stanine occurred over the summer (Table 11 ). Both Pasifika and non-Pasifika males and females improved significantly, although across the cluster, Pasifika females had consistently higher achievement levels than males (nearly one stanine difference in 2007). It is important to note that Pasifika female achievement was within the average bands of achievement at the end of 2006 and the beginning of 2007, whilst Pasifika males were on average still in the below average band.
| * p < .05, ** p < .01, *** p < .001. | |||||||||||
| Ethnicity / Gender | Time 1 Term 1 2006 | Time 2 Term 4 2006 | Time 3 Term 1 2007 | School Year (2006) | Summer (2006 - 2007) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| t | ES | t | ES | ||||||||
| Pasifika - Male | M | 3.03 | 3.41 | 3.16 | -5.15 | *** | 0.27 | 3.37 | *** | -0.17 | |
| SD | 1.42 | 1.43 | 1.51 | ||||||||
| n | 152 | 152 | 152 | ||||||||
| Pasifika -Female | M | 3.68 | 4.06 | 4.03 | -4.30 | *** | 0.29 | .423 | 0.00 | ||
| SD | 1.38 | 1.25 | 1.47 | ||||||||
| n | 145 | 145 | 145 | ||||||||
| Pasifika -Average | M | 3.35 | 3.73 | 3.58 | -6.64 | *** | 0.27 | 2.62 | ** | -0.10 | |
| SD | 1.44 | 1.38 | 1.55 | ||||||||
| n | 297 | 297 | 297 | ||||||||
| Non-Pasifika - Male | M | 3.33 | 3.88 | 3.73 | -2.27 | * | 0.37 | .85 | -0.10 | ||
| SD | 1.58 | 1.40 | 1.54 | ||||||||
| n | 40 | 40 | 40 | ||||||||
| Non-Pasifika - Female | M | 4.09 | 4.33 | 4.33 | -1.75 | 0.16 | .00 | 0.00 | |||
| SD | 1.62 | 1.48 | 1.47 | ||||||||
| n | 55 | 55 | 55 | ||||||||
| Non-Pasifika - Average | M | 3.77 | 4.14 | 4.07 | -2.86 | ** | 0.24 | 0.62 | -0.05 | ||
| SD | 1.64 | 1.46 | 1.52 | ||||||||
| n | 95 | 95 | 95 | ||||||||
| Overall - Male | M | 3.09 | 3.51 | 3.28 | -5.38 | *** | 0.29 | 3.31 | ** | -0.16 | |
| SD | 1.45 | 1.43 | 1.53 | ||||||||
| n | 192 | 192 | 192 | ||||||||
| Overall - Female | M | 3.80 | 4.14 | 4.11 | -4.61 | *** | 0.24 | 0.37 | -0.02 | ||
| SD | 1.46 | 1.32 | 1.47 | ||||||||
| n | 200 | 200 | 200 | ||||||||
| Overall - Average | M | 3.45 | 3.83 | 3.70 | -7.07 | *** | 0.26 | 2.58 | * | -0.08 | |
| SD | 1.50 | 1.41 | 1.56 | ||||||||
| n | 392 | 392 | 392 | ||||||||
The three main Pacific ethnic groups (i.e., Samoan, Tongan, and Cook Island Māori) made significant progress within the school year and only Samoan students exhibited a significant drop in achievement level over the summer holiday (Table 12 ).
| Ethnicity | Time 1 Term 1 2006 | Time 2 Term 4 2006 | Time 3 Term 1 2007 | School Year (2006) | Summer (2006 - 2007) | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| t | ES | t | ES | |||||||
| Samoan | ||||||||||
| M | 3.37 | 3.78 | 3.61 | 5.12 | *** | 0.29 | -2.29 | * | -0.11 | |
| SD | 1.43 | 1.42 | 1.55 | |||||||
| n | 161 | 161 | 161 | |||||||
| Tongan | ||||||||||
| M | 3.23 | 3.63 | 3.33 | 2.65 | * | 0.31 | -1.73 | -0.24 | ||
| SD | 1.34 | 1.22 | 1.34 | |||||||
| n | 43 | 43 | 43 | |||||||
| Cook Island Māori | ||||||||||
| M | 3.23 | 3.65 | 3.49 | 4.30 | *** | 0.29 | -1.72 | -0.11 | ||
| SD | 1.48 | 1.39 | 1.63 | |||||||
| n | 77 | 77 | 77 | |||||||
| Other Pasifika | ||||||||||
| M | 4.00 | 3.81 | 4.38 | -0.68 | -0.13 | 2.06 | 0.38 | |||
| SD | 1.51 | 1.42 | 1.59 | |||||||
| n | 16 | 16 | 16 | |||||||
| * p < .05, ** p < .01, *** p < .001. | ||||||||||
Cluster E
Cluster E used STAR to examine its achievement in reading comprehension in Years 4 - 8. The cluster aimed to “enhance the achievement outcomes for all students and target the achievement of Māori and Pasifika students”. There were, however, no specific targets for Pasifika students in the reports provided to the researchers. The target for 2009 was “77% of Year 3 - 8 students achieving within the STAR “average” stanine band or higher (stanine 4 - 9)”. The cluster has recently engaged the services of the local university to support them in their analysis of the cluster and individual school data.
Two reports were produced per year between 2005 and 2007. The first reported students’ achievement at the beginning of a school year (e.g., March 2007), and the second report assessed students’ achievement at the end of a school year (e.g., November 2007). These reports provided projections of student achievement progression to examine whether cluster aims would be achieved. In addition to the nine cluster analysis reports, the researchers also received over 150 data files for individual schools from 2005 to 2007.
The cluster collected data at an individual student level in a way that would allow for cohorts of matched students to be tracked over time. The cluster had some excel files titled “tracked data” where the analyst matched students who had sat all the required tests in one year. In those files, however, there was missing data (e.g., subtests for some students) and some columns were not labelled, thereby making the matched data more difficult to interpret.
Data from schools were in two forms: the first were data at an individual student level in 2006 and 2007 (but not 2005), and the second were in the form of frequency charts (e.g., frequency counts of students grouped by ethnicity, gender and stanine level). The data files had consistent ways of recording and storing the achievement information even though lists of variable names were not available (i.e., there were no explanations of what the database labels meant). The cluster’s achievement reports were all similar in format and followed a structure similar to a standard scientific report with an Executive Summary at the front, followed by cluster demographics, analysis of the data, summary of findings, recommendations to the cluster, and lastly a technical report. Higher level statistical analyses (linear regression for achievement projects) were performed by the statistician that worked with the cluster. Data gathering and quality assurance of the dataset were of good standard, although not completely free of calculation errors (e.g., the numbers of students in one frequency table sampled for checking did not match the number of students in the database).
While the cluster reported a cluster-wide system for standardising the administration of the tests, there was no school-wide standard in administrating the tests. Data accuracy was checked within each school and across the whole cluster against the gender and ethnicity of the enrolment data.
The researchers did not receive a collated database for the whole cluster over the three years; rather, over 150 separate school files were provided. Matched student data across two school years was provided but records were separated by year levels in separate worksheets, and there were some missing data and missing column names. Given the financial constraints involved in re-analysing the data (i.e., having to create a cluster database from the separate files) and given the high quality of the analysis reports, the researchers did not re-analyse the data. Instead, the results presented here were extracted from Cluster E’s most recent analysis reports to the cluster.
Over time an upward trend in achievement is evident. Figure 4 shows the trend graphically whilst Table 13 displays the mean stanines by ethnicity group. (Note that as the report did not cite number of students or standard deviations our analysts could not, therefore, use the matched datasets to recalculate the number of students or standard deviations from the separate school files due to financial and time constraints.) The data in Figure 4 and Table 13 are not of matched students, therefore we cannot conclude from the cluster data if gains had been made because individual students were not tracked over time. The differences in achievement over time are conflated with student absenteeism and transience, and it is not clear how student absence and transience influenced the results.
| Ethnicity | 1/03/2005 | 30/11/2005 | 1/03/2006 | 30/11/2006 | 1/03/2007 | 30/11/2007 |
|---|---|---|---|---|---|---|
| Pasifika | 3.58 | 3.81 | 3.77 | 3.94 | 3.69 | 4.02 |
| European | 4.91 | 5.59 | 5.32 | 5.65 | 4.95 | 5.55 |
| Māori | 3.7 | 4.01 | 3.94 | 4.42 | 3.92 | 4.19 |
Pasifika students achieved at a lower level when compared to New Zealand European and Māori students, and the gap did not appear to close over time. On average, Pasifika students achieved below national expectations, with the exception of the end of 2007 when their achievement levels were within the average band.
Figure 4: Mean stanines by ethnicity groups over three years (Cluster E).
The cluster tracked the achievement of both the same students across years for Years 3 - 4 and Years 7 - 8, as well as students within year levels. As indicated in Figure 5 and Figure 6 , Pasifika students showed lower achievement levels than New Zealand European students but achieved similarly to Māori students in Years 3 - 4 and Years 7 - 8. Pasifika students, like other ethnic groups, made accelerations during the school year. There did, however, appear to be a summer effect in Years 3 - 4 ( Figure 5 ). Both male and female students of the different ethnicities examined dropped in achievement over summer. In Years 7 - 8 this drop in achievement over summer (Figure 6 ) was slightly more pronounced for girls than boys.
Figure 5: Mean stanines by gender and ethnicity for tracked Year 3 cohort (Cluster E).1
Figure 6: Mean stanines by gender and ethnicity for tracked Year 7 cohorts (Cluster E).2
3.1.3 Clusters with weaker evidence of achievement
This section summarises the results for clusters with weaker evidence of achievement. To protect cluster anonymity, we have not named the specific issues and errors found in the dataset and/or reports through our verification process. The specific issues have been fed back to clusters that are now in the process of cleaning their databases. Given the issues identified within these clusters, the results in this section must be interpreted with caution.
In this section of the report we summarise the achievement results of only four clusters. (Appendix P contains tables showing the achievement results for each cluster.) We are unable to report on two clusters for the following reasons:
- One cluster did not provide any cluster data at the age levels we were examining as the data at those levels was sampled from high and low achieving classrooms and was, therefore, not representative of the cluster (designations of high and low were made by the school). There was cluster data at the other age levels; however, we were not focusing on those levels in this report.
- One cluster did not provide us with sufficient data to verify their reports (e.g., no cluster analysis report was provided) and as such they were excluded from this section of the analysis. The cluster leader reported that there were neither cluster-wide nor school-wide systems for standardising the administration of the tests. Data accuracy was checked across the cluster but not in each school.
It is also worth noting that the researchers had to reanalyse the data in three of the four clusters to clearly show the achievement of Pasifika students. (We could not reanalyse the data in one cluster because of the volume of data and the structure of the data files.)
There were some general trends in achievement which are summarised here:
- Pasifika student achievement was lower than national norms across the four clusters, although there was variation across clusters in the ‘distance’ from national norms. For example, in one cluster the Pasifika students in 2006 were at stanine 4.18 (SD = 1.58), whilst in another cluster the Pasifika students in 2006 were at stanine 2.67 (SD = 1.36).
- Across clusters the amount of progress varied.
In two clusters there were accelerations in achievement for the majority of cohorts (e.g., higher than the expected progress in asTTle, statistically significant progress in STAR).
One cluster did not make expected progress for Pasifika students when compared to the asTTle norms.
One cluster made expected progress; however, the levels of achievement were roughly two years behind national norms. Therefore, accelerations in achievement are required if the cluster is to progress achievement for Pasifika students further.
-
In general, where we had sufficient data to examine gender and ethnicity, Pasifika males achieved less well than Pasifika females. For example, in one cluster Pasifika males scored stanine 2.31 (SD = 1.17) compared to stanine 3.00 (SD = 1.46) for Pasifika females. Non-Pasifika males scored stanine 3.68 (SD = 2.03) and Non-Pasifika females 4.23 (SD = 1.82).
3.1.4 Conclusions: What do we know about Pasifika achievement across clusters?
Our researchers were unable to provide as much data on Pasifika achievement across the clusters as we would have liked given the cluster databases, the analyses produced and the time and finance required to re-analyse the information. Clusters, however, have been cleaning their databases and conducting new analyses since our initial analysis. It must also be emphasised that clusters may have made significant improvements in achievement. Cleaning the databases and reanalysing the data would help clusters clearly demonstrate the improvements made. The Pasifika Schooling Improvement – Policy Paper (Lai, McNaughton & Amituanai-Toloa, 2009) expands on policy implications and provides recommendations to the Ministry. The Building Evaluative Capability in Schooling Improvement Project will further support schools in the analysis of their own data, as well as produce a picture of achievement gains across clusters.
Three of the nine clusters had stronger evidence of achievement. (One cluster, the Focus Cluster, is reported in a later section.) Data in one non-Focus Cluster suggested increasing improvements in student achievement of up to four months in addition to expected national progress, with Pasifika females at the expected national band (stanine 4) by 2007. Pasifika males, on the other hand, remained below national expectations and were much lower than Pasifika females. The other non-Focus Cluster with strong evidence of achievement showed an upward trend in achievement for their cohorts which were not matched, albeit the gap between Pasifika student achievement and other ethnic groups (in particular, New Zealand European students) remained large. Pasifika achievement was within the national bands by 2007 (although the data provided did not allow us to examine any gender differences). The cluster tracked achievement of the same students across years for Years 3 - 4 and Years 7 - 8 and also tracked students within year levels. Pasifika students achieved lower than New Zealand European students and similarly to Māori students in Years 3 - 4 and Years 7 - 8. Pasifika students, like students of other ethnicities, made accelerations during the school year. There did, however, appear to be a summer effect in Years 3 - 4 where achievement dropped over the summer months.
3.2 Phase Two
This section seeks to answer Research Questions 2 - 4:
- What differences, if any, occur between the gains in the Schooling Improvement initiatives for different student groups within Pasifika (ethnicity, gender, generation in New Zealand, language)?
- What are the practices in schools and initiatives that work, and the practices that do not work, for Pasifika students and under what conditions?
- What are the barriers to schools achieving positive learning outcomes for Pasifika students?
For the Cluster A achievement analyses (section 3.2.1) we used the Years 4 - 8 cohorts as this was what the scope of the project covered. As the main focus was on Years 4 - 8 and not Year 9, for the general statistical analysis of Cluster A we excluded the fifth cohort. In order to increase the power of statistical modelling we included as much data that were available, therefore section 3.2.2 includes students who were Year 9 in 2008. This additional cohort was Year 8 in 2007 and Year 9 in 2008. We included as many cohorts in the data modelling as possible, because part of our interest was to see whether cohort made a difference to student achievement. Further to this, the fifth cohort in the data modelling came from a middle school and this provided the capability of looking at school-to-school differences. Without the fifth cohort this school would have had a very small sample size and we wanted to use a more representative sample for their mean. School difference became more apparent when looking at the overall mean in the data modelling.
3.2.1 Cluster A results
A total of 649 Years 4 - 8 Pasifika students sat all four tests in 2007 and 2008 (Pre-test 2007, Post-test 2007, Pre-test 2008, Post-test 2008). These students came from six schools in the cluster (one school in the cluster declined to be involved in this project). The cohort was further separated into subsets: Cohort 1 (Year 4 2007 - Year 5 2008), Cohort 2 (Year 5 2007 - Year 6 2008), Cohort 3 (Year 6 2007 - Year 7 2008) and Cohort 4 (Year 7 2007 - Year 8 2008). The largest cohort was Cohort 4, with 254 students (39%), followed by 147 students in Cohort 1 (23%), 146 students in Cohort 2 (22%) and 102 students in Cohort 3 (16%). The reason for Cohort 3 being a smaller proportion of the total sample is due to the inclusion of one contributing primary school and two intermediate schools who do not have students in Year 6.
Nearly half of these students were Samoan, with 305 students (47%). There were 175 Tongan students (27%), 108 Cook Island Māori students (17%), and 61 students from ‘Other Pasifika’ groups (9%) including 48 Niuean students (7%), 9 Tokelauan students (1%), and 3 Fijian students (less than 1%). Only one student was from a Pasifika group other than those listed here. There were 349 females (54%) and 300 males (46%).
As a baseline, the mean stanines of all students present at Pre-test 2007 (including those who sat all tests plus those who sat some but not all tests) can be broken down by ethnicity. ‘Other Pasifika’ groups had the highest mean stanine (M = 3.54, SD = 1.58), followed by Samoan (M = 3.47, SD = 1.53), Cook Island Māori (M = 3.15, SD = 1.51) and Tongan (M = 3.01, SD = 1.38). Overall the mean stanine was 3.29 (SD = 1.50).
Achievement
As seen in Figure 7 and Table 14 , the percentage of students in the higher bands increased and the percentage in the lower bands decreased from Pre-test 2007 to Post-test 2008, which indicates an improvement in achievement. At both time points, however, there were fewer students in the higher bands than national norms.
These differences in the distribution between Pre-test 2007 and Post-test 2008 were tested using the chi square (χ2) test. This was found to be significant (χ2 (df = 5, N = 649) =194.01, p < .001). Additionally, the distribution at both Pre-test 2007 and Post-test 2008 was found to be significantly different from national norms ( χ 2 (df = 5, N = 649) = 545.77, p < .001 for Pre-test 2007 and χ 2 (df = 5, N = 649) =131.58, p < .001 for Post-test 2008). As seen in Figure 7 , at both time points there were more students in the lower bands and less in the higher bands than national norms, indicating that although the distribution is slowly moving toward the national norm distribution, greater improvements are still needed.
Figure 7: Mean percentages of students scoring within achievement bands at Pre-test 2007 and Post-test 2008 (Cluster A).
| Low (Stanine 1) | Below Average (Stanine 2-3) | Average (Stanine 4-6) | Above Average (Stanine 7-8) | Outstanding (Stanine 9) | |
|---|---|---|---|---|---|
| Expected % | 4 | 19 | 54 | 19 | 4 |
| Expected (number) | (25.96) | (123.31) | (350.46) | (123.31) | (25.96) |
| Pre-test 2007 % | 12.79 | 44.99 | 39.91 | 2.31 | 0.00 |
| Pre-test 2007 (number) | (83) | (292) | (259) | (15) | (0) |
| Post-test 2008 % | 7.55 | 28.81 | 55.62 | 7.40 | 0.62 |
| Post-test 2008 (number) | (49) | (187) | (361) | (48) | (4) |
Note that in all χ2 tests, individual stanines from 1 to 5 were included, and stanines 6 - 9 were collapsed into one band. This was necessary as each stanine band needed to contain at least 5 students for valid statistical analyses.
As seen in Table 15 , Table 16 and Figure 8 , achievement improved significantly from Pre-test 2007 to Post-test 2008, and throughout each academic year (Pre-test to Post-test 2007 and Pre-test to Post-test 2008) for all cohorts.
| ***p<.001. **p<.01. *p<.05 | |||||
| Pre-test 2007 | Post-test 2008 | t | d | ||
|---|---|---|---|---|---|
| Cohort 1 | |||||
| M | 3.14 | 4.09 | 9.39 | *** | 0.69 |
| SD | 1.33 | 1.44 | |||
| n | 147 | 147 | |||
| Cohort 2 | |||||
| M | 3.92 | 4.70 | 6.80 | *** | 0.45 |
| SD | 1.60 | 1.88 | |||
| n | 146 | 146 | |||
| Cohort 3 | |||||
| M | 3.40 | 3.70 | 2.55 | * | 0.17 |
| SD | 1.85 | 1.75 | |||
| n | 102 | 102 | |||
| Cohort 4 | |||||
| M | 3.02 | 3.96 | 13.12 | *** | 0.62 |
| SD | 1.31 | 1.68 | |||
| n | 254 | 254 | |||
| Total | |||||
| M | 3.31 | 4.12 | 16.52 | *** | 0.50 |
| SD | 1.51 | 1.72 | |||
| N | 649 | 649 | |||
| ***p<.001. **p<.01. *p<.05 | ||||||||||
| Pre-test 2007 | Post-test 2007 | t | d | Pre-test 2008 | Post-test 2008 | t | d | |||
|---|---|---|---|---|---|---|---|---|---|---|
| Cohort 1 | ||||||||||
| M | 3.14 | 3.83 | 5.94 | *** | 0.48 | 3.78 | 4.09 | 3.53 | *** | 0.21 |
| SD | 1.33 | 1.52 | 1.48 | 1.44 | ||||||
| n | 147 | 147 | 147 | 147 | ||||||
| Cohort 2 | ||||||||||
| M | 3.92 | 4.40 | 5.39 | *** | 0.28 | 4.34 | 4.70 | 3.58 | *** | 0.21 |
| SD | 1.60 | 1.78 | 1.60 | 1.88 | ||||||
| n | 146 | 146 | 146 | 146 | ||||||
| Cohort 3 | ||||||||||
| M | 3.40 | 4.05 | 5.56 | *** | 0.33 | 3.18 | 3.70 | 6.07 | *** | 0.32 |
| SD | 1.85 | 2.07 | 1.52 | 1.75 | ||||||
| n | 102 | 102 | 102 | 102 | ||||||
| Cohort 4 | ||||||||||
| M | 3.02 | 3.65 | 10.69 | *** | 0.43 | 3.59 | 3.96 | 6.19 | *** | 0.22 |
| SD | 1.31 | 1.60 | 1.62 | 1.68 | ||||||
| n | 254 | 254 | 254 | 254 | ||||||
| Total | ||||||||||
| M | 3.31 | 3.92 | ||||||||
Figure 8: Mean stanine scores for cohorts over two years (four time points) (Cluster A).
While all cohorts made losses over the summer break (Post-test 2007 to Pre-test 2008) (see Table 17 ), these were very small, and did not reach statistical significance for Cohorts 1, 2 and 4. There was, however, an overall drop in achievement across the cluster as a whole, which may be due to the large loss in achievement over summer for Cohort 3. Cohort 3 was the cohort of students who progressed from Year 6 to Year 7. We have noticed a consistent drop in achievement over the summer for students progressing from Year 6 to Year 7 across various projects, which may suggest that the drop between Year 6 and 7 is due to the change in the STAR test used rather than a loss of learning over summer (Lai & McNaughton, 2008). The critical issue is whether the achievement for the Year 6 to 7 cohort improved enough to ‘recover’ from that drop over summer.
| ***p<.001. **p<.01. *p<.05 | |||||
| Post-test 2007 | Pre-test 2008 | t | d | ||
|---|---|---|---|---|---|
| Cohort 1 | |||||
| M | 3.83 | 3.78 | -0.41 | -0.03 | |
| SD | 1.52 | 1.48 | |||
| n | 147 | 147 | |||
| Cohort 2 | |||||
| M | 4.40 | 4.34 | -0.78 | -0.04 | |
| SD | 1.78 | 1.60 | |||
| n | 146 | 146 | |||
| Cohort 3 | |||||
| M | 4.05 | 3.18 | -6.97 | *** | -0.48 |
| SD | 2.07 | 1.52 | |||
| n | 102 | 102 | |||
| Cohort 4 | |||||
| M | 3.65 | 3.59 | -1.03 | -0.04 | |
| SD | 1.60 | 1.62 | |||
| n | 254 | 254 | |||
| Total | |||||
| M | 3.92 | 3.73 | -4.05 | *** | -0.11 |
| SD | 1.73 | 1.61 | |||
| N | 649 | 649 | |||
Gender and ethnicity
The combined effect of gender and ethnicity was tested using a univariate ANOVA, reported in Table 18.. The interaction between gender and ethnicity was significant for Cohort 2 at Pre-test 2007, and Cohort 3 at Pre-test 2007 and Pre-test 2008, and was not significant for Cohorts 1 or 4 at any time point. The main effect of gender was significant for Cohort 2 at Post-test 2008, and for Cohort 4 at all time points. The main effect of ethnicity was significant for Cohort 2 at all time points and Cohort 4 at Post-test 2008 only. These effects indicate that the effect of gender and ethnicity, and various combinations of gender and ethnicity, varies by cohort. This suggests that gender and ethnicity differences are not ‘fixed’ and further research is needed to understand the conditions under which no such differences will occur.
| Time | Pre-test 2007 | Post-test 2007 | Pre-test 2008 | Post-test 2008 | ||||
|---|---|---|---|---|---|---|---|---|
| F (p value) | ES | F (p value) | ES | F (p value) | ES | F (p value) | ES | |
| Cohort 1 | ||||||||
| Main Effect | ||||||||
| Gender | 2.06 | 0.01 | 0.62 | 0.00 | 3.31 | 0.02 | 1.87 | 0.01 |
| (df =1, 180) | (0.15) | (0.43) | (0.07) | (0.17) | ||||
| Ethnicity | 1.38 | 0.03 | 2.24 | 0.05 | 1.69 | 0.04 | 1.83 | 0.04 |
| (df =3, 180) | (0.25) | (0.09) | (0.17) | (0.14) | ||||
| Interaction | ||||||||
| G x E | 0.45 | 0.01 | 1.17 | 0.02 | 0.40 | 0.01 | 0.51 | 0.01 |
| (df =3, 180) | (0.71) | (0.32) | (0.76) | (0.67) | ||||
| Cohort 2 | ||||||||
| Main Effect | ||||||||
| Gender | 2.35 | 0.02 | 2.58 | 0.02 | 0.17 | <0.01 | 4.90 | 0.03 |
| (df =1, 171) | (0.13) | (0.11) | (0.68) | (0.03) | ||||
| Ethnicity | 2.85 | 0.06 | 4.67 | 0.09 | 3.22 | 0.07 | 5.21 | 0.10 |
| (df =3, 171) | (0.04) | (<0.01) | (0.02) | (<0.01) | ||||
| Interaction | ||||||||
| G x E | 1.59 | 0.03 | 0.37 | 0.01 | 0.62 | 0.01 | 1.45 | 0.03 |
| (df =3, 171) | (0.19) | (0.77) | (0.61) | (0.23) | ||||
| Cohort 3 | ||||||||
| Main Effect | ||||||||
| Gender | 0.37 | 0.00 | 0.73 | 0.01 | 0.24 | 0.00 | 0.76 | 0.01 |
| (df =1, 94) | (0.55) | (0.39) | (0.62) | (0.39) | ||||
| Ethnicity | 2.03 | 0.06 | 1.51 | 0.05 | 1.34 | 0.04 | 1.45 | 0.04 |
| (df =3, 94) | (0.12) | (0.22) | (0.26) | (0.23) | ||||
| Interaction | ||||||||
| G x E | 3.40 | 0.10 | 2.18 | 0.07 | 4.54 | 0.13 | 1.36 | 0.04 |
| (df =3, 94) | (0.02) | (0.10) | (0.01) | (0.26) | ||||
| Cohort 4 | ||||||||
| Main Effect | ||||||||
| Gender | 11.02 | 0.04 | 13.04 | 0.05 | 13.34 | 0.05 | 13.12 | 0.05 |
| (df =1, 246) | (<0.01) | (<0.01) | (<0.01) | (<0.01) | ||||
| Ethnicity | 2.12 | 0.03 | 4.94 | 0.06 | 1.96 | 0.02 | 2.33 | 0.03 |
| (df =3, 246) | (0.10) | (<0.01) | (0.12) | (0.07) | ||||
| Interaction | ||||||||
| G x E | 0.98 | 0.01 | 0.39 | 0.00 | 0.56 | 0.01 | 0.31 | 0.00 |
| (df =3, 246) | (0.40) | (0.76) | (0.64) | (0.82) | ||||
| Note: The effect size reported in this table is partial eta squared. | ||||||||
While gains were similar for males and females (see Figure 9 ), as seen in Table 19 , there were significant differences between males and females at each time point. On average, female students scored higher than males by around 0.5 stanine.
Figure 9: Mean stanine scores for each gender over two years (4 time points) (Cluster A).
| ***p<.001. **p<.01. *p<.05 | |||||||
| Gender | Gender Difference | ||||||
|---|---|---|---|---|---|---|---|
| Male | Female | M | t | d | |||
| Pre-test 2007 | |||||||
| M | 3.06 | 3.52 | 0.46 | -3.86 | *** | -0.31 | |
| SD | 1.53 | 1.48 | |||||
| Post-test 2007 | |||||||
| M | 3.65 | 4.15 | 0.50 | -3.70 | *** | -0.29 | |
| SD | 1.78 | 1.64 | |||||
| Pre-test 2008 | |||||||
| M | 3.44 | 3.99 | 0.54 | -4.30 | *** | -0.35 | |
| SD | 1.69 | 1.50 | |||||
| Post-test 2008 | |||||||
| M | 3.76 | 4.42 | 0.66 | -4.93 | *** | -0.39 | |
| SD | 1.76 | 1.62 | |||||
A one-way ANOVA, reported in Table 20, found significant differences between ethnic groups at all four time points. Post-hoc comparisons using a Bonferroni adjustment found Samoan students’ mean stanine scores were significantly higher than Tongan students’ scores at Pre-test and Post-test 2007 and Post-test 2008, and Samoan students scored significantly higher than Cook Island students at Post-test 2007 and Post-test 2008. No other significant differences were found. This is illustrated in Figure 10 . It is worth noting that students of other Pasifika groups had similar scores to Samoan students.
| Note: The effect size reported in this table is partial eta squared. | ||
| F | ES | |
|---|---|---|
| (p value) | ||
| Pre-test 2007 | 3.59 | 0.02 |
| (df = 3, 645) | (0.01) | |
| Post-test 2007 | 7.85 | 0.04 |
| (df = 3, 645) | (<0.01) | |
| Pre-test 2008 | 3.34 | 0.02 |
| (df = 3, 645) | (0.02) | |
| Post-test 2008 | 6.58 | 0.03 |
| (df = 3, 645) | (<0.01) | |
Figure 10: Mean stanine scores for ethnic groups over two years (4 time points) (Cluster A).
As seen in Table 21 and Table 22 , all ethnic groups made significant gains from Pre-test 2007 to Post-test 2008, and throughout each year (Pre-test to Post-test 2007 and Pre-test to Post-test 2008). The three main ethnic groups made less gain in 2008 than in 2007.
| ***p<.001. **p<.01. *p<.05 | |||||
| Pre-test 2007 | Post-test 2008 | t | d | ||
|---|---|---|---|---|---|
| Tongan | |||||
| M | 3.07 | 3.77 | 7.80 | *** | 0.46 |
| SD | 1.42 | 1.60 | |||
| n | 175 | 175 | |||
| Cook Island Māori | |||||
| M | 3.11 | 3.81 | 6.85 | *** | 0.43 |
| SD | 1.49 | 1.73 | |||
| n | 108 | 108 | |||
| Samoan | |||||
| M | 3.49 | 4.39 | 11.83 | ** | 0.55 |
| SD | 1.55 | 1.73 | |||
| n | 305 | 305 | |||
| Other Pasifika | |||||
| M | 3.44 | 4.30 | 5.22 | *** | 0.53 |
| SD | 1.55 | 1.72 | |||
| n | 61 | 61 | |||
| Total | |||||
| M | 3.31 | 4.12 | 16.52 | *** | 0.50 |
| SD | 1.51 | 1.72 | |||
| N | 649 | 649 | |||
| ***p<.001. **p<.01. *p<.05 | ||||||||||
| Pre-test 2007 | Post-test 2007 | t | d | Pre-test 2008 | Post-test 2008 | t | d | |||
|---|---|---|---|---|---|---|---|---|---|---|
| Tongan | ||||||||||
| M | 3.07 | 3.55 | 5.41 | *** | 0.30 | 3.53 | 3.77 | 3.35 | *** | 0.15 |
| SD | 1.42 | 1.72 | 1.68 | 1.60 | ||||||
| n | 175 | 175 | 175 | 175 | ||||||
| Cook Island Māori | ||||||||||
| M | 3.11 | 3.58 | 4.54 | *** | 0.30 | 3.49 | 3.81 | 3.43 | *** | 0.19 |
| SD | 1.49 | 1.67 | 1.70 | 1.73 | ||||||
| n | 108 | 108 | 108 | 108 | ||||||
| Samoan | ||||||||||
| M | 3.49 | 4.24 | 11.47 | *** | 0.46 | 3.93 | 4.39 | 7.28 | *** | 0.28 |
| SD | 1.55 | 1.71 | 1.54 | 1.73 | ||||||
| n | 305 | 305 | 305 | 305 | ||||||
| Other Pasifika | ||||||||||
| M | 3.44 | 4.02 | 4.45 | *** | 0.37 | 3.80 | 4.30 | 3.74 | *** | 0.31 |
| SD | 1.55 | 1.62 | 1.53 | 1.72 | ||||||
| n | 61 | 61 | 61 | 61 | ||||||
| Total | ||||||||||
| M | 3.31 | 3.92 | 13.82 | *** | 0.38 | 3.73 | 4.12 | 9.35 | *** | 0.23 |
| SD | 1.51 | 1.73 | 1.61 | 1.72 | ||||||
| N | 649 | 649 | 649 | 649 | ||||||
Gains in achievement
Gain scores for each year were calculated for individual students by subtracting the Pre-test stanine from the Post-test stanine. In 2007, over half of the students accelerated in stanines (51%), meaning that their end of year stanine was higher than their stanine at the start of the year. A further 36% of students maintained the same stanine from the start to the end of the year, and only 13% had lower stanines at the end of the year than at the beginning.
In 2008, a large proportion of students accelerated also (43%). A further 41% maintained the same stanine, and 16% had a lower stanine at the end of the year. These proportions are illustrated in Table 23 and Figure 11 .
| 2007 | 2008 | ||
|---|---|---|---|
| Loss | |||
| Frequency | 83 | 107 | |
| Percent | 12.79% | 16.49% | |
| Maintain | |||
| Frequency | 235 | 266 | |
| Percent | 36.21% | 40.99% | |
| Accelerate | |||
| Frequency | 331 | 276 | |
| Percent | 51.00% | 42.53% |
Figure 11: Percentage of students, who lost, maintained and accelerated achievement in 2007 and 2008 (Cluster A).
Gains were also calculated separately by initial stanine at Pre-test as seen in Table 24 and Figure 12 for 2007, and Table 25 and Figure 13 for 2008. For example, of those students who had a stanine score of 1 in Pre-test 2007, 44.58% maintained their score (i.e., had a stanine of 1 in Term 4, 2007) and 55.42% accelerated (i.e., had a stanine score higher than 1 in Term 4, 2007). In general, across both 2007 and 2008, students who began on lower stanines were more likely to accelerate than those on higher stanines. This might reflect a focus on lifting stanine 3 students, or a more general effect whereby the interventions are mostly effective for students in these bands, and more targeted interventions are needed for higher bands. Small numbers and regression to the mean effects might also explain the smaller gains in the upper stanines.
| Stanine 1 | Stanine 2 | Stanine 3 | Stanine 4 | Stanine 5 | Stanine 6 | Stanine 7 | Stanine 8 | ||
|---|---|---|---|---|---|---|---|---|---|
| Loss | |||||||||
| Frequency | 13 | 23 | 22 | 11 | 12 | 1 | 1 | ||
| Percent | 11.02% | 13.22% | 15.94% | 13.58% | 30.00% | 9.09% | 25.00% | ||
| Maintain | |||||||||
| Frequency | 37 | 46 | 45 | 48 | 35 | 17 | 5 | 2 | |
| Percent | 44.58% | 38.98% | 25.86% | 34.78% | 43.21% | 42.50% | 45.45% | 50.00% | |
| Accelerate | |||||||||
| Frequency | 46 | 59 | 106 | 68 | 35 | 11 | 5 | 1 | |
| Percent | 55.42% | 50.00% | 60.92% | 49.28% | 43.21% | 27.50% | 45.45% | 25.00% |
| Stanine 1 | Stanine 2 | Stanine 3 | Stanine 4 | Stanine 5 | Stanine 6 | Stanine 7 | Stanine 8 | ||
|---|---|---|---|---|---|---|---|---|---|
| Loss | |||||||||
| Frequency | 11 | 16 | 25 | 23 | 17 | 12 | 3 | ||
| Percent | 11.70% | 11.27% | 17.61% | 19.66% | 25.37% | 57.14% | 50.00% | ||
| Maintain | |||||||||
| Frequency | 34 | 30 | 54 | 64 | 50 | 27 | 4 | 3 | |
| Percent | 56.67% | 31.91% | 38.03% | 45.07% | 42.74% | 40.30% | 19.05% | 50.00% | |
| Accelerate | |||||||||
| Frequency | 26 | 53 | 72 | 53 | 44 | 23 | 5 | 0 | |
| Percent | 43.33% | 56.38% | 50.70% | 37.32% | 37.61% | 34.33% | 23.81% | 0.00% |
Figure 12: Percentage of students, who lost, maintained and accelerated achievement by Pre-test stanine in 2007 (Cluster A).
Figure 13: Percentage of students, who lost, maintained and accelerated achievement by Pre-test stanine in 2008 (Cluster A).
The percentage of students who accelerated varied only slightly between the different cohorts, as seen in Table 26 and Table 27 , and Figure 14 and Figure 15 . These percentages varied from year to year. For example, Cohort 1 had the highest percentage of students who accelerated compared with other cohorts in 2007, but the lowest percentage in 2008.
| Cohort 1 | Cohort 2 | Cohort 3 | Cohort 4 | ||
|---|---|---|---|---|---|
| Loss | |||||
| Frequency | 28 | 23 | 12 | 20 | |
| Percent | 19.05% | 15.75% | 11.76% | 7.87% | |
| Maintain | |||||
| Frequency | 35 | 56 | 41 | 103 | |
| Percent | 23.81% | 38.36% | 40.20% | 40.55% | |
| Accelerate | |||||
| Frequency | 84 | 67 | 49 | 131 | |
| Percent | 57.14% | 45.89% | 48.04% | 51.57% |
Figure 14: Percentage of students, who lost, maintained and accelerated achievement by cohortin 2007 (Cluster A).
| Cohort 1 | Cohort 2 | Cohort 3 | Cohort 4 | ||
|---|---|---|---|---|---|
| Loss | |||||
| Frequency | 31 | 30 | 10 | 36 | |
| Percent | 21.09% | 20.55% | 9.80% | 14.17% | |
| Maintain | |||||
| Frequency | 58 | 56 | 38 | 114 | |
| Percent | 39.46% | 38.36% | 37.25% | 44.88% | |
| Accelerate | |||||
| Frequency | 58 | 60 | 54 | 104 | |
| Percent | 39.46% | 41.10% | 52.94% | 40.94% |
Figure 15: Percentage of students, who lost, maintained and accelerated achievement by cohort in 2008 (Cluster A).
Classroom gains
Classroom gain scores for each year were calculated by subtracting the classroom’s Pre-test mean stanine from the Post-test mean stanine. This section only includes students who did not change class within any one year, leaving a total of 617 students in 2007 and 620 students in 2008.
As seen in Figure 16 , almost every class made accelerated gains from Pre-test 2007 to Post-test 2007. While more classes made losses in 2008 than in 2007 (see Figure 17), the majority of classes still made gains during this year also. It is important to note that a stanine loss does not indicate negative learning. Rather, it indicates that on average, students made less than expected progress for that year. In summary, 92.16% of classes made gains in 2007, compared to 73.47% of classes in 2008.
Figure 16: Mean gains from Pre-test to Post-test 2007 by classroom (Cluster A).
Note: The gap between those who made losses and gains represents 2 classes whose mean stanine scores were identical from Pre-test 2007 to Post-test 2007, i.e., they neither gained nor lost in stanines.
Figure 17: Mean gains from Pre-test to Post-test 2008 by classroom (Cluster A).
Note: The gap between those who made losses and gains represents 5 classes whose mean stanine scores were identical from Pre-test 2008 to Post-test 2008, i.e., they neither gained nor lost in stanines.
Achievement by school
All schools made gains between the beginning of 2007 and the end of 2008. Mean scores by school are presented in Table 28 . Effect sizes ranged from 0.38 to 0.77.
| ***p<.001. **p<.01. *p<.05 | |||||
| Pre-test 2007 | Post-test 2008 | t | d | ||
|---|---|---|---|---|---|
| School A1 | |||||
| M | 3.25 | 4.01 | 8.01 | *** | 0.50 |
| SD | 1.45 | 1.60 | |||
| n | 163 | 163 | |||
| School A2 | |||||
| M | 3.04 | 3.57 | 4.45 | *** | 0.38 |
| SD | 1.26 | 1.49 | |||
| n | 84 | 84 | |||
| School A3 | |||||
| M | 3.00 | 4.03 | 6.68 | *** | 0.56 |
| SD | 1.68 | 1.96 | |||
| n | 105 | 105 | |||
| School A4 | |||||
| M | 3.86 | 4.59 | 7.57 | *** | 0.48 |
| SD | 1.55 | 1.49 | |||
| n | 131 | 131 | |||
| School A5 | |||||
| M | 3.63 | 4.41 | 5.71 | *** | 0.45 |
| SD | 1.52 | 1.93 | |||
| n | 98 | 98 | |||
| School A6 | |||||
| M | 2.72 | 3.85 | 9.88 | *** | 0.77 |
| SD | 1.22 | 1.69 | |||
| n | 68 | 68 | |||
Almost all schools made significant gains (see Table 29 ) from the beginning to the end of the 2007 academic year (with the exception of School A5, whose Pre-test to Post-test 2007 gains did not reach significance).
| ***p<.001. **p<.01. *p<.05 | ||||||||||
| Pre-test 2007 | Post-test 2007 | t | d | Pre-test 2008 | Post-test 2008 | t | d | |||
|---|---|---|---|---|---|---|---|---|---|---|
| School A1 | ||||||||||
| M | 3.25 | 3.76 | 6.49 | *** | 0.34 | 3.55 | 4.01 | 6.14 | *** | 0.30 |
| SD | 1.45 | 1.57 | 1.52 | 1.60 | ||||||
| n | 163 | 163 | 163 | 163 | ||||||
| School A2 | ||||||||||
| M | 3.04 | 3.65 | 5.12 | *** | 0.42 | 3.19 | 3.57 | 3.65 | *** | 0.27 |
| SD | 1.26 | 1.65 | 1.36 | 1.49 | ||||||
| n | 84 | 84 | 84 | 84 | ||||||
| School A3 | ||||||||||
| M | 3.00 | 3.39 | 4.06 | *** | 0.23 | 3.76 | 4.03 | 2.24 | * | 0.14 |
| SD | 1.68 | 1.70 | 1.81 | 1.96 | ||||||
| n | 105 | 105 | 105 | 105 | ||||||
| School A4 | ||||||||||
| M | 3.86 | 4.99 | 11.22 | *** | 0.68 | 4.21 | 4.59 | 4.68 | *** | 0.25 |
| SD | 1.55 | 1.75 | 1.55 | 1.49 | ||||||
| n | 131 | 131 | 131 | 131 | ||||||
| School A5 | ||||||||||
| M | 3.63 | 3.90 | 1.87 | 0.18 | 3.99 | 4.41 | 3.23 | ** | 0.24 | |
| SD | 1.52 | 1.54 | 1.56 | 1.93 | ||||||
| n | 98 | 98 | 98 | 98 | ||||||
| School A6 | 7.04 | |||||||||
| M | 2.72 | 3.44 | *** | 0.50 | 3.51 | 3.85 | 3.19 | ** | 0.20 | |
| SD | 1.22 | 1.62 | 1.71 | 1.69 | ||||||
| n | 68 | 68 | 68 | 68 | ||||||
There was some variation between schools in terms of mean scores. This was confirmed by a one-way ANOVA, which indicated a significant effect of school on mean score at every time point (see Table 30 ).
| 1The effect size reported in this table is partial eta squared. | ||
F | ES1 | |
|---|---|---|
| Pre-test 2007 | 8.37 | 0.06 |
| (df = 5, 643) | (<0.01) | |
| Post-test 2007 | 15.33 | 0.11 |
| (df = 5, 643) | (<0.01) | |
| Pre-test 2008 | 5.59 | 0.04 |
| (df = 5, 643) | (<0.01) | |
| Post-test 2008 | 4.88 | 0.04 |
| (df = 5, 643) | (<0.01) | |
A second ANOVA was performed on gain scores to assess whether there were any differences in gains between schools (see Table 31 ). There was a significant effect of school overall in Pre-test to Post-test 2007 gains. A post-hoc Scheffe analysis divided schools into two groups – the higher gain schools, which were School A6 and School A4, and the other schools, whose gains were still substantial but not quite as high as these two. There was no significant effect of school overall in Pre-test to Post-test 2008 gains, indicating that all schools made similar gains.
Previously, School A1 was selected as the high gain school. This was based on an analysis using pre-post students who were present from 2006 to 2007. As this analysis was based on different students, the results here are slightly different. Despite the fact that the Scheffe analysis did not identify School A1 as being the highest scoring school overall, the school did have the highest gains in 2008, indicating that it is indeed a school that continues to make large significant gains for its students.
| 1 The effect size reported in this table is partial eta squared. | ||
F | ES1 | |
|---|---|---|
| Pre-test to Post-test 2007 | 9.03 | 0.07 |
| (df = 5, 643) | (<0.01) | |
| Pre-test to Post test 2008 | 0.46 | 0.00 |
| (df = 5, 643) | (0.80) | |
3.2.2 Statistical modelling
‘Gap difference’ models
The aim of the ‘gap difference’ model was to establish predictors of student reading achievement growth over the four tests across 2007 to 2008 where each pair of tests is a ‘gap’ to be analysed. Here, the results of the four repeated tests were used to calculate the three ‘gap difference’ observations on each student, i.e., change in stanine between the beginning and end of 2007 (gap 1), change in stanine over the Summer of 2007 to 2008 (gap 2), and the change in stanine between the beginning and end of 2008 (gap 3). Two models were developed; one based on the ‘entire’ dataset, which was based on the longitudinal sample of 715 Cluster A students who were in Years 4 to 8 at the beginning of 2007, as part of the preliminary analysis, and the other based on the ‘genuinely complete cases’ dataset.
The preliminary analysis based on the ‘entire’ dataset showed that school, ethnicity, the interaction of school and ethnicity (F(45, 2070) = 1.60, p < 0.01, Pillai’s trace V = 0.10), and starting level of reading achievement were significant predictors of the ‘gap differences’. In particular, starting achievement level was significant over and above the interaction of school and ethnicity (F(3, 688) = 17.00, p < 0.001, Pillai’s trace V = 0.07). The starting achievement level factor was determined by the stanine score on the first test, at the beginning of 2007. Students were categorised into two groups; ‘low’ was assigned for a stanine less than or equal to 3, and ‘mid-high’ assigned otherwise. There were insufficient students in the stanine 7 - 9 bands to create a separate ‘high starting stanine’ group.
Figure 18 illustrates the difference in starting level, where the most prominent difference was that the low starting level group made a greater gain at the first gap (gain of about 0.7 stanine) than the mid-high starting level group (gain of slightly under 0.4 stanine; t(714) = 4.47, p < 0.001). The difference between the two groups, however, was not significant at the second and third gaps (t(714) = 1.25, p = 0.21, and t(714) = 0.87, p = 0.38, respectively). While the significance of school and ethnicity was fairly straightforward to interpret statistically, the interaction between ethnicity and school makes it hard to interpret in practical terms. There were perceptible differences in the patterns of achievement change between various school-ethnic group combinations. This means that the patterns of achievement gains or losses were not uniform across all six schools for any ethnic group. However, given the number of possible combinations (6 schools x 4 ethnic groups = 24 combinations), pairwise comparisons were impractical. Note that as the data were extremely unbalanced with respect to student cohorts, cohort was not considered as a predictor for the ‘gap difference’ models.
Figure 18: Mean stanine changes (with 95% confidence intervals) for the two starting levels at each time gap based on the ‘entire’ dataset (Cluster A).
The ‘gap difference’ model that was based on the ‘genuinely complete cases’ dataset restricted attention to the effect of language, country of birth and time spent in New Zealand, and its association with the changes in reading achievement levels. In this reduced dataset (the ‘genuinely complete cases’ dataset was a subset of the ‘complete’ dataset), ethnicity was no longer a significant predictor (F(9, 1041) = 0.89, p = 0.53, Pillai’s trace V = 0.02) while starting achievement level and the school factor remained significant (F(3, 348) = 7.12, p < 0.001, Pillai’s trace V = 0.06). Amongst the information collected by the student survey, time lived in New Zealand and their combined effects with the school factor were significant predictors (F(30, 1050) = 1.80, p < 0.01, Pillai’s trace V = 0.15). Time lived in New Zealand was a four level categorical factor, with students grouped into those who had lived in New Zealand for less than a year, between 1 and 5 years (inclusive), more than 5 years, or those born in New Zealand. No student, however, had lived in New Zealand for less than a year.
Figure 19 illustrates the difference in starting level, where the major difference was that on average, the low starting level group made greater gains in stanine at the first gap than their mid-high starting level peers (t(379) = 3.10, p < 0.01), but the gains (or losses) were similar at the second and the third gaps (t(379) = 1.18, p = 0.24, t(379) = 0.70, p = 0.49, respectively). The starting achievement level of students continued to account for a sizeable amount of variance. However, the interaction between how long a student had lived in New Zealand (i.e., the time lived in New Zealand factor) and school in practical terms was difficult to interpret as the patterns of achievement changed for different groups. In other words, the patterns of achievement were not uniform across all schools and student-to-student variability was again large.
Figure 19: Mean stanine changes (with 95% confidence intervals) for the two starting levels at each time gap based on the ‘genuinely complete cases’ dataset (Cluster A).
Across the two ‘gap difference’ models, no evidence of a language effect, country of birth effect, or gender effect was found (test statistics for the ‘genuinely complete cases’ dataset: home language, F(6, 694) = 0.35, p = 0.91, Pillai’s trace V = 0.006; first language, F(9, 1041) = 0.59, p = 0.81, Pillai’s trace V = 0.02; country of birth, F(6, 694) = 0.80, p = 0.57, Pillai’s trace V = 0.01; gender, F(3, 347) = 1.37, p = 0.25, Pillai’s trace V = 0.01). There was a starting achievement level effect for both the ‘entire’ and the ‘genuinely complete cases’ data sets; that was the clearest and most easily described phenomenon observed. There was an ethnicity by school interaction for the ‘entire’ dataset, and a time lived in New Zealand by school interaction for the ‘genuinely complete cases’ dataset. The nature of these interactions, however, appeared to be too complicated to be amenable to any simple description. With respect to the starting level effect, this effect was largely driven by the fact that the low starting level students made significantly larger gains at the first gap, the beginning to end of 2007. At the latter two gaps there were no significant differences between the two starting achievement groups as illustrated by Figure 18 and Figure 19 . Higher and lower starting students made similar drops in achievement over the summer holidays, and gained at similar rates during the academic year of 2008. In 2008, low starting level students made lower gains than in 2007.
This suggests that instruction initially may have been more effective for low starting level students in 2007, but after an initial larger effect and when students are at higher levels, the rate of gain evens out. Further study would be useful to tease out if low starting level students were recent arrivals to the school or their starting levels were artificially low due to transition effects.
‘Level difference’ models
The ‘level difference’ models aimed at investigating whether the schools varied in their mean reading achievement stanines across the four tests. Subsequently, the ‘level difference’ models aimed to answer the question of whether the strength of association between reading achievement and measurable student information was similar across schools. Measurable student information included variables such as ethnicity, gender, students’ language usage (home language and first language) and time spent in New Zealand. Given that the interest here was mainly around the association between the available predictors and the schools, two models were developed and compared. The first ‘level difference’ model treated school as a fixed effect, and the explanatory power of the factor ‘school’ was limited to the dataset. The second ‘level difference’ model then treated school as a random effect to encompass the notion that the six schools were a random selection of all schools.
There were two variants of language usage: first language and home language. Through the model selection process, the results showed that home language contained more explanatory power than first language, and therefore, home language was used as the choice of language variant in both models.
Table 32 contains a summary of the estimated coefficients for the first ‘level difference’ model where school was modelled as a fixed effect. The mean stanine of those that spoke only a Pasifika language at home were estimated to be 0.41 of a stanine less than those that spoke English at home. Those that spoke two or more languages (i.e., both Pasifika language and English) at home were estimated to be 0.58 of a stanine less than those that only spoke English at home. The differences between different home language usages in mean stanines in each cohort are illustrated by Figure 20. The figure also illustrated a downward trend in achievement where older cohorts achieved lower stanine scores. Note that Figure 20 does not represent a single cohort tracked longitudinally from Year 4 to Year 9. The figure presents cohorts and their means over two years.
There are two interesting features in these data that require further analyses. The first is that differences between home language groups are minimal at some time points, especially at higher ages. This finding from mixed longitudinal and cross sectional data here is consistent with data from the case studies. Secondly, the largest difference occurs in the cohort progressing from Year 6 to Year 7. The effect of having a ‘stronger’ English background may be felt especially when the length and nature of the STAR test changes at Year 7.
1 Intercept represents the baseline of male students that speak English at home and had lived in NZ between 1 and 5 years from the baseline school. 2 School (n = 6 schools; F(5, 361) = 3.68, p < 0.01) estimates not reported here for anonymity. | ||||
| Fixed Effect | Coefficient | SE | t | p |
|---|---|---|---|---|
| Intercept1 | 2.98 | 0.33 | 8.99 | < .001 |
| Home language | ||||
| Pasifika language | -0.41 | 0.16 | -2.62 | <.01 |
| Two or more languages | -0.58 | 0.25 | -2.30 | 0.02 |
| Gender - female | 0.51 | 0.14 | 3.68 | <.001 |
| Time lived in NZ | ||||
| More than 5 years | 1.02 | 0.28 | 3.70 | <.001 |
| Born in NZ | 0.88 | 0.24 | 3.60 | <.001 |
| School2 | - | - | ||
Figure 20: Cohorts’ overall mean stanine levels across the four tests by home language for the ‘complete’ dataset (Cluster A).
With all other factors held constant, students that had lived in New Zealand for more than five years were on average, achieving 1.02 stanine higher than those that had lived in New Zealand between one and five years (i.e., basis of comparison); students that were born in New Zealand achieved an average of 0.88 of a stanine higher than the group of students that was used as the basis of comparison in the model (see Table 32 ). Figure 21 displays the mean stanines of each cohort split by the groups of time lived in New Zealand. Ethnicity was not a significant predictor (F(3, 358) = 1.71, p = 0.16) in a model that already included the stated predictors. Note that students that had lived in New Zealand for more than five years had all or the majority of their schooling in New Zealand, similar to those that were born in New Zealand.
There were significant differences amongst the schools (t(361) = 4.69, p < 0.001). For example, the largest estimated mean stanine difference was 0.82 stanine (SE = 0.18). This was consistent with the one-way ANOVA results that tested for significant differences between schools’ mean scores at each time point (see Section 3.2.1).
Figure 21: Cohorts’ overall mean stanines across the four tests by time lived in New Zealand for the ‘complete’ dataset.
Note ‘t’ represents years lived in New Zealand (Cluster A).
The large drop at Year 6 to Year 7 mirrors the drop found for home language and may reflect similar reasons. However, these mixed cross sectional and longitudinal data suggest an exacerbated effect when relative unfamiliarity with the New Zealand schooling system is considered.
The second ‘level difference’ model included school as a random effect in an HLM. The model estimated that, with all else held constant, those that spoke only Pasifika languages at home scored 0.42 (SE = 0.16) of a stanine less than those that spoke English at home. Those that spoke both Pasifika language and English at home scored 0.58 (SE = 0.25) of a stanine less than those that spoke English. On average, female students scored 0.51 of a stanine (SE = 0.14) higher than their male peers. Students that had lived in New Zealand for more than five years were on average, achieving 1.04 stanine (SE = 0.27) higher than those that had lived in New Zealand between one and five years (i.e., baseline), and students that were born in New Zealand achieved an average of 0.90 stanine (SE = 0.24) higher than the group of students used as base comparison. These estimated coefficients and results of the second ‘level difference’ model were consistent with the first ‘level difference’ model (see Table 33 ).
| 1 Intercept represents the baseline of male students that speak English at home and lived in NZ between 1 and 5 years from the baseline school. | ||||
| Fixed Effect | Coefficient | SE | t | p |
|---|---|---|---|---|
| Intercept1 | 2.81 | 0.27 | 10.39 | <.001 |
| Home language | ||||
| Pasifika language | -0.42 | 0.16 | -2.69 | <.001 |
| Bilingual (Pasific and English) | -0.58 | 0.25 | -2.27 | <.001 |
| Gender - female | 0.51 | 0.14 | 3.66 | <.001 |
| Time lived in NZ | ||||
| More than 5 years | 1.04 | 0.28 | 3.75 | <.001 |
| Born in NZ | 0.90 | 0.25 | 3.68 | <.001 |
Overall, the mean scores for the students that spoke only Pasifika languages and both Pasifika and English languages at home were significantly lower than that for the English-speaking students. The mean scores for females were significantly higher than that for males. With respect to the length of time lived in New Zealand, the mean scores for those that had lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than those that had lived in New Zealand between one and five years.
The treatments of school effect presented no change to the coefficient estimates of the two ‘level difference’ models; the difference in treatments of the school factor between the two ‘level difference’ models therefore made no substantial difference in how we are interpreting our model results.
‘Level Difference’ Model with Cohort as Predictor
Cohort differences were not considered in the previous two ‘level difference’ models, as the combination of school and cohort presented theoretical and practical difficulties (e.g., modelling with unbalanced numbers of students). As an extension to the two ‘level difference’ models, the cohort predictor was brought in for a series of extended analyses of the ‘level difference’ model. The aim was mainly to look for an expected phenomenon of students who speak languages other than English (using both first language and home language) closing the achievement gap on English speaking students as their year level increases (i.e., the expected ‘catch-up’ pattern). However, a simple ‘catch-up’ pattern did not appear. The cohort effect was not linear in time, which means students’ mean stanines did not increase at a constant rate as their year level increased. The other result that could be conjectured from these extended analyses was that the differences in cohorts amongst the schools constituted a large part of the explanation for the between school differences.
Summary of the modelling results
Students’ starting level of achievement was the most prominent and readily interpretable effect for all the ‘gap difference’ models (i.e., the models which examined the amount of gain made within each academic year and between academic years). Students with lower starting levels (stanines 1 - 3) made greater gains than students with middle to higher starting achievement levels (stanines 4 - 9) from the beginning to end of 2007, but the two starting level groups made similar drops over the summer break (end of 2007 and beginning of 2008) and within year gains from the beginning and end of 2008. The higher and lower starting students made similar drops in achievement over the summer holidays, and gained at similar rates during the academic year of 2008. In 2008, low starting level students made lower gains than in 2007. The results may indicate that instruction initially may have been more effective for low starting level students in 2007. Discussions with the cluster leader indicated that the cluster placed far less emphasis on reading in 2008 than 2007, and that the imbalance in instructional focus was being addressed in 2009.
Another possible explanation is that there might be stronger effects for low achieving students initially, but that after the initial effects different groups may respond at similar rates. This has important implications for judging effectiveness (which needs to be over the longer term and to look at different groups). The initial higher gain may be as much due to regression effect as it is to more explicit and targeted instruction having an immediate impact.
There was an ethnicity by school interaction for the ‘entire’ dataset, and a time lived in New Zealand by school interaction for the ‘genuinely complete cases’ dataset. This effect was largely driven by the fact that the low starting level students made significantly larger gains in 2007. Interpretation of these two interaction effects, however, turned out to be too complicated. There were no gender or language (whether language used at home or first language) effects on the achievement gains. In other words, males and females, and students who spoke different languages made similar gains throughout the three ‘gaps’ examined.
For the ‘level difference’ models where we examined the achievement levels aggregated across four tests, the four main effects, gender, time lived in New Zealand, home language, and school turned out to be optimal in estimating the mean stanine level over the four tests. In other words, these factors were associated with significantly different levels of achievement. Overall, the mean scores for the students that spoke mainly Pasifika languages and those that spoke two or more languages (Pasifika language as well as English) were significantly lower than that for the mainly English-speaking students. The finding likely reflects collapsing across year levels and there is evidence for the case study schools that differences can disappear when achievement levels are considered over school years. The mean scores for females were significantly higher than that for males. With respect to the length of time lived in New Zealand, the mean scores for those that had lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than those that had lived in New Zealand between one and five years. The mean levels of achievement differed significantly between schools, and part of this difference could be due to the different year levels (i.e., cohorts) that the schools catered for.
3.3 Phase Three Results – Four Case Study Schools
This section seeks to answer Research Questions 2 - 4 with more depth:
- What differences, if any, occur between the gains in the Schooling Improvement initiatives for different student groups within Pasifika (ethnicity, gender, generation in New Zealand, language)?
- What are the practices in schools and initiatives that work, and the practices that do not work, for Pasifika students and under what conditions?
- What are the barriers to schools achieving positive learning outcomes for Pasifika students?
3.3.1 Focus Cluster School: Case Study 1
Overview of the school
This is a medium performing decile 1 primary school when considered in terms of gains and levels of achievement for Pasifika students (see Table 34 ). Overall achievement when considered at the school was medium, with levels across time between stanine 3 and 4 in reading comprehension, and in general the Pasifika students consistently made higher than nationally expected gains within years. Within the different Pasifika groups there were differences in achievement when tracked longitudinally over two years.
The total Pasifika group at the school overall is around 80%. The breakdown for the Years 4 - 8 groups we examined was Samoan 37%, Tongan 29%, Cook Island Māori 21% Niuean 12% and Other Pasifika 1%. More than half (57%) of these students had English as first language, with Samoan being the next most common other first language (20%). Of the total Pasifika group more than two thirds (73%) of students were born in New Zealand.
| 1 School Year uses Pre-test to Post-test scores. 2 Summer uses Post-test and the following year’s Pre-test scores, e.g., Post-test 2007 - Pre-test 2008. | ||||||||||||||
| School Year1 (2007) | Summer2 (2007 - 2008) | School Year (2008) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pre-test 2007 | Post-test 2007 | Pre-test 2008 | Post-test 2008 | t | ES | t | ES | t | ES | |||||
| Tongan | M | 3.19 | 3.55 | 3.62 | 3.85 | 2.41 | * | 0.25 | 0.36 | 0.04 | 1.80 | 0.16 | ||
| SD | 1.41 | 1.53 | 1.45 | 1.40 | ||||||||||
| N | 47 | 47 | 47 | 47 | ||||||||||
| Cook Island Māori | M | 3.09 | 3.57 | 3.54 | 3.86 | 2.50 | * | 0.31 | -0.15 | -0.02 | 1.93 | 0.17 | ||
| SD | 1.46 | 1.65 | 1.88 | 1.82 | ||||||||||
| N | 35 | 35 | 35 | 35 | ||||||||||
| Samoan | M | 3.25 | 3.90 | 3.58 | 4.20 | 5.60 | *** | 0.47 | -2.12 | * | -0.23 | 4.98 | *** | 0.42 |
| SD | 1.36 | 1.43 | 1.33 | 1.61 | ||||||||||
| N | 60 | 60 | 60 | 60 | ||||||||||
| Other Pasifika | M | 3.67 | 4.14 | 3.33 | 4.05 | 2.22 | * | 0.26 | -2.88 | ** | -0.47 | 3.63 | ** | 0.44 |
| SD | 1.74 | 1.88 | 1.56 | 1.69 | ||||||||||
| N | 21 | 21 | 21 | 21 | ||||||||||
| Total | M | 3.25 | 3.76 | 3.55 | 4.01 | 6.49 | *** | 0.34 | -2.19 | * | -0.14 | 6.14 | *** | 0.29 |
| SD | 1.45 | 1.57 | 1.52 | 1.60 | ||||||||||
| N | 163 | 163 | 163 | 163 | ||||||||||
Connectedness – community and school
The theoretical prediction was that an effective school would have well-developed connections with communities and families. The connections would be reciprocal, that is, two way with considerable flow of information both ways. In addition, general models of parent ‘involvement’ identify a range of types of involvement, from volunteering, participating in decision making, and communicating with the school, to active academic support including involvement with homework at home, with some forms likely to be more effective than other forms.
In this school more active roles for parents were still being formulated and the evidence from the interviews was that the school promotion of roles had not been focused on specific involvement at home in academic support. There were some initiatives to develop involvement at school to facilitate the conditions of learning with a recognition that practices had mostly been one way but strategies were being put in place to create more two way systems.
The Principal indicated that parental involvement was an area that still needed work. He commented that a large group of parents did not get involved, thinking that it was the school’s job. The Literacy Leader noted that attendance at meetings fluctuates but that that they get a good turnout for parent interviews, with 90% of the parents attending.
We are still doing a lot of work around developing those links and trying to maximise those to help the kids. [Principal]
The Principal however did not want to use the ‘community’ as an excuse for not getting even higher levels of achievement and didn’t want the teachers distracted by a focus on the community from being learning-centred at school. This was echoed by the Literacy Leader who felt that the primary role for parents would be to get children to school.
Getting the kids to school, getting the kids to be present when they are at school, getting them here on time and making sure they’ve got their equipment; as long as they do that, that’s all we can ask. You don’t have that control over homework and things. There’s lots of families that have issues around homework, either they don’t understand it or they want more, or they think there’s too much and that’s no different to any other school. [Literacy Leader]
Several strategies to increase the conditions for learning had been developed and were being monitored. The Literacy Leader noted a community initiative was that they had organised breakfasts, and they provide lunches and tried diet control with fruit in an attempt “to make kids comfortable and well fed, (to enable) the best learning conditions possible”. Another was a change in interview structure which was solely focused on learning (not behaviour problems) and where the child was included in the discussion. Since 2006 they had used the Home School Partnership initiative to promote talking with parents. Using this model they had realised that the information flow was typically one way.
On the one hand we were saying, we want to share with you and talk with you and invite you into the school, and then in practice what we were doing was, yeah come and sit down and I’m going to tell you what happens and when I’m finished speaking you can go home and sort it out. [Principal]
Given this realisation from the Home School Partnership model, school meetings with parents are now run by a group of teachers and parents who have a shared role. Discussion groups based on ethnicity are formed and the discussion is shaped by a set of questions. The Literacy Leader commented that some really good ideas have come from this change.
The school is planning new initiatives. A new development is a group of parents who are to review how the school reports to parents. The review is to enable the school to evaluate how effective the current system is, although the Literacy Leader commented that there was little interest indicated from a survey so far. They send portfolios home and even previews (although it has turned into something of a “showcase” of finished copies). Another plan is to use the inquiry projects that groups of teachers run and present at the cluster conference as a basis for a discussion evening with parents. Yet another was to run workshops with parents during a whole day on the new curriculum.
The school did employ teacher aides who run adjunct programmes for children who might be struggling. It was not clear whether these were from the local community but they weren’t identified as part of the connection with the community. The evidence was that the school had viewed parent involvement primarily as motivational and as situated at home, but was moving to increase involvement at school and reciprocity.
Parents’ views
The concerns about parent involvement and participation and how the school is beginning to address these concerns highlighted by the Principal are also partly echoed by parents of the school. The talanoa with parents showed that they had high aspirations for their children’s education. Their beliefs about the school are positive and they see the school as ‘experts’ in their field. One parent puts it this way:
...O se kaimi muamua fo’i lea ua faia ai se kalakalagoaga aua o lea e kuu aku lava e makua fagau i a’oga aua e kele e iga aumai lakou homework...e moi foi a kake fesoasoagi fo’i lea ae pei lava o le mamafa lava e kuu lava i susuga a faiaoga ma a’oga i le a’oa’oina o fanau ma alo a le kakou atunuu. Pei lava o makua ia ga o siga fesoasoagi lava ae pei o le mamafa kele lava e pau lava lea i faiaoga ma le aoga i lea kausaga ma lea kausaga ile aoaoga o fagau.
[Translation] This is the first time a talanoa has happened because parents let the children attend school and they bring homework… it is also true that I help [with homework] but the weight [of responsibility] is left to the teachers and the school to teach our children. It’s like parents could support a little but the major work to teach our children is the school’s and teachers’ year in year out. [Samoan Parent]
However, there are certain areas, according to parents, where suggested improvements could be made. These areas were noted to be in reporting, assessment and support. In the reporting area, whether school reporting, parent interview reporting or face-to-face parent-teacher reporting, parents felt that the school had not been explicit in how they fed back the information about the achievement of their children. For example, one parent with six small children believed that the school should be frequently reporting back to parents instead of her waiting for the end of year report or when her child ‘gets naughty’. She thought that if her child was not achieving during the year, the end of the year report would arrive too late to allow her to provide support. She stated:
… I’d like to hear more from the teacher [during the year] other than just the parent interview. And if I can’t make it to that then I’d still like to get some feedback to see how she is going or any of my children. Yeah, but just to say that if there is anything that she is falling behind on within that you know then I can help her with, I suppose you only ever hear from a teacher when your child is naughty eh? [Cook Island Parent]
At the parent interviews, teachers discussed with parents how ‘good’ or how helpful the student is to others and the teachers, or the sports in which their child has been involved, but they never discussed the child’s achievement. One parent conveyed the issue this way:
E kalagoa mai a le faia’oga ‘o lae lelei lau kama. E fa’alogo foi. Soo se mea foi e fai aku ai e fai. E kele a iga fesoasoagi ae maise a le kamaikiki lea o [child’s name]...e kele a iga fesoasoagi Reg i isi kamaikiki. E kele foi a ina help i faiaoga i mea kau kaaloga ma mea fa’apena. ...Po o le a kogu lava le mea la e vaivai ai [her child] ... e leai se okooko mai ole mea kogu la e vaivai ai...poo le a le makaupu, pau a le mea o la e alright uma mea uma...
[Translation] Teachers say to me that everything is alright with my child. He listens too. I hear too that he does everything he’s asked to do. He helps other children most of the time and helps other teachers with sports and the like. [But in terms of] the exact academic weakness [her child] there is no specific identification of that or in what subject… only that everything is alright. [Samoan Parent]
Another parent commented on the need to see daily homework from school for his children. But what he wanted was examples and clear instructions on how to carry out homework. He claimed that reading Duffy books and the Bible also helped. However, when children had difficulty, the parent’s sister-in-law who is a lawyer then helped the children instead. When told that his child’s dream was to be ‘just like Daniel Carter and a playmaker’ he laughed and pointed out:
It’s good … but whether sports, you still need a good brain. [Niuean Parent]
One caregiver, a Tongan grandmother, thought the informal face to face reporting was good because the teacher was of the same ethnicity and she could understand what the teacher was saying. The grandmother explained that she got feedback from a Tongan teacher in the Tongan language which led to greater understanding of the progress being made by her grandchild at school.
The parents clearly had ideas about how they could contribute. When asked what the school could do to improve the achievement of their children, one parent suggested that the school should be looking into creating opportunities for children to compete academically.
O le isi mea e tatau ona fai i totonu o a’oga, e tatau ona fai ‘comptetition’ fo’i ia e aoga e tauva ai le tamaititi. E le fa’apea e compete fo’i lea, e leai aua o la e laiti. A alu fo’i la le tamaititi i le taeao i le a’oga, o la e tinou e fa’apea, ‘oute alu atu loa fai loa le...’ tai pei o mea fo’i e fai e tatou i Samoa fa’apea o le ‘spelling’ aua e iai tamaiti Māori, Samoa ma isi e leiloa sipela se upu ae lelei ituaiga polokalame na e tu’u atu se taimi fo’i lea i syndicate ma year levels eseese e ... pe fai fo’i le Maths competition fa’apea fa’aopoopo, fa’atele, ...e fiafia fo’i tamaiti e su’esu’e ma a’o mea aua o la e ua fiafia ... pei lava o le potopoto fo’i lea o le a fai se fiafia e fa’apena fo’i lo latou fiafia o le a fai le competition. Pei lava o le competition fo’i lea o le a mua la latou House fa’apea year 4 year 5 year 6 fa’apea ‘o lea o le a fai le tou spelling is tu’u i ai le sekone i le tamaititi e spell mai tle upu ...pei la e usu atu le tamaititi i le a’oga fa’apea o le a fai le competition a ia e tauloto atu lava le sipeli i le fale e o’o lava i le a’oga...a o la e ua look forward uma i ai.
[Translation] One thing the school must implement is the establishment of academic competition where children could be motivated to learn. Not in the true sense of competition because they are young. But when children go to school in the morning aware that there will be a competition they will say, ‘I am going to study really hard’ something like that … just like how we did in Samoa, because there are Māori, Samoan and other children who cannot spell and those programmes would be useful for different syndicates or different year level competitions …or it could be Maths [competition] such as ‘addition’ and ‘multiplication’. Children would be happy to learn to do well – and just as they are happy to get together for a cultural ‘fiafia’ they will also be happy to get together for an ‘academic’ competition or a House competition…because they all look forward to it. [Samoan Parent]
Parents had designed practices at home to support school achievement. One parent did not allow his children to watch television all week except Friday to Sunday. Similarly, the children had limited access to the internet and parents made sure to check on children’s homework and what they were doing.
O le lagolago lava lena a faia’oga e una’i e faitau tusi. O le tele lava o taimi e fai mai faia’oga e lelei tele le faitau ma e toa’ga pea e faitau tusi ina ia tumau ai. [Samoan Parent]
[Translation] Teacher support to me has been ‘keep pushing her to read’. Most of the time the teacher says she [his child] reads well, so push her to read and continue to read to sustain her reading. [Samoan Parent]
Other forms of teacher support occurred through the advice they gave to parents. One suggestion was for parents to tell the child to be selective in whom they choose to be around. One parent stated:
O le isi la support a faia’oga o le fai mai lea e fai i le fanau e pipi’i i le tamaititi e lelei ae aua le alu i le tamaititi e le faia ai se mea a’oga.
[Translation] The other support teachers give is they say, ‘tell your child to associate with good children and not to be around children who don’t do homework. [Samoan Parent]
One parent believed that it is the balance between school and home support that is most important and it is that which results in achievement. From her observations of her child’s academic development, she concluded that her child’s love of school and her teacher coupled with the family support had resulted in the child’s high academic achievement. In that sense, her child’s academic needs were met at school. She explained:
I know the teacher was very good. She liked the teacher. O le fiafia ia i le a’oga. A kuu la i lau fesili lea ga e fai mai e uiga I le support a…poo le support a makua poo le aoga [She loves school. If I have to respond to your question about support …albeit parent or school] I’m pretty sure its both. Aua aga fa’apea e lelei le support a makua ae fa’alekogu le a’oga ia kailo. A fa’apea foi la e lelei le a’oga ae leaga le support a makua ia e fa’apega lava. Ae ou ke iloaiga lava … [Because if there was good support from parents and the school support was not [good], I don’t know. Just as if the school support was and parents’ was not, well it would be like that. But I know…] I know her needs were catered for.
Inquiry processes and collective efficacy
It was predicted that effective schools would have robust and well-developed inquiry processes operating. Recent research both locally and internationally has implicated collaborative inquiry processes in improved outcomes for students, particularly for students in low socio-economic communities which are culturally and linguistically diverse (e.g., Lai, McNaughton, Amituanai-Toloa et al., 2009; McNaughton & Lai, 2009; Taylor et al., 2005; Timperley et al., 2007). Several of these interventions where inquiry processes have been central show improvement for their predominantly Pasifika student population with gains of up to one year in addition to expected national progress (Lai, McNaughton, Amituanai-Toloa et al., 2009). Professional learning communities are implicated in these interventions as the inquiry is often collaborative between teachers, researchers and other key stakeholders. A strong inquiry process is associated with a strong sense of efficacy held within the professional learning community.
The Principal and Literacy Leader described a school professional learning community that was very well versed in inquiry learning and which had as its basis a strong evidence base for the processes in place influencing achievement. Evidence for this collective efficacy came from detailed descriptions of how achievement evidence was used and how teams in the school were engaged in inquiry into the effectiveness of their practices. The senior team knew in great detail patterns of achievement over several years and could identify cohorts and individual classroom variations.
The data show that [the inquiry / learning focused approach] is helping, 8 odd years ago there was a target set to get 70% of our kids at or above, and last year we managed to get 2 of our year groups there. We’re still not at target, we still haven’t quite reached it but we’ve come a long way to get it….the data has gone up and down. Like 2006 we actually dropped right down. [Principal]
There was evidence from the descriptions of coherence across the school in the inquiry processes. From the junior team that had a focus on assessment through to the senior team there was both a general collective analysis and use of achievement data as well as specific projects which systematically gathered classroom data including data on instruction.
Looking to see whether or not, where the shifts have been made and for some of them and some of theories that the teachers and students have come up with, have worked really, really well, and some of them not so well. We are currently doing projects at the moment, same inquiry type thing and as part of filling the gaps we try and get them to find an alternative initiative or strategy that’s got some research backing behind it so that we can actually see how wherever that comes from, how their research went against teaching in the class. [Principal]
The deliberate maintenance of the professional learning community and its focus on inquiry was apparent in the commitment to induction of new staff. A very detailed plan including guided involvement in analysing classroom data was part of the induction.
We have cluster wide induction. A lot of it is around analysing the data… it doesn’t matter whether you are a beginning teacher or an experienced teacher… then our, they used to be called staff meetings, we now call them professional discussion forums – PDFs - also centre on building that, that kind of pastoral side or it but there’s also getting the classroom culture and the learning culture going… Then new teacher have their release, senior teachers are released as whole day a week to work with teacher and tutor teachers also get release time. [Literacy Leader]
Quality instruction that is culturally responsive
Our theoretical view outlined in the review section was that in effective schools there would be evidence for what the literature suggests is generically high quality instruction. But we also had the view that a generic feature of high quality instruction would be that it was culturally responsive. That is, distinctively effective ‘Pasifika pedagogical styles’ would reflect the generic need to have culturally responsive features in instruction. In the case of Pasifika students, like Māori students, that would be reflected not only in a deliberate use of background knowledge and styles of interacting, but also in mutual respect and positive relationships held by the Principal and Literacy Leader.
As in the other primary case study school, part of the view was that the pedagogy was not specifically Pasifika-focused. It was a more generically effective pedagogy focused on individualised instruction which had been tailored for Pasifika students.
What we’ve found is that there’s not so much an individual programme that is a Pasifika type programme but it’s more looking at pedagogy and the practice and taking or having an understanding of how we take that information or that knowledge from the teachers and get them to use it best with Pasifika children…rather than looking at ethnic-based type initiatives…we’ve gone from a different point of view…looked more at learning…every child no matter what ethnicity ..are all learners …we treat everyone as a learner…and design the learning to fill those gaps. [Principal]
Nevertheless the Team were very aware that there were important differences within the general label Pasifika and were wary of a general approach. The Principal was strongly of the view that they were not trying to use what would be deficit thinking about the backgrounds, and what he described as the “baggage that the kids might bring from outside”. The Literacy Leader held this generic view too but again implied that this meant it had a Pasifika focus.
The majority of the children…are Pacific Island so everything has to be about Pacific Island children because that’s our community. It’s all about good teaching, effective teaching. [Literacy Leader]
The observations in the classroom showed a marked difference between the two teachers. Of the two teachers observed, one had a relatively high total score (T1 = 84.5%) and the second teacher had the lowest total score (T2 = 58.3%). The percentages averaged for the three lessons are shown in Table 35.
| Classroom Features % | Instructional Dimensions % | Cultural Responsiveness % | |||||
|---|---|---|---|---|---|---|---|
| Talk | Knowledge | Strategy | Vocab | Feedback | |||
| T 1 | 94.4 | 88 | 73.3 | 66.7 | 75 | 83.3 | 100 |
| T 2 | 58.3 | 55.6 | 61.1 | 55.6 | 60.0 | 33.3 | 66.7 |
As noted in the overall results for the classroom instruction (Section 6), the scores are not very good indicators of specific gains in specific classrooms, even over three lessons. Across the schools the ratings of the instruction appear to be associated with the overall levels of the schools’ achievement and the gain over the most recent year, 2008. The overall score for the two teachers in this school was moderate (71%) and this school had moderate overall levels of achievement in 2008 (between stanine 3.4 and 4.1). It made a large above expectation gain in 2008 (M = 0.64 stanine).
| 1 STAR mean stanine. 2 Above expected gains (EG = 0). 3 STAR mean stanine. | |||||||||||||
| Teacher | Total % | Mean % | Class Achievement1 | School Achievement3 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n | Pre-test 2008 | Post-test 2008 | n | Pre-test 2007 | Post-test 2007 | Gain | n | Pre-test 2008 | Post-test 2008 | Gain | |||
| T1 (Y5-6) | 84.5 | 22 | 3.37 | 3.47 | |||||||||
| T2 (Y5-6) | 58.3 | 19 | 4.18 | 4.32 | |||||||||
| T1, T2 | 71.4 | ||||||||||||
| School | 318 | 3.34 | 3.92 | 0.58 | 403 | 3.44 | 4.08 | Above2 0.64 | |||||
The transcripts for both these teachers have elements of ‘best practice’ instruction. The dimension of Cultural Responsiveness was strongly present in T1’s classroom as recorded by the observers (M = 100%). In part this was made possible by the selection and use of particular texts or tasks for instruction. He drew on familiar artifacts (e.g., a Kiwiana theme for descriptive writing involved an ‘L & P’ bottle) and experiences (e.g., a word finding task included Niuean referents such as “Niue is a small…”) to activate and build vocabulary, background knowledge and thematic understandings in both descriptive writing and reading comprehension. The observer notes include comments on how warm and positive the class atmosphere was. There was a lot of humour but very explicit expressions of respect and clear rules. It was noted that this teacher (a Pasifika teacher) required children’s shoes to be off while inside and they were neatly placed at the back of the room. The observer notes also identify high engagement. Teacher 1 was very accepting but not at the cost of being uncritically affirming. However, achievement levels for his classroom were no higher than T2, and the rate of gain in the years was similar to T2’s classroom. T2 was particularly low on the dimension of feedback primarily because of low overall rates and a preponderance of instances of non-descriptive affirmation (“good”).
The students’ views about teachers and their instruction
Thoughts about the teacher
Teacher instruction and practices were supported by students’ comments. There was a sense of being affirmed and being respected in positive relationships. When asked what they thought about school, almost all students commented on how ‘fun’ school was because of the new learning, which motivated them to fulfil their dreams. For example:
[school] it’s … fun when you learn. [Year 5 Student]
I feel that it’s really good. My mum believes that I can get better and she believes that I can reach my goals if I go to school and it’s much easier if I come to school because then … she wants me to get a good job and yeah… [Year 6 Student]
It’s cool … cool teachers. [Year 5 Student]
With regards to the teaching, one student particularly was explicit about why she liked her teacher. She states:
…[my teacher] believes in me because he knows that I can get better in any … every subject…um …helping me if I’m like …if I have trouble understanding something, he’ll just come and help me. [Year 6 Student]
When the same student was asked whether there was anything she would like the school to do more of to help her learn, she replied:
No. It’s [the school] done a lot. I’m already smart.
Another student liked her teacher because of the clarity of teacher instructions on the mat before they do a task.
Before we do something, we always get it explained to us on the mat and stuff…
However, she also wanted the school to “do more stuff that we often do”. In this sense, the student’s preference is for academic work that is more challenging than what she currently gets.
One student, however, did not share what most of the students said. Although she believed the school was ‘a cool school’ she observed that students in the school are somehow treated differentially to others resulting in everyone not being equal:
But sometimes some students get treated more than other students…um…special treatment and we don’t get equal…like if you ask questions and things, it won’t be answered straight away and …things like that. [Year 6 Student]
Pasifika learners
The theoretical and empirical review provides a basis for assuming that the students themselves are sources of variance in achievement. That is, attributes of the students including their own beliefs and values as well as cultural and linguistic resources would contribute to effective learning and instruction. Given the limited amount of evidence however, our predictions were deliberately open-ended. We argued on the basis of the literature that there may very well be features of the Pasifika learners such as their language status that would be associated with achievement.
Language features
In the general analysis in one Focus Cluster (with Years 4 - 9) and when looking at gap differences we found that language, whether first language or language at home, was not associated with achievement. However, when looking at level differences (that is, differences in overall level achievement), we found that there were four main effects - gender, time lived in New Zealand, home language, and school - that were associated with significantly different levels of achievement. Overall, the mean scores for students that spoke mainly Pasifika languages at home and those that spoke two or more languages (Pasifika language as well as English) at home were significantly lower than that for the mainly English-speaking students. The general analysis results are reported in Section 3.2.2.
Language features and their relationship with achievement were examined in both the overall 2008 cohort of students as well as a smaller two-year longitudinal cohort from 2007 to 2008. The former consisted of students from Years 4 - 8 who had achievement scores from both the beginning and end of 2008 (n = 150). The latter consisted of students in Years 4 - 7 in 2007 and Years 5 - 8 in 2008 and present for both Pre-test and Post-test in 2007, and Pre-test and Post-test in 2008 (n = 91). These are small sample sizes and these analyses should be read as indicative and interpreted with considerable caution.
Unlike the general analysis there was no positive relationship between having English as a first language and higher achievement (see Table 37 ). Unlike the general analysis there was a relationship between having both a Pasifika language and English spoken at home and level and gain over a year (see Table 38 ). However, given the very small numbers these patterns are difficult to interpret.
| 1 Three students missing first language information, therefore not included in this table. | ||||
| First language spoken1 | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | ||
| English Only | ||||
| M | 3.71 | 4.05 | 0.34 | |
| SD | 1.48 | 1.61 | 0.95 | |
| n | 77 | 77 | 77 | |
| Pasifika Language Only | ||||
| M | 3.38 | 3.84 | 0.46 | |
| SD | 1.42 | 1.69 | 1.03 | |
| n | 63 | 63 | 63 | |
| Pasifika and English | ||||
| M | 3.17 | 4.50 | 1.33 | |
| SD | 1.17 | 1.52 | 0.52 | |
| n | 6 | 6 | 6 | |
| Other | ||||
| M | 5.00 | 4.00 | -1.00 | |
| SD | . | . | . | |
| n | 1 | 1 | 1 | |
| First language spoken1 | 2007 | 2008 | ||||||
|---|---|---|---|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | Pre-test | Post-test | Stanine Gain | |||
| English Only | ||||||||
| M | 3.39 | 3.95 | 0.56 | 3.85 | 4.15 | 0.29 | ||
| SD | 1.50 | 1.63 | 0.84 | 1.71 | 1.77 | 0.90 | ||
| n | 41 | 41 | 41 | 41 | 41 | 41 | ||
| Pasifika Language Only | ||||||||
| M | 3.07 | 3.60 | 0.53 | 3.49 | 4.02 | 0.53 | ||
| SD | 1.47 | 1.42 | 0.96 | 1.44 | 1.74 | 1.05 | ||
| n | 43 | 43 | 43 | 43 | 43 | 43 | ||
| Pasifika and English | ||||||||
| M | 3.60 | 4.40 | 0.80 | 3.40 | 4.80 | 1.40 | ||
| SD | 1.52 | 1.34 | 0.45 | 1.14 | 1.48 | 0.55 | ||
| n | 5 | 5 | 5 | 5 | 5 | 5 | ||
| 1 Two students missing first language spoken information, therefore not included in this table. | ||||||||
There was some indication that by the time students were in the upper levels of the school (Years 5 - 8), an earlier gap between children with only a Pasifika language at home and those with only English at home had closed (see Figure 22 ).
Figure 22: Mean stanines by first language for longitudinal cohort (Case Study 1).
Education in the New Zealand system
It could be predicted that greater familiarity with New Zealand educational practices (which is almost always confounded with immersion in English instruction) would be associated with higher achievement. This was found in the overall analysis, however, students who had lived in New Zealand for more than five years had similar levels of achievement and gain to those born in New Zealand. In this Case Study School we found that students who were born in New Zealand had higher mean stanines than those born elsewhere at both time points (see Table 39 ). The highest gains, however, were for those born in the Pacific region. This was complimentary to the general finding whereby the length of time lived in New Zealand was associated with different levels of achievement. The mean scores for those that lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than those that had lived in New Zealand between one and five years.
| 1 Four students missing birth country information, therefore not included in this table. | ||||
| Birth country1 | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | ||
| Pacific Region | ||||
| M | 3.17 | 3.83 | 0.66 | |
| SD | 1.42 | 1.67 | 0.77 | |
| n | 29 | 29 | 29 | |
| New Zealand | ||||
| M | 3.66 | 4.10 | 0.44 | |
| SD | 1.46 | 1.60 | 1.02 | |
| n | 112 | 112 | 112 | |
| Other | ||||
| M | 3.60 | 3.20 | -0.40 | |
| SD | 1.52 | 1.48 | 0.55 | |
| n | 5 | 5 | 5 | |
The general interpretation of this finding is supported by examining the patterns for students who had spent differing amounts of time in New Zealand. Students who were not born in New Zealand but who had lived in New Zealand between one and five years had higher mean stanines at all time points than students who had lived in New Zealand for five or more years (see Table 40 ).
| 1 Ten students missing time in New Zealand information, therefore not included in this table. | ||||
| Time in New Zealand1 | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | ||
| 1 - 5 years | M | 3.36 | 4.00 | 0.64 |
| SD | 1.43 | 1.73 | 0.67 | |
| n | 11 | 11 | 11 | |
| > 5 years | M | 3.21 | 3.68 | 0.47 |
| SD | 1.58 | 1.70 | 0.84 | |
| n | 19 | 19 | 19 | |
| Born in NZ | M | 3.65 | 4.09 | 0.44 |
| SD | 1.46 | 1.61 | 1.03 | |
| n | 110 | 110 | 110 | |
Student views
As noted earlier, all students had high hopes of a good future and they all believed that education would get them there. When asked whether the school was preparing them for their future, students were almost all in consensus that it was. They preferred to have “more homework and harder stuff”. When asked whether they let the teacher know about this, one replied, “Yeah, but he’s trying his best to make it hard”. Overall, there was evidence that students positively appreciated teaching and learning at the school. Consistent with existing research reviewed in this report, Pasifika students expressed high motivation. They wanted more challenging teaching and to achieve highly.
3.3.2 Focus Cluster School: Case Study 2
Overview of the school
This was a medium to high performing decile 3 secondary girls’ school. Overall achievement at the end of Year 9 for Pasifika students on asTTle reading comprehension was 627.95, which was close to the national norm (634) and at Year 10 it was 708.7 (national norm 728) (see Table 41 ). It is expected that from the end of Year 9 to the end of Year 10 students would gain 94 points. In general, the Pasifika students made higher than nationally expected gains within years. Within the different Pasifika groups, however, there were differences in achievement when tracked longitudinally over two years. Case Study 2 Pasifika students gained a mean of 80.77 points from the end of Year 9 to the end of Year 10. Within Pasifika groups, Tongan students made the greatest mean asTTle point gain, above the nationally expected gain (M = 153.33). The Samoan group also had a mean gain marginally above the nationally expected mean asTTle points gain (M = 94.65).
| 1We used ethnicity information from students’ self-report in the language surveys. The asTTle data provided only reports four main ethnic groupings: Māori, Pasifika, New Zealand European and Other. 74 of the pre-post students were identified as being Pasifika from the asTTle files. Of these, only 43 had completed language surveys. Therefore, the remaining 31 Pasifika students are listed as ‘unspecified’. 2These students were Year 9 in 2007 and Year 10 in 2008. | ||||
| Pre-test 2007 | Post-test 2007 | Post-test 2008 | ||
|---|---|---|---|---|
| Tongan | ||||
| M | 459.00 | 520.67 | 674.00 | |
| SD | 136.17 | 176.66 | 93.21 | |
| N | 3 | 3 | 3 | |
| Cook Island Māori | ||||
| M | 572.50 | 664.50 | 750.50 | |
| SD | 61.52 | 9.19 | 40.31 | |
| N | 2 | 2 | 2 | |
| Samoan | ||||
| M | 528.59 | 624.35 | 719.00 | |
| SD | 81.52 | 71.19 | 65.42 | |
| N | 17 | 17 | 17 | |
| Other Pasifika | ||||
| M | 534.71 | 638.57 | 715.24 | |
| SD | 52.70 | 68.50 | 47.67 | |
| N | 21 | 21 | 21 | |
| Pasifika Unspecified | ||||
| M | 523.29 | 630.74 | 699.32 | |
| SD | 51.57 | 50.18 | 62.68 | |
| N | 31 | 31 | 31 | |
| Total | ||||
| M | 526.47 | 627.95 | 708.72 | |
| SD | 64.34 | 69.54 | 60.06 | |
| N | 74 | 74 | 74 | |
The overall total population of the Pasifika group at the school is estimated to be 55%3 including Year 9 and Year 10 Pasifika students. The breakdown for the Year 9 and 10 groups we examined was Samoan 45%, Tongan 10%, Cook Island Māori 7%, Niuean 5% and ‘Other Pasifika’ (including more than one Pasifika identity) 32%. Exactly half of these students had English as first language, with Samoan being the next most common first language (15%). Of the total Pasifika group, two thirds were born in New Zealand.
Connectedness – community and school
The theoretical prediction here was outlined for Case Study 1. The evidence in the present Case Study was that the Principal had thought strategically about involvement and close connectedness. This had come after a series of ‘hit and miss’ strategies with different Literacy Leaders at different times, where they had not been able to identify strategies that would encourage parents to participate. This is because the initial focus was directed inwardly - on strategies and programmes to be developed for achievement of students and not on strategies to increase connection.
The particular strategies that we’ve used to meet the needs of Pacific Nations students have really developed over time. Because we really, I think we just didn’t understand how deeply we had to delve into this. I think people had been lulled into a false sense of security with the literacy across the, language across the curriculum that everyone used to do. And sort of thought that that was what it would be all about. [Principal]
This admission was in many ways a sign of relief and one which set the school in a new direction thus signalling a new beginning.
It very quickly become apparent to me and I guess it helps because I’m an English teacher by [specialisation], you know, that’s what I was when I was a teacher - classroom teacher … became very clear to me that it was the pedagogy that needed to change. It was a much deeper thing than simply dishing out a few strategies and saying this is what you need to do. [Principal]
A new beginning was to start with student engagement and new forms of inquiry.
The way we’ve done things in the past does not work, cause we do not get that sort of engagement that we need to get in the Junior school. It doesn’t suddenly happen when they get into the Senior school. What we’ve seen, because we, it’s quite a change in the type of student we get coming into our school, we really need to get them engaged at Year 9 and Year 10. That’s why we’re really keen on getting the Gifted and Talented teacher involved because she has, she does a focus on this whole business of Inquiry learning and thinking strategies, thinking skills, theory of knowledge. We want to do that with all of our students. [Principal]
Nevertheless, several specific programmes designed to increase connections were identified. For example, the delivery of information about asTTle to parents and the community extended to much more focused events during an informal barbecue. Others were classroom-specific activities where Form teachers held term meetings with their parents in smaller groups, thus creating opportunities for parents to talk one-on-one. Although the information exchanged was concerning analyses of achievement from asTTle and the form and subject teachers’ focus for the following term, these were also attempts at making bigger connections with the community.
A wider community connection, for example, is evident in a yearly Fono at which achievement data is again, with the help of a translator, discussed in addition to NCEA results being compared with national norms and decile averages. In addition, there are mentoring schemes run by community members – one for Year 12 - 13 and one for Year 10 students. There was not, however, a sense from the interview that the parents were given specific information about how to help support their children academically or how the school could learn from the parents about cultural and linguistic backgrounds and resources. Thus, the school leadership was aware of the community and had several programmes in place to involve both parents and the wider community, but it was unclear how reciprocal this relationship was in terms of mutual learning and how specifically parents might have been given information with regard to how to provide academic support for students.
The Principal felt the school now had more engagement with students and noted the generalised effectiveness for Pasifika students of the Te Kotahitanga (TK) initiative. There is a sense that the school is perhaps ‘testing the waters’ in an attempt to understand its community better; to see which programme fits and which strategy could work best for parents and the community at this stage for reciprocity to occur.
Inquiry processes and collective efficacy
The theoretical prediction was outlined for Case Study 1. In this school, the Principal acknowledged the impact Te Kotahitanga has made to teachers and their practice, but most importantly to student achievement. Though the tendency of teachers was to go the ‘middleclass European way of doing things’ by default, she admits that the Te Kotahitanga initiative has made teachers realise the importance of inquiry learning, of self-examination and critique and questioning why things are the way they are. The comment below illustrates the evidence of implementing Te Kotahitanga.
… at the end of 2006 where we started on Te Kotahitanga taking it into last year we decided we wanted to focus on Pacific Nation students using that same strategy so the teachers were asked to, the teachers of literacy were asked to consider, make an effort to consider at least 4 of the students, in the same way as they were doing for Māori students, at least 4 of the students in their classes to focus on their needs in terms of literacy and numeracy to get them to where we want to go to show that interest. We found that that had a very empowering effect on their learning as well. Then that plus the fact that we really started to push the idea of group work or students working in groups, student centred learning all that sort of stuff, they were then, the students were then sharing their knowledge, what they’ve learned, helping other students in the class. [Principal]
The Principal referred to detailed forms of evidence and communicating that evidence to parents. Clearly through the Schooling Improvement initiative, and especially Te Kotahitanga, there has been a clear focus on pedagogic styles in addition to gathering and managing robust databases for tracking students. The Principal used evidence to support claims regarding improvement.
There’s a lot of things that have been happening. It’d be hard to pull out one thing and say this is what made the difference from the students’ point of view, but I do believe that from the teacher point of view as far as pedagogy is concerned it has been TK that has made the big shift and teacher pedagogy in Year 9. [Principal]
The Principal also made claims about visible differences in relation to her broader educational philosophy and goals for students in the school.
And it’s been really good to see that difference. It’s certainly supported my particular philosophy I suppose you would call it. Which is that we’re in a low decile school, we’re very multicultural, and if we want to break the poverty cycle we would want to create young women who are leaving school who are going to be beneficial and do great things for our communities then they’ve got to leave with qualifications. [Principal]
Sustaining the achievement, however, and getting teachers to fully understand the importance of change in the way they teach and in the way they do things, given the less effective pedagogical practices of the past especially for Pasifika students, is another goal for the school.
Getting that change starting to happen and realising that in fact it was making far bigger difference than anything they’ve done in the past that’s been the big challenge. I think that taking a step back and looking from further away our retention rate of Pacific Nation students has increased enormously. I do think that having that Year 10 high-motivation class has made a big difference cause they continue in that class in Year 11, 12, 13. I think that’s been a huge help. [Principal]
Parents’ views
Whilst the Principal’s views about ‘change’ focus mainly on the internal practices of the school, generally parents agreed that what the school is doing is ‘pretty good’ by frequently informing them about developments. This is illustrated by the responses of two of the parents:
… I think what I’ve seen … what they do is pretty good like keeping us informed regularly and frequently and constantly kind of thing and that is really all what we need. We just need to know she [her child] is not wagging class and in 4 weeks time they are going to be taught sex education that kind of thing. [Parent]
But if there is something that school can do. [School B1’s] actually quite good, they have been keeping us informed. Sometimes sending us letters like just to give an understanding of where they are at in school, the forms and what they are about to teach and they are really good. [Parent]
The home-school partnership was important to the majority of parents as one way to realise their aspirations for their children. The parents had specific ideas to contribute about home and school practices that might help achievement. It was apparent that there might be areas that the school could focus on in order to increase parent participation. This is conveyed by one parent:
… a lot of parents come over with their ‘fanau’ not being able to speak English very well, teaching the parents English to help them teach their children you know, that would be a help. [Parent]
When asked whether she was suggesting that the school run workshops for parents, she responded:
Well yes, because a lot of the parents that come to school don’t speak English and the girls are translating for them or their English is very limited. [Parent]
One parent did not agree that parents should come to the school. She claimed that although volunteering in the school is a good strategy to get parents involved, she did not think that this was applicable to parents of teenage students:
No, I don’t think, at that level I don’t think getting parents to come and volunteer their time at school, in school, is not a good idea. I know with my youngest at her school ever since kindy they have always tried to get parents involved like volunteering their time and get around their kids, and that was good for their growth at that age, but at school it’s more of a teenage world there where everything has to be so cool or if you are not cool you are not in. So it’s not cool to have your mum come [laughs] in [her child’s] eyes it’s not cool to have your mum come to school and do something there. It’s just not cool. [Parent]
In the case of students not performing to school expectations, parents suggested that the school should find out from them what the cause of that might be - as conveyed by one parent in the school:
Like if there’s anything the child is not to expectations, then the school should ring parents and let them know then the parents tell them what’s going on…and what should be done. By doing that, the parents will know the degree in which the relationship between the school and the home is at. What the kids between them or what’s happening at home that the school is unaware of. [Parent]
This was also supported by one parent when she indicated the importance of having Pasifika teachers in the school at meetings:
Because that way they understand more of what Pasifika Islands are going through…to help in the school with sorting out students…can help parents who are not fluent English speakers, and work as go between home and school…That’s what I think anyway… [Parent]
In the case of their support for their children’s homework, parents would also prefer that the school give explicit examples of how these should be done in order for them to guide their children. For example:
It’s just that some parents say that teachers should be able to give examples of homework and how it should be done so that parents could follow it so… [Samoan Parent]
But perhaps a general desire by parents especially the grandparents is to see visible evidence of the education success of their grandchildren before they pass over. The comment below is furnished by one of the parents concerning her father’s wishes for her child.
[Of her father] I do actually, he never said specifically what, but one day they were talking and he said, “if all I want for you is get to Uni, graduate before I go” [laughs] and that’s all he wants, he says, “try hard at school and you get to university, but when you are there you finish it and if that’s the last thing I see before I leave this earth then you have done it, you have made me proud” [laughs]. [Cook Island Parent]
Quality instruction that is culturally responsive
Our theoretical view was outlined for Case Study 1. The school has a deliberate focus on Pasifika learning and achievement, which has followed a strong commitment to Te Kotahitanga. The Principal believed that over time a distinctive pedagogy that was more effective for Pasifika students had been developed. It was comprised of an amalgam of several strands which included elements of a new inquiry model drawn from Te Kotahitanga and explicit instruction in literacy content and features, such as main ideas and use of text structure (borne out in the observations in classrooms see below). It was in part content-based drawing on text resources.
We had Pacific Nations writers coming to assemblies to read stories they’d written, books they’d had published. [Principal]
The push in reading included buying many new books and short stories that had a Pacific nations focus for SSR. But again this was one prong of a coherent set of strategies. One Literacy Leader had visited every Year 9 class to read a book in Samoan, explaining how she worked out some meaning from her limited knowledge of Samoan as a means of modelling how to get the gist and making an ‘informed guess’. The effect of Te Kotahitanga had been felt in two areas. One was in the area of an inquiry-based pedagogy and the other was the need for showing an interest in cultural background and individual learning needs. Other strategies included group-based work to shared knowledge and also an understanding of the socialisation of Pasifika girls.
In summary this account (supported in part by the classroom observations) suggests a deliberately designed pedagogy that has generic features such as inquiry and explicit teaching and includes cultural responsiveness. The observations described below support this overview in part, such as the explicit literacy teaching and the use of familiar textual themes, but do not fully support a sense of heightened cultural responsiveness or extended inquiry (which, in the observations, would be tapped by the dimension of Extended Talk).
Of the two teachers observed, one had a relatively high total score (T1 = 87.3%) and the second teacher one of the lowest total scores across schools (T2 = 60.4%). The percentages averaged for the three lessons are shown in Table 42 . In some instances the scores reflect the unique characteristics of secondary organisation. For example, T2’s classroom was scored low on features partly because it was the home room for German teaching and was not a ‘rich’ environment for English teaching. By contrast T1 was observed in her home room and there were many examples of student work, reminders and parts of projects.
| Classroom Features % | Instructional Dimensions % | Cultural Responsiveness % | |||||
|---|---|---|---|---|---|---|---|
| Talk | Knowledge | Strategy | Vocab | Feedback | |||
| T 1 | 100 | 61.2 | 94.4 | 93.3 | 86.7 | 66.7 | 83.3 |
| T 2 | 61.0 | 38.9 | 83.3 | 50 | 38.9 | 88.9 | 61.1 |
As noted in the overall results across schools for the classroom instruction, the scores are not very good indicators of specific gains in specific classrooms, even over three lessons. The patterns observed in this Case Study school were not as expected. The teacher with the higher instructional score (T1) had lower gains in 2008 (classroom gain = 76.40), but in 2008, the students in her Year 9 classroom were above the national norm for asTTle scores (634). The second teacher (T2) followed the opposite pattern, with higher gains (2008 classroom gain = 99.35) but lower levels.
In general across schools, however, the single teacher ratings of the instruction appear to be associated with the overall levels of the schools’ achievement and the gain over the most recent year, 2008. The overall score for the two teachers was moderate (74%) and this school had moderate overall levels of achievement in 2008. At the end of the school year in both 2007 and 2008, the mean asTTle reading score equates to curriculum level 4P. This is at national norm but below the Curriculum expectation. The school made a below expectation gain in 2008 (asTTle reading mean gain = 83.58). See Table 43 for the summary.
| 1 STAR mean stanine. 2 Below Expected gains = 117, but levels similar to nationally expected levels. | |||||||||||||
| Teacher | Total % | Mean % | Class Achievement1 | School Achievement1 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n | Pre-test 2008 | Post-test 2008 | n | Pre-test 2007 | Post-test 2007 | Gain | n | Pre-test 2008 | Post-test 2008 | Gain | |||
| T1 (Y9) | 87.3 | 5 | 568.80 | 645.20 | |||||||||
| T2 (Y9) | 60.4 | 16 | 510.81 | 610.63 | |||||||||
| T1, T2 | 73.9 | ||||||||||||
| School | 88 | 525.06 | 626.85 | 101.80 | 77 | 533.23 | 616.82 | 83.582 | |||||
The transcripts for these two teachers have elements of ‘best practice’ instruction. One teacher had noticeably different scores in all areas apart from Feedback (illustrating that teachers who rated highly on any one dimension tended to rate highly across all areas). Some of the opportunities to observe specific instructional dimensions were restricted because of the prevalence of independent and/or silent study in the classrooms. However, it is worth noting that even when opportunities were present, High Level Talk did not occur very frequently for either teacher whereas a Core Knowledge Focus was noticeably present (e.g., specific questions on worksheet about novel). The dimension of Cultural Responsiveness was not strongly present in either classroom as recorded by the observers. For T1 in two of the lessons there was “no particular appreciation made of ‘backgrounds’, but evidence of high positive expectations”. T1 used more engaging texts and tasks such as writing a newspaper article. Aspects of her instruction, including her expectations and dimensions, are captured in the introduction to the series of lessons.
Now, you may have done newspaper article writing last year during your intermediate school. What we are looking at this year is progressing in our writing skills. We are looking at a higher standard of writing. A really good title. Good introduction. Indirect speech, quotes. Detailed description of the event. What I'm going to give you today is a sample of a newspaper article for you to look at and analyse it just before you get onto your work. Or you may refer to it in between your work at your convenience. What we also have later is a checklist, you are going to keep the checklist in front of your group so you know what we are looking at, at the finished product. [Teacher]
The lesson involved specific instruction in core knowledge (such as writing in the third person, use of pronouns, use of headers and so forth).
The students’ views about teachers and their instruction
Thoughts about the teacher
The students had distinct views about teachers, teaching and learning. There are some consistencies between their views at a general level but comments about individual teachers show inconsistencies. When asked about their views of their teachers, one student preferred to have a good relationship with the teacher because this is important to her. She likes some of her teachers, particularly the ones that are “really nice, they motivate us to learn more”. She described her English teacher as “not bad” although she sometimes uses words that they don’t understand well. She does ask her teacher to clarify on such occasions and her teacher obliges.
A second student noted that her English teacher was very helpful in instruction. Yet another girl said what she liked most about her English teacher was that she was prepared to go over material as many times as necessary so that the students “get it”. She feels that her teacher really wants her students to pass their tests and encourages them to stay on at school. Another noted that her English teacher explains to them what to expect next year and prepares them for it. Although she thought her English teacher is good at what she does, she didn’t find English to be a very interesting subject. Of two other students, neither liked their English teacher because they have difficulty understanding the way she talks because of her accent. They liked the way their English teacher taught except when she let them do what they wanted. They felt that things are not explained well enough in English and that they move on too quickly through topics before they have come to understand them.
One student described her teacher as “the best” and “the bomb” because she’s not strict:
Like if you forget your homework or something, she’s like, okay, bring it tomorrow and if you forget it tomorrow, she’s like, okay, bring it the next day. She’s not that strict.
She claimed that her teacher gets onto the students’ level at times and makes the work fun.
She’s just cool. She acts like she’s our age sometimes, she treats us like normal people, not like children.
Students had views on teachers’ pedagogical styles, making reference to the emotional connection and support. One of the students liked the way her ICT teacher teaches her because “she’s really nice and she knows what she’s doing and she really supports our class”. Her English teacher tries to motivate the students and allows more time for completion of work if necessary. Another student, on the other hand, liked the fact that her teacher intersperses fun with the serious work:
…there’s times where we could have, like fun, but then there’s times where we have to be serious.
One student identified another aspect of the relationship in the way the teacher “trusts us, like if we don’t bring homework she can tell us to bring it another day”. She also likes the way her teacher explains things, although the students don’t always get it all the same, mainly because she “talks too fast”.
The students did comment on the limited use of background knowledge. Two students did not find the topics interesting because their teachers don’t relate to things they are interested in, such as “our favourite singers”. They would like to have greater choice and have their opinions listened to. They do however like working in small groups, something which happens often in their English class. As one student explained, this is preferred because “if [the other students] haven’t done any work and I haven’t done any work then it’s not just me getting into trouble.” Another reason for preferring group work is that “you get more help when you’re with your friends”.
Pasifika learners
Language features
Language features and their relationship with achievement were examined in a two-year longitudinal cohort who were Year 9 in 2007 and Year 10 in 2008 (n = 43). In some analyses for ethnicity and language patterns these students were augmented by those students who were Year 9 in 2008 and had beginning and end of year asTTle scores (n = 39). These are very small sample sizes and these analyses should be read as indicative only and, therefore, interpreted with considerable caution.
Unlike the general analysis for level differences there was not a positive relationship between having English as a first language and higher achievement (see Table 44 ). Again unlike the general analysis the relationship between having both a Pasifika language and English spoken at home, and achievement level over a year was not negative (see Table 45 ).
Given the numbers these patterns are difficult to interpret. At the very least these data show that by Year 9 having spoken a Pasifika language first or having a home that is bilingual is by no means an impediment to high achievement.
| First language spoken | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | ||
| Year 9 National Norm | M | 517 | 634 | 117 |
| English Only | ||||
| M | 536.42 | 600.58 | 64.16 | |
| SD | 115.32 | 94.90 | 61.40 | |
| n | 19 | 19 | 19 | |
| Pasifika Language Only | ||||
| M | 539.73 | 649.36 | 109.64 | |
| SD | 86.99 | 58.91 | 77.97 | |
| n | 11 | 11 | 11 | |
| Pasifika and English | ||||
| M | 486.60 | 544.20 | 57.60 | |
| SD | 83.72 | 150.78 | 103.01 | |
| n | 5 | 5 | 5 | |
| Other | ||||
| M | 623.25 | 681.50 | 58.25 | |
| SD | 92.25 | 47.65 | 58.27 | |
| n | 4 | 4 | 4 | |
| Language spoken at home | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | ||
| Year 9 National Norm | M | 517 | 634 | 117 |
| English Only | ||||
| M | 539.91 | 625.73 | 85.82 | |
| SD | 110.23 | 110.23 | 74.77 | |
| n | 22 | 22 | 22 | |
| Pasifika Language Only | ||||
| M | 467.00 | 509.50 | 42.50 | |
| SD | 84.28 | 100.36 | 79.92 | |
| n | 6 | 6 | 6 | |
| Pasifika and English | ||||
| M | 561.78 | 646.67 | 84.89 | |
| SD | 69.47 | 55.10 | 62.92 | |
| n | 9 | 9 | 9 | |
| Other | ||||
| M | 659.50 | 679.00 | 19.50 | |
| SD | 129.40 | 80.61 | 48.79 | |
| n | 2 | 2 | 2 | |
Education in the New Zealand system
It could be predicted that greater familiarity with New Zealand educational practices (which is almost always confounded with immersion in English instruction) would be associated with higher achievement. This was found in the overall analysis. Due to the small numbers in this case study school a simple summary is not possible. We found that being born in New Zealand was not associated with a higher asTTle level score or gain (see Table 46 ). This was contrary to the general finding whereby the length of time lived in New Zealand was associated with different levels of achievement. The mean scores for those that lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than for those that had lived in New Zealand between one and five years.
| Birth country | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | ||
| National Norm | M | 517 | 634 | 117 |
| Pacific Region | ||||
| M | 551.11 | 607.44 | 56.33 | |
| SD | 120.32 | 113.29 | 80.33 | |
| n | 9 | 9 | 9 | |
| New Zealand | ||||
| M | 533.86 | 611.18 | 77.32 | |
| SD | 103.41 | 92.19 | 71.13 | |
| n | 28 | 28 | 28 | |
| Other | ||||
| M | 573.50 | 710.50 | 137.00 | |
| SD | 43.13 | 54.45 | 11.31 | |
| n | 2 | 2 | 2 | |
| Time in New Zealand | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | ||
| National Norm | M | 517 | 634 | 117 |
| < 1 year | ||||
| M | 369.00 | 477.00 | 108.00 | |
| SD | . | . | . | |
| n | 1 | 1 | 1 | |
| 1 - 5 years | ||||
| M | 620.67 | 676.67 | 56.00 | |
| SD | 113.56 | 57.14 | 72.02 | |
| n | 3 | 3 | 3 | |
| > 5 years | ||||
| M | 553.71 | 625.86 | 72.14 | |
| SD | 86.75 | 120.13 | 91.26 | |
| n | 7 | 7 | 7 | |
| Born in NZ | ||||
| M | 533.86 | 611.18 | 77.32 | |
| SD | 103.41 | 92.19 | 71.13 | |
| n | 28 | 28 | 28 | |
Student views
What students thought about their school
Students’ views have been described above. Here we identify their thoughts about school more generally and ideas about making school better for them. When asked what their thoughts were about school, student responses varied from ‘not bad’ to ‘boring’. One student thought that school is “not bad”. She likes drama because it’s fun, and also likes maths and English. She claimed that the other good thing about school is that she gets to spend time with her friends. She doesn’t always feel really good about going to school but when she’s there “it’s okay”.
Another student described school as “alright”, adding that she mostly comes to school for her friends, but that “education is cool too”. Other girls enjoyed a variety of subject areas including social studies, PE, ICT, music, English and German and because they get to see friends, meet new people and learn “different stuff”. Two students think school is boring because the teachers “don’t make it fun” for them and “don’t like listening to [their] point of view”. Subjects were liked for a variety of reasons: because they can get to “do [their] own thing” in between working and because they “get away with talking”; or because science and maths was interesting but English was not interesting. These two students did not feel very good about school. They don’t get excited about their learning and sometimes simply “can’t be bothered” because they feel that they are not learning anything. The main reason they come to school is to be with their friends.
Clear views about homework were held, varying from “okay” and “pretty straightforward” to quite bad. There were differences between teachers in how much homework was set. For example, three girls said they didn’t get much homework in English, and it only takes about half an hour to complete it. The homework is sometimes hard, but one student thinks that is preferable to getting easy work because it is more “challenging”. This student does her homework independently, in her bedroom. One girl does her homework by herself and usually works in her room or does it at school in the morning before school starts. Some of the homework is challenging but their teacher goes through it with them. Both girls usually do their homework without help, but will get help from a sibling or parent if necessary. Homework is usually completed in a bedroom.
The girls said that the homework which they do get often does not get checked by their teachers, so they generally don’t bother to do it. If the homework is unfinished class work, they will do the work at lunchtime if they have to. If they get told their homework is going to be checked, the girls will do it before school or sometimes at home. If the girls understand the homework, they will do it by themselves. Otherwise, they may copy it from friends to get it done faster so they won’t get into trouble.
The girls were asked about what they would like changed at school to assist learning. Again, relationships, the style of teaching and the challenging nature of the work together with higher expected levels of difficulty were often among the responses. One said she wanted her teachers to have “a better attitude” towards the students and “be more supportive instead of yelling”. Two girls echoed her concern and believed that they would learn better if their teachers had a better attitude instead of being “grumpy”. Anotherwould like teachers to teach in a more interesting manner rather than talking from the front of the class. She felt that more small group work would make learning more interesting. These features are contained in the following statement:
They'll take their anger out on us, so that's when we backchat them… yeah it's easy to get into an argument with the teachers here cause they're not as, they don’t understand you. Like when you say something they just tell you to be quiet and do your work and you get angry and then you backchat them and it's all because they don’t talk to you about what happens.
For the most part, the students get along with the other students who they think are “really cool” but some students don’t behave well in class, which can be a bit distracting. One of the girls described their peers as being “like family”. Two other students said they feel sorry for the other students because they fight over boys a lot. They believed there was “too much hate at this school and bullying”.
All students expressed a preference for being taught in English.
3.3.3 Case Study 3
Overview of the school
This was a decile 1 contributing primary school. Overall achievement was high, with levels across time between stanine 4 and 5 in reading comprehension. The Pasifika students as a group consistently made higher than nationally expected gains within years. When tracked longitudinally over two years, there were differences in achievement for the different Pasifika groups within years (see Table 48 ).
The total Pasifika group at the school overall is approximately 74%. The breakdown for the Years 4 - 6 groups we examined was Samoan 48%, Cook Island Māori 28%, Tongan 21%, and ‘Other Pasifika’ 3%. Almost half (48%) of the students had English as first language, with Samoan being the next most common first language (29%). Out of the total Pasifika group, more than two thirds of students (77%) were born in New Zealand.
| Pre-test 2007 | Post-test 2007 | Pre-test 2008 | Post-test 2008 | ||
|---|---|---|---|---|---|
| Tongan | |||||
| M | 4.00 | 4.17 | 4.83 | 3.67 | |
| SD | 0.63 | 0.75 | 0.75 | 1.03 | |
| n 1 | 6 | 6 | 6 | 6 | |
| Cook Island Māori | |||||
| M | 5.00 | 5.00 | 4.63 | 4.38 | |
| SD | 1.07 | 0.76 | 1.06 | 1.41 | |
| n | 8 | 8 | 8 | 8 | |
| Samoan | |||||
| M | 4.14 | 4.57 | 4.21 | 4.50 | |
| SD | 1.29 | 1.60 | 1.81 | 1.61 | |
| n | 14 | 14 | 14 | 14 | |
| Other Pasifika | |||||
| M | 3.00 | 4.00 | 4.00 | 4.00 | |
| SD | . | . | . | . | |
| n | 1 | 1 | 1 | 1 | |
| Total | |||||
| M | 4.31 | 4.59 | 4.45 | 4.28 | |
| SD | 1.17 | 1.24 | 1.40 | 1.41 | |
| n | 29 | 29 | 29 | 29 | |
| 1 Numbers too small to run t tests. | |||||
Connectedness – community and school
Our theoretical prediction was outlined for Case Study 1. The evidence in the present case indicated that the school had considerable involvement and close connectedness. In terms of Epstein’s 6 types of involvement, the deliberate strategies the school helped activate were both the ‘help’ and ‘motivational’ types as well as the specific academic input types (Epstein, Coates, Salinas, Sanders, & Simon, 1997). The latter two types, involvement that impacts on motivation and on academic skills, are consistently related to academic achievement.
The school leadership deliberately and strategically planned to include the community. The school has a community initiative around Parents as Reading Tutors (PART) which involves parents assisting as reading tutors using the pause, prompt, praise programme. The school is also participating in the Home-School Partnership programme. Parents are seen as a key ingredient in the success of these initiatives and clear roles have been carved out for them on the understanding that developing a culture through leadership is a priority, and enables the school to give them a place to belong to and be valued.
I think we've developed a culture here where they feel part of the [school name] community. They feel, you know, a sense of belonging and a sense of that they are valued as a partner. And I think that the fact that the parent interviews that we had at the beginning of the year, we had 95% parent turnout – which is quite significant. [Literacy Leader]
The involvement also draws on their particular cultural expertise. As part of the Home-School Partnership initiative they have identified their key ethnic groups and harnessed the cultural and pedagogical expertise of their Pasifika teachers and leaders in their communities to communicate with parents, share the achievement data, and disseminate in their language strategies other parents can use to support their children’s literacy and numeracy development.
The Principal’s view of the teachers as having specific cultural expertise is conveyed in the following comment:
[We call on] staff from different Pacific ethnic groups as well so their contribution or their knowledge, you know, we value in terms of doing topics and trying to use their expertise. [Principal]
In addition, the school has worked to utilise key leaders within the various Pasifika communities to liaise with the wider Pasifika community and disseminate information relevant to children’s learning:
[We identify] key parent leaders, and then provide those key parents with strategies that we've focused purely around literacy and numeracy. So a few of the sessions were providing those key parent strategies that we use within the school around literacy and them being able to deliver that in their home languages to the parents. And… being able to do that has seen the school have more parent involvement because it's almost like the school is saying that we acknowledge that [they have a part to play]. [Principal]
This strategy acknowledges the unique skills that these community members have, and conveys to the Pasifika communities that their input is both necessary and valued:
The great thing is when we run those sessions that the teacher is not seen as the lead person, someone in the community is seen as the lead person, so we are acknowledging that we really value the parents out there, and when the parents see that one of their own is leading the discussion they sort of feel really comfortable and so their involvement in the school has become somewhat ... what it's allowed, really, is more parent involvement in the school, and parent conferences or parent interviews, we have between 90 and 95% of our parents turn up. And that's great for us, for our little school. So that's been a really great strategy. [Principal]
‘Trust’ is known as one of the strategies for ‘acceptance’ between people and it is important to note that the school was evidently using this strategy to get parents involved. They were able to deliberately empower parents by strategically disempowering themselves as professionals (Amituanai-Toloa, 2007a). They did this by admitting to the community that the school and staff generally have limitations and that they are not knowledgeable enough about what could be done. By this admission the balance of power is tipped, thus creating for both school and community acceptance and equal space for problem-solving and decision making. Below is an example of disempowerment:
[We] can't do it alone, we can't do it without you, we desperately need you. You know. We can do one little bit of it but we can't without the kind of parent support that we get. There's so much that we can't do. And they know that, they understand that. They feel that. [Literacy Leader]
The rigorous home-school partnership that the school aspires to seems to embody a strong school belief and understanding that a good and workable partnership with parents is one that is similar to that of ‘aiga’where family and extended family members work together through relationships for the betterment of the group. For example, the school’s plans for continued strength of the home-school partnership in order to raise and sustain student achievement gains has led to the identification of specific Pasifika groups which are not achieving as well as others, thereby creating a focus and continued dialogue with their parents. The notion that ‘it takes the whole village to raise a child’ is validated and seen in the school as an important focus for deliberations and agreement on workable solutions.
Just continuing to discuss what more can we do, what works. Looking at what works in other areas, that's why this research is so exciting. We need to talk to the Tongan parents about that and find out what we can do together to raise that achievement. I think that's a drilling down. [Literacy Leader]
An important aspect of leadership in this school is the underpinning rationale for connection made by the Principal himself. As he himself is Pasifika, he is the first point of connection to the community and parents and his cultural knowledge and expertise is made known by the partnership processes and practices he adopts. Epstein notes that ‘the development of a partnership is a process, not a single event’ (Epstein, 2001).
The connection even extends to the involvement in academic skills throughout the school, for example, with struggling students. The school has an English for Speakers of Other Languages (ESOL) programme for students struggling academically and employs parents from the community as teacher aides as part of the PART programme.
In the following quote, the Literacy Leader elaborated on the teacher aide programme. She agreed that it has had an impact on students but only because student needs are able to be identified earlier at this level for follow up action before the needs increase.
[The] teacher aide programme where the teacher has identified certain needs and those are worked on with students. [Literacy Leader]
Pasifika parents are involved in more traditional ways too. For instance, their expertise is recognised and utilised to assist with cultural activities, arts and school sports.
We found that many of our parents were really great sewers and really great weavers, so their involvement and their expertise and supporting kids with sewing and weaving and seeing the final product is just I think, utilising them in that way, through the things like coaching sports teams and their involvement in the school cultural group as well. [Principal]
These are not added extras or useful adjuncts to core business. Parents are seen as a key resource and central to students’ achievement gains. The Principal’s view is that the most effective strategies for connection involve strong relationship building with parents through their understanding of what the initiatives have enabled, integrated with the greater understanding they have obtained of student data and how to use the school-based evidence to improve academic achievement for Pasifika students. This idea has seen an increase in the number of parents participating in school activities.
School is about people and building good relationships and I think if you can build those really good relationships and have a real good understanding of the child holistically, knowing a bit about their background, what's happening in their home life, but also knowing where they are academically… it's all about growing really strong relationships so that allows…more people to really be involved in the school in many, many ways. The second part to that would be really understanding the child academically, so really knowing your data and knowing how you are going to move them from A to B. But relationships really, is the one for me. [Principal]
Importantly, the planning for involving parents is not static. The school plans to continue to strengthen many of the initiatives which have already begun, such as parents’ understanding of student achievement data:
What this means for them and their children and what it means for the school… and where are we all going next in terms of continuing to raise student achievement… the more knowledgeable the parents are, the better it’s going to be, and the easier it’s going to be for us to continue to raise student achievement… of the things that we've got planned for next year is looking at our school wide achievement data, looking at our different ethnic Pasifika groups and looking at their student achievement as individual, like Samoan data, the Tongan, the Niuean, having a look at that and maybe looking at talking to those groups of parents individually, and find out what we can do together to raise that achievement. [Literacy Leader]
The leadership team’s focus on achievement and the strategies involved were echoed by parents. Four mothers were interviewed; two worked at the school, the other two were housewives. The talanoa process with parents confirmed that the Pasifika parents at this school had high aspirations for their children, matched by the desire to provide for their education, even if it meant working at the school to be close to their children or walking the distance to take the children to school because of transport issues. For one mother, it meant that her children had to go away from the home to someone more educated in order to get the academic help they needed.
These parents were in agreement on two things. Firstly, their main responsibility is the protection of their children. Secondly, they know that as parents they are depended upon for the academic success of their children. With regard to protection of their children, mothers believed that whilst ‘change is inevitable’, their closeness to their children could kerb the peer pressure their children are exposed to so that they do not end up on the street like other children. One mother stated:
There are a lot of kids on the street nowadays who don’t make good use of their time…and they don’t care. In Samoa, the kids come home after school and know exactly where to go, i.e., homework then chores. That is why [names her child] I have to take him to school and pick him up. The only time I let him walk is when he has to lose weight…but in doing that, he plays games with me…he detours to the shop asks for one dollar…but he asks me in front of his friends so that I don’t have to say ‘no’ and growl him. Then I say, ‘wait till we get home eh?’ [Parent]
With regard to the second aspect, parents who are close to their children are aware of their academic needs. On the topic of the support she gives her son in maths, one mother suggested that the school should make children aware of different mathematical terms that are known to parents too. She stated:
Ma’imau pe ana fa’apea e a’oa’o e le a’oga ia upu tutusa uiga i le Maths. Pei o le ‘minus’ to’ese mo le subtraction; fa’atele ‘mulitply’ mo le times ma mea fa’apena. Aua a ou fai atu minus e le iloa po o lea le uiga. Na tilotilo mai lea ia te a’u pei o a’u e valea.
[Translation] I wish they [the school] could teach the children other words [synonyms] for maths concepts e.g., ‘minus’ for subtraction; ‘multiply’ for times and that sort of thing. Because when I say minus, he doesn’t even know what that meant. And he looks at me as if I’m stupid [laughter]. [Parent]
The parents see their role as one which should complement the school’s role. They understand the importance of the work the school is doing and support the school in raising achievement for their children, hence the need to do what they do to keep their children out of trouble. But they also need synergy between school and home strategies and practices so that what they do at home benefits both the school and their children too.
The mothers’ aspirations were generally similar, but in one case differed in that one of the mothers wanted her son to be ‘exactly like her’. Her profession was in accounting (which she had worked in previously) and she felt that the reason her son did not want to take up the same profession as her was because his mathematics ability with numbers was not strong enough. This she put down to the different way the school taught maths to her son compared to the way she taught her son at home - which is the way she was taught while at school. When asked to explain further, she replied:
Like…I worked at a good job in Samoa at the bank – and when I look at my children, no one seems to be good at maths and going that way – no one is good at accounting. Maths terms differ to those taught [to her] at school. [Parent]
The following example came from an older mother in this school who does not speak English. She wept from beginning to end of talanoa. After she had said that her aspiration for her child was to see her succeed in life she was asked to describe the kind of support she gave her daughter in order for that success to be realised. Below was her response in Samoan:
[still crying] …Ou ke vili o’u kuagage fo’i ga e o la e ou ke iloa e lelei lakou a’oga ...ae maise o lakou fagau e fesoasoagi mai ia ke a’u ...ia ae sa’o lelei lava lau saugoaga ia ke a’u e pei oga e va’ai mai ua makukua ae o la e fa’ako’a valu kausaga o la’u keigeikiki makua. O la’u fa’agaugauga ia lelei age a lakou a’oga a e ou ke leiloa po’o le a sa’u fesoasoagi ou ke faia i la’u fagau...o a’u fo’i e vaivai fo’i. O le kele o kaimi e fa’afekai i le Ali’i ...oga o le kele o kaimi i la’u fagau laiki e kele iga le malamalama oga ou fai lea i la’u keige lea e fa’asa’o oga kago lea o ia e fesoasoagi i oga kei. O le kele o kaimi e fai ai lakou meaa’oga pe a o aku fo’i poo ile po...a ou le malamalama oga ou fai lea e kago e ka’u le upu...ma ou fai i ai...”kou ke iloa o le kou kiga e le ave ka’avale – ou ke malosi lava e ave e momoli oukou. Ia popoko ma malolosi oukou e fesoasoagi ... e kakau lava ia lakou oga ka’u mai po o a fesoasoagi e fai i la’u fagau ile gumela ae maise a fo’i le faikau kusi aua o la’u kamaikiki la e le poku 5, vaivai ia i le faikau kusi, ae avaku fo’i i le aiga, ou ke fai i gai ou kei e fesoasoagi. Aua e iai lakou polokalame faikau kusi o lea ga ou fai lea iai ‘fa’amolemole pe mafai oga avaku la’u kama iiga e a’oa;o mai ai laga faikau kusi?” Malie lau susuga, ou ke ...
[Translation] I call my brothers whom I know are well-educated and especially their children to help me out…it’s like when I see that I’m getting old and my eldest daughter is only 8 years old. My desire for them is to have a high level of education but I don’t know what sort of support I should give my children… I am also weak [unknowledgeable]. Most of the time I thank the Lord…and most of the time my little ones don’t understand and I tell my eldest to help correct them. Other times when they don’t understand and if I don’t understand either, I say to them, ‘say the word’ and at the same time I say, “You know your mother can’t drive a car – but I am committed to taking you to school. You be clever and strong to help”… they [the school] must tell me what kind of help I should give to my children in maths and also reading because my child is weak at reading so at home, I ask my nieces to help because they have reading programmes… so I asked if my child could come to learn to read. [Parent]
This mother engaged the help of her brother’s daughters for her child’s reading. Her nieces are older and they are what she considered ‘well-educated’. She desired a successful education for her child but as a mother, she did not know what to do as she was less educated. She considered driving the children to and from school as the only form of support she could give them. In addition, she gives constant encouragement when her child reads at home. She would prefer the school to inform her of the sort of help she should give to her child and at the same time provide some guidance through a school mentoring workshop for parents.
This is not to say that the school was not supporting them in any way. Rather, there are other factors which parents indirectly referred to as benefits. For example, one mother was glad that her child’s teacher was honest in telling her why he had lowered her daughter’s reading level. Her child was originally in another class where she was told that her child’s reading level was “very high at level 13”. Citing low comprehension as a cause and with the latter teacher being Samoan, the mother was happy that she was able to understand where her daughter was at and the rationale behind lowering her child’s reading level.
...ae ua ou fiafia…aua o le palagi…ua aumai i le Samoa la’u kamaikiki, fai mai le faia’oga lea, ‘e sa’o e lelei le faikau kusi, e avaku ua i luga i level 13, ae vave le faikau ae le o malamalama i le mea lae faikau ai. O lea ga gofo ai ma koe ku’u i lalo [sobs] O lea ua fiafia ai, fa’amalie aku....
[Translation] ...but I was happy ... because [the previous teacher] was a Palagi... now my child is with a Samoan [teacher] and he said, it’s true her reading is improving and that when she came [to me] her reading was at level 13, but she reads fluently but does not understand what she reads. This is why I put her down another level. I was happy then. Please excuse me [sobs]… [Parent]
The mothers believed that their support roles were vital and reciprocal in relation to the school and their children. In that sense, the work of the majority of parents, although indirect, could be seen as another form of connection to understanding the child as a ‘holistic’ being.
Inquiry processes and collective efficacy
The theoretical prediction here was outlined for Case Study 1. The school analysed school-wide data to monitor student progress, and identify and target needs in view of the fact that the Principal believed that using data to inform teaching was what had raised achievement of their students. Additionally, teachers, as part of their professional inquiry, tracked the progress of the Pasifika students in their classrooms and used the data to firstly identify needs and next to determine strategies to put in place for modification of their teaching practice.
Like we look at data and we are able to sort of analyse the data and pick out, identify the weaknesses, identify those kids who are doing really, really well, and then talk about the types of strategies that we could put in place to support those kids…With the data that we get back…that we collate from the research centre, seems to come back and then broken into the ethnicities. Now, we get that information there, but we also do it at a school sort of level where teachers have, in their classrooms, they've identified all the Pasifika kids in their class where they're sitting at the moment...it's almost like a tracking form for Pasifika kids, showing their progress... it's like a chart, a graph chart that sort of shows the movement or the progress that these kids have made within that sort of 12 years…So that's been a really great strategy, the work that we've done through [the Schooling Improvement cluster], the literacy and numeracy and PART work that we've been doing, our involvement in that, and understanding data more, and looking in closely at data, and moderating writing samples. I think that has been a great strategy because that has allowed great consistency for school but also it's allowed teachers to understand strategies to support Pasifika students. [Principal]
The Principal and the Lead Teacher’s claims that the initiatives have been very effective for Pasifika students were backed up by specific evidence in the interview:
What we've noticed in our school is that… back in 2004, a lot of our students were sort of hovering around stanine 1 to 3, and when you look at the results today, a lot of the students are hovering around stanine 4, 5 and 6… [for literacy] as well as numeracy, yeah, we’ve seen huge gains in our student achievement. [Principal]
The Principal and Lead Teacher believed the improvements were due to the consistency in the approach taken by the school in the strategies that have been applied in the classroom. This view is well supported by the classroom observation.
I think the effectiveness comes from the fact that we participate in the PD [professional development], we participate in the clustering and then it’s brought back to school and it’s followed up, so we don’t just go to the PD, come back, that’s done, out of the way…what happens is we come back together, we plan and how that PD is going to be effectively implemented in our classrooms and it happens on a school-wide basis, so everybody is doing and saying the same thing. [Lead Literacy Teacher]
The school has programmes in place to help socialise new teachers into the school’s programme. For example, through the cluster-wide initiative, an induction programme is provided for new teachers to the area and where Lead Teachers spend time explaining how the programmes operate in the school. In this sense there is a clear professional learning community dynamic at play where ‘novices’ are initiated into the roles and values of the community. To retain good teachers and encourage the standardisation of teacher knowledge within the cluster and its schools is an approach that the Ministry of Education might consider a positive move.
Like the views of parents and community involvement, the inquiry process for connectedness is not static. The school is constantly examining data in addition to testing new ideas and solutions. For example, the focus for the future is to link up the new Pasifika Education Plan with what the school has already implemented and to consider more closely how to incorporate more inquiry learning into the classroom.
… the new Pasifika Education Plan that's come out, trying to tie that into what the school is all about. So how it links back to the, our strategic plan, so we try to link a lot of good work, that all the strategies or the key ideas from the Pasifika plan into the school strategic plan. [Principal]
Quality instruction that is culturally responsive
Our theoretical view was outlined for Case Study 1. The Leadership team’s views are somewhat contradictory. On the one hand the Literacy Leader claimed that although the majority (75%) of the students were Pasifika the programmes were not Pasifika specific:
[Since] I’ve been here, there’s nothing specifically focused on Pasifika students, but I mean the bulk of the students are Pasifika students so, what seems to work here is working for Pasifika students. [Lead Literacy Teacher]
But, as noted above, there were specific strategies in place both in the connectedness with communities and in the inquiry processes in the school that reflected a keen sense of adapting and being responsive not just to Pasifika generally but to specific groups. This may reflect a general problem in the concepts of pedagogy that schools are confronting, which is to resolve professional development views of generically effective instruction with what is effective specifically with the cultural groups represented in the school. With the clear and weighty involvement of parents and the community, the school is arguably being cautious so as not to be seen as favouring one particular pedagogy over the other for specific groups. Clearly, the Lead team are aware of the need to build inquiry more into the pedagogy of the school, as noted earlier, and this is not seen as inconsistent with a need to be responsive to the specific backgrounds and cultural values and resources of students. Our argument for the resolution is given in the Introduction.
The observations of classroom instruction provided ratings of instruction using the empirical and theoretical literature (including attributes of cultural responsiveness). A summary of all the observations across schools is contained in Section 6. It shows several things. The first is that there was little relationship between individual teachers’ scores and achievement levels or gains in any year. Nevertheless, the observations also show that there is a pattern in which greater coherence between the (only) two teachers observed in any one school, coupled with the overall quality, was associated with greater effectiveness, which supports the school-wide focus on coherent instruction in this Case Study. Secondly, while there were differences between teachers, teachers who score high in one area tended to score high in other areas. That is, ‘good practice’ teachers were good across the board.
The two teachers observed at Case Study 3 school (T1 and T2) had the highest scores of all observed teachers across Case Study Schools - essentially near perfect scores on all categories (T1, M = 91.0% and T2, M = 98.6%). The percentages averaged for the three lessons are shown in Table 49 .
| Classroom Features % | Instructional Dimensions % | Cultural Responsiveness % | |||||
|---|---|---|---|---|---|---|---|
| Talk | Knowledge | Strategy | Vocab | Feedback | |||
| T 1 | 94.4 | 83.3 | 100 | 83.3 | 83.3 | 88.9 | 100 |
| T 2 | 100 | 94.4 | 100 | 100 | 100 | 94.4 | 100 |
As noted in the overall results for the classroom instruction, the scores do not provide very good indicators of specific gains in specific classrooms, even over three lessons (see Table 50 ). But consistent and high ratings of the instruction appear to be associated with the overall levels of the schools’ achievement and the gain over one year. This school had highest overall levels of achievement in 2008 (above stanine 4.0) and made an above expectation gain in 2008 (M = 0.34 stanine gain).
| Teacher | Total % | Mean % | Class Achievement1 | School Achievement3 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Pre-test 2008 | Post-test 2008 | N | Pre-test 2007 | Post-test 2007 | Gain | N | Pre-test 2008 | Post-test 2008 | Gain | |||
| T1 (Y4-5) | 91.0 | 11 | 3.82 | 5.00 | |||||||||
| T2 (Y5-6) | 98.6 | 16 | 4.94 | 4.44 | |||||||||
| T1, T2 | 94.8 | ||||||||||||
| School | 56 | 4.30 | 4.54 | 0.23 | 64 | 4.06 | 4.41 | Above2 (0.34) | |||||
| 1 STAR mean stanine. 2 Above expected gains (EG = 0). | |||||||||||||
The transcripts for these two teachers are models of ‘best practice’ instruction. As noted above they have near perfect scores across Classrooms Features, Instructional Dimensions and Cultural Responsiveness. The dimension of Cultural Responsiveness was strongly present as recorded by the observer – a dimension associated with successful secondary schooling for Māori students (Bishop et al., 2007). In part, this was made possible by the selection and use of particular texts. The two teachers drew on familiar artefacts (e.g., colour of the Tongan flag) and experiences (e.g., a barbecue at beach with Church) to activate and build vocabulary, background knowledge and thematic understandings in both poem writing and reading comprehension. In addition, observer notes include comments on how positive, respectful, and reciprocal the relationships were. Teachers were very accepting but not at the cost of being uncritically affirming. One interaction captures this in the context of vocabulary building and strategy use (clarifying/predicting/using context) as an example of accepting but correcting meanings.
Can you find the word ‘fossicking’ for me, halfway down, ‘fossicking’. What do you think it might mean? Can you find the word ‘fossicking’ for me? In that fourth paragraph down it says, mum says I could have the... Caleb, he began ‘fossicking’ through a cupboard in the garage.
- Searching
- Could be...
- Stomping...
Could be stomping, now stomping through a cupboard in the garage?
Yeah, Searching through the cupboard... so ‘fossicking’ just means moving stuff around and looking, searching, I’m afraid not stomping. You’d have to be quite a big cupboard for that. Right okay...
- Searching through the cupboard
In summary, the two teachers at this school were excellent teachers as judged by the scores for three areas of their instruction. This was associated with high levels of achievement in their classrooms, and at the school. These consistently high levels may be the critical finding. The school’s success may be at least in part attributed to consistently high levels of instruction contributing to the presence of ‘instructional coherence’ (Raphael et al., 2006).
The students’ views about teachers and their instruction
Thoughts about the teacher
The students’ comments confirm these features of instruction too, and elaborate on the styles which were well matched to their needs. Their comments were uniformly positive, typically describing the teachers as “good”, and “cool”. A strong emotional relationship was identified:
Oh my teacher is like my mum, she’s way better than my mum actually. She helps me out, she only growls me ‘cause so I know what I'm doing wrong. She’s like a really good teacher. So I love her like she’s a mum as well.
The teachers were seen to use humour as one device to express this. Each of the students referred to their teachers in some way as using this device and being fun.
This sense of belonging was coupled with instruction that was challenging, direct and explanatory. Two students appreciated the way their teacher helped them to understand and teach them new things, such as different ways to solve maths problems and to speak in “full sentences”.
She can help us in many different ways like if we’re stuck on a maths problem, she tells us different ways to solve it or a faster way to solve it, and writing, we always stick in worksheets, like a structure, and then when we look at our work and we don’t know what to do, we just flip back to the structure to look at it.
One student said she understood why her teacher “growls” sometimes when the other children won’t listen because it makes her “frustrated”. Another said her teacher is “smart” and “good at maths too”. She likes the way her teacher teaches her, particularly the way she explains things and makes difficult things easier to understand. Yet another student expressed these balanced attributes believing her teacher was “tough” but “good”. She thinks that her teacher probably holds high expectations for her students. Another girl said her teacher is “really funny”. She likes the way he helps when they get stuck on something and helps them to understand what they are reading. She understands that her teacher has high expectations for them:
He treats us very nice and he respect[s] all of us and he says that when he growls us, he is growling us because he cares for us and he wants us to work hard so we can be good and we can go to university.
Another girl likes the fact that her teacher gives them the “learning intentions and the success criteria” and that she gets to learn new things. She particularly likes the way her teacher helps them to learn new words and find their meanings in the dictionary.
It was not just girls who expressed this combined view of positive affect and rigorous, challenging but explicitly supportive instruction. Two boys said the teacher was “cool” and thought the teacher was great because “she pushes us hard so that we could earn a scholarship, because now that it’s intermediate, it’s going to get harder for our subjects to go higher in our standards.”
Another boyliked the way his teacher teaches, especially the way she provides them with the “success criteria” for a lesson as this makes it “more easier for [him] to understand”. He thinks his teacher is really good at teaching maths and reading. “She reads with us and she keeps on going until we understand”. The boys could be very specific about the instruction: one liked the way he is taught reading and narrative writing. He likes the way his teacher gives them challenging tasks because it helps him to learn. He would like to learn science so that when he gets to college he will know how to do it.
One of the girls said that her teacher is “cool” but he is also “sometimes a bit mean”. Despite this, she thinks he’s funny and enjoys the extra homework that he gives them if they’ve got nothing to do. She likes the way she is taught. She believes her teacher is doing his best “to help us get to university”. He helps them to learn and he helps them to focus: “when we are not focusing he says that we have to say the word ‘focus’ so that we can focus on our work”. She also likes the fact that her teacher lets them play on the computers and the smart board when they are good and at lunchtime. Her teacher also encourages them to “get active instead of just being lazy”.
Perhaps not surprisingly, given these comments, the students all said they wanted to learn more. One said she wanted to learn about science, and “how to make an explosive with chemicals, so I know what I can do and when there’s something wrong I can just figure it out.” Another wantedto learn “different kinds of languages” especially French, so that she can “talk different languages to people who don’t speak English and I can know what they mean” and Samoan “cause a lot of people around here are Samoan and they talk a lot of Samoan”. And she would also like to spend more time on maths. One of the boys would like to learn more about “how does life work” and science. And in his terms, to succeed further in his education, he would like to learn more maths.
A final statement of this multifaceted view is given by one the boys: “He’s like the bestest teacher I ever met in the whole world.” What he liked best at school is learning new things—“information and facts”. He likes the way his teacher shares the learning intentions and success criteria with the class because that helps them to learn. He also likes the way his teacher provides clear instructions so that everyone knows what they are expected to do.
In summary, there is a description emerging from this school of instruction that is challenging, explicit and very rich, and this is coupled with a close positive relationship which means that the students both respect and appreciate their teachers, often through the device of their teachers’ humour. The instructional dimensions and the cultural responsiveness as observed in the classroom lessons are validated by the students’ views of their teachers.
Pasifika learners
Language features
In the general analysis in one Focus Cluster (with Years 4 - 9) and when looking at gap differences we found that language, either first language or language at home was not associated with achievement. However, when looking at level differences (that is differences in overall level achievement), we found that there were four main effects - gender, time lived in New Zealand, home language, and school - that were associated with significantly different levels of achievement. Overall, the mean scores for students that spoke mainly Pasifika languages at home and those that spoke two or more languages (Pasifika language as well as English) at home were significantly lower than that for the mainly English-speaking students. The general analysis results are reported in Section 3.2.2 and Table 51 provides a summary of the language spoken at home.
| Language spoken at home | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | ||
| English Only | ||||
| M | 4.04 | 4.57 | 0.54 | |
| SD | 1.45 | 1.00 | 1.37 | |
| n | 28 | 28 | 28 | |
| Pasifika Language Only | ||||
| M | 3.79 | 4.05 | 0.26 | |
| SD | 1.47 | 1.31 | 1.48 | |
| n | 19 | 19 | 19 | |
| Pasifika and English | ||||
| M | 3.80 | 4.60 | 0.80 | |
| SD | 1.92 | 1.67 | 0.45 | |
| n | 5 | 5 | 5 | |
The longitudinal patterns in this Case Study school provided further indicative data on developing bilingual status (see Figure 23 and Table 52 ). Those students who had a Pasifika language as first language were by Year 5 resembling in their achievement those students who had English only, a finding consistent with other developmental analyses of bilingual development (Amituanai-Toloa, McNaughton, & MacDonald, 2007).
| First language spoken | 2007 | 2008 | ||||||
|---|---|---|---|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | Pre-test | Post-test | Stanine Gain | |||
| English Only | ||||||||
| M | 4.73 | 5.00 | 0.27 | 4.27 | 4.45 | 0.18 | ||
| SD | 1.01 | 1.18 | 1.01 | 1.01 | 0.82 | 0.60 | ||
| n | 11 | 11 | 11 | 11 | 11 | 11 | ||
| Pasifika Language Only | ||||||||
| M | 4.00 | 4.31 | 0.31 | 4.54 | 4.38 | -0.15 | ||
| SD | 1.08 | 1.32 | 0.85 | 1.33 | 1.26 | 1.46 | ||
| n | 13 | 13 | 13 | 13 | 13 | 13 | ||
Figure 23: Mean stanine by first language for longitudinal cohort (Case Study 3).
Education in the New Zealand system
It could be predicted that greater familiarity with New Zealand educational practices (which is almost always confounded with immersion in English instruction) would be associated with higher achievement. This was evident in the present school. We found that students born in New Zealand (see Table 53 ) had a higher mean stanine at all time points (M = 3.95 (Pre-test) and M = 4.46 (Post-test)) than students born in a Pacific country (M = 3.73 (Pre-test) and M = 4.18 (Post-test)). Students born in New Zealand made a greater mean stanine gain (M = 0.51) than students born in a Pacific country (M = 0.45). This was contrary to the general finding where the length of time lived in New Zealand was associated with different levels of achievement. The mean scores for those that lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than those that had lived in New Zealand between one and five years.
| Birth country | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | ||
| Pacific Region | ||||
| M | 3.73 | 4.18 | 0.45 | |
| SD | 1.56 | 1.17 | 1.51 | |
| n | 11 | 11 | 11 | |
| New Zealand | ||||
| M | 3.95 | 4.46 | 0.51 | |
| SD | 1.47 | 1.23 | 1.30 | |
| n | 39 | 39 | 39 | |
| Other | ||||
| M | 4.5 | 4 | -0.5 | |
| SD | 2.12 | 0 | 2.12 | |
| n | 2 | 2 | 2 | |
The general interpretation of this finding is supported by examining the patterns for students who had spent differing amounts of time in New Zealand. Students who were not born in New Zealand but who had lived in New Zealand for five or more years had higher mean stanines at all time points than students who had lived in New Zealand between one and five years (see Table 54).
| Time in New Zealand | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Stanine Gain | ||
| 1 - 5 years | ||||
| M | 3.22 | 3.89 | 0.67 | |
| SD | 1.39 | 1.05 | 1.58 | |
| n | 9 | 9 | 9 | |
| > 5 years | ||||
| M | 5.25 | 4.75 | -0.50 | |
| SD | 0.96 | 0.96 | 1.29 | |
| n | 4 | 4 | 4 | |
| Born in NZ | ||||
| M | 3.95 | 4.46 | 0.51 | |
| SD | 1.47 | 1.23 | 1.30 | |
| n | 39 | 39 | 39 | |
Student views
Further evidence of engagement in the school, connectedness with families and the role of students can be gathered from students’ views regarding their aspirations and how they believed the school was motivating them towards their life goals. As noted above, the students viewed the teachers and the Principal in a positive light. Here we add their overall ideas and beliefs about school and their aspirations.
The students had big dreams for their future including being an “air hostess”, “a lawyer”, “an actor and a singer”, “a chef” and, without having any specific occupation in mind, being “good at maths”. Each of the children believed that school is helping them get closer to achieving their dreams.
An interesting additional feature which may also be seen as a Pasifika dimension of their learning was the sense that learning would involve service. The student who wanted to be a chef said partly this is because she gets to help out the “the people who do the special lunch orders”. More directly, one of the boys said he wanted to be a billionaire so that he can “give some [money] to charity, and to the poor, and give some to my family”. He believes that he needs to “get high standards” at school and “try to get to high school so that [he] could get graduated and get a job”. One of the girl’s big dreams is to become a missionary and “travel around countries, and tell people about the church and how they have to respect people.” But before she becomes a missionary, she would like to go to university. She believes that her schooling is helping her to achieve her dream through learning about reading and writing, and improving her understanding. One of the boys said very formally:
My opinion about school is that it’s a good place for children to learn so that when we grow up we will get better education and earn money for our families.
Each of the students liked being at school because it was connected with their family life: “you can learn heaps and go home and tell your parents about it”, “you learn [even] more than what you learn at home”. One girl said “I don’t like to miss school even if I’m sick. This school makes me feel safe and I love the school more than my family.” Another thought school is “very cool, cause you get to learn lots of good things. You get to know other things that you didn’t know before, make new friends.” She is happy to come to school because if she were to stay at home she would have nothing to do.
One of the boys even preferred school to holidays. He likes coming to school for the learning. He likes to read books and learn different subjects. He especially likes physical education and writing.
The students were mostly happy about the other children at school. For one student this was because many of her relatives also go to the school, although sometimes she doesn’t like to be around the other children at school because they are “back stabbers”. The sense of peers being appreciated who were similarly focused on learning led to both positive and negative comments. One liked the other students because they help her out, but another was not so keen on the ones who talk about things other than their school work while they are in class. Some of the children in her class are “really smart” and she would like to be like them.
The students’ discussion about homework had four features which illustrate further their commitment to learning and add to the earlier descriptions of preferred pedagogy. One feature was that they often did homework daily and secondly they enjoyed doing this. Two professed to “love homework”, not the least because it provides an excuse to get out of doing household chores. Other reasons given for liking homework were that “it’s just fun, something to do after school” and “it makes me know that I am brainy”. The students do their homework most often soon after they get home.
The third feature was the involvement of the family in supporting homework, directly through guidance and in terms of resourcing and hence parental/family involvement which impacts on academic skills. The type and degree of support interestingly was often controlled by the students. One student said she always did her homework by herself while she usually works with her cousins or, if they get stuck, they get an older cousin or a parent or aunt to help out. These students did their homework in their rooms or sometimes in other places such as at the computer, in the sitting room, the car, or the library.
The fourth feature was a concern that it be meaningful and challenging homework. One student said she was not too keen on the easy homework. She usually does her homework when she gets home from school; it takes her about half an hour to complete. She works in her bedroom and does her homework on her own when others in the house are busy and it is noisy. When things settle down, if she needs to, she asks her mum or her older brother to help her.
Even though one boy said he found the homework hard at times, he still liked it. He usually does his homework when he gets home from school, either in his room or with his mum. He often spends about 2 hours doing his homework but that doesn’t bother him. Indeed, he would welcome more homework “so he can follow his dream”.
One boy didn’t want too much challenge; he doesn’t like homework sometimes because he finds it difficult, especially some of the maths homework. He attends a physical activity after school programme so he does his homework when he gets home after the programme. When the homework is hard it takes him up to an hour to complete, but he doesn’t always complete all his homework on one night. His Mum helps him with his homework: “but she doesn’t really give the answers, she just makes it easier for me.” He usually does his homework in his room but sometimes he works in the sitting room.
Two of the girls said they usually spend about 20 minutes on homework, and like other students they do this before other commitments. Again the sense of family involvement was highlighted. She sits near her parents to do her homework, usually at the table so her mum can help her while she is cooking. The sense of being in control and determining the type and level of help is given in the following comment by one of the girls: she sometimes asks her parents or her sister to help her with her homework but “not give the answer, just help”. She tries to do her homework by herself. She likes to work somewhere quiet, in her bedroom or sometimes in the kitchen. Similarly, one of the boys said his mum sometimes helps him with his homework in which case he works at the table. Otherwise he works in his bedroom.
The students did not have a long list of changes they wanted made; they mostly felt the school was doing a good job. The most consistent change wanted was in the attitude of some of the other students and lower levels of bullying.
Their views about the language of instruction and the match with their languages were mixed. A Māori/Cook Island student said she would prefer to speak Māori in class, although this child prefers her teacher to speak in English when she is having difficulty understanding. One Samoan student thought that having things explained in Samoan would be helpful sometimes when things are difficult to understand, while another Samoan student was happy for her teacher to speak in English all the time, even when explaining difficult things. However, the latter noted that her teacher often gets her to speak in Samoan to other children when they don’t understand. Several others preferred English to be spoken in class, even when they may have difficulty understanding.
3.3.4 Case Study 4
Overview of the school
This decile 1 secondary school has had a particular focus on Māori students and has only recently turned its focus also to Pasifika students. Pasifika students at this decile 1 secondary school achieve somewhat below national levels (see Table 55 ). Overall achievement at the school at the end of Year 9 for Pasifika students on asTTle reading comprehension was 523.36 (national expectation 634) and at Year 10 it was 594.78 (national expectation 728). It would be expected that from the end of Year 9 to the end of Year 10 students would gain 94 points. Overall Pasifika students at Case Study 4 gained mean = 71.42 points from the end of Year 9 to the end of Year 10. The ‘Other Pasifika’ group was the only group who made close to the nationally expected gain in asTTle points (M = 93.67).
The total Pasifika group at the school overall is around 35%. The breakdown for the Year 9 and 10 groups we examined was Samoan 60%, Tongan 14%, Cook Island Māori 8%, Niuean 8%, Fijian 8%, and ‘Other Pasifika’ 2%. More than half (67%) of these students had English as first language, with Samoan being the next most common first language (15%). Of the total Pasifika group, more than two thirds of the students (71%) had been born in New Zealand.
| 1 These students were Year 9 in 2007 and Year 10 in 2008. | ||||
| Pre-test 2007 | Post-test 2007 | Post-test 2008 | ||
|---|---|---|---|---|
| Tongan | ||||
| M | 468.14 | 545.71 | 621.71 | |
| SD | 35.33 | 65.23 | 104.79 | |
| n | 7 | 7 | 7 | |
| Cook Island Māori | ||||
| M | 468.25 | 598.00 | 662.75 | |
| SD | 39.63 | 73.23 | 101.30 | |
| n | 4 | 4 | 4 | |
| Samoan | ||||
| M | 454.50 | 512.70 | 577.27 | |
| SD | 54.45 | 74.44 | 91.21 | |
| n | 30 | 30 | 30 | |
| Other Pasifika | ||||
| M | 457.67 | 508.33 | 602.00 | |
| SD | 73.93 | 112.40 | 92.33 | |
| n | 9 | 9 | 9 | |
| Total | ||||
| M | 458.08 | 523.36 | 594.78 | |
| SD | 54.10 | 82.64 | 94.69 | |
| n | 50 | 50 | 50 | |
Connectedness – community and school
Our theoretical prediction was outlined for Case Study 1. Both the Principal and the Head of Department (HOD) were clear that there was more work needed to involve parents. Specific parent groups were formed but haven’t lasted. Mostly parents were involved in receiving school information; for example, the parents are involved in parent teacher evening where data is represented and shared and interpreted. Specific information provided included explaining the assessments (asTTle) and the HOD commented that an outcome would be that parents would become more involved in assessing the improvement of their children. There had been a barbecue with the Samoan students attended by the parents but the need expressed by the Principal is to capitalise on that success across terms to boost involvement in the information sharing meetings. The most successful involvement at school is not through these information sharing sessions but rather through the Auckland Secondary Schools Festival (Polyfest). There is keen involvement with large attendances at schools and at the festival.
The parents are seen as being key to a substantial problem in attendance which is particularly marked in the upper school years. Though the feeling (according to the school) is that the poor NCEA results are at least in part attributable to the erratic attendance, which is due to family commitments, it is clear from the parents’ keen involvement in the Polyfest that this is the primary involvement they might have with the school given its relevance to their culture and language.
The school has appointed a new deputy Principal who has, as part of his portfolio, the task of developing Pasifika parents’ roles. The meeting formats are being trialled, and the Pasifika teachers are involved in reaching out. The meetings are late afternoon and early evening with food, and the content is an important educational topic.
One of the new initiatives is a bilingual Samoan programme in Years 9 and 10, which if it develops like the Māori bilingual programme will enable strong involvement to occur which they have found is associated for the Māori group with higher achievement. It was not clear from the interviews about what the specific roles in the involvement might be. But the Principal commented:
I haven’t got the parents yet in the way we have been able to win over Māori parents to a bilingual programme. They have got a different view of education, but it is something we are working on. [Principal]
Other strategies are planned which increase the familiarity and welcoming properties of the school, such as individual telephone invitations to meetings, and the building of a Fale. Teacher aides who are members of the Samoan community help in classes. However, they were identified primarily to provide support for teaching, rather than because they were members of the community coming into school.
Parents’ views
Of the twelve parents identified by the school to be interviewed, only two parents managed to keep the talanoa times, the others citing family commitments. This highlights the concern the school has about attempting to ‘win over’ Pasifika parents. Although the views and beliefs of parents who were able to be interviewed are reported here, they should be interpreted with caution given the small number of parents involved.
The talanoa sessions revealed three outcomes that parents believed were important to their child’s life. These are a good education, a good life and good employment. They mostly stressed involvement as support and motivation. They believed the school was catering to their needs through ‘looking after’ their children. One parent explained:
To you know, them to have good education, good life, good work you know look after their family, that’s, that’s my dream. But yeah I’m, I’m so happy at [school name]. They look after my children and they were good. [Tongan father with a Samoan mother]
This might support what the Principal meant by, ‘they have a different view of education’. Whilst in a sense, this parent by his comments might have given the Principal a view that the school is a caregiver and not an educator it is important to interpret this from the parent’s point of view. He stated:
…the school is very helpful because we have two children that are at Universities at the moment. Yeah all my children that went to the same school, they okay. Yeah they’re alright.
He acknowledged that the school is working on improving their understanding of the intentions of the parent, but was very happy with an emphasis on the caring role:
You know that’s why I think that they really care. But the kids, because if um, we forgot, if they sick or we are busy doing something else and forgot probably 11 o’clock, 12 o’clock the phone ring and it’s the teacher, we say sorry, we forgot to ring up, she’s sick and we forgot to ask the other big girl, she always go to school to ask the teacher if she is not in.
Furthermore, the evidence that the other two children had already been through the school successfully resulting in obtaining scholarships are also noted by parents as part of the holistic viewpoint:
The school is alright but if you go there as a good kid, I mean as a good person, the school will be alright. And that son encouraged the other one who’s in university now. He got… he had… 5 years… 5 year scholarship over there and that is his you know, when he got the scholarship 2 or 3 years ago aye.
The first parent believed that the support they had given all their children is through encouragement and by physically taking them to school to make sure they attend.
The three that we already, we just encourage them and plus and we take her after school, make sure they are at school. I don’t know exactly what we… But we’ll support in what she want.
The other parent believed that his child should be equipped with life skills, hence he supports his son in martial arts, music and animation, noting that animation is the child’s strength. Little support is given to schoolwork.
He is a gifted boy. See what he’s done [showing his son’s animation work]. He did it all by himself and he’s gifted in this area…and the belts he got from karate…So I support him in all these because I believe that if he doesn’t achieve at school at least he’s got another pathway to go. [Parent]
Despite the strong sense of supporting the school, when asked what he thought the school might offer to ensure his son’s dream is fulfilled he did refer to more specific achievement and progress information:
I don’t know. I only get the report at the end of the year so I can’t help him because then it’s too late. I wait to see where he’s at every term. [Parent]
Another talk I said no, no son, no, not you. You go to university and you got a scholarship, we can’t afford to pay you. You got to go on a loan. The scholarship to pay for everything.
Inquiry processes and collective efficacy
The theoretical prediction was outlined for Case Study 1. There was evidence from the interviews with the Principal and the HOD that collecting, sharing and using evidence from achievement data occurred. There is a cycle of repeated assessments and patterns of achievement aggregated for the Pasifika group were well known. But it was unclear how much of the school’s practices which specifically involve inquiry and developing shared efficacy from the inquiry occurs. The HOD was concerned that not enough was known in the school about the learning needs of students and that there was a need for more professional conversations with other experts who have research evidence about “how Pasifika students learn best”. The HOD was very aware that the best vehicle for this was the professional learning community which is inquiry focused. She used an example from her own hypotheses:
Where people say look this is the data that we have, this is our understanding of the kind of what’s going on for those students in your classroom and why they might be sitting there and it looks like they are taking it all in, but actually they are not asking questions because they are not comfortable asking question and I think that is one of the issues personally I have come across with Pasifika students. They don’t want to ask questions, they are still not quite sure I think about the teacher student relationship and they have been taught there is a bit of a separation and that the teacher, Pasifika students in general I think are quite respectful, but almost too much in a way. [Head of Department]
The HOD noted the need to have more targeted professional development about the needs of Pasifika students and for that to be held with departments. The new ‘Professional Leaning Centre’ lead by a coordinator was seen by both the Principal and the HOD as a vehicle for building the professional learning communities within the school. The Principal’s comment above about the attendance suggests that some of the challenges in raising achievement are seen as located in the community, although the Principal was very explicit that the key focus in the school was on teaching and more effective instruction, especially for literacy.
Quality instruction that is culturally responsive
The theoretical prediction was outlined for Case Study 1. As with other schools, the Principal and the HOD had a generic view of effective instruction for Pasifika students.
Given the large number of Pasifika students that we have in the school, [our programmes] relate directly to them even though they weren’t specifically designed for Pasifika students. [Principal]
I don’t think that specific literacy strategies geared to Pasifika students are in place…we tend to gauge prior knowledge (and other strengths and weaknesses) vocabulary is a particular area …because of their NESB background… and also inference. [Head of Department]
The view of the HOD was more differentiated as could be seen in her description of the effects of Te Kotahitanga. As in the other secondary case study school, the Te Kotahitanga programme had been present in the school. The school’s facilitator was one of the teachers taking the Samoan classes at Year 9 and 10. The Principal and the HOD had differing views on its effectiveness with Pasifika students. The Principal felt it was equally beneficial, while the HOD’s view was:
We are actually finding with the empirical data that Pasifika students are very different in the classroom in terms of how their cultural experience relates to a classroom situation, very different for the TK kind of approach which tends to work very well with Māori, but also very well I think with European students. [Head of Department]
The HOD also identified specific areas of cultural knowledge and values, including a description of Pasifika students finding it uncomfortable being focused on and preferring to not stand out or be seen to be too proud.
Both the Principal and the HOD viewed specific instructional strategies including the major development of the bilingual classes as core to effective teaching for Pasifika students. Examples of the former included small class sizes, deliberate hiring of Pasifika teachers and the support for teachers to gain TESOL training and then using them to work on the language needs (including vocabulary).
| Classroom Features % | Instructional Dimensions % | Cultural Responsiveness % | |||||
|---|---|---|---|---|---|---|---|
| Talk | Knowledge | Strategy | Vocab | Feedback | |||
| T 1 | 72.2 | 91.7 | 75.0 | 66.7 | 58.3 | 80.0 | 55.6 |
| T 2 | 84.9 | 38.9 | 55.6 | 66.7 | 38.9 | 61.1 | 66.7 |
In terms of the observations in classrooms one teacher had a relatively low total score (T2 = 62%) while the other had a higher total score (T1 = 71%). As noted in the overall results across schools for the classroom instruction, the scores are not very good indicators of specific gains in specific classrooms, even over three lessons. Although in this school, T1 had substantially higher reading achievement than both the other teacher’s class and the overall Year 9 Pasifika students’ averages. However, there were only three Pasifika students in her class. In general across schools, however, the single teacher ratings of the instruction appear to be associated with the overall levels of the schools’ achievement and the gain over the most recent year, 2008. The overall average score for the two teachers in this secondary school was the lowest across the four schools (M = 66.2%) and this school had the lowest gains generally among the four Case Study Schools. At the end of the school year in 2008, the mean asTTle reading score (554.89) equates to curriculum level 3A. Pasifika students in Year 9 at this school made a below expectation gain in 2008 (asTTle reading mean gain = 97.10). See Table 57 for the summary.
| 1 STAR mean stanine. 2 Below Expected gains = 117, but levels similar to nationally expected levels. | |||||||||||||
| Teacher | Total % | Mean % | Class Achievement1 | School Achievement1 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Pre-test 2008 | Post-test 2008 | N | Pre-test 2007 | Post-test 2007 | Gain | N | Pre-test 2008 | Post-test 2008 | Gain | |||
| T1 (Y9) | 70.8 | 3 | 510.00 | 658.67 | |||||||||
| T2 (Y9) | 61.6 | 22 | 458.68 | 554.55 | |||||||||
| T1, T2 | 66.2 | ||||||||||||
| School | 57 | 453.12 | 517.39 | 64.26 | 90 | 457.79 | 554.89 | 97.102 | |||||
It is interesting to note that these two teachers had relatively low scores for cultural inclusiveness, which may reflect the more generic view espoused by the Principal. For both teachers the observation notes indicate high scores for the relationships dimension (with some humour) but low on examples of incorporation of students’ cultural and linguistic resources. In keeping with the focus on designing the most effective instruction, in T1’s lessons there were many instances of analysis and explicit teaching of language features and structures (such as how words convey opinion versus fact in writing), with a focus on technical terms (such as prepositions; definitions of ‘fact’ and opinion’). T1 also had very high scores for High Level talk (there was an extended sequence around the question asked by a student: “Why do you do lateral thinking?”) and Feedback, perhaps reflecting the influence of Te Kotahitanga. But this was not the case in the second teacher’s lessons.
The students’ views about teachers and their instruction
Thoughts about the teacher
All students believed that their education is “going well”. In one class, students enjoyed all their subjects except Maths which they put down to the teacher being unreasonable. When asked why they thought about the teacher that way they said two things: that the teacher had high expectations and that she doesn’t explain things. When the same students were asked what they thought the best solution was, they responded by referring to two features. One was the cultural responsiveness dimension to do with positive respectful relationships (to be nicer and warmer and have fun). This is important because the criticism adds to the previous descriptions by primary children that a feature of the preferred pedagogy is the use of devices for conveying positive affect such as humour. The other feature made reference to enabling elaborative talk and inquiry by the students.
Nah she’s just like…strict all the time and she doesn’t let us talk once
She doesn’t let us talk, she doesn’t let us
You can’t speak to her… and we can’t ask questions because she’s too scary
She doesn’t let us speak to each other. She says that we have to be like really quiet
She never lets us have a little bit of fun like the other teachers
Their definitions of a good and effective teacher reinforce the dimensions and attributes examined in the observations. A teacher has to have ‘humour’; has to respect them in order to be respected, has to understand the students, does not talk too much, and teaches in a rigorous way:
Is open and funny
Has a humour when teaching like…
Isn’t really boring…yeah…kind of strict
And doesn’t talk too much…yeah
Yeah, yeah …respect and respected
And a teacher that understands us
Yeah [teaching] hard out
By contrast other teachers, “They make it fun. They make learning fun”, and interestingly used devices such as shared histories to personalise instruction: “Um…Um…telling us about how they learnt what we learnt today when they were younger”.
The issue of non-attendance was raised by the students linking it to boredom and being classroom bound:
More outdoor…like practical – instead of just being inside the class all the time
It’s good but sometimes it gets boring and we get restless and we don’t want to come back to class and heaps of kids wag
Pasifika learners
Language features
The theoretical prediction here was outlined for Case Study 1. Language features and their relationship with achievement were examined in a two-year longitudinal cohort who were Year 9 in 2007 and Year 10 in 2008 (n = 19). In some analyses of ethnicity and language patterns these students were augmented by those students who were Year 9 in 2008 and had beginning and end of year asTTle scores (n = 54). These are very small sample sizes and these analyses should be read as indicative and interpreted with considerable caution.
The patterns are very difficult to relate to the general analysis. In terms of rates of gain (‘gap differences’) there are different patterns in different years. In terms of level differences students with English only at home were consistently higher than others but there were only two students who identified both Pasifika and English being used at home.
Education in the New Zealand system
It could be predicted that greater familiarity with New Zealand educational practices (which is almost always confounded with immersion in English instruction) would be associated with higher achievement. This was found in the overall analysis. Due to the small numbers in this case study school a simple summary is not possible. We found that being born in New Zealand was associated with a higher asTTle level score or gain (see Table 60 ). This was complementary to the general finding whereby the length of time lived in New Zealand was associated with different levels of achievement. The mean scores for those that lived in New Zealand for more than five years and those that were born in New Zealand were significantly higher than those that had lived in New Zealand between one and five years.
| First language spoken | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | ||
| National Norm | M | 517 | 634 | 117 |
| English Only | ||||
| M | 468.14 | 574.86 | 106.71 | |
| SD | 48.70 | 83.50 | 70.00 | |
| n | 28 | 28 | 28 | |
| Pasifika Language Only | ||||
| M | 444.74 | 533.04 | 88.30 | |
| SD | 55.37 | 55.47 | 57.16 | |
| n | 23 | 23 | 23 | |
| Pasifika and English | ||||
| M | 462.00 | 521.00 | 59.00 | |
| SD | . | . | . | |
| n | 1 | 1 | 1 | |
| Other | ||||
| M | 423.50 | 538.50 | 115.00 | |
| SD | 140.71 | 157.68 | 16.97 | |
| n | 2 | 2 | 2 | |
| Language spoken at home | 2007 | 2008 | |||||
|---|---|---|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | Post-test | Total Score Gain | |||
| National Norm | M | 517 | 634 | 117 | 727 | 93 | |
| English Only | |||||||
| M | 454.42 | 516.50 | 62.08 | 549.83 | 33.33 | ||
| SD | 63.32 | 82.34 | 71.45 | 79.78 | 53.32 | ||
| n | 12 | 12 | 12 | 12 | 12 | ||
| Pasifika Language Only | |||||||
| M | 427.80 | 437.40 | 9.60 | 505.40 | 68.00 | ||
| SD | 35.96 | 59.09 | 31.76 | 44.98 | 34.69 | ||
| n | 5 | 5 | 5 | 5 | 5 | ||
| Pasifika and English | |||||||
| M | 437.00 | 457.50 | 20.50 | 532.00 | 74.50 | ||
| SD | 35.36 | 126.57 | 91.22 | 0.00 | 126.57 | ||
| n | 2 | 2 | 2 | 2 | 2 | ||
| Birth country | 2008 | ||||
|---|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | |||
| National Norm | M | 517 | 634 | 117 | |
| Pacific Region | |||||
| M | 442.81 | 521.54 | 78.73 | ||
| SD | 51.51 | 61.46 | 54.31 | ||
| n | 26 | 26 | 26 | ||
| New Zealand | |||||
| M | 471.15 | 589.04 | 117.89 | ||
| SD | 55.99 | 74.82 | 66.91 | ||
| n | 27 | 27 | 27 | ||
| Other | |||||
| M | 412.00 | 490.00 | 78.00 | ||
| SD | N/A | N/A | N/A | ||
| n | 1 | 1 | 1 | ||
| Time in New Zealand | 2008 | |||
|---|---|---|---|---|
| Pre-test | Post-test | Total Score Gain | ||
| 1 - 5 years | ||||
| M | 429.08 | 501.85 | 72.77 | |
| SD | 47.22 | 63.09 | 46.87 | |
| n | 13 | 13 | 13 | |
| > 5 years | ||||
| M | 446.09 | 538.00 | 91.91 | |
| SD | 57.67 | 60.79 | 64.12 | |
| n | 11 | 11 | 11 | |
| Born in NZ | ||||
| M | 471.15 | 589.04 | 117.89 | |
| SD | 55.99 | 74.82 | 66.91 | |
| n | 27 | 27 | 27 | |
Student views
Like students in the other schools, these students also had big dreams: to be pilots, professional sportspeople, or professionals such as lawyers, social workers, or teachers. As evident in their previous views about teachers and teaching, some students believed that their dreams would be fulfilled, while others did not. This shows that while some students were motivated and engaged with learning, other students were not.
When asked about what the school could do to make a change almost all students supported the school’s plan. They wanted the school to have a facility which all Pasifika students could call their place, although there was some debate as to whether it should be Samoan-specific or not. This comes at a time when Māori students already have a facility which Pasifika students are not permitted to use. One student put it this way: “Yeah because the Māoris have got their canteen and we’re not allowed to go in... in the Māori block. They come to ours but we’re not allowed to go into theirs. We got kicked out. They said that we were buying too much of their food and they come into our canteen and take all of ours.”
The holistic view is that teaching and learning entails the whole being. Pasifika students consider a place of their own as a vital part of their education and feel strongly about a place which they can utilise where they can socialise and interact with peers. As part of engagement for learning they needed more Pasifika teachers whom they could approach for academic help but noted that there were fewer numbers of Pasifika teachers especially those who teach Pasifika students in ESOL classes.
More island teachers
We only have 3 island teachers
…yeah, yeah cause it’ll be better for them
Better understanding English
Because there are Palagis and Indians that are teaching the ESOL groups, but the ESOL groups are islanders and they don’t understand and they can’t really … yeah
They recognised that the home support was about motivation and extended that to the church’s role in their education. For example, when asked about the sort of home support they get, they responded that their parents ‘push hard’ to achieve and part of that push is attending church activities.
They’re pushing you
Do it for church
Put God first and then education
3.3.5 Summary
Taken together these case studies suggest a number of themes or conclusions relating to the general hypotheses. The conclusions that follow are not made using the logic of systematic comparisons between schools. There are too few schools, and they differ both in known and unknown ways that make direct comparisons risky. Nevertheless, because the schools are each in Schooling Improvement clusters and because they vary in the overall levels as well as rates of gain in achievement, it is possible to draw themes, given a model that these clusters are designing more effective systems and some are more developed than others. These themes are drawn from the patterns of developing greater effectiveness given the evidence and given the hypotheses. A final caveat is our view that schools should develop solutions to effectiveness that are firmly rooted in their local contexts. Given variations in context, their solutions and therefore the findings around these hypotheses are likely to vary.
Connectedness
The relationship between a school and its families is important. The case studies suggest greater effectiveness deriving from practices that involve sharing knowledge and resources with a degree of reciprocity, with the specific outcome of increasing parent involvement which may then improve students’ motivation and academic skills. Putting together the evidence across the various sources, three conclusions are suggested: (a) parents’ understanding of information about their own individual child’s learning and achievement, both strengths and weaknesses as well as progress across time, can increase parental impact on motivation and skills; but (b) parents need guidance and advice on both motivational and academic involvement; and (c) parents are keen to receive advice and have ideas about practices both at home and at school that could contribute. These may or may not be effective but they are important ideas that can be the basis of reciprocity - an example is the role and forms of homework.
Inquiry and collective efficacy
The hypotheses about developing inquiry practices that are evidence-based and outcomes-focused is well illustrated in the case studies. The schools are all engaged in clusters of Schooling Improvement which focus on inquiry and it would be expected that these practices would be in place. But the schools varied in how deeply ingrained, extensive and coherent their practices are. The patterns suggest that greater coherence will be associated with greater effectiveness. Coherence matters: (a) between levels in the schools, across members of the school professional community, and between different instructional parts including teachers; (b) for new members of the system so that detailed induction as a member sharing values and skills is important; and (c) so that all programmes – existing and new – are integrated into the inquiry practices and are ‘tested’ by the inquiry process. The coherence between teachers appears to be especially significant so that there is consistency in pedagogical approaches as well as in focus and goals.
Pedagogy and cultural responsiveness
There is some ambiguity detected in the data in how these terms are used, and there is a need to clarify more specifically what is meant by these terms. However, in general, the evidence across schools was that the schools to varying degrees taught using generically effective forms of instruction but adapted them to be applicable to and responsive to different Pasifika learners. At a general level, cultural responsiveness is a dimension of generically effective instruction. Having said that, it is possible to identify elements of what the model is that the schools are moving towards. Clearly, schools are effective to the degree that they use known attributes such as explicit instruction for both basic knowledge and strategies, high levels of elaborative talk and inquiry are promoted, there is a focus on the language needs including those for vocabulary and there are well-developed forms of feedback. Running across these is the need to be clear and explain goals and needs for learning. On the other hand specific dimensions of cultural responsiveness are clearly part of more effective teaching. The twin dimensions of positive relations and incorporating students’ resources are identified. Importantly, these themes are echoed by the students. Pasifika pedagogies that are being developed in these schools, in the sense of being culturally responsive, draw on background knowledge including topics and event knowledge, language patterns and activities, and the students and teachers are aware of this. But in addition there is the dimension of a strong emotional relationship which, together with the instructional attributes, has elements of being both rigorous and challenging as well as being respectful and empathetic. The former includes the high expectations and the latter a Pasifika sense for the students of education being service-oriented and, from the teacher, positive affect expressed with devices such as Pasifika-oriented humour.
Pasifika learners
The student voices are very similar to those from the Te Kotahitanga project but the adaptations suggested above include a need for teachers to provide a strongly supportive base enabling the students to take risks and be critical and engaged. The evidence supports previous research showing Pasifika learners to be generally highly motivated to succeed and to be willing to learn across the schools. Students are more consistently positive at primary schools (but this is true generally, and there is a more general need to consider how to increase engagement and emotional connection at secondary levels). If we only look at language status from the point of view of achievement, and putting the conclusion negatively, there is no evidence that having two or more languages is an impediment to high success either at primary or at secondary. The patterns of development may look different for those students with a Pasifika language or both a Pasifika and English language background in the earlier years, compared with English only students. But from the middle and upper primary and into the secondary years the sense is that bilingualism may (under important conditions not tested here, such as level of bilingualism) lead to similar outcomes as having a strong English only status, and in the wider sense indicated in the Introduction confer other advantages. There is perhaps an obvious suggestion in the data that more familiarity with the New Zealand education system is advantageous and we take this to mean that for newly arrived students there is a need to have very explicit induction and support to develop the knowledge and skills required for schooling.
Downloads / Links
Sections
Contact Us
For more publication-related information, please email: information.officer@minedu.govt.nz
Search Publications
Copyright © Education Counts 2011 | Contact information.officer@minedu.govt.nz for enquiries.