Diagnostic assessment tools in Māori-medium education: A stocktake and preliminary evaluation Publications
This project was undertaken as a partnership between researchers of the Māori Educational Research Institute (MERI) in the School of Education at the University of Waikato and the research whānau of the Specialist Education Services Poutama Pounamu Research and Development Centre of Tauranga.
Author(s): R. Bishop; M. Berryman, T. Glynn & C. Richardson
Date Published: 2001
The project: The purpose of the stocktake/evaluation was "to determine which Māori-medium diagnostic tools are currently used in New Zealand to assess student's achievement in reading, writing and mathematics in the first 4 to 5 years of instruction." This involved an evaluation of existing Māori-medium diagnostic tools in terms of the information provided by these instruments and the uses that teachers make of the information. The information was provided at both the level of strengths and at the level of specific weaknesses. This study also sought to identify gaps in current provisions and potential areas for development.
Diagnostic tools or instruments in this report are broadly defined as one that allows each child's strengths and weaknesses in a particular learning area to be identified and provides information for future teaching. It also provides information to help parents support their children's learning.
Consideration was given to approaches fundamental to Kaupapa Māori research in that issue of ownership of knowledge and intellectual property rights and cultural legitimation were addressed. The research group that undertook the project was constituted as a Kaupapa Māori "whānau of interest" research group (see Bishop, 1996a and Bishop, 1996b). In this manner, the activities of the researchers, when understanding this evaluation, addressed the research issues of initiation, benefits, representation, legitimation and accountability because the issues were addressed from within Māori cultural frames of reference. For example, ownership of ideas was carefully identified and any development of ideas, were collaborative and beneficial for both the participants and the researchers. In other words, information gathered was to be part of a process of development for the betterment of young people in Māori-medium settings. This process in itself forms part of the overall evaluation of the diagnostic tools.
Potential for Conflict of interest
The small number of people who work in the field of Māori-medium education sometimes creates an apparent problem of conflict of interest. Indeed, this might well appear to be the case in this report, as some of the diagnostic tools (approximately 50% of those most frequently used), were developed by some of the members of the research team. However, we feel that a number of procedures undertaken in the project will give the assurance that we were not just identifying and rating highly those diagnostic tools developed by our colleagues.
The first procedure was to fax the list of diagnostic tools we identified to a sample of Maori Medium schools beyond the boundaries of the project. The purpose of this activity was to check if we had found a complete list of diagnostic tools being used in schools, and to ask if there was any developments they saw as essential. The feedback from this fax survey was very encouraging, identifying that our list was complete and giving us a number of valuable suggestions for future developments.
The second procedure was that the Project Director and first author of the report was not involved in any way in the development of any of the diagnostic tools. As a result he was able to complete the analysis and development of the final report in relationship to the usefulness of the tools as identified in the evaluation criteria in an objective manner.
A third procedure was that the very process of conducting the research allowed ownership of the representations contained in this report to be with the research participants. In other words, the research team adhered to Kaupapa Māori research principles that sought to facilitate self-determination of the research participants at all stages.
Lastly, we had an independent reviewer make an extensive evaluation of the report and the procedures used in this research. His response was very positive, extensive and supportive of the process; apparent or potential conflict of interest or bias did not enter into his deliberations.
This study of diagnostic tools, found an overall pattern of limited availability, limited cultural validity, limited confidence and understanding of the linkages of the assessment procedures to teaching programmes and limited models for and access to professional development.
Teachers spoken to were, in the main hesitant to suggest the development of further diagnostic tools unless those that are available become more readily accessible to teachers, are clearly linked to teaching programmes, are legitimated by and within Māori cultural processes and are accompanied by appropriate teacher development support.
The information in this study was mainly garnered from a survey of some 20 Māori-medium settings in three areas where the Māori population is highly concentrated (Northland, Bay of Plenty, East Cape and Waikato/Hamilton) and consequently so also is Māori-medium education. Due to the time scale and agreed to (contracted) scope of the study, some areas of Māori concentration e.g. Auckland, and East Coast were not part of the study. Nevertheless, the picture that was indicated from the areas we did sample was one of limited availability and use of diagnostic tools. Further, this research has identified that there are a few diagnostic tools that are being used by most of the schools in the sample of 20 and there are a number that are used less frequently. Indeed there are really only three sets of diagnostic tools that have widespread currency in the sample of schools and these we have described and evaluated in detail in this report. Other diagnostic tools have warranted less extensive analysis due to their limited use.
We also had aspirations, as stated in our original scoping exercise, to identify how the diagnostic tools might link into the curriculum statements and objectives. However, these aims could not be fully realised, given the scope and scale of the information available. Nonetheless, despite the limited information and emergent understandings, there are some indicators of what might constitute useful conditions for the future development of diagnostic tools and their distribution to and implementation by Māori-medium teachers.
The ongoing debate within Māori-medium education over the purpose of education through the medium of Māori language needs to be acknowledged within the context of diagnostic assessment. Māori-medium education developed in New Zealand as the result of a strong determined resistance movement to the ongoing colonisation of the minds of Māori people (Smith, 1995). One of the basic tenets of the whole Māori-medium education movement was to afford Māori learners and their whānau, tino rangatiratanga (self determination) over what constitutes an appropriate model, as well as medium of education (Smith, 1995).
In this sense then, the research team considered it vital that a Kaupapa Māori model of evaluation be used to identify in what ways the diagnostic tool was meeting Māori cultural aspirations, preferences and practices. We used the model developed in Bishop and Graham (1997) and expanded in Bishop and Glynn (1999). This model raises critical questions about power and control over issues of initiation, benefits, representation, legitimation and accountability (IBRLA model) by asking a series of questions such as Who initiated the development of the diagnostic tool and for what purpose? Who specifically was expected to benefit from the use of this tool and in what ways? Are Māori cultural aspirations, preferences and practices evident in development, presentation and use of the diagnostic tools? What authority does the diagnostic tool have in terms of Māori cultural aspirations? To whom are the developers and users of the tool accountable?
These issues are currently at the centre of contestation within education in New Zealand. To bypass this starting point would be to ignore the primary message of Māori-medium education, that is the call for the validation of Māori language and culture. The implication is that we as researchers needed a means of identifying in what ways Māori cultural aspirations, preferences and practices were central to the development and analysis of diagnostic tools. The questions above proved extremely useful in this endeavour.
By including this evaluative framework we have been able to conclude why some tools have been implemented successfully and some less so. For example, those tools developed in accordance with cultural aspirations eg. Ngā Kete Kōrero, have acceptance. However others do not appear to have this degree of 'ownership' by the recipients. The researchers found that to be the second significant outcome of the evaluation.
Linkages to Teaching
The third major finding from this research was at an instrumental level. Teachers made little use of any tools where there was no clear linkage of assessment to the ongoing teaching and learning programme of the students. However, where there was such a clearly identified linkage (and this may have to be identified as part of a programme of professional development), then teachers had little trouble using the tool.
The fourth major finding is that professional development programmes that do not include on site modeling of effective practice followed by ongoing reflection and feedback are not successful. Those who said that they had engaged in successful professional development identified that these were models where trained facilitators accompanied them to the classroom and worked alongside them while they learned to implement the diagnostic tool. Teachers considered it would be of great help if these facilitators could also be available on an ongoing or 'call-out' basis.
Successful design and implementation of tools (or future developments) is not a matter of instrument distribution and teacher implementation but rather a matter of working within a dynamic model of professional development. The professional development model that was observed to be most successful involved a sequential, four-element approach. First instruction and demonstration, followed by opportunities to perform or practice the procedure in an authentic context, then opportunities for ongoing and informed reflection and feedback between teachers and supporters in the classroom context. Where teachers have gone from training (instruction, demonstration and practice) to immediate implementation and practice in their own classroom setting with ongoing reflection and feedback, the procedures are learned and implemented with greater reliability than if the instrument is distributed to the school without training or with training components alone implemented. The period between training and usage of the procedure in an authentic classroom context is also crucial to the successful uptake of the skills and knowledge required to use the procedure reliably in the future.
In this study and in the experience of the research team, trainers need to be informed (knowledgeable about the related theories) and competent (able to practice the procedures). The amount of time spent in training appears to be a less significant factor towards effectiveness. Retired teachers, teachers from English medium, and graduate students who have fluency in te Reo (the Māori language) have been observed and identified by teachers as successful trainers. This identified that successful trainers do not necessarily have to mean a further drain on already depleted Māori-medium staffing roles.
Professional development needs to focus on an inclusive education model that seeks to demystify existing packages. This will require moving away from an expert-novice model of professional development to one perhaps more in line with ako (Pere, 1982) where the roles of the teacher (trainer) and learner (teacher) are interchangeable and both roles take into account what can be learned from the direct observation of students during the administration of the diagnostic tools in an authentic setting. The expert-novice model does not allow this to happen and results in the expert missing valuable opportunities for reflection and further learning.
Because of the dynamic and developing nature of Māori-medium education, current teacher education practices are seen as inadequate for providing skills and understandings that practicing classroom teachers need. This means that the knowledge of "expert" providers is rapidly outdated because of their lack of ongoing interaction with students and teachers. Professional development and teacher education needs to be conducted differently from the way it is at present.
In summary, the overall picture from this study is that a successful diagnostic tool addresses an interlocking matrix of concerns, where
- cultural legitimation is seen in terms of the IBRLA model.
- assessment outcomes clearly link back into teaching and curriculum development.
- in-school reflection, feedback and support is essential to the professional development model.
There was not a great call for more diagnostic tools. Rather there was a call for more in-depth and successful implementation of existing diagnostic tools. This does not mean Māori-medium educators do not want more diagnostic tools, especially in the area of mathematics, but it does mean that to expand the quantity at this time when quality of delivery and implementation are in doubt could be problematic. However, it must be emphasised that the model that is suggested above by this research is one that works within Māori cultural practices and principles, this being an essential initiating process for the development of any new diagnostic tools.
A key direction for future research into diagnostic tools within Māori-medium education lies in establishing a professional development model that targets new tools to areas of national curricula, and that provides for competent and ongoing mentoring and feed back on their appropriate use.
One reason for teacher resistance to the use of diagnostic tools may well be associated with the inappropriateness of using translated tools. With translation comes the expectation of "what all X year old children should know." So all-pervading is English in the wider community, it is the language likely to be used by the majority of parents to communicate in the home (Benton, 1985), and it is still also the first language of the majority of children attending education in Māori immersion settings (Benton, 1985; Education Review Office, 1995; Hollings, Jefferies & McArdell, 1992). Māori learners coming from English language dominated communities and a variety of education settings demonstrate a wide competency range in te Reo and cannot reasonably be expected to perform at equivalent levels to their English medium counterparts. Persistence with this approach of translating resources developed for a different language and culture could mean that schools will continue to avoid using these types of tools or students will continue to be misrepresented and misinterpreted by the results of these tools and found to be deficit (Berryman, 1996). However, having said that, some of the successful tools are those that are worked anew from English medium examples. However, what does appear to be crucial is that alternative models are best where resources for Māori-medium education are developed and legitimated from within the framework of the Māori language and culture.
Education Data Requests
If you have any questions about education data please contact us:
Email: Requests EDK
Phone: +64 4 463 8065