Evaluation of the Manaiakalani Digital Teaching Academy Publications
This report outlines findings from an evaluation of the first year of a pilot initiative called the Manaiakalani Digital Teaching Academy (hereafter MDTA). The Manaiakalani Education Trust and the schools within its network are described in Section 2 to provide a context for the initiative and the evaluation.
An addendum to this report contains a brief update, carried out after graduation from the MDTA towards the end of 2015.
Author(s): Rosemary Hipkins, Jenny Whatman and Jo MacDonald, New Zealand Council for Educational Research. Report prepared for the Ministry of Education.
Date Published: October 2015 Addendum 2016
About Manaiakalani and MDTA
The MDTA pilot was conceived as a proactive response to a growing frustration from many school leaders in the Manaiakalani schools cluster. These schools have been working hard to transform their practice for the digital era. From their perspective, initial teacher education (ITE) programmes have not kept up with 21st century changes in education.
During 2014, 10 newly qualified beginning teachers (BTs)1 were each paired with a mentor teacher, working alongside each other in the same learning space. The aim was to support these first-year teachers to begin their teaching career working with the digital pedagogies their mentors already skilfully use, thus accelerating their progress to becoming the sorts of highly effective teachers needed by the schools in the cluster. In 2015—their second year of full-time teaching—the 10 BTs now have responsibility for their own group of students2 but continue to work in the same school, with ongoing close support as needed.3
A practical component was embedded within the programme by releasing the BTs on Wednesdays to enhance and extend their digital skills. In 2014 BTs and their mentors also attended The University of Auckland to take part in an academic programme of postgraduate study specifically designed to support the initiative. Most mentors are working towards an MEd qualification and BTs towards an Honours qualification in education. In 2015 most members of both groups are carrying out research projects initially devised as part of their 2014 learning.
The Evaluation: Evaluation questions and methodology
The purpose of this short evaluation was to understand how the MDTA has been playing out so far, from the perspectives of all the different participants, and to shape our findings in ways that will be useful to both MDTA and to the Ministry of Education (the Ministry).
We posed the following evaluation questions:
- From the perspectives of different participants, how well prepared are the BTs to take full responsibility for their Manaiakalani class (or classes) and students' digital learning?
- What role has the BT's apprenticeship with a mentor teacher played in how well prepared they are?
- What role has their university work played in how well prepared they are?
- (How) has their involvement in a teaching inquiry influenced their developing practice?
- How has the mentor role impacted on participating mentor teachers' professional practice?
- Do they report differences in their understanding, confidence and practice of digital learning as a result of their mentor teacher role?
- Do they report differences in their understanding, confidence and practice of digital learning as a result of their university work?
- (How) has their involvement in a teaching inquiry influenced any changes they have made in practice?
- What (other) benefits have there been for BTs, mentor teachers and students?
- How does the MDTA approach fit with what is known of good practice in other ITE4 work designed to provide knowledge and confidence with digital pedagogy for beginning teachers?
As a first step in designing the evaluation, two NZCER researchers worked with a representative group of participants and cluster leaders to develop a "theory of change" for the project as a whole. The agreed model is shown as Figure 1 on page xii. By clearly identifying the anticipated outcomes—and how they were expected to be achieved—this model served as referent for devising interview questions and for guiding the analysis. In March, three field workers interviewed Manaiakalani and MDTA personnel, principals, mentors and BTs involved in the MDTA pilot. Interview notes were recorded on a Google spreadsheet.
We conducted a scan of the literature to answer the final research question. We drew on our own knowledge of ITE (in particular, field-based programmes) and professional learning and development (PLD), on digital learning in New Zealand and links on the Manaiakalani website.
The success of the pilot
This did indeed prove to be a successful pilot. The BTs are now working as effective, engaged second-year teachers, all but two of them in low decile schools. MDTA and school personnel consider they are coping better than most other BTs in these schools (who were supported more traditionally to complete their first year of teaching). Some of the BTs in MDTA are markedly more skilful than might be expected at this early career stage. All of them hope to continue to work in the Manaiakalani cluster, at least through the next few years. Some BTs are taking on significant leadership roles within the school, the cluster and beyond. All BTs are well positioned as teachers capable of accelerating students' progress through the use of digital pedagogies.
The three aspects of MDTA (in-class pairing with a mentor, digital upskilling and university study) have all played a significant role in the BTs' preparation and the strength of the model relies on the interplay between the three aspects. It is not possible to say which has had most impact as each has played out very differently for the 10 BTs.
BTs valued different things about their Wednesdays—some rating the digital upskilling highly, and others preferring the university study, especially the course on accelerating learning and the summer course on digital pedagogies. The Wednesday digital learning component clearly accelerated BTs' digital knowledge and skills for effective teaching and learning. Some of the group are now providing professional leadership in digital learning in the wider education context, well beyond what might be expected at this early career stage.
The university lecturers have scaffolded BTs into the second year of the qualification—an intervention inquiry into an aspect of accelerating learning through digital pedagogies. BTs appear to be confident about completing their qualification and enhancing teaching and learning through their study. All 10 BTs successfully completed the first year of the academic programme towards an Honours degree in education designed for them by The University of Auckland. Most mentors enrolled in and completed the first year of the academic programme towards a Master's degree in education.5 While their study proved to be demanding—particularly in terms of finding the time to do justice to the learning—most participants can clearly see the value that this component added to the overall programme. They acknowledge that the initiative would not have been as strong without the academic component. A few mentors have postponed the second year (research project stage) of their university study because of competing demands or difficult life circumstances. This decision is unlikely to have a lasting negative impact on their development as mentors or on the programme providing they do not postpone their study indefinitely. The structures that MDTA has put in place for the future to support mentors will help mentors meet their academic commitments.
In the strongest mentor/BT collaborations, both partners mutually enhanced their pedagogical repertoires as they worked and studied together. Three mentor/BT relationships were extremely successful and the mentors in those relationships (and in most other mentor/BT combinations) commented positively on their own learning because they were a mentor, their learning from the BT, especially in relationship to using digital tools, and the positive impact that having two teachers with the class had had. They also valued the university component, in particular the course on accelerating learning, and could see the importance of now having the theoretical background to critically examine their practice.
A small number of mentor/BT partnerships were less fruitful and the BTs involved were supported more intensively by MDTA personnel. MDTA has recognised that providing ongoing professional learning support for all mentors needs to be prioritised for any future developments. In addition, MDTA acknowledges the need for careful pairing of BTs and mentors.
The role of the MDTA professional adviser has been pivotal in supporting BTs personally and professionally and in making connections between the different parts of MDTA.
The supernumerary position of the BT provided significant benefits to the school in the form of an additional teacher. There appear to have been benefits for students, especially in classes where the mentor/BTs worked well together and where the BT was perceived as another bona fide teacher. Schools did not need to rely on relievers in these classrooms as BT or mentor could take full responsibility when one or the other was absent. BTs shared their digital pedagogical knowledge within their team and the school, and some have demonstrated new learnings beyond the Manaiakalani cluster. Some challenges for 2014 included less successful mentor/BT partnerships which were unable to capitalise on having two teachers sharing a teaching space and group of students. In part this will be ameliorated by the mentor PLD that is already in place and by different BT selection processes planned.
Timetabling all three components of the programme has also been a challenge—partly because withdrawal of BTs on Wednesdays was seen by many to interrupt their classroom learning and their ability to contribute more widely to the school. Almost all BTs, along with mentors who were undertaking a university qualification, identified workload as a major challenge, particularly when assignments were due. Despite this, the university was seen in the main to have provided relevant courses based on the Manaiakalani experience.
MDTA is unique
MDTA is unique in New Zealand, and, as far as we can ascertain, internationally. The digital enhancement aspect is loosely modelled on a University of Alabama university lecturer (Dr John Strange) whose assignments require students to present and share work in a variety of digital modes. Conceptually it exhibits all the characteristics of high-quality ITE and PLD. As with most other programmes, there have been teething problems and some of the intentions of the programme have not been realised for all participants. MDTA personnel are aware of the difficulties individuals have experienced and have worked hard to provide support and to rethink structures so that these difficulties are minimised in future. The model is expensive and purpose-built—it is designed to provide digitally skilled teachers for schools in Manaiakalani. To date its success has partly relied on the vision, hard work and goodwill of a small number of people and a very fruitful relationship between the Manaiakalani Education Trust, the Manaiakalani schools and The University of Auckland. Where the right BT has been selected, and placed with the right mentor in the right school, and where the principal, mentor and BT have fully embraced the MDTA vision and approach, the results have been very positive and the intentions of MDTA fully realised.
- We have used the term BTs throughout rather than the more widely accepted PRTs (provisionally registered teachers). This is because Manaiakalani and the Ministry RFP refer to BTs and because our evaluation is not concerned with teachers meeting registration requirements but is focused on the BT journey into teaching.
- This should not necessarily be read as meaning a group of students and a teacher in a single-cell space. Some BTs worked in this way but others began their second year working in larger spaces as one of a team of teachers.
- One BT has moved to a different school with full support from both schools involved.
- Manaiakalani and the Ministry RFP describe MDTA as an ITE programme. We consider MDTA to be a PLD programme for early career teachers rather than a preservice programme.
- Two mentors were exempt because they already had Master's degrees. All mentors completed one paper: Accelerating Learning alongside PRTs.
Education Data Requests
If you have any questions about education data then please contact us at:
Email: Requests EDK
Phone: +64 4 463 8065