Evaluation of Teacher Professional Development Languages (TPDL):
For teachers of languages in years 7-10 and the impact on language learning opportunities and outcomes for students
This study was carried out during 2008 and aims to inform the Ministry about the TPDL and the impact on language learning opportunities and outcomes for students.
Author(s): Sharon Harvey, Clare Conway, Heather Richards & Annelies Roskvist, AUT University
Date Published: December 2009
This report is available as a download (please refer to the 'Downloads' inset box). To view the individual chapters please refer to the 'Sections' inset box. For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Chapter 3: Impact of the TPDL programme Years 7-10
On the development of teachers' proficiency in the teaching language (TL)
Teachers are 'busy and often overworked'. In order to sustain their involvement, they need a catalyst or 'urgency'. The learning experience needs to be meaningful, practical and relevant to their classroom with their particular students (Stoll, Fink & Earl, 2003, p. 85).
Studying the TL is a core component of TPDL for teachers who do not have advanced proficiency. This has many benefits both in developing participants' language knowledge as well as providing further insights into the challenges of being a language learner. For the language study component of the TPDL, teachers study their TL at an appropriate level from a local institution or from an extramural provider, in order to improve their language proficiency and prepare for an internationally recognised language examination (Thomson, 2008d).
In this chapter we consider the progress teachers made in developing their TL proficiency. We give the background of the participants, including their teaching TL and type of language study undertaken. We note the length of study and participation in internationally recognised examinations. Also considered is the teachers' proficiency in the TL, their motivation to learn the TL, the teachers' knowledge compared with their students' knowledge of the TL and teacher confidence in using the TL in the classroom and in the community. The effectiveness of the language study component of TPDL including teacher suggestions for improvements is also presented.
The cohort of TPDL participants who began in 2008 consisted of 58 teachers, 50 of whom completed the programme (see Figure 1). The majority were teaching European languages (21 French, 21 Spanish and five German) with five teaching Chinese and four teaching Japanese (Thomson, 2008b). Most of these 58 teachers were also studying the language concurrently.
Figure 1: Number of teachers on TPDL programme and their teaching language (Thomson, 2008b and Thomson, 2009)
The TPDL participants undertook their language study courses in a range of settings (see Figure 2). Half (11 of 22) of the survey one core respondents were studying the language through a credit bearing course (e.g. through a university, polytechnic or high school), while seven were studying in local community classes and four others were engaged in independent language study. Survey three showed that three participants had made changes during the year; two to independent study and one to a local community class. One of the reasons that some of these changes occurred was that programme directors made new arrangements to 'accommodate individual preferences and circumstances' (Thomson, 2008e, p. 2).
Figure 2: Teachers and type of language study course
Length of study and exam achievement
Becoming proficient in another language takes time. Learning language is a long process, and it can be a challenge to sustain the learning, so motivation may fluctuate over the time. The fact that the TPDL teachers committed to learn and teach the language concurrently may have been the powerful 'catalyst' needed (Stoll et al., 2003) to help them to pursue their language study in addition to their already demanding workload.
In survey one respondents indicated that their time in language study varied. However all of them devoted a minimum of two hours a week, and over half spent more than two hours a week studying language. Thomson (2008e) reports that nearly half the TPDL teachers studied at UNITEC (an Auckland polytechnic), and feedback on attendance results indicated that they missed very few classes over the semester. In addition, UNITEC's customised distance programmes worked well, particularly the French programme which had an increased number of teachers wanting to take part in the course (Thomson, 2008e).
Data from the 25 respondents in survey three indicates that just under half (11) studied their TL for one to two terms, another 11 studied for three to four terms, while two did no language study. So just under half the respondents continued their language study into semester two. While Milestone Report Eight suggests that 90% of the teachers who had studied in semester one would continue in semester two (Thomson, 2008e, Appendix 16), the researchers' data indicates that a second semester of TL study had not eventuated for all of them. Case study interviews revealed that one teacher would have liked to continue her language study for a second semester, but was unaware that TPDL would pay for her to continue.
Language study was not considered necessary for two of the survey three respondents, as they were either native speaker level or advanced TL speakers. However, other participants found the language study course valuable not just because of the language learning. For example, when asked what was most useful in their language study, four of the 20 respondents mentioned aspects other than acquiring the language. Two said they benefited from being a learner, and two said they gained good teaching ideas from taking part in the language classes.
According to the most recent milestone report (Milestone Report Nine), the TPDL programme provides support and encouragement for teachers to sit an internationally recognised language examination. Twenty two of the 50 teachers who completed TPDL in 2008 sat a relevant language examination, including one who sat Level 1 NCEA in the TL (Thomson, 2009). Nineteen teachers passed their examination, with three still awaiting results (Thomson, 2009). The remaining teachers did not sit an examination either because they already had a degree in the language, were not confident or ready yet, or there was no examination available at an appropriate level (Thomson, 2009). However, there is continuing development in the area of language examinations, and as further suitable examinations became available, the TPDL directors facilitated the teachers' enrolment. For example, one teacher of Chinese sat and passed the new Chinese Proficiency Test 1 (CPT1) which was being trialled in the Auckland area. The introduction of this Chinese examination and the new A1 Spanish Diploma de Español como Lengua Extranjera, (DELE) offered more teachers the opportunity to set language proficiency goals and work towards achieving them. As noted by Thomson (2008e), language examinations can have a motivating effect on language teachers.
Teacher perception of TL proficiency
A further way of obtaining insights into teachers' proficiency in the TL is to investigate their perceptions of their gains in the TL. A comparison of data from surveys one and three indicated some progress. Survey one, administered shortly after the start of the TPDL course, asked respondents to categorise their proficiency level in their TL on a five point Likert scale from beginner to bilingual. The results (see Figure 3) revealed that the majority (20 of the 25 respondents) described themselves as being at 'intermediate' level or below at the beginning of the course. Only five of the core 25 respondents described themselves as highly proficient, at 'advanced' or 'bilingual' level. Results from survey three, administered at the end of the course, showed the advanced level had remained constant and there had been movement at the lower levels, with the clearest upward movement being from elementary to intermediate level. By the end of the course, eight of the 24 respondents described themselves as 'intermediate' level, compared with just three at the start of the course. Half (12 of the 24 core respondents) perceived themselves at beginner or elementary level.
While teacher perceptions of their language level are not an objective measure of TL proficiency, they do indicate that teachers themselves feel they improved in their language skills. This sense of achievement can only be good for their confidence in using the TL in the classroom.
However, the fact that half the survey respondents indicated they were still at elementary level or below raises the question of whether they have sufficient knowledge of the subject (i.e. the TL) to meet the Ministry of Education's Statement of Intent regarding effective teaching: 'Effective teaching focuses on maximising learning outcomes for all learners in every situation. Effective teaching requires knowledge of subject and teaching practice. The heart of effective teaching is where these three areas of influence intersect' (Ministry of Education, 2007b, p.3). In this particular programme, and at this particular stage of the Ministry of Education's intent to upskill generalist teachers, it is not possible for all language teachers to have considerable subject knowledge. However, language learning is an ongoing 'dynamic, developmental process' (Crabbe, 2005, p. 6) and teachers at the lower levels of TL proficiency need to be encouraged to continue learning the TL until they do have considerable knowledge so they can be fully effective.
Figure 3: Perceived language proficiency level (Survey 1 and Survey 3)
The case study data gathered from surveys one, two and three across the year reveals further insights into the type, timing and amount of change in language proficiency teachers perceived (see Table 1 below). Of the teachers who perceived no change in their language proficiency level, none of them sat an exam and studied the language only for one term or not at all. The teacher who noted a decrease in proficiency had one term of study, and sat and passed the examination. Although orally fluent, through the language study course, the teacher became more aware of the need to develop the written area of her language. Of the three teachers who noticed increases in their proficiency, two sat examinations, one learned the language for three terms and one for four terms. The teacher who studied for the full two semesters and sat an international exam indicated she had made a steady increase in her level of language proficiency throughout the year. As could be expected, the length of study and the preparation for an exam resulted in the greatest perception of change. Sitting an external exam appears to have made a difference to teachers' perception of proficiency. One reason could be that passing the exam gave the teachers a new measure of confidence. Another reason could be that the exam provided external motivation for the teachers, thus increasing their perception of proficiency.
Table 1: Teacher perceptions of change in language proficiency level
Motivation to learn TL
Motivation is an important factor in language learning as it is seen as responsible for '… why people do something, how long they are willing to sustain the activity and how hard they are going to pursue it' (Dornyei, 2000, p. 8). Motivation is not constant and data from surveys one, two and three shows the fluctuations among participants over the year. At the start of the TPDL course, survey one revealed that all of the respondents except one were motivated or very motivated to learn the TL (see Figure 4).
By survey two, eight of the 24 core group were no longer studying the TL. A number of reasons were given for stopping: time pressure (personal and/or work commitments); no available course at the next level; course not meeting their needs; had already fulfilled the TPDL requirements; being at a very high level of TL proficiency and so there was no need for any further instruction. Of the 16 teachers who responded to the question about their level of motivation to learn the TL, all except one were motivated or very motivated to learn.
Data from survey three revealed that three quarters (17 of the 22 respondents) were still motivated to learn the language, but the number who felt 'very motivated' had dropped from 10 teachers to six teachers. This may reflect teachers' 'end of year' feeling with a degree of tiredness, or the fact that they had already completed the TPDL requirements. It is worth noting that when teachers were asked in survey three about their intentions for further TL study, 21 of the 24 respondents planned to continue to study their TL in 2009, either through a local community class, a credit bearing course or independent self-study. Three teachers had no plans to continue: two because of the cost, and one because she was already bilingual.
Figure 4: Teacher motivation to learn TL
One factor which may help to sustain motivation for some teachers is the Ministry of Education Language Immersion Award (LIA). There is considerable interest in the LIA with 19 of the 23 survey three respondents stating they are, or may be, interested in the award to travel and study their TL abroad. All but one of the seven case study teachers were interested in applying or had applied for the LIA. However, comments from those who expressed interest but had not yet applied, indicated concerns such as family commitments and the 'monster time' required to complete the application. The one teacher who was not interested was a beginner in the TL, and wanted to wait until she was more competent so that she could fully benefit from the cultural experience. Five of the 50 TPDL teachers (three French, one German, one Japanese) received an LIA award in 2008 (Thomson, 2009) and will be undertaking their immersion experience in 2009. Further evidence of the benefits of this award will be reported on in Evaluation of the Language and Culture Immersion Experiences for Teachers Programmes: Their impact on teachers and their contribution to effective second language learning (Harvey, Roskvist, Corder & Stacey, 2009).
Teacher knowledge of TL compared with student knowledge of TL
At the beginning of the TPDL course 17 of the core 25 survey participants described themselves at beginner or elementary level in their knowledge of the TL. In most cases, these teachers were teaching the TL to students who were also beginners. It was important to find out whether the teachers' knowledge of the TL continued to be at least equal to, but preferably greater than, the students' knowledge throughout the course. From the beginning, all but one respondent felt they had 'a bit more' or 'considerably more' knowledge of the TL than their students (see Figure 5). This was to be expected, as all but two respondents had had some previous TL instruction. Of the remaining two, one had experienced living in the target culture and had undertaken self-study in the TL, and the other one, although not familiar with the TL, had previously learned other languages. Survey three data indicated that there had been movement for three respondents who now indicated they had considerably more knowledge than their students.
Figure 5: Teacher perceptions of their TL knowledge compared with their students' TL knowledge
As mentioned before, knowledge of subject is one of the requisites for effective teaching (see section 3.3) and it is desirable for teachers to have more knowledge than their students. There is the potential for students (through independent study or prior language knowledge) to be at a more advanced level than the teacher. However, the core survey respondents indicated this was not the case in their classes. On the TPDL programme, nearly all teachers had more knowledge, although some had only a little more. Where teachers had only a little more knowledge than their learners, it is possible that language teaching could be seen as less effective. However, it is reported (Scott and Butler, 2007) that when teachers and their students are learning the language at the same time, they empower one another as language learners; together they take risks, increase their knowledge and support each other in learning.
Confidence and use of TL language
The development of teachers' TL proficiency can be investigated through an enquiry into teacher confidence in using the TL with their learners in the classroom. A further means of investigation is to consider the teachers' opportunities for TL use when participating in the TL community either directly or indirectly.
Confidence in using the TL in the classroom
Participants were asked about their level of confidence in using the TL with their learners at the beginning and at the end of the TPDL programme. There was an increase in teachers' level of confidence with the majority (22 of 24 respondents) indicating they were 'confident' or 'very confident' in using the TL with the students, compared with 17 teachers at the start of the course (see Figure 6).
Figure 6: Teacher confidence in using the TL in the classroom
Overall, the surveys indicated the majority of the teachers were feeling confident about their use of the TL with their students. They were not only feeling more confident, but were also using the TL more in class. The TPDL course facilitators gathered evidence across four visits on the amount and purpose of teacher use of the TL and at each visit, the expected standard was raised (Thomson, 2009, Appendix 13). According to milestone report nine, the majority of teachers (43 of the 47) observed in visit four, were either meeting or exceeding the progress standards for teacher TL use in the classroom (Thomson, 2009, Appendix 27). The researchers in their final observations of case study teachers noted varying degrees of teacher TL input which is reported on in 4.2.1.
Teacher participation in the TL community
Participating in the TL community is an integral part of language learning. A study of more than 1700 highly motivated foreign language learners found that interethnic contact influenced learner attitudes to TL in a positive way, assisted the development of the learners' language competence, increased their motivation and reduced their anxiety when using the TL (Kormos & Csizér, 2007). The study also identified two main types of intercultural contact; direct and indirect. Direct contact includes visit to TL country, contact with non native or native speakers of the TL, internet chat and email or mail. Indirect includes contact with a teacher who has had experience in TL country and interaction with television, the internet, books and magazines. Both kinds of contact were found to build positive attitudes to language learning (Kormos & Csizér, 2007). It is therefore important for TPDL participants, as learners of the TL themselves, to have contact with their TL community.
In survey one, respondents were asked if they had opportunities for participation in the TL community. Over half the respondents indicated they had none. However, the researchers believed that participants may have had a narrow concept of 'TL community'. Therefore, in survey two, a more detailed question was asked, categorising contact with the TL community according to types of intercultural contact situations, 'direct' and 'indirect' (Kormos & Csizér, 2007).
The data from survey two showed that more than two thirds of the teachers were engaging indirectly with the TL community through the internet, DVDs and books. In addition, some were reading magazines and watching television in the TL. Similarly, direct contact was mentioned by 20 of the 28 teachers (71%) who indicated they had direct spoken contact in New Zealand with TL expert users. Eight of the 28 teachers indicated they participated in the TL community through visiting the TL country. A small number of teachers had direct written contact through email and conventional mail in the TL. The results from survey three indicated that this engagement with the TL community remained constant for the rest of the course.
Data gathered from the case studies provides further elaboration on the participation in the TL community, both direct and indirect (see Table 2).
Table 2: Teacher participation in the TL community
Limited participation in TL community. Not confident to explore TL community. Has practised language in TL restaurant.
Reads TL books and uses the internet. Watches TV.
Full participation. Speaks to other TL colleagues, relatives, and practises in TL country.
Watches TL television, movies and DVDs and reads books.
Limited participation in TL community. Not aware of any opportunities to practise speaking TL and not confident to go the wider TL community. Practises with one class member in TL Study class.
Watches movies and DVDs, reads books and magazines and uses the internet.
Limited participation in TL community. Writing emails in TL.
Reads books and magazines. Uses the internet. Downloads daily TL news programme on to iPod. Watches movies and DVDs.
Limited participation. Sometimes speaks TL to colleague in school. Doesn't go into the wider TL community.
Reads TL magazines, books and uses the internet.
Some participation in TL community. Member of TL community group; weekly meetings with TL speaking friends.
Reads TL books, uses the internet and watches TL movies.
Full direct participation with TL community. Member of TL community group. Weekly meetings with TL speakers. Speaks to TL colleagues and language assistants.
Uses TL internet, books, DVDs, magazines and writes letters.
Table 2 shows combined data gathered from interviews two and three with case study teachers. The researchers noted there was a range of direct participation, from limited practices of speaking with or writing to other TL users, through to full, regular oral and written contact with the TL community. Researchers also noted a slight increase in the variety of indirect contact during the year. For example, one teacher who mentioned using the internet and reading books earlier in the course indicated she was also watching movies and reading magazines in the TL by the end of the course. By the end of the course, all had established some indirect contact with the TL community. It is difficult to ascertain the extent to which the increased participation in the TL community was a result of TPDL. However, increased motivation and participation appears to correspond with studying the language and working with other colleagues interested in the same language and culture.
Effectiveness of the language study course
As well as gathering data on the teachers' developing language proficiency, motivation and confidence in use of the TL, the researchers sought to understand the extent to which the teachers believed their language study course was effective (see Figure 7).
Figure 7: Extent to which participants feel their language study course is effective
Figure seven shows that teachers' perceptions of language study effectiveness remained constant. The majority (17 of 21) indicated at the end of the TPDL programme that their language study course was effective 'to some extent' or 'a great deal.' However, there were still four respondents who believed the language study they undertook was only 'a little effective' or 'not effective at all'. All five of the seven case study teachers who undertook language study were positive about their language study courses. However, two were disappointed, but commented that it was no fault of the TPDL programme directors, as there were no suitable classes in the area.
As noted by Thomson (2008e), there were occasions where teachers chose to go to particular community classes against the Project Director's advice. These classes proved unsatisfactory for three teachers. In addition, one teacher found the customised distance programme did not suit her learning style because of a slow start and difficulties communicating with the tutor. The TPDL course directors went to considerable lengths to organise alternative tuition. However, difficulties in finding suitable tutors in convenient locations broke the continuity of language learning, which may have accounted for the lack of effectiveness recorded by some survey respondents.
Useful aspects of the language study course
Surveys two and three asked qualitative questions requiring participants to reflect on useful aspects of their language study course at the end of each semester of learning. A number of categories emerged as being important in fostering language study. The teachers appreciated the opportunities to use the TL and to communicate with other participants or the class teacher when they were studying the language. Teachers noted the usefulness of the language content they were taught. Receiving language teacher feedback on the accuracy of their language use encouraged them as language learners. In survey two, some participants indicated they found studying and preparing for an exam motivating.
The answers from the survey three qualitative question on useful aspects of the language study course are summarised below (Table 3) and are in ranked order of mention. One area that did not emerge in survey three was the motivation arising from examination preparation. By the time survey three was administered, teachers had completed their internationally recognised language exam and so the motivating aspect of this seems to have become less important.
(x No of Participant Comments)
|Sample Teacher Comments|
|Opportunities to practise (x 12)||
Opportunity for conversation practice; |
conversation with my tutor most useful;
opportunity to speak with my language instructor regularly.
|Language course content (x6)||
Hearing correct pronunciation and grammar; |
|Class teacher feedback on language (x5)||
Immediate feedback; |
the help from tutor;
having an instructor available to clarify information such as verb tenses;
regular feedback and feed-forward.
Teacher suggestions for improvements to language study course
As well as understanding the benefits teachers received from their language study course, it is important to consider teacher suggestions for improvements. Suggestions for ways to improve the course were gathered in surveys two and three. Not all respondents made comments. However eight teachers said they were happy with the course and did not make any suggestions for improvement. All other suggestions are reported below in Table 4 in ranked order (the same comment made by the one teacher in the two surveys is indicated only once).
|Improvements||Sample Teacher Comments|
|More practice (x6)||
More conversational time. |
Maybe a bit more oral interaction.
More face-to-face oral practice.
|Delivery (x5)||Method of teaching was dull - more variety and reduce the amount of theoretical analysis. Slow down. Didn't suit my learning style.|
|More grammar (x4)||More structure of the language and how things fit together. More emphasis on grammar.|
|Other (x1 comment each unless otherwise stated)||
Need local area class (x 2).|
A course at the appropriate level (x 2) – too wide a range of speaking level.
Better technical support for distance classes e.g. video cam.
Clearer assessment guidelines.
More written work in the course.
Supply of DVDs for language lessons.
More feedback on language use.
Face-to-face university credit-bearing course in the TL that fits around full-time work.
While there were many positive aspects of the language study component of the TPDL, there were still areas where the language courses were not fully meeting the needs of the teachers. This is an area that may be difficult to address as not all the language courses are able to be influenced by the TPDL directors. This is further elaborated in chapter eight when considering sustainability of the TPDL programme.
Teacher proficiency in the TL is a key determinant in the quality of student language learning (Gibbs & Holt, 2003). The languages learning component of the TPDL programme had a positive impact on the TL proficiency of most teachers. By the end of the programme, most teachers perceived they had increased their level of TL proficiency and at the time of writing this report, nearly all of those who sat internationally recognised examinations had received confirmation of passing. The data showed that by the end of the course there was a decrease in those describing themselves at lower levels, and an increase in those perceiving themselves at intermediate level. Amongst the case study teachers, those who perceived positive change had studied the language for more than two terms and sat an examination. Those who perceived no change had not sat an exam and had studied the language for the minimum time required, or not at all. Along with this perception of increased level of proficiency came a gain in confidence in using the TL in the classroom. As well, all the teachers indicated at the end of the course they had more knowledge of the TL than their students. TL use was also reported on by the TPDL directors who confirmed that the majority of teachers were at the expected or an accelerated level of TL use in the classroom. Outside the classroom, teachers were engaged in a range of direct and indirect forms of participation in the TL community. Motivation to study the TL was sustained over the course. Although after semester one some participants had stopped learning, by the end of the TPDL programme more than three quarters of survey respondents intended to continue to study in 2009. Similar numbers expressed possible interest in the LIA as a means of developing their TL fluency and understanding more about the culture. Some TPDL participants had already been awarded LIA for 2009.
The language study component of the TPDL programme was considered by the majority of respondents to be effective to some extent or a great deal. The main way the language study course fostered TL proficiency was the opportunity provided for teachers to communicate in the TL in the language class. In addition, teachers appreciated the accurate language input they received and the feedback on their own TL accuracy. While the majority of teachers were satisfied with their language study courses, some respondents felt their course was only 'a little effective' or 'not effective at all'. Teachers thought language study courses could be improved by including more oral interaction, more engaging teaching methods and more focus on the structure of the TL. At this stage it would appear that some of these improvements are not within the immediate control of the TPDL directors who were very aware and pro-active in trying to meet the language needs of the teachers.
Although participants had made progress in their TL learning, around half of survey three respondents perceived themselves to be at Elementary or lower in their TL proficiency. Another eight participants were at Intermediate level. The Ministry of Education Statement of Intent, 2006-2011 notes that knowledge of subjectis one of the three conditions for effective teaching to occur (Ministry of Education, 2007b). With half of the core survey participants having relatively low levels of TL proficiency, it is difficult for student language learning to be maximised.
Recommendations and implications for the TPDL programme
- It is desirable for teachers who are teaching the TL to be at an intermediate level of TL proficiency or above. Therefore, teachers who are below an intermediate level of TL proficiency should be encouraged to study the TL during and beyond the course, and to sit exams to reach intermediate level.
- Sitting an external language exam is desirable for those teachers who are not bilingual as sitting an exam was found to increase teacher perceptions of TL proficiency.
- The Language Immersion Awards are motivating for many of the TPDL participants and should be strongly promoted on the programme.
- Language study classes need to be of a high standard and to meet the needs of the TPDL teachers, so there is continuity of learning.
- Teachers' prior interests and experiences in relation to the target language community need to be acknowledged and harnessed during the TPDL course. These can be explicitly built upon to encourage participants to seek out opportunities to engage directly and indirectly with target language communities.