Evaluation of Teacher Professional Development Languages (TPDL):

For teachers of languages in years 7-10 and the impact on language learning opportunities and outcomes for students

Publication Details

This study was carried out during 2008 and aims to inform the Ministry about the TPDL and the impact on language learning opportunities and outcomes for students.

Author(s): Sharon Harvey, Clare Conway, Heather Richards & Annelies Roskvist, AUT University

Date Published: December 2009

Please consider the environment before printing the contents of this report.

This report is available as a download (please refer to the 'Downloads' inset box).  To view the individual chapters please refer to the 'Sections' inset box.  For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.

Chapter 6: The perceived impact of TPDL on student learning and outcomes

Opportunities for teachers to engage in professional learning and development can have a substantial impact on student learning (Timperley, Wilson, Barrar & Fung, 2007, p. xxv).

Introduction

A key focus of the TPDL research is evaluating 'the effectiveness of the programme in improving learning opportunities and outcomes for students of languages' in Years 7-10 (Ministry of Education, 2007b, p. 4). Chapter five reports on TPDL participants' increase in their knowledge of teaching pedagogy and effective classroom practice. As Alton-Lee suggests, 'quality teaching (is) identified as the key system influence on high quality outcomes for diverse students' (2003, p.89). This chapter analyses the impact of TPDL on students' language learning, experiences and outcomes. It is important to note that the design of the study has not enabled the researchers to collect data on individual student academic achievement and learning; and that findings are limited to teachers' perceptions both through the case studies and surveys, as well as milestone report data.

The term 'outcomes' encompasses more than academic achievement outcomes. It also includes social outcomes such as an enhanced sense of personal identity, improved self-esteem, and more positive attitudes towards learning as well as improvement in interactions with both fellow students and with teachers (Timperley et al., 2007; Guskey, 2002; Thomson, 2008a and 2009). Outcomes can also include increased student engagement, development of learner independence and acquisition of language learning strategies. The latter is of significance since the vision articulated in the LL area includes (alongside learning to communicate in an additional language and exploring different world views) development of students' 'capacity to learn further languages' (Ministry of Education, 2007a, p. 24).

The following sections report on the impact of TPDL on student learning, achievement in learning languages and on student motivation. They describe teachers' views on what they believe the majority of their students know and can do in the language, and how they are assessing and reporting progress made by their students. Pertinent at this point is an awareness of the distinction between learning and achievement. Some teachers in the study appeared to lack an awareness of this distinction. Poskitt and Taylor argue that the nuanced difference between the two is important.

In simple terms, learning is the process or experience of gaining knowledge or skill. It can be likened to the journey towards a destination…. By contrast, achievement refers more to the successful completion of something (especially by means of exertion, skill, practice or perseverance). It can be likened to arrival at a destination (2008, p. 14).

Moreover, any discussion of student output should be considered in the light of the classroom context; the primary and intermediate school context (Years 1-8) is significantly different to that of the secondary context (Years 9-13). One key difference is the amount of time dedicated to language teaching and learning; although there are variations across schools, researchers noted primary and intermediate schools generally timetabling one period weekly of 45 minutes (and this could be for one, two or more terms), while secondary schools timetabled three periods of 50 minutes per week or thereabouts for the year. The length of time students have studied the TL is another significant contextual factor. The LL area has eight levels of achievement with students who have no prior knowledge of the language, beginning at Level 1. 'Curriculum level 1 is the entry level, regardless of school year' (Ministry of Education, 2007c). Students learning languages at primary and intermediate schools are likely to be at Level 1 as could be those students in Year 9 who are learning the language for the first time. Obviously the proficiency descriptor for students at Levels 1-2 and corresponding expectations of language knowledge and cultural knowledge will be considerably lower than expectations of students who have studied the TL for several years and who are at Levels 3-4 or higher. These differences have implications for teachers and students. Students with more learning time over the year are likely to make better progress and realise higher levels of achievement than those who are taught for just one period a week for two terms.

Perceived impacts of TPDL on student learning

Initially in this section, teachers' perceptions of overall student learning are discussed and then this is followed by comment on teachers' perceptions of their students' learning of specific skills and knowledge.  Survey data showed that teachers' expectations of students' learning were higher towards the beginning of the TPDL programme than at the end of the year when TPDL was completed.  As shown in Figure 17, in comparing the responses of teachers who responded to both the first and final surveys, the majority of teachers (21 of 24 respondents) in the first survey believed that their TPDL learning would have 'considerable' or 'high' impact on the majority of students' learning. However, the final survey revealed that only 14 of 24 respondents saw their learning as having 'considerable' or 'high' impact. Corresponding with this decrease was an increase in the number of teachers who thought the programme would have only 'some' impact on their students' learning.

Figure 17 : Teachers' perceptions of impact of teachers' TPDL learning on student learning

Image of Figure 17 :
  Teachers' perceptions of impact of teachers' TPDL learning on student
  learning.

There are a number of possible reasons for this difference between initial high expectations and lower perceptions at the end of the course. As indicated in Chapter four, initially survey respondents when asked to indicate how they expected the programme to change their practice, anticipated that the main impact would be increased TL proficiency. Subsequent surveys showed a shift in teachers' perceptions with an increasing focus on the pedagogical aspects of the classroom and their developing classroom skills (see 4.2.1). However, although teachers in the survey reported less optimism about the effects of TPDL on overall student learning, other sources indicated a more positive picture. Milestone reports, in particular the observations by the TPDL facilitators, and the case study observations undertaken by the researchers, suggested improved student outcomes across several areas. The following sections describe these in more detail.

Perceived student use of the TL

TPDL's stated aim is to develop teacher language proficiency and second language teaching capabilities in order to improve student language learning outcomes in Years 7-10 (Thomson, 2009, p. 1). The facilitator visits had a strong focus on teacher and student use of the TL. This data is 'discussed with teachers in learning conversations focused on maximising student learning and outcomes' (Thomson, 2009, p. 5). Facilitators who observed lessons in the fifty TPDL teachers' classrooms recorded their observations using three progress standards which are considered critical components of effective language teaching and learning: TL input, student (oral), TL output and meaningful (oral) communication (Thomson, 2009, p. 14). As noted in Chapter four, the expected and accelerated progress standards under each of the three categories is raised for each visit. Of particular relevance to this chapter is student TL usage; almost 20% of classes in visit four demonstrated accelerated progress in student TL usage while 70% of classes in visits three and four showed expected progress (Thomson, 2009, Appendix 26). Similarly, the final milestone report on 2007 (Milestone Report Four) reports that in 30 of 31 classes (Years 7-8), student utterances in the TL had qualitatively and quantitatively improved through the year (Thomson, 2008a, p. 10).

Perceptions of students' ability to understand and use familiar expressions and engage in interactive tasks

Survey questions asked about teachers' perceptions of their students' abilities to use common expressions and everyday language and engage in interactive tasks, to make connections between their own language and the TL and to recognise that the TL is organised in particular ways. Most survey respondents answered questions about levels one and two, with only six teachers in survey one and four teachers in survey three responding to those questions relevant to levels three and four. Several of those teachers also answered the questions related to levels one and two indicating they taught across all four levels.

One of Ellis' principles for effective instructional practice is that teachers need to ensure that learners develop 'a rich repertoire of formulaic expressions' (2005a, p. 33) since these play a key role in language acquisition particularly at the early stages. In looking at the core group of teachers who answered both survey one and the final survey, the number of teachers who perceived that their students' ability to understand and use familiar expressions and everyday vocabulary to a 'considerable' extent increased from one teacher (out of 23 respondents) in the first survey to almost a third of teachers (seven of 22 respondents) in the final survey. There was a corresponding decrease in the number of teachers who thought their students could understand and use familiar expressions and everyday vocabulary 'a little', that is 10 of 23 respondents in survey one to four of 22 respondents in survey three. The number of teachers, who believed the majority of their students could understand and use familiar expressions and everyday vocabulary to 'some extent', remained the same.

Figure 18: Teachers' perception of students' ability to understand and use familiar expressions and everyday vocabulary

Image of Figure 18: Teachers' perception of students' ability to
  understand and use familiar expressions and everyday vocabulary.

In early case study observations, researchers noted the use of formulaic language by students mostly in predictable situations such as roll call and greetings. In some instances students were able to use formulaic language effectively in freer practice when well supported (e.g. with reference to cue cards). A number of classrooms also displayed formulaic chunks in the TL on large cards and posters so that students could refer to them and students were observed doing this. By the final observation, researchers noted some students using formulaic language both more often and more confidently in predictable situations such as if they were late, and in free language use for example, creating a role play.

Resonating with the above are milestone reports for 2006, 2007 and 2008 which reported increases in student TL production over the respective academic years. Milestone Report Nine (for 2008) reports baseline data (obtained in visit one) showing that in the majority of classes, either no formulaic language was heard or it was restricted to greetings. By visit four, in all but five out of 47 classes (i.e. 90%) student utterances demonstrated 'substantial improvement in quality and sophistication' and in almost 20% of classes, student TL use dominated and showed 'a rich repertoire of formulaic expressions' (Thomson, 2009, p. 12).

Alongside the increase in the use of familiar expressions and everyday vocabulary, survey findings (the core group of teachers who answered both survey one and survey three) demonstrated that the proportion of teachers who perceived the majority of their students as able to interact in a simple way in supported situations to 'a little' extent decreased from more than a third (nine of 23 respondents) in the first survey to four of 22 respondents in the final survey. Correspondingly the number of teachers who viewed their students as being able to interact in a simple way to 'some' or a 'considerable' extent showed a small increase over the three surveys from just under half (13 of 23 respondents) of teachers to more than four fifths (18 of 22 respondents) in the final survey.

Figure 19: Extent to which the majority of students in teachers' classes can interact in a supported way using the TL

Image
  of Figure 19: Extent to which the majority of students in teachers'
  classes can interact in a supported way using the TL.

This critical role of interaction is recognised in the new curriculum:

Interaction in a new language, whether face to face or technologically facilitated, introduces (students) to new ways of thinking about, questioning, and interpreting the world and their place in it. Through such interaction, students acquire knowledge, skills, and attitudes that equip them for living in a world of diverse peoples, languages, and cultures (Ministry of Education, 2007a, p.24).

But as Ellis points out, 'creating the right kind of interaction for acquisition constitutes a major challenge for teachers' (2005a, p. 41). As noted in Chapter four, case study observations showed teachers setting up group activities with varying degrees of success. This resonates with the baseline data obtained from the first visit to teachers' classrooms by TPDL facilitators. Observations 'showed no group or pair work and no opportunities for meaningful interaction in two thirds of classes' (Thomson, 2009, p. 12). By visit four, facilitators reported that in 80% of classes, students were using the TL to negotiate meaning and in 37% of these classes, there was evidence of students taking ownership of interactions (Thomson, 2009, p. 2). Further data provided in the final milestone report for 2008 shows that close to 50% (n=24) of teachers visited were at the 'expected' level as regards providing opportunities for student interaction and 30% at the 'accelerated' level (Thomson, 2009, Appendix 26). In the final survey, most teacher respondents (75% of 24 respondents) indicated that the majority of their learners were able to complete TL interactive tasks set up in the classroom while 21% (five of 24 respondents) of teachers believed the majority of their students could do this 'to some extent'. Case study observations showed some excellent examples of student interaction but unrealised opportunities were also noted. As stated in Chapter four, there is a need for increased scaffolding, clear and staged instructions and monitoring of activities in order to maximise the potential of interactions for effective practice.

Perceived opportunities for TL output and levels of student engagement (evidenced in classroom observations)

In the case study observations researchers recorded a variety of opportunities for output and levels of student engagement. Examples were seen of case study teachers encouraging TL output from restricted practice activities to extended TL interactions, with the former appearing to be more prevalent. The extent of student TL output and level of engagement observed by the researchers in a selection of activities is indicated in Table 9. High levels of student engagement were observed in restricted TL practice activities such as songs and acting out a play (script in TL provided) and in freer language activities, specifically games. However researchers also noted that in some free practice activities, teachers' intentions for students' TL output were not always fully maximised (see also 4.2.2). It is useful to briefly comment on the value of 'games'; these are activities with a strong language focus where the TL is used purposefully and authentically. Reasons for their success in engaging students and providing opportunities for authentic language use include the competitive element, the rules or structure of the game which are familiar because students may have played similar games in their own languages as well as the opportunity to use repeated formulaic expressions in realistic contexts, for example, 'it's your turn'. 

Table 10 : Examples of opportunities for student TL output observed by researchers
Activity observed Extent of student TL output Level of student engagement
Class Level:  5 to 8
Information Gap
(Groups)
Semi-restricted language practice: use of formulaic language to share likes and dislikes (food) for about 10 minutes. Students totally engaged while teacher working with their group. Some other groups also engaged but about 40% of class off task at some point (needed more scaffolding in order to undertake the activity).
Songs
(Whole Class)
Restricted language practice: repetition with actions in some cases; 15 minutes of extended language practice. High level of engagement; all students observed to be on-task.
Role Play
(Groups)
Free language practice: use of formulaic language for five minutes in groups using known and new formulaic language and then presentation of role play to class. Most groups engaged and on-task but several were not able to produce the new phrase, relying instead on limited number of known phrases (needed opportunity to listen and repeat/choral repetition to build fluency).
Card Game
(Groups)
Free language use: use of formulaic language for extended period ( 20 minutes). Students provided with phrase cards as support. High level of engagement by all students; students were forced to use the TL phrases to play the game.
Dialogue Writing
(Pairs)
Free language use based on a dialogue students had listened to earlier. Some pairs used TL effectively in their dialogues while other pairs had difficulty and did not seem to have sufficient TL to undertake the task. Some off-task behaviour noted (needed more scaffolding).
Crossword
(Individually)
Restricted language practice: students work individually to complete the crossword using familiar vocabulary; students could also access words from wall charts. Some students highly engaged; others unsure what to do (needed further instructions in order to complete the task).
Class Level: 9
Writing article for newsletter in TL
(groups)
Free language use: students access all known vocabulary and grammar as well as TL sources on internet and in books to write the article in TL (topic of their choice). Engaged for 35 minutes. High level of engagement. Some groups wrote in TL while others wrote in English and then translated into TL. Teacher worked individually with groups and used TL almost exclusively. Students talked with teacher about TL in English.
Vocabulary Building
(Individually)
Restricted language practice; students needed to find vocabulary related to current topic and record; gave teacher one word answers. Students on task and generally engaged but limited output; no reason to communicate or negotiate meaning.
Ordering sentences on cards making up a story; then read aloud (pairs) Restricted language practice for 15 minutes. Students highly engaged and on-task reading TL sentences aloud.
Acting out a Play
(groups)
Restricted language practice for 20 minutes.
  Opportunity to focus on pronunciation and fluency.
High level of engagement.


Perceptions of other student learning outcomes and experiences

The impact of TPDL can also be seen in teachers' perceptions of other student learning (other than language learning) that occurred as well as the impact on student learning experiences. The researchers noted that in some case study classes, teachers were encouraging the enhancement of learner independence and a culture of mutual learning and support. In one class for example, students were working in groups playing different language games. Once they had mastered the game, they then taught the rest of the class their game in the TL. In another class, learners led class activities in the TL (e.g. calling out colours for a game of colour snap). Students were also observed working in groups and undertaking mutual error correction on their group poster. At the end of one of the classes, students gave class and individual feedback on how well they had been able to do the tasks, with teacher input in the form of suggestions as to how students could improve their learning.

An analysis of impacts on student learning should also include the acquisition and use of language learning strategies since, as discussed in the introduction to this chapter, language learning strategies enhance acquisition and, being transferable, will have a positive effect on the acquisition of further languages. Chamot defines learning strategies as 'the thoughts and actions that individuals use to accomplish a learning goal' (2004, p. 1).  As Table 11 shows, the most frequently used strategies observed by the researchers were recording new vocabulary in exercise books, and using other sources of TL information (e.g. wall charts, text books, sheets in exercise books and dictionaries).

Table 11: Language learning strategies used by students across three observations
Strategies Observed Instances
Across 3
Observations
Recording new vocabulary in exercise books. 14
Using other sources of TL information e.g. wall charts, text books, sheets in exercise books. 9
Using dictionaries (including picture dictionaries), and internet translation software such as Language Tools (via Interactive Whiteboard) to find TL words. 9
Reflecting on learning e.g. discussing how much TL was used and what they needed to know to play the game. 3
Listening to teacher explain the purpose of activities. 2
Guessing words using context. 2
Learning vocabulary using laminated word cards. 1
Using pictures to help remember words. 1


Teachers were observed explicitly teaching strategies, for example encouraging learners to find vocabulary in dictionaries or on the internet, and teaching vocabulary learning strategies. In interview three, teachers also mentioned teaching students to use mnemonics and associations to learn vocabulary, making links between the shape of characters (for Chinese or Japanese) and meaning, discussions with students of ways to learn and the purpose of activities, looking for patterns, making links across other school subjects and looking for common words in English and the TL.

Finally, a further and very important outcome is the development by students of the Key Competencies (Ministry of Education, 2007a). Key Competencies are defined as 'the capabilities people have, and need to develop, to live and learn today and in the future' (Ministry of Education, 2007a, p.12). Teachers are expected to ensure that the Key Competencies, viewed as the 'key to learning' in every learning area,  develop alongside language and intercultural skills. According to Crabbe, second language learning has a 'unique contribution to make to the development of the generic key competencies' (2005, p. 9). One Key Competency of particular significance to language learning and the development of intercultural competence is Managing Self. Involving far more than just organising oneself, this Key Competency is linked with the concept of identity. As students 'move between, and respond to, different languages and different cultural practices, they are challenged to consider their own identities and assumptions' (Ministry of Education, 2007a, p.24). Additionally this Key Competency involves students having the opportunity to enhance their sense of personal identity, 'seeing themselves as capable learners', as being able to undertake self-assessment, set learning goals, make plans and know how to act independently (Ministry of Education, 2007a, p.16).

The table below provides examples from case study observations where students applied Key Competencies. As can be seen in regards to Managing Self, although students were observed taking some responsibility for managing their learning, there was little evidence of developing intercultural competence.

Table 12: Examples of students in case study classes developing Key Competencies
Key Competency: Thinking Year(s)
Examples of activities where students were observed developing key competencies:
  • Answering questions relating to causes of differences between
    NZ and TL country's climate. Year 9
  • Asking questions about the language and culture. Year 9
  • Making connections between own and target culture. Year 9
  • Thinking about structure of the TL and making own corrections. Year 9
  • Guessing meaning of new phrases and words. Year 7
Key Competency: Using Language, Symbols, and Texts Year(s)  
Examples of activities where students were observed developing key competencies:
  • Reading texts and identification of words related to different seasons. Year 9
  • Using language orally and in written form. Year 7
  • Use of ICT to access information. Years 8 & 9
Key Competency: Managing Self Year(s)  
Examples of activities where students were observed developing key competencies:
  • Working to time limits to complete activities. Year 9
  • Finding opportunities to learn by locating vocabulary in dictionaries
    and internet resources. Years 7 & 9
  • Able to work effectively in groups. Years 7, 8 & 9
  • Organising own written work. Years 7, 8 & 9
Key Competency: Relating to Others Year(s)
Examples of activities where students were observed developing key competencies:
  • Group work: listening to each other contributions. Years 9 & 7
  • Encouraging each other to present. Year 7
  • Playing game co-operatively Years 7, 8 & 9
Key Competency: Participating and Contributing Year(s)
Examples of activities where students were observed developing key competencies:
  • Building up class knowledge of climate and creating a group poster of the
    four seasons. Year 9
  • Allocating parts and reading a play aloud in groups. Year 9
  • Presenting a role play to the class. Year 9
  • Producing (in groups) articles for a Class Newsletter (for parents) in TL
    on topics of their choice. Years 7, 8 & 9


In terms of the perceived impact on student language learning experiences, case study teachers reported changes in students' responses to their teaching. Final interviews with case study participants revealed that five of the seven teachers had noticed very positive changes in the students' responses. As shown in Table 12, these changes were related to their students being more positive, confident, motivated and engaged. The remaining two teachers recognised their own teaching had changed (e.g. they reported more recycling of language and more focus on oral production) but had not noticed a change in students' responses. One teacher, as noted in chapter four, is a first year teacher who admitted difficulty in noticing changes. She noted, 'I can't really compare because this is my first year. Some of the class are really interested and others aren't but this is the same in other subjects'.

Table 13: Case study teachers' perceptions of changes in their students' responses
Case study teachers' perceptions of changes in students' responses (Interview three)
Increased student TL output (x 5)
  • Greater uptake in speaking TL.
  • More use of language outside the classroom – they see the practical applications of the language.
  • Increased TL usage but need more time to practise.
  • More independent learning, for example, one student found an idiomatic phrase for 'you're crazy' and tries to use it in various situations.
  • Use of the TL has grown exponentially especially with this particular topic of food.
Increased engagement /enjoyment (x4)
  • Students can see the teacher is enjoying teaching so will buy in if they notice you have fun and are enjoying teaching.
  • More engaged compared to the beginning of the year.
  • Students are always very enthusiastic and have responded well.
  • Students always pleased and eager for more.
Increased confidence (x2)
  • Increased confidence to give it a go, especially the boys.
  • Students not scared to speak.
No noticeable change (x2)
  • Don't think the students would notice any difference.


Earlier interview responses from case study teachers included changes such as students having more fun in their language learning, more student involvement with the language, greater use of formulaic expressions, students being more motivated and becoming more independent as language learners. One teacher, for example, reported that students were getting TL books out of the library while another talked of students not wanting to give up their TL lessons for other timetable subjects.

As discussed in chapter four, all case study teachers remarked on positive changes in their language teaching. In terms of teachers' perception of their own effectiveness at teaching the TL, a considerable increase was reported (see 4.2.1) over the period of the programme. That this has impacted on student learning is indicated in the comment by one teacher who noted increases in student engagement, student output and motivation:

It has completely changed the way I teach my language class ... I am also using task based activities as much as possible which has really engaged the students. Finally, I am letting go of the 'direct teaching' in class by providing more student   directed activities which has shown an increase in student production of the TL instead of me talking all the time. This programme has changed my whole approach which has made the experience exciting for me and my students.

Perceived impact of TPDL on students' learning: Cultural Knowledge strand

As discussed in chapter five, making connections between students' own cultures and the target culture is a new component in The New Zealand Curriculum (2007). The Cultural Knowledge strand talks of students learning about culture and the interrelationship between culture and language.

They grow in confidence as they learn to recognise different elements of the belief systems of speakers of the target language. They become increasingly aware of the ways in whichthese systems are expressed through language and cultural practices. As they compare and contrast different beliefs and cultural practices, including their own, they understand more about themselves and become more understanding of others (Ministry of Education, 2007c).

It appears from observations and interviews that for many teachers this is proving to be a challenging shift. In terms of students' ability to recognise that the target culture is organised in particular ways, survey one showed almost half of teacher respondents (12 of 23 core respondents) perceived that the students in their classes had 'no' or 'a little' recognition that the target culture is organised in particular ways. However, there was an increase from survey one to the final survey, with almost three quarters of teachers (15 of 21) in the final survey viewing their students as having this recognition to 'some extent' or to a 'considerable extent' and a corresponding decrease to six respondents perceiving the majority of their students having 'no' or 'a little' recognition.

Figure 20: Teachers' perceptions of recognition by most students of organisation of target culture

Image
  of Figure 20: Teachers' perceptions of recognition by most students of
  organisation of target culture.

Similarly, the proportion of teachers, whose students were able to make connections with the target culture to 'some extent' and 'considerably' increased. In the observations of case study teachers, researchers noted teachers making connections such as linking the use of waiata to tell stories and songs in the TL and inviting native speakers to the classroom. The statement from one teacher reported in an interview is an example of linking cultural knowledge with language teaching:

When we were doing modal verbs, I used the context of youth culture i.e. what you are allowed to do and not do in (country) as a teenager e.g. you can go clubbing as a 16 year old in (country) but you have to be home by 12.00. We made comparisons with NZ… so activities revolved around how you would say that in TL.

However, although researchers observed some effective opportunities for students to make links between their own cultures and the target culture, case study observations also noted missed opportunities for students to make explicit connections. One such example was when students watched a video which showed TL speakers greeting each other. Discussion of what is appropriate in greeting people in the TL culture accompanied by comparisons with students' known cultures would have maximised the opportunity for making cross-cultural links.

Overall, students did not appear to be getting many opportunities to make cultural connections. This may be for two reasons: firstly because as noted previously, teachers do not yet seem to have a robust understanding of the new curriculum as regards the emphasis on cultural knowledge and the movement to intercultural competence. Moreover, TPDL in 2008 did not appear to have much of a focus on this other than its inclusion in the Evidence of Principles and Strategies form used by facilitators when observing the classes (Thomson, 2009, Appendix 15). In Milestone Report Nine, Thomson (2009, p. 16) mentioned cultural knowledge but it is not obvious that 'making connections' or 'intercultural competence' were integrated into the programme. Secondly, it is important that teachers themselves are interculturally competent. As Byram points out, 'Language teaching with an intercultural competence dimension presupposes that teachers themselves will have acquired intercultural communicative competence to a reasonable level' (2008, p. 83). A stronger focus on this competency in TPDL and accompanied by opportunities to participate in immersion experiences would be beneficial to improving opportunities and outcomes for students.

Perceived impact of TPDL on student achievement

The perceived impact of TPDL participants' learning on student achievement is a difficult area to comment on given the data available (limited to teachers' perceptions). In terms of the perceived impact of TPDL on student achievement, there is a decrease between teachers' expectations in the first survey and their perceptions in the final survey. When comparing responses for the core 23 teachers who answered both survey one and the final survey, around three quarters (18 of 23 teacher respondents) believed their TPDL learning would have 'considerable' or 'high' impact on student achievement. The final survey showed a decrease with almost half of the teachers in the third survey (12 of 23 respondents) seeing the TPDL learning as having 'considerable' or 'high' impact and close to half indicating only 'some' impact.

Figure 21: Teachers' perceptions of the impact of TPDL learning on student achievement

Image of
  Figure 21: Teachers' perceptions of the impact of TPDL learning on
  student achievement.

The final survey also asked teachers for examples of the impact of the programme on their students' achievement. Twenty two teachers responded and of these, the majority (18) listed increased student use of the TL and/or increased TL output quality.

  • summarises their responses. Note that some teachers recorded several examples.
    Table 14: Teachers' examples of the impact of the programme on their students' achievement
    ImpactTeachers Who Listed Impact
    Increased use of TL by students increased quality of TL output18 Teachers
    • Students are greeting and chatting in L2 in the playground.
    • Students show interest in extending their L2 knowledge for their own day to day use.
    • Increased fluency and knowledge of TL (formulaic expressions).
    • Greater use of formulaic expressions has helped the more diffident to communicate orally.
    • They have moved from single words to formulaic chunks to full sentences and multi-clause sentences.
    • More language heard in class formally and informally.
    • Students are able to write letters about themselves and their families.
    • Increased willingness to use the TL.
    • Increased use of task-based learning. Suits boys particularly.
    • Communicative skills have improved with more task-based learning.
    • Due to students knowing exactly what and why they are doing things, they produce better results.
    Students' Enjoyment, Enthusiasm and Motivation 8 Teachers
    • Students are asking for extension activities.
    • Higher achieving students are more motivated to learn.
    • Insistence on use of TL is self-perpetuating motivationally.
    • Retention appears to be much higher in this group for 2009.
    Confidence2 Teachers
    • More confident and enthusiastic about speaking.
    Recall1 Teacher
    • Students are recalling more language.

    Assessment of student progress in learning languages and what accounts for these practices

    Hattie and Timperley argue that the importance of assessment to successful learning and teaching cannot be underestimated:

    Effective teaching not only involves imparting information and understandings to students (or providing constructive tasks, environments, and learning) but also involves assessing and evaluating students' understanding of this information, so that the next teaching act can be matched to the present understanding of the students. (2007, p. 88).

    In this section, types of assessment and means of reporting are discussed.  It should be noted that there is variability across primary, intermediate and secondary schools as regards recording achievement in the TL. This varies from no requirement to the regular recording of achievement. This is perhaps not surprising given the different school contexts as well as individual school requirements.

    Survey data shows that both formal and informal assessments of students are undertaken by teachers. The surveys indicated that formal assessment of students' progress was achieved through a variety of means: speaking tests were used most frequently followed closely by listening, vocabulary and writing tests. Grammar tests were the least used means of assessing students' progress. From the interviews with case study teachers, it appears the tests were generally written by the teachers and based on classroom learning. 

    Figure 22: Means by which teachers assess students' progress formally (Survey 2)   (Note: Teachers responded to more than one category)
    Image of
  Figure 22: Means by which teachers assess students' progress formally
  (Survey 2)   (Note: Teachers responded to more than
  one category).

    Interview data showed teachers of classes at levels one and two used more informal assessment, for example 'through conversation as I monitor the class and through checking of exercise books'. In the observations of case study teachers, all demonstrated considerable use of informal feedback given in the TL to individuals and to groups/class as a whole. As indicated in Table 15, case study teachers had a range of responses when asked about their assessment practices, from no formal assessment (only informal oral assessments during the lesson) through to one-to-one conferencing with students (Year 7) and regular formative assessments with summative unit tests set by the department (Year 9). In one class, students were asked to self-assess at the end of the unit (Year 7). Another teacher gave an end of year exam assessing listening, reading and writing as well as assessing oral language where students used digital voice recorders (Year 9).

    Milestone Report Nine reported evidence of an 'increased awareness of the importance of formative assessment' (Thomson, 2009, p. 17). Of the 21 responses to the post-test survey (undertaken as part of EDPROFST360), all teachers undertook some kind of student assessment but this was largely of oral production with less than a quarter undertaking some kind of written test. The data showed evidence of a shift in formative assessment with about 75% in the post-test survey reporting monitoring individual students as they worked in pairs/groups (as opposed to teachers asking individual students for an answer to a question they posed (Thomson, 2009, p. 18). Given the importance of assessment as (along with quality teaching) 'at the heart of increased student engagement, learning and achievement' (Poskitt & Taylor, 2008, p. 10), TPDL could benefit from an enhanced focus in this area.

    A majority of teachers (22 of 27 respondents in the second survey and 15 of 23 respondents in the final survey) recorded the results of formal tests twice a term. Although three of 27 teacher respondents in the second survey were not keeping any records of formal testing, this had decreased to just one teacher by the time of the final survey. This is reflected in Milestone Report Nine which reports anecdotally that Years 9-10 teachers have a greater awareness of the importance of diagnostic testing while Years 7-8 teachers have increased awareness of 'the importance of reporting on student achievement in the TL and the inclusion of TL language work in students' portfolios' (Thomson, 2009, p. 17).

    For 14 of 24 teacher respondents in the final survey the results were made known to parents and 10 teachers submitted test results to the Head of Department (HOD) or Principal. No teacher reported submitting results of formal tests to the following year TL class teacher which potentially has implications for student transition to higher class levels. Table 15 shows how case study teachers were assessing students' progress and also indicates how the progress was reported. Teachers of students at Years 5-8 tended to have no requirement to report student progress while teachers of students in Years 9-10 upwards had more formal means of reporting progress. The table indicates a range of reporting methods, from no formal record, through to regular recording in either mark book or electronic database. One teacher kept a duplicate copy receipt book and after one-to-one conferencing with a student she gave them a copy of her comments. Two Years 7-8 teachers took advantage of school portfolios to include results of TL tests and or school work. In some cases assessment records went into school reports. One HOD used results in a Board of Trustees report and several teachers commented on the use of assessment records to inform further planning. One teacher intended for her students to take a record of their learning to their language class the following year.

    Table 15: How case study participants are assessing and reporting on progress of learners
    Years 5-8
    How students' progress is being assessed by case study teachers (From interview two)
    • No formal assessment (x2). 
    • Informal oral assessment – Assessing as teacher monitors the class (x2). 
    • Checks students' books and gives feedback (x2). 
    • Other:  
      • 1:1 conferencing on a weekly basis (School policy) where the language is assessed.
      • Prepares pre-tests for new units of work.
      • Self assessment by students at the end of units.
    • In Action Research project, teacher noted students' questions and answers using formulaic language and gave individualised feedback.
    Method of reporting progress by case study teachers (From Interview Three)
    • No formal requirement to report at the school (x2).
    • Has recorded comments (positive for all children).
    • Plans to report informally at end of year learners' 'fete' and students may receive a certificate at the end of the year.
    • Parents may get feedback at the 3-way conferencing.
    • End of year report will have comments on what students can do in the TL.
    • Comments in intermediate general school report for able TL learners only
    • Next year, teacher plans to report on all students in the school report.
    • School prize for TL student.
    • Results of assessment (e.g. asking students for colours at the weekly 1:1 conferencing) written in duplicate receipt book and gives a copy to students to show them what they know.
    • Database to track raw data from pre- and post-tests.
    • Portfolio of all school work including language learning (x2).
    • Written TL samples included in student portfolios for Secondary school.
    • Would like to extend her Action Research project so that she can give feedback to parents on student output.
    Year 9
    How students' progress is being assessed by case study teachers (From interview two)
    • Assesses all four skills with this level.
    • Gives unit tests and mini-orals, with students using digital voice recorders saving files on the computer.
    • The end of year exam assesses listening, reading and writing.
    • Formative tests all the time (e.g. vocabulary tests).
    • Summative unit tests (departmental tests).
    Method of reporting progress by case study teachers (From Interview Three)
    • Exam results in form of grades are entered on school database and reported in school report. Includes detailed suggestions to parents on ways they can support their children in language learning.
    • Records students' progress in electronic mark book and uses this to give feedback and feed-forward to students.
    • Reports to parents both in Progress Report and in end of year report with grades and comments.
    • Also reports to HOD who writes reports for School Board.
    • Reports at Parent/teacher interviews with Year 10 students and makes recommendations regarding students continuing with TL study.

    Perceived impact of TPDL learning on student motivation

    As discussed in chapter three, motivation is known to have considerable impact on second language acquisition (Dornyei, 2000). In this section, teachers' perceptions of the impact of TPDL on student motivation is explored first through the eyes of the TPDL teachers and then in light of researchers' observations of case study teachers and interviews with them. Survey respondents perceived that nearly all of their students were 'motivated' or 'very motivated' to learn the TL.

    Figure 23: Teachers' perceptions of the extent of student motivation to learn TL
    Image of Figure 23:
  Teachers' perceptions of the extent of student motivation to learn TL.

    A level of student motivation was also confirmed by the researchers' observations and interviews. Observations revealed varied examples of students being enthusiastic and engaged in a number of activities, requesting more class time on TL learning, and asking questions about the language and culture.

    Two thirds of teachers (limited to the core group who answered both the first and final surveys) in the first survey (17 of 24 respondents) expected that their learning would have 'considerable' or 'high' impact on the majority of students' motivation while six respondents believed it would have 'some' impact (see Figure 24). In contrast, in the second and final surveys, this was reversed. About one third of teachers in the final survey (nine of 24 respondents) indicated 'considerable' or 'high' impact and almost two thirds (15 of 24 respondents) viewed the programme as having 'some' impact on their students' motivation. This downward trend resonates with those findings discussed earlier in the chapter where teachers' initial expectations exceeded what they perceived as actually occurring at the time of the second and final surveys. Interestingly when a cross tabulation was done, there was a positive relationship between teachers' level of motivation to learn the TL and their perceptions of their students'  motivation to learn the language.

    Figure 24: Teachers' perceptions of the impact of TPDL programme on students' motivation
    Image of Figure 24:
  Teachers' perceptions of the impact of TPDL programme on students'
  motivation.

    Student motivation and student contact with TL and culture

    A more recently researched extension to motivation is 'willingness to communicate' (WTC) which also plays a significant role in second language use and hence acquisition. Dörnyei argues that generating a willingness to communicate is 'a central, if not the most central objective of modern L2 pedagogy' (2001, p. 51).  Figure 25 reveals that the majority of teachers (21 of 24 respondents) who responded to the final survey perceived that the majority of their students were 'willing' or 'very willing' to communicate in the TL. Researchers' observations also showed that students were willing to communicate i.e. to use the TL with each other and with their teachers.

    Figure 25: Teachers' perceptions of students' willingness to communicate
    Image of Figure 25:
  Teachers' perceptions of students' willingness to communicate.

    Motivation and willingness to communicate can be enhanced through contact with the TL and culture (Csizér & Kormos, 2008). It is unclear if the TPDL programme explicitly explores how student motivation can be enhanced, however given its importance in language learning, the programme could well benefit from a heightened focus on motivation and WTC. Survey data, observations and interviews with teachers indicated that their students have considerably more opportunities for a number of indirect TL cultural contacts mainly in the form of the internet, books, movies/DVDs and magazines than opportunities for direct contact with the TL community. In terms of direct spoken contact, less than half of the teachers surveyed (24 respondents), reported opportunities for their students to have direct contact with TL expert users such as visitors from TL countries while very few respondents reported direct written TL contact. The final survey also indicated that 66% of teachers' classes had no access to a native speaker language assistant.

    Table 16: Opportunities provided by TPDL teachers for their students to participate with the TL community
    Indirect TL Contact Number of
    Opportunities
    Internet 18
    Books 14
    Movies/DVDs 14
    Magazines 13
    TV 4
    Direct spoken TL Contact
    In NZ with TL expert users 11
    Visit to TL country 5
    Direct written TL Contact
    Email 2
    Chat rooms in TL 0
    Post 0


    Classroom observations of the seven case study participants showed mainly indirect opportunities for students to have TL contact. Use of the Ministry-provided LLS, which provides multi-media materials designed for teachers and students who are new to language learning at Years 7 and 8 (Levels one and two achievement objectives in the curriculum) such as Sí! An Introduction to Spanish, were noted in a number of observations with teachers commenting on the usefulness of the accompanying videos, CDs and print materials. In regards to the Oui! An Introduction to French (DVD), one teacher commented 'it does the things that I can't do - it provides a springboard for further learning… also brings in cultural learning… it's powerful'.

    Other indirect opportunities observed included the use of songs in the TL from the Hello World website (Hello World, 2009) which provides TL games, activities and songs. One case study teacher reported in an interview that she used the internet to obtain authentic texts such as weather reports in the TL about the TL country, as well as international soccer results which were of interest to her class. She also took her students to TL films. Her students were observed accessing the internet themselves to locate information on the TL country's results in the Olympic Games. However in general, case study observations showed limited opportunities for students to participate directly with the TL community. As noted in chapter four, two case study teachers had access to native speaker language assistants. Interestingly both teachers were the most fluent of the case study teachers. Increased direct participation has the potential to augment the quality of language learning opportunities and experiences for students and to further increase their motivation to interact in meaningful ways. This is an area TPDL could usefully make a contribution through for example, promoting the application process for language assistants, and raising teacher awareness of the benefits of technology-facilitated communication.

    Conclusion

    TPDL has resulted in changed teaching practices which in turn appear to have led to enhanced language learning opportunities and outcomes for students. Although some survey findings demonstrated there were decreases between teachers' initial optimistic expectations of the impact of TPDL on the students' learning, achievement and motivation and their perceptions of what they believed occurred by the end of the teaching year, other survey results indicated positive findings. These included teachers' perceptions of increases in their students' abilities to understand and use familiar expressions, to interact in supported situations, to complete interactive tasks and to recognise that the TL and culture are organised in particular ways. Positive outcomes for students were recorded from classroom observations and interviews by the researchers and from milestone reports. Gains other than those directly related to language acquisition as listed above, included some development of learner independence, some development of Key Competencies and a modest acquisition of language learning strategies. Most case study participants noted positive changes in their students' responses to their teaching with the students being more positive, confident, motivated and engaged.

    Most opportunities to participate with the TL community were of an indirect nature through the internet, DVDs, books and magazines. Few opportunities for direct contact were noted. However teachers perceived their students to be willing to communicate in the TL where opportunities arose.

    In terms of assessment practices, findings revealed differences in the amount and type of assessment undertaken. Both formal and informal assessments of students were administered by teachers with teachers of students at Levels one and two undertaking more informal assessment. Formal assessment of students' progress was achieved mainly through speaking tests followed closely by listening and vocabulary tests. Recording of students' progress and achievement also showed considerable variation. For some teachers it was not a school requirement while others regularly recorded results of tests or took advantage of school portfolios to include results of TL tests and/or work. Despite teachers in the survey perceiving the impact of TPDL on student achievement was less than they had originally expected, they also indicated that their students had achieved in their language learning in several ways with an increase in the amount and quality of TL output being the most widely reported achievement.  Overall, the research provides evidence that the TPDL programme has impacted positively on student language learning opportunities and outcomes.

    Recommendations and implications for the TPDL programme
    • Given the importance attributed to motivation and willingness to communicate in TL use and acquisition, TPDL could benefit from an enhanced focus on these.
    • Contact with the TL and culture can result in increased motivation and willingness to communicate. Greater access to direct authentic interaction opportunities through language assistants, the use of technology, and as an outcome of LIAs for teachers through school to school links (with immersion country schools) would likely lead to increased student motivation, widely recognised as integral to successful second language acquisition.
    • Given the recognised importance of the role of assessment in learning and in achievement, its focus in TPDL should be re-assessed so that student learning and motivation can be maximised.
  • Where to find out more

    Contact Us

    Education Data Requests
    If you have any questions about education data then please contact us at:
    Email:      Requests EDK
    Phone:    +64 4 463 8065