Executive Summary

This study was commissioned by the New Zealand Ministry of Education (MoE) to investigate the literacy and language knowledge and teaching practices of mathematics and science teachers in Year 7, 9 and 11 classrooms. One reason such a study is important is because there is an apparent mismatch between theory and policy that emphasises the importance of subject-literacy instruction, and actual classroom practice. This mismatch is evident in New Zealand, for example, in a recent Education Review Office Report which describes literacy instruction in Years 9 and 10 as “somewhat bleak” (Education Review Office, 2012), as well as overseas, for example, in the seminal article “Why content literacy is difficult to infuse into the secondary school: complexities of curriculum, pedagogy, and school culture” (O’Brien, Stewart, & Moje, 1995). Despite these reports, there appears to have been little systematic collection and analysis of evidence of classroom-based literacy practices in the upper primary and secondary levels in New Zealand.

Four key research questions therefore guided the study:

1. What knowledge, beliefs and understandings do teachers have about the literacy and language of science and mathematics?
2. In light of their knowledge, beliefs, and understandings about the literacy and language required, what practices do teachers use in the teaching of science and mathematics to support students’ learning and achievement?
3. What are the areas of strengths and weaknesses in literacy and language pedagogy within science and mathematics?
4. What understandings do students have of the literacy and language of science and mathematics?

To answer these questions, we employed a mixed-methods approach to investigate the literacy knowledge and teaching practices of 12 teachers through classroom observations, interviews with teachers and students, and a measure of Subject Literacy Pedagogical Content Knowledge (SLPCK).

The following sections summarise: Previous research and theories that framed our study; our methods; findings; and some implications for policy and practice.

Literacy and language in science and mathematics: background

Our starting point for this study of literacy and language teaching in mathematics and science classrooms is the New Zealand Curriculum (NZC) (MOE, 2007) because this “[s]tatement of official policy” sets “the direction for student learning” in English-medium schools and therefore provides one important benchmark against which judgements about the observed literacy and language teaching might be made.

The NZC is unequivocal in its position that not only does each learning area have “its own language”, it is the responsibility of every subject-area teacher to develop the knowledge that students need to meet these specialised literacy and language demands (NZC, p. 16). As well as this general statement, subject-specific literacy and language demands are identified in the ‘Learning Area’ statements and achievement objectives for both science and mathematics and statistics. These demands include vocabulary knowledge and the ability to access and communicate ideas through reading and writing, as well as deeper demands of literacy and language, particularly applying knowledge from texts to real-world and novel situations and critically reading subject and popular texts. All in all, in the NZC, literacy and language learning and teaching in mathematics and science has an importance that “cannot be overstated” (NZC, p. 16).

The statements pertaining to subject literacy in the NZC are consistent with a large body of recent research in the field of adolescent literacy (Draper, 2008; Fang & Schleppegrell, 2010; Moje, Stockdill, Kim & Kim, 2011; Shanahan & Shanahan, 2008). One of the main tenets of this body of research is that students are faced with increasingly sophisticated and subject-specialised literacy demands as they progress through their schooling years. Shanahan and Shanahan (2008) have represented this visually in Figure 1.

Figure 1:  Shanahan & Shanahan’s (2008) model of the increased demand for specialization of literacy development


The common thread in this line of research is that the ‘disciplinary literacy’ skills needed in different content areas are “more sophisticated but less generalisable” (Shanahan & Shanahan, 2008, p. 45) than those needed in the earlier years of schooling. An important implication of this model of literacy development is that while literacy at the ‘basic’ and ‘intermediate’ levels (which is not used here to refer to intermediate schools) can be taught by a specialist literacy or English teacher, the kind of specialised-literacy knowledge presented in the top section of the triangle can only be taught by subject teachers.

So, what kinds of literacy knowledge and teaching practices are likely to contribute most to achieving these ambitious goals? Our collection and analysis of data for this study is framed around the following factors that have been identified as most catalytic for effective literacy instruction (Parr, McNaughton, Jesson, Wilson, Amituanai-Toloa & Oldehaver, 2011). Firstly, there is a need to know about the texts that students and teachers use: their length, purposes, features and modes. Importantly, we would also need to know what teachers and students do with these texts, whether they use them to find information, compare and contrast, critique or evaluate; and how frequently.

Secondly, we would want to know the sites for these practices, the sorts of activities, groupings and differentiation patterns that occur. Teacher instruction and support for the use of texts theoretically mediates text use (Parr et al., 2011). Therefore, inquiry into teachers’ literacy instructional strategies is implicated, as is investigation into teachers’ goals and knowledge for their lessons. Crucially, the outcomes from the students’ point of view warrant investigation: what students understand and take from instructional sequences.

Methods

Participants

We selected participants with a view to developing the field of subject-specific literacy. The setting of the study was 12 classrooms in three secondary and two intermediate schools in Auckland. The 12 classrooms comprised two mathematics and two science classes from each of Years 7, 9 and 11. The schools were mid to low decile with ethnically-diverse student populations.

The three secondary schools were invited to participate because our analysis showed they had high enrolment and pass rates in selected NCEA Level 1 achievement standards from mathematics and/or science. Secondary principals and heads of department then identified teachers in their school who they judged to be effective, as well as one feeder intermediate school.

There were two reasons for employing a teacher effectiveness criterion. Firstly, an important purpose of this study was to identify future directions for literacy and language pedagogy, and we reasoned that these would be most fruitfully built on a foundation of already effective teaching. More specifically, anecdotal evidence from Schooling Support Services Literacy Facilitators involved in the Secondary Literacy Project (in reports to the National Co-ordinator, and co-author of this report, Aaron Wilson) suggested that without such an effectiveness criterion the amount of literacy and language instruction we would observe might be very limited.

It is important to note that because of both the small number of participants, and the effectiveness criterion, we make no claims that these teachers are representative of other mathematics and science teachers. In order for educators to similarly investigate the subject-specific literacy teaching within their own settings we have developed a supporting tool (see Appendix G).

Measures

We relied on four sources of data: Teacher observations, Secondary Literacy Pedagogical Content Knowledge Tool, teacher interviews, and student interviews.

Teacher observations

As a measure of practice, we observed each teacher over three consecutive lessons using an observation template (Wilson, McNaughton, & Jesson, nd) designed to record instances of literacy teaching that occurred within three minute blocks. We recorded details about:

• texts used in the lesson, such as their source, word length and form
• teaching activities, such as whether the teacher was lecturing, modelling or conferencing
• how students were grouped and what forms of differentiation were observed
• the content of literacy instruction that was observed, for example whether it was focused on vocabulary, text structure, audience and purpose, language features or spelling and punctuation
• instructional depth, for example whether the literacy instruction was focused on item knowledge (eg, teaching a definition), activation of students’ prior knowledge (APK), providing opportunities to practise, developing students’ metacognition/strategy use, or critical literacy.

Observer inter-rater checks determined high reliability (> 90% inter-rater agreement).

Subject Literacy Pedagogical Content Knowledge (SLPCK)

As a measure of teacher knowledge we employed a SLPCK Tool (Wilson, nd), which asked teachers to identify aspects of  language and literacy in a subject text that might act as potential barriers for students’ reading, and to suggest teaching moves they could make in response. Completed tools were qualitatively assessed to identify themes within the responses.

Teacher interviews

We also interviewed teachers about their goals for lessons, and whether they had achieved those goals. Teachers were asked about the literacy goal of the preceding lesson, the methods the teacher had used to assist the students in achieving that goal, and what measures they used to understand whether they had achieved their goal.

Student interviews

Finally, we interviewed students about particular aspects of the lessons, and what they perceived the literacy focus to have been.

Summary of findings

Finding 1: Most classroom texts were short texts created by the teacher
Working from an assumption that texts should be at the heart of literacy instruction, we analysed the frequency with which texts were used and what the features of those texts were. We took a reasonably broad view of texts and included all texts with any written words, including instructions, diagrams with labels or headings and symbolic expressions that included at least one word, but did not count texts that had no words whatsoever and were therefore solely oral or visual.

We observed a total of 91 texts delivered to students to read overall. Overwhelmingly, the texts being used in classrooms were created by the teacher, often as notes or worksheets, or modelled examples on the whiteboard. Teacher created texts were more evident in mathematics (93% of cases, compared with 72% for science). The whiteboard was the most common way texts were delivered (82% of texts in mathematics and 40% of texts in science) while the second most common form of delivery was via photocopied text resource (11% in mathematics and 32% in science). Published print materials were seldom used in either subject (2% of texts in mathematics and 6% in science).

Texts were predominantly short, with fewer than 10 words. In mathematics classes, the majority of texts contained fewer than 10 words (64%) and these were often one or two word instructions (eg, “simplify” followed by a series of equations). In science classes the highest proportion of texts contained between 11–50 words (38%), followed by fewer than 10 (26%), 51–100 (16%), 101–300 (12%) and 301–600 (2%).

Unsurprisingly, in mathematics classes the highest proportions of texts were mainly number based (41%) although there was also a relatively high proportion of running written text (32%). These were often short-sentence, instruction-based text (eg, “calculate the angle”). There were few examples of the extended contextualised word-based problems typical of NCEA mathematics assessments.

There was very little use of any real world texts or electronic or internet text, and very little evidence of students learning to use texts in ways that are valued by the discipline (eg, reading or writing ‘like a scientist’) or working across multiple texts.


Finding 2: The teachers know and teach much more about subject specific vocabulary than they do about other aspects of literacy and language
We analysed each three-minute block to identify whether any literacy instruction occurred and what the focus of that instruction was. In mathematics classes, the total number of instances of literacy instruction was fewer (observed in 38% of blocks) than the total number of instances of literacy instruction in science classes (observed in 69% of blocks).

Vocabulary knowledge was clearly the dominant subject literacy concern of teachers and was observed in 26% of blocks in mathematics and 61% of blocks in science.

Interviews and SLPCK responses demonstrated that teachers considered knowledge of vocabulary as an important subject-specific literacy goal. In particular, knowledge of challenging or conceptually important subject-specific vocabulary was a strong theme that emerged from both interviews and the SLPCK Tool.

There was less evidence that teachers discriminated between receptive and productive vocabulary learning, and teachers did not always seem to distinguish ‘to understand’ subject vocabulary from ‘to use’ that vocabulary. There was also less evidence that teachers had strategies for identifying which words would be most productive or catalytic to teach, or were aware of everyday words that had specialised uses.

Instruction about text structure or audience and purpose was rarely observed, and no instance of instruction about language resources, such as expected level of writing or sentence structure, was observed at all.


Finding 3: Lessons were characterised by whole class question and answer (Q & A) sessions, followed by individual work where teachers rove and assist
The teachers in our study predominantly used whole-class sessions and engaged students in Q & A sequences. Students also spent a large amount of time working individually, with teachers roving and assisting through brief exchanges. While natural discussion did occur between students who were supposed to be working alone, there were comparatively few instances of students working collaboratively in groups, except in Year 7, and in particular Year 7 mathematics, where students worked in ability groups, and participated in group interaction with the teacher.

Overall, there were no instances of teachers and students participating in extended discussions about the content of the texts, let alone aspects of literacy and language, other than those requiring short answer responses by students.


Finding 4: Instruction in both subjects was largely undifferentiated, especially in Year 9 and 10 classrooms
Students in both subjects were undifferentiated for the majority of blocks observed, that is, they were working towards the same class objectives or learning intentions. Students in mathematics lessons spent 76% of observed blocks in undifferentiated activity in which all students were engaged in the same activity, and were working in ability-grouped activities for 24% of observed blocks, while students in science classes spent 98% of observed blocks in undifferentiated activity and only 2% of blocks working in ability-grouped activity.
We observed no instance of differentiation to the level of individual students. Students in Year 7 classes were engaged in ability-grouped activities for a higher proportion of time (35%) than Year 9 (3%) and Year 11 classes (0%).


Finding 5: Formative assessment mainly consisted of Q & A sessions or checking
Formative assessment is needed if teachers are to be able to effectively identify the learning needs, including the literacy and language needs, of their students and to monitor the effectiveness of their teaching in response to those needs. This is the vision of pedagogy articulated in the NZC through the ‘Teaching as Inquiry’ cycle (NZC, p. 35).

Teachers described their strategies for knowing whether students were achieving the teaching goals largely in terms of Q & A sessions, or roving to check whether students were getting the correct answers. Some conferencing was observed, largely at Year 11, and teachers at Year 7 identified ‘working with’ students and self-assessment techniques.

In general, the students in our study were able to identify the teacher’s goal for their learning, most often identifying words or concepts that the teacher wanted them to learn.

In the secondary context particularly, we saw few practices that might allow teachers or students to diagnose whether difficulties were literacy or content based, or to monitor or regulate literacy learning. No teacher at any level referred to a standardised measure of literacy achievement, such as asTTle reading or writing.


Finding 6: There was little focus on developing students’ critical literacy or strategy use
One of the aspects of literacy instruction that we analysed was what we called ‘instructional depth’. Critical literacy is an explicit focus of the NZC, in terms of the key competencies, as well as in the learning areas, “for each area, students need specific help from their teachers as they learn … how to listen and read critically, assessing the value of what they hear and read” (NZC, p. 16). When literacy instruction was observed, instructional depth in both mathematics and science was dominated by practice (57% of literacy instruction observed in mathematics classes and 48% in science classes). In mathematics, the remaining categories were fairly evenly distributed between item teaching (14%), Activating Prior Knowledge (APK) (14%), strategy instruction (9%) and critical literacy (7%). In science classes, a relatively high proportion of literacy teaching was categorised as item teaching (31%), with lower proportions of APK (8%), strategy instruction (8%) and critical literacy (5%).

Implications and recommendations

Before discussing implications and recommendations arising from our findings we wish to acknowledge a tension that is at the heart of this study. On one hand, we have evidence that the science and mathematics teachers in this study are more than normally-effective subject teachers. On the other, we have identified evidence of gaps in the teachers’ literacy knowledge and practice. How do we explain this apparent mismatch?

One interpretation is that the kinds of literacy and language knowledge and pedagogy suggested by the literature, and implied by NZC, are not, in fact, necessary conditions for effective subject teaching. Our observations confirm that these teachers had deep subject knowledge, provided well planned and purposeful lessons, and created a positive and productive classroom learning environment. Is it the case that these attributes of quality teaching are sufficient to promote valued subject outcomes without a need to provide opportunities for rich engagement with written subject texts or literacy instruction other than that related to receptive knowledge of subject vocabulary?

We cannot discount this possibility with the evidence currently available to us. To test this would require us to compare the literacy and language teaching practices of this group of teachers with others who teach similar groups of students but whose students do not achieve so well.

We do think it likely, on the basis of anecdotal evidence (from our own experience as literacy researchers and professional developers as well as those reported to us by Schooling Support Service Literacy Facilitators in the Secondary Literacy Project), that despite the gaps we identified, the teachers in this study might still know more about, and do more and higher quality, literacy teaching (especially of vocabulary) than a randomly selected group of teachers would.

The combined results indicate that teachers provide content instruction, often by mediating texts. In general this seemed to be a feature of teachers attempting to prepare their students by building knowledge of the content area, therefore identifying what students need to know, and summarising this in the form of teacher-made notes or modelling for students.

We have evidence that the teachers in this study have high expectations for their students and want them to achieve valued subject outcomes, including outcomes of the sort assessed in NCEA. We suggest that the teachers in this study do not want students who might struggle with the complex literacy and language demands to be excluded from opportunities to engage with valued scientific and mathematical knowledge because of these literacy and language demands. Thus, one explanation for the relative low frequency of literacy and language teaching in these classrooms is that the teachers have responded to challenging aspects of reading and writing by minimising the amount and challenge of reading and writing that the students do.

In one sense, such an approach is understandable. After all, teachers are encouraged to provide scaffolding that enables students to access subject learning at a higher level than they can currently access independently; avoiding or minimising the literacy and language demands of the subject, and providing alternative means for students to access the subjects is one form of teacher scaffolding.

We have no evidence that the teachers in this study were in any way antagonistic to the idea that they have a responsibility to teach the literacy and language of their subject. Indeed, the frequency and quality of teaching receptive subject vocabulary that we observed suggests to us that when the teachers were confident in their knowledge about an aspect of literacy and language they taught it and taught it well. We contend, rather, that the reasons that these teachers adopted such an approach are understandable and were made with the interests of their students in mind.

We are not suggesting either that these were not evidence-based decisions. Our sense is that the teachers did have evidence that minimising the literacy and language aspects of their subjects in the way we observed would facilitate students’ learning and achievement in mathematics and science in the year levels they taught. In mediating text use by restricting or controlling the use of texts, however, there are a number of inherent risks.

Firstly, such an approach would appear to be premised on the idea that subject-content knowledge is the exclusive (or at least primary) valued student outcome in mathematics and science. We contend, rather, that being able to read, write, talk and think critically about mathematics and science texts are themselves important subject outcomes, and indeed, knowledge of language and content are so interwoven as to be inseparable. We have identified the potential for students to be denied opportunities to engage with text, thereby decreasing their opportunity to develop literacy skills to use texts independently, and also decreasing opportunity to develop subject-specific skills, in terms of using texts in ways that are valued in the disciplines.
Secondly, we think that while possibly expedient (at least up to Achievement in NCEA Level 1), such an approach might unintentionally place a ceiling on students’ achievement in that discipline. Our analysis supports the teachers’ view that a more sophisticated language and literacy demand is one aspect that discriminates the criteria for Excellence from the criteria for Achievement, and Level 3 from Level 1. Such a ceiling might limit students’ ability to achieve the Merit and Excellence grades at Level 1 NCEA; or it might not limit students’ achievement at Level 1 at all, and kick in only at Level 2 or 3; or it might suffice more for internal and less for externally-assessed standards; or it might not be noticeably restricting until such a time that no teacher is available to mediate the texts students read and write, such as at university, in the workplace, or in everyday life.

Clearly, some students are able to apply their generic reading and writing knowledge and skills to the specialised-subject literacy demands of science and mathematics without very much deliberate instruction by the teachers of those subjects. However, we contend that more frequent and more deliberate subject-literacy instruction is part of what will be needed if we are to achieve ambitious equity goals and increase the number and range of students who can achieve highly in these subjects (eg, gain Excellence) at the higher levels of NCEA and tertiary education.

Thus, a number of implications emerge related to text use, textual knowledge, pedagogy surrounding text use, and inquiry into student learning.


1. Text use: students need opportunities to engage with text in ways that are valued by the disciplines and by the New Zealand Curriculum document
Having sufficient time to learn, and repeated opportunities to practice, is essential when learning any complex subject matter (Bransford, Brown, & Cocking, 2000) so, clearly, students need repeated opportunities to read, write, think about, and discuss the types of texts valued in science and mathematics if they are to become skilled users and producers of such texts.
There is ample scope for more time spent engaged in reading texts for subject-specific purposes, and both gathering and applying content knowledge. We see as concerning the lack of alignment between the time and types of texts students actually encountered in class and those that the NZC implies would be important, and that students will encounter in NCEA, in the disciplines themselves, and in ‘real world’ contexts.
We are not suggesting at all that written texts supplant other ways of teaching content or providing meaningful contexts, however  we are suggesting that written texts should be used more often for these two purposes.


2. Textual knowledge: students need opportunities to develop knowledge of how important types of subject texts work
In the service of using texts in subject-specific ways, as effective readers and writers, students develop and use knowledge of how important types of subject texts work. Knowledge of how texts work consists of knowledge about audience and purpose, vocabulary, organisational features and language resources.

It is important that teachers also have this knowledge. They need to know this so they can diagnose reading and writing problems, employ appropriate teaching strategies to address these problems, and evaluate the effectiveness of these actions.

Our findings show that the teachers knew and taught much more about vocabulary than they did other features of literacy and language. There was little to no evidence that the teachers had deep knowledge about, or taught students about, other important aspects of texts such as audience and purpose, structure, or features of language at a sentence level. One explanation for why there was so little teaching about these aspects is that teachers do not know as much about these aspects at an explicit level as they do about vocabulary. Specifically, this study suggested that they might not know how gaps in students’ knowledge of these features might affect students’ ability to comprehend or produce written texts, how to diagnose such problems, and what instructional practices to employ.

While teachers need to know these features, and students need to develop such knowledge, we do not agree that teachers should, or need to, teach all of these features as a matter of course. For example, “Repeated studies have demonstrated that instruction in isolated grammar, decoding or comprehension skills may have little or no impact on students’ activity while actually reading” (Schoenbach, Greenleaf, Cziko & Hurwitz, 1999, p. 7). Rather, teachers need to know how these features (including, but not limited to, receptive, specialised vocabulary) may act as barriers to making or creating meaning from texts and how to diagnose and address problems identified through inquiry.


3. Strategy learning: students need opportunities to develop a toolbox of cognitive strategies they can use flexibly to make and create meaning
There was very little evidence of strategy instruction in the classrooms we observed. The strategies that we think will be most pivotal for students to learn in mathematics and science are more specific to each subject’s texts and purposes and will be strategies that students can employ when features of those highly-specialised text forms become barriers to making and creating meaning. In science, for example, as well as hearing, learning and using vocabulary items, students might develop strategies for solving unfamiliar words they encounter by integrating morphological word level strategies with text level context based strategies.


4. Pedagogy: students need opportunities to participate and contribute in rich literacy learning experiences
In the classes we observed, teachers of both science and mathematics frequently modelled, and often created texts to support this modelling. However, there was less evidence that teachers used grouping or extended discussion to build understandings. We would therefore argue for a greater balance of approach. We see at least two potentially valuable purposes in incorporating greater use of participatory or dialogic teaching approaches. The first is creating opportunities for greater support through co-constructive approaches. The second is to disrupt the traditional ‘Initiate, Respond, Evaluate’ (Mehan, 1979) classroom discourse pattern to build richer, more authentic and more cognitively-challenging discourse patterns.


5. Critical literacy: students need opportunities to develop the kinds of critical literacy valued in the subject areas
Critical literacy is an explicit focus of the NZC, in terms of the key competencies, as well as in the learning areas. Critical literacy involves a shift away from ‘getting the correct answer’ to questioning the assumptions in texts, critiquing, and challenging. In our observations of teachers we saw no evidence of any instruction that could be characterised as critical literacy. We would therefore argue, from a position of instructional depth, that students need opportunities to engage with issues, ideas and concepts, to challenge and critique them as part of deep learning within their subject areas.


6. Independence: students need opportunities to develop self-regulation (in reading and writing)
Alongside instructional support and instructional materials, students also have a vital role to play in their own learning. An environment that supports self-regulation requires that students participate in discussions and other learning tasks that focus on the learning. Teacher responses indicated that when students were faced with literacy difficulties, they supported students to solve the literacy issues. In order to foster self-regulation, however, students need to be able to develop strategies for independent solving. To do so, students need to know what literacy skills or strategies they are trying to develop. We therefore argue for strategies that develop students’ awareness of the literacy demands of their subject area, beyond knowing the meanings of words.