Ua Aoina le Manogi o le Lolo: Pasifika Schooling Improvement Research - Final Report

5. Pedagogical Content Knowledge Patterns

5.1 Primary Schools

In total 96 teachers across 7 primary schools completed the Pedagogical Content Knowledge Survey (PCK). Demographics for teachers who completed the PCK survey across all schools are presented in Table 67 . A total of 63 (66%) respondents were teachers, with a further 16 (17%) respondents who were Senior or Lead Teachers. More than half of the respondents (82%) had a bachelors degree or higher. The ages included approximately a quarter in each of the three younger age brackets (30 or under, 31 - 40, 41 - 50). Most teachers (75%) were trained in New Zealand, with 45% of all respondents having taught for 10 or more years.

5.1.1 Mean scores

Across all questions in the survey (Appendix H), the mean score was 1.35 (SD = 0.42; n = 67), or between 1 (correct) and 2 (correct with rationale). Mean scores for each individual question and combined across similar questions are presented in Table 68 to Table 71 below. See Section 2.3.2 for a description of the questions. Note that means are made up of different numbers of teachers because some teachers did not answer all questions. Means combined across sections are created only for those teachers who answered every question in that particular section.

The highest mean score for any question was Section 2, Question 4, in which teachers were asked what to do next following analysis of STAR results. The lowest mean score was Section 2, Question 2, which involved pointing out additional information in STAR results.

In general, scores were higher in Section 1, which involved identifying teaching moves and explaining what could be done differently, than Section 2, which involved analysing and applying STAR data. Scores were higher in explaining what could be done differently (Section 1, Question 2a - c) than in identifying effective teaching moves (Section 1, Question 1a - c). Scores were also higher in applying STAR data to teaching (finding other information and explaining what to do next; Section 2, Question 3 - 4) than analysing STAR data (Section 2, Question 1 - 2).

5.1.2 Mean scores by school

Results indicated much variation between schools. Mean scores across all sections were as high as 1.52 (Case Study 3) and as low as 0.97 (Case Study 1). There was also variation between areas of strength. For example, Case Study 1 and School A4 both have very similar mean scores overall, yet Case Study 1 had higher scores in Section 2 while School A4 had higher scores in Question 1.

Two of the Case Study Schools, Case Study 3 and Case Study 5, had higher means for the survey than the overall average of all primary schools. Scores in Section 1 were slightly lower for Case Study 5, but higher for Section 2, whereas scores for Case Study 3 were almost identical across the two sections.

5.1.3 Score frequencies

Figure24 and Figure 25 displayed the number of teachers at each possible score (0 - 3) for each question for Sections 1 and 2. The number of teachers who did not answer each question varied from none (all teachers answered Question 1a) through to 18 (18 teachers did not answer Question 2c). Note that as Section 1’s Questions 1a - c and Questions 2a - c required teachers to list 3 responses, fewer teachers answered 1c and 2c (not all teachers provided 3 responses) while most answered 1a and 2a.

Questions 2a - c (explaining what could be done differently) generally had more scores of 2 and 3 than Questions 1a - c. However, there were also more scores of 0 in Questions 2a – c. Scores were more consistent in Questions 1a – c (identifying effective teaching moves), where in each question most teachers had a score of 1 (correct but without an explanation).

Figure 24: Number of teachers with each score for each question in Section 1

Figure 25: Number of teachers with each score for each question in Section 2

Section 2 had great variation between different questions. Questions 1 - 2 (to explain what paragraph comprehension results meant and point out other information from the results) were generally similar, with a roughly even spread of scores. Question 3 (suggest further information that a teacher could use in making decisions about comprehension) had the greatest number of teachers scoring 0, while Question 4 had only 1 teacher scoring 0 and most scored 1 or 2.

5.1.4 Score correlations for Focus Cluster A

Correlations with class achievement measures

There was no significant correlation between total PCK scores and any of the achievement measures (level or rate of gain, see Table 73 ). It may be that, like the classroom observations, the more significant relationship might be with school level of gains, reflecting the dimension of coherence across teachers. It may also be, as noted above, that the ‘inert’ measures of PCK, that is, pencil and paper checks, are not useful.

A significant correlation was found between class stanine gains in 2007 (Pre-test to Post-test) and section 2, question 4 of the PCK survey. This question asks what the teacher should do with STAR results, so high marks on this question involve understanding how to apply the results of testing to further teaching. However, this correlation did not occur in 2008, suggesting that it may have been due to chance considering the large number of correlations conducted.

Correlations with teacher demographics

There was no significant correlation between total PCK scores and any of the demographic indices. There was no consistent pattern of relationship with subsections either, and so we assume this measure of PCK does not accurately distinguish between different forms of effective knowledge. It may be that we will need to design measures which tap ‘enactive’ knowledge rather than this ‘inert’ form of knowledge to find relationships.

Significant positive correlations were found between years of experience teaching and Section 2, Question 3, and between years of experience teaching and the average of Section 2, Questions 3 - 4 (presumably due to the correlation with Question 3). Question 3 asks the teacher to identify other information, in addition to STAR results, which could be used to make decisions about teaching comprehension.

Significant positive correlations were also found between position in the school and several PCK scores including: Section 1, Question 1b; Section 2, Question 4 ; the mean score across Section 2, Questions 3 - 4; and the mean score across all questions in Section 2. Question 1b in Section 1 asks teachers to identify effective teaching moves, and Question 4 in Section 2 asks teachers to point out information from STAR results. The other correlations with Section 2 mean scores are probably due to the strong correlation with Question 4.

Significant positive correlations were also found between teacher qualifications and PCK scores including Section 1, Question 2b and Section 1, Question 2c. Qualifications also correlated with the mean of Section 1, Question 2a - c presumably due to the previous correlations. Questions 2b and c ask teachers to suggest things which they would do differently in teaching a particular lesson.

Additionally, one negative correlation was found to be significant: teacher qualifications and Section 2, Question 1. This question involves analysing paragraph comprehension results in STAR.

5.2 Secondary Schools

In total 77 teachers across six schools completed the Pedagogical Content Knowledge Survey (PCK). Demographics for teachers who completed the PCK surveys across all schools are presented in Table 75. A total of 44 (57%) respondents were teachers, with a further 20 (26%) respondents who were Senior or Lead Teachers. More than half of the respondents (87%) had a bachelors degree or higher. The age range was approximately even in the three younger age brackets (30 or under, 31 - 40, 41 - 50), however, respondents in the 51 - 60 age bracket accounted for 42% of all respondents. Most teachers (71%) were trained in New Zealand, with 57% of all respondents having taught for ten or more years.

5.2.1 Mean scores

Across all questions, the mean score was 1.74 (SD = 0.41; n =59), or between 1 (correct) and 2 (correct with rationale). Mean scores for each individual question and combined across similar questions are presented in Table 76 and Table 81 . See Section 2.32. for a description of the questions. Note that means are made up of different numbers of teachers because some teachers did not answer every question. Means combined across sections are created only for those teachers who answered every question in that particular section.

The highest mean score for any question was Section 2, Question 2b, in which teachers were asked to suggest ways in which to help students improve gaps in their asTTle results for the subtest ‘inference’. However, as seen in the Frequencies section below, many teachers did not answer this question. Those teachers who chose not to answer the question may have received a lower score if they had answered the question. The lowest mean score was Section 1, Question 2, which involved identifying effective teaching actions.

In general, scores were very similar in all three sections. Scores were higher in identifying less effective teaching moves (Section 1, Question 3 - 4) than in identifying effective moves (Section 1, Question1 - 2). Scores in suggesting another effective move (Section 1, Question 5) were very similar to those for identifying effective moves. Scores were only slightly higher in improving ‘inference’ (Section 2, Questions 2a - b) than in improving ‘finding information’ (Section 2, Question 1a - b). Note that as the four questions in Section 3 had identical content, its sub-sections cannot be compared.

5.2.2 Mean scores by school

Mean scores across all sections were as high as 2.00 (Case Study 4) or as low as 1.33 (School B8). Despite the similarities in mean scores across all schools, there was variation between sections within some schools. For example, School B4 had higher marks in Section 1 than Section 2, while School B1 had much higher marks in Section 2 than Section 1. School B8, on the other hand, had identical mean scores across all three sections.

In general, the two Case Study Schools had higher marks than secondary schools overall. Case Study 2 had some variation between sections, with higher marks in Section 2 and lower marks in Section 3. Case Study 4, however, had identical mean scores for Sections 2 and 3, and only slightly lower scores for Section 1.

5.2.3 Score frequencies

Figure 26 to Figure 28 display the number of teachers scores for each question for Sections 1, 2 and 3. The number of teachers who did not answer each question varied from none (all teachers answered Question 1 and 3 in Section 1) through to 15 (15 teachers did not answer Question 2b in Section 2).

Scores were relatively consistent in Section 1, with most teachers scoring 2 in each question and many scoring 1. No teachers received a score of 0 for this section.

Figure 26: Number of teachers with each score for each question in Section 1.

Figure 28: Number of teachers with each score for each question in Section 3

There was some variation in Section 2, the most marked of which was the large number of teachers who did not answer Question 2b. Scores were more consistent in Section 3, which had similar results to Section 1. Note that these were not analysed for correlation with student achievement due to a lack of class level data.