Laptops for teachers: An evaluation of the TELA scheme in Otago schools
The purpose of this evaluation was to investigate the impacts of the Laptops for Teachers Scheme (TELA) on primary schools within the Otago region.
Author(s): Keryn Pratt, Kwok-Wing Lai & Ann Trewern with Fiona Concannon & Harriet Sutton. Report prepared for the Ministry of Education.
Date Published: May 2010
This report is available as a download (please refer to the 'Downloads' inset box). For links to related publications/ information that may be of interest please refer to the 'Where to Find Out More' inset box.
Section 3: Method
The objective of this case-study evaluation was to explore not only how teachers have used the laptop computers provided to primary school teachers in Otago through the TELA scheme, but also why they have an impact on their professional practice. In broad terms, this evaluation asked about personal factors affecting laptop use, such as teacher goals and expectations of laptop use and how this changed over time. In addition, the impact of the laptop on teachers' professional growth and collaboration opportunities, access to, and creation of quality ICT-based teaching and learning and assessment resources was explored, as well as on their lesson planning, preparation and administration. Attitudes and beliefs of teachers about the use of ICT in teaching and learning as a result of the TELA initiative, and pedagogical approaches they used as a result of the expected increases in ICT skills and confidence, were also examined.
The second area of exploration related to school factors affecting laptop use. In this respect, teachers were asked to what extent the school had and was supporting their participation in the TELA initiative. The importance of work culture and school leadership to promote successful adoption of the technology was explored, along with considerations of the additional demands such as access to the Internet or to peripheral devices at home or at school. All of these issues are illustrated graphically in Figure 1.
Figure 1: Broad research questions addressed in this evaluation
The objective of the current evaluation is to provide more detailed information about the impact of the scheme on individual teachers and at a school level through the examination of the 93 individual interviews conducted in five rounds between 2005 and 2008. Reference will also be made to information gathered fromsix focus group interviews conducted at three schools in rounds three, four and five, and from observations made in 38 classrooms in rounds three, four and five.
The methodology adopted in this evaluation is of an instrumental, collective, case-study approach to collect data from a range of stakeholders (teachers, principals and ICT coordinators) to evaluate the success of the TELAinitiative. Instrumental case studies focus on an issue or issues (Stake, 1995), and as such, this case was instrumental as it centred its examination on the TELA scheme's impact in the schools. Collective case studies involve more than one case or instrument, and in this instance, five schools or cases were involved. A three-year longitudinal approach has been adopted in this study to capture the change in the laptop use over time, and the long-term effects this use has on teaching (Appendix B details the timeline of the evaluation project).
Participant information: Schools
Initially, three schools were invited to participate in the research. In 2006, this was expanded to five, as the TELA scheme was extended to all levels of primary school teachers. This resulted in five schools participating in the study. At the beginning of the research, the numbers of teachers per school who gave their consent to participate in this evaluation ranged from two to seven. In addition, the principal and ICT coordinator of each school agreed to participate. These participants were invited to take part in individual interviews throughout the period of the study, but the number decreased over time owing to teachers leaving their positions. In addition, all other teachers at these schools were invited to participate in focus group interviews at various times throughout the research.
Schools that were invited to participate in a large-scale study of the TELA scheme by the Waikato research team were excluded from this study. Three criteria were used to select schools from those remaining:
- information from the Ministry of Education on the uptake of the TELA scheme in schools within the Otago region. Schools that had a higher proportion of teachers participating in the TELA scheme were given priority, as were schools that had only recently joined the scheme
- location – the research team classified the schools as rural, provincial or urban (see Appendix C) and sought two urban schools, two provincial schools and one rural school
- school type (a mix of contributing and full primary schools).
Taking these criteria into account, the sample was narrowed, and schools were invited to participate. Those schools that responded and agreed to participate then became the purposive sample.
The research team was faced with a degree of difficulty in obtaining five schools to participate in the evaluation. This was initially owing to the limited number of the schools in the region that had only just received their laptops in May 2005 when the evaluation project first began. Hence, there were only a small number of schools eligible for the purposive sample. Secondly, time restrictions, pressures on teachers, and their participation in other projects (for example, the ICT Professional Development Clusters) that were perceived to be seeking comparable information, acted as barriers to schools taking part. Thirdly, the lack of a tangible incentive or reward was also cited by schools as a reason to decline the invitation to become involved in this evaluation.
However, the schools that agreed to take part in the evaluation gave willingly of their time. Several teachers sacrificed their lunch period to participate. Others gave time after school, occasionally waiting until after five o'clock for an interview session. One of the teachers visited the university to participate in an interview on a public holiday. In another school, the teachers coordinated cover for each other's classes whilst the interviews took place. All in all, the participating schools and teachers were extremely generous in efforts to take part in interviews and in all later communications.
Table 1 provides an overview of each of the participating schools. A synopsis of each school follows.
School A is a contributing primary school, classified by the research team as an urban school. The school is a co-educational state school with a roll of between 350 and 400 students. Two members of the research team conducted a preliminary school visit to discuss the project with staff at the school in 2005. The school agreed to take part, and information packs were distributed to the teachers who were participating in the laptop scheme. Although two teachers initially agreed to participate in the evaluation, they were unable to be interviewed in 2005. In April 2006, the school was again contacted, and interviews were carried out with the principal, the lead ICT teacher and two additional teachers. In November 2006, two teachers participated in interviews, while two focus group interviews were conducted, owing to 12 teachers wishing to be involved in the focus group interviews. The large number of teachers involved in the focus group interviews may have been simply a function of the relatively large number of teachers at the school and the limited number who were taking part in the individual interviews. Observations were also conducted in three classrooms. In the fourth round of interviews (October 2007), the principal, ICT coordinator and one teacher participated in interviews, while a focus group interview was conducted with a further six teachers. Observations were conducted in two classrooms. In the final round of interviews, only the principal and one teacher were interviewed, as the other teacher and the ICT coordinator had both left the school. Observations were carried out in the teacher's classroom. No additional teachers wished to participate in focus group interviews.
School B is a full primary school, classified by the research team as being a rural school. The school is a co-educational state school, with a roll of between 235 and 285 students. Two members of the research team conducted a preliminary school visit to discuss the project with staff at the school in 2005. This was followed by a visit by one member of the research team to conduct face-to-face interviews in June 2005 with five staff members. At the time of this first round of interviews, the school had undergone leadership changes and, as a result, the principal did not participate in the initial stage of interviewing. The second round of interviews took place in May 2006. At this stage, the principal was interviewed, along with the ICT lead teacher of the senior syndicate and four other teachers. In the third round of interviews, conducted in November 2006, four teachers participated in individual interviews, while one focus group interview was conducted with four teachers. Observations were conducted with two classes. In October 2007, the fourth round of interviews was conducted with the principal, ICT coordinator and two teachers. In addition, two focus group interviews were conducted, one with five teachers from the junior syndicate and one with two teachers from the senior syndicate. Observations were conducted in two classrooms. In the final round of interviews, the principal, ICT coordinator and one teacher were interviewed. The other teacher interviewed at round four was on study leave and unable to be interviewed. Observations were carried out in one teacher's classroom, but no focus group interviews were conducted.
School C is a full primary school, classified by the research team as a provincial school. The school is a co-educational integrated school, with a roll of between 125 and 175. In agreement, and after discussions with the principal at the school, the research team determined that a preliminary visit was not necessary. A member of the team conducted face-to-face interviews in May 2005. Interviews took place with the principal, the ICT syndicate lead teacher and two other teachers. The school was contacted again and a second round of interviews took place in May 2006. In this second round of interviews, a single research member again carried out interviews with the same principal, the same teachers and an additional three teachers, two of whom had joined the school in the past six months. Five teachers participated in the third round of interviews, conducted in November/December 2006. No focus group interview was conducted at this school, as almost all teachers at the school were already involved in the individual interviews being conducted as part of the research, while six classes were observed. In round four (October 2007), interviews were conducted with the principal, ICT coordinator and five teachers, while observations were conducted in six classrooms. The final round of interviews in late 2008 involved the principal and four teachers. The other teacher interviewed at round four had left the school, as had the ICT coordinator. Observations were unable to be carried out at the school and no focus group interview was conducted.
School D was one of two additional schools approached in 2006 owing to the expansion of the evaluation after the TELA scheme was widened to include Years 1 to 3 teachers. As such, it was not part of the first round of interviews that took place in 2005. The school is a contributing primary school, classified by the research team as being an urban school. The school is a co-educational state school, with a roll of between 150 and 200 students. The school was contacted by email and phone. This was followed by a visit by one member of the research team to conduct face-to-face interviews with the principal, the ICT lead teacher and two other teachers who had laptops. In November/December 2006, interviews were conducted with two teachers and a focus group interview was conducted with three teachers. Three classrooms were observed at School D. Conducted in October 2007, the fourth round of interviews involved the principal, ICT coordinator and one teacher. Two classes were observed and one focus group (with three teachers) was conducted. In late 2008, the principal and one teacher were interviewed. The ICT coordinator was not interviewed, as she no longer held the position, however, a focus group interview was conducted, with five teachers involved.
School E was the other additional school approached in 2006 owing to the expansion of the evaluation project after the TELA scheme was widened to include Years 1 to 3 teachers. The school is a full primary school, classified by the research team as being a provincial school. The school is a co-educational state school with a roll of between 300 and 350 students. The school was contacted by email and phone. This was followed by a visit by one member of the research team to conduct face-to-face interviews with the principal, the ICT lead teacher and five other teachers who also had laptops. In November/December 2006, interviews were conducted with five teachers. No focus group interview was conducted at this school, as no other teachers wished to participate. Six classrooms were observed at School E. The fourth round of interviews, conducted in October 2007, involved the principal, ICT coordinator and three teachers. Observations were conducted in four classrooms but no teachers wished to participate in focus group interviews. In the final round of interviews, the principal and three teachers were interviewed at School E. One of these teachers had previously been the ICT coordinator, but she had given up this position. No staff wished to take part in focus group interviews.
Participant information: Individuals
A total of 29 people (five principals, six ICT coordinators and 18 teachers) took part in individual interviews as part of this research. The number of participants decreased over the course of the research, as teachers left participating schools. A summary of those involved and their participation in each round of the research is presented in the following table. Whilst the sample size for these observations is small, it allows for the exploration of some of the characteristics of this purposive sample and for the consideration of some general trends that arose in the interviews.
As this table shows, six male and 23 female teachers participated in the research. At the time of their first interview they had a range of teaching experience (3–37 years) and taught at a range of levels (Year 1 to Year 8).
|ICT coordinator||Y||Y||Female||5 yrs||Year 2|
|Teacher||Y||Y||Y||Y||Female||17 yrs||Year 6|
|Teacher||Y||Y||Female||6 yrs||Years 3-4|
|ICT coordinator||Y||Y||Y||Female||6 yrs||Year 5|
|Teacher||Y||Y||Y||Female||7 yrs||Years 2-3|
|Teacher||Y||Y||Y||Female||10 yrs||Years 6 & 7|
|Teacher||Y||Y||Y||Y||Female||12 yrs||Year 6|
|ICT coordinator||Y||Male||37 yrs||Years 2-3|
|ICT coordinator||Y||Y||Y||Female||13 yrs||Years 7 & 8|
|Teacher||Y||Y||Y||Female||10 yrs||Years 4 & 5|
|Teacher||Y||Y||Y||Y||Female||30 yrs||Years 2-3|
|Teacher||Y||Y||Y||Y||Y||Female||7 yrs||Years 3-4|
|ICT coordinator||n/a||Y||Y||Female||19 yrs||Year 6|
|Teacher||n/a||Y||Y||Female||27 yrs||Years 3-4|
|Teacher||n/a||Y||Y||Y||Y||Female||7 Yrs||Year 1|
|ICT coordinator||n/a||Y||Y||Y||Female||Years 2-3|
|Teacher||n/a||Y||Y||Y||Y||Female||6 yrs||Year 4|
|Teacher||n/a||Y||Y||Female||3 yrs||Years 1 & 2|
|Teacher||n/a||Y||Y||Male||5 yrs||Years 5 & 6|
|Teacher||n/a||Y||Y||Y||Y||Female||29 yrs||Years 7 & 8|
Data collection procedures
Multiple data collection methods were employed in this evaluation to allow for triangulation. These included:
- semi-structured interviews
- needs assessment (Goals and objective statements)
- class observations
- focus group meetings.
An overview of the research timeline, and when each type of method was employed, can be found in Table 3.
|Round 1||Round 2||Round 3||Round 4||Round 5|
|Teachers||mid 2005||mid 2006||late 2006||late 2007||late 2008|
|ICT Coordinators||mid 2005||mid 2006||late 2007||late 2008|
|Principals||mid 2005||mid 2006||late 2007||late 2008|
|Goals & Objective Questionnaires||mid 2006|
|Class Observations||late 2006||late 2007||late 2008|
|Focus Group Interviews||late 2006||late 2007||late 2008|
Semi-structured interviews were conducted with up to five teachers in each of the five participating case-study schools. The intention was to interview participating teachers four or five times throughout the lifecycle of the evaluation project, although this was not always possible owing to teachers leaving the participating schools. Principals and ICT coordinators were interviewed at least three times, at the beginning of the first year, and at the end of the third year and fourth years. The interviews covered both personal and school factors affecting laptop use (see Appendix D for the themes and open-ended questions that served as a guideline for the researchers).
In rounds one and two, a needs assessment was conducted with teachers participating in the individual interviews, collecting information on teachers' goals and objectives (see Appendix E for an example). Teachers were provided with the form during the interview sessions, and were asked to reflect on and complete it. This was then to be posted back to the research team, to be kept on file for discussion and revision during the subsequent interviews with the teachers. After the second round of interviews teachers were asked about their goals and objectives rather than being given a form, owing to the low response rate in the first two rounds.
Classroom observations were also conducted in the classrooms of willing teachers as part of rounds three, four and five of data collection to gain information on classroom ICT practices. Information gained from the observations was used to help in our analysis and interpretation of transcripts. These observations took place in a range of classrooms and subjects within each of the five schools.
As part of the third, fourth and fifth rounds of data collection, focus group interviews were also conducted. Focus groups provided an opportunity for teachers within the case-study schools who were not participating in the individual interviews to give their opinion on matters pertaining to the TELA initiative. The focus group interviews were semi-structured and covered similar areas to the individual interviews (see Appendix F for the questions that served as a guideline for the researchers). Data from these interviews was compared with the personal interview data. The information gained from these interviews was in line with that gained from the individual interviews, so has been included in the general findings sections rather than discussed separately.
The individual and focus group interviews were transcribed, coded and analysed to identify factors affecting laptop use and changes of pedagogical approaches. The unit of analysis selected for coding constituted a 'unit of meaning' after Chi (1997). The data was coded in line with a priority of factors that arose from the research questions; however, these categories were adapted in light of participants' responses. The graphical representation of the research questions was also adapted, with the new version shown below. This version arose from analysis of the transcripts and is used to present this analysis.
Figure 2: Adapted version of broad research questions addressed in this evaluation
Where to find out more
For more information about this publication please email the: Research Mailbox