E-learning for adult literacy, language and numeracy: a case study of a polytechnic

Appendix A: Methodology

A.1. Approach

The research methodology was adapted from the success case method (Brinkerhoff, 2003, 2005). While our reporting of the research focused on success, we also noted unexpected outcomes and failures. For example, in the m-learning example, we noted an unsuccessful innovation. In order to gather data relating to the polytechnic’s adoption of innovations and commensurate systemic change, we asked polytechnic leaders questions based on the three change models described in Appendix B.

The urban polytechnic at the centre of this case study is becoming increasingly known for its innovative endeavours in the provision of quality education. Initial informal requests for exploratory discussions were made to staff known to be including e-learning or mixed media learning as part of their programmes.

These meetings confirmed that the polytechnic was a good site for a case study of a tertiary institution that had, for some years, been successfully using e-learning to promote the literacy, language and numeracy (LLN) skills of adults. The Ministry of Education agreed with this conclusion and told us that the polytechnic uses the Tertiary Education Commission Learning Progressions. Further details of the selection process are provided under procedures.

We began our investigation by seeking initial permission from the polytechnic’s chief executive officer to proceed with the study. He advised us on procedures for gaining approval from the institution’s ethics committee. We also sought permission from the University of Canterbury College of Education ethics committee. (Further details of the ethical considerations are provided under the procedures section below.) We then put in our applications for ethical approval, which were granted. The polytechnic’s Dean of Research announced the research project to staff.

A.2. Criteria used to select case study organisation and programmes

The selection criteria we used to select the polytechnic for our case study, and the criteria we used to select programmes and courses for sub-case studies, included as many of the aspects of good practice as possible that we identified in the literature available at the time the research was proposed (Ministry of Education, 2005). We accordingly looked for an institution and for programmes and courses that had a range of the following attributes:

• Includes e-learning;
  
• Has been in place for at least two years and has documented evidence of outcomes;
  
• Targets students from groups of people who are either in work or about to begin work;
  
• Has staff with relevant qualifications and expertise in adult education, elementary education, community education, and/or educational administration;
  
• Offers teaching based on sound principles of adult learning;
  
• Connects its curriculum to students’ needs and interests;
  
• Provides (to some extent) content in collaboration with a range of relevant agencies such as schools, employment agencies and educational groups;
  
• Has explicit and clearly structured modes of teaching literacy and/or numeracy and provides individualised tuition;
  
• Uses individual learning plans;
  
• Has high expectations of students;
  
• Provides students with formal credit and accreditation for their study.

A.3. Participants

Our research team consisted of two researchers and a research assistant. Together, we conducted interviews with staff and undertook observations in all areas of the polytechnic where e-learning or mixed media learning were being used or developed at the time of our investigation. We also covered those areas of the polytechnic with programmes or initiatives directed at New Zealand adults with LLN needs. These areas were:

• The ESOL centre in the library (including one day class);
  
• Trades faculty (including two trades classes);
  
• Food and hospitality faculty (including one programme for apprentices);
  
• Māori and Pasifika faculty (including the dean);
  
• Adult literacy (including one evening class);
  
• Applied sciences and health faculty (including one pre-health course).

Interviewees

We formally interviewed 34 people:

• Ten leaders, some of whom were also managers and tutors. The leaders included the CEO, three deans, two heads of schools, three unit managers, and the e-learning coordinator and designer;
  
• Six tutors (two literacy, two ESOL, two numeracy/trades);
  
• Eighteen students (seven adult literacy, four ESOL literacy, four maths, three trades).

The students represented a range of backgrounds, skills and needs, including ESOL, low literacy, behavioural and physical disabilities, and various ethnicities (including Māori, Pasifika, Asian, and European), ages (late teens to 50s) and subject areas (from trades to professional). More females than males volunteered to participate in our research; approximately two-thirds of our respondent students were female.

Procedures

Interviews with staff

The interviews were conducted by one or more of the three researchers. Face-to-face staff interviews occurred in a variety of situations, including interviewees’ own offices, classrooms, a café and labs. On a small number of occasions, other people were present, or the interview was conducted by phone. 

Potential questions covered several areas, including (but not restricted to) the nature of good practice in e-learning, work-based learning, success factors for e-learning, impacts on learners, motivation of learners, barriers to learning, and the value of a free, online site for developing LLN skills, etc. Our interview of leaders included questions on the three change models described in Appendix B.

Interviews were semi-structured, allowing for flexibility to follow new leads or ideas raised by the interviewees. All interviews were transcribed. Each interviewee received a copy of his or her transcript for checking and amendment.

Interviews with students

These interviews, which were semi-structured, were conducted by one of the two researchers or by the research assistant. We interviewed students either by phone (after they had told their tutors that they wanted to participate in the study) or face to face in their labs or classrooms. Most student interviews were done on a one-to-one basis. However, we conducted two group interviews in the adult literacy class and one in the ESOL centre.

The questions we asked sought students’ reasons for wanting to improve their LLN skills. The questions also focused on employment aspirations, current or past use of e-learning, and whether or not the student would use a free, public online LLN site.

All student interviews were transcribed, but it was not possible to have students check their transcripts because of their literacy limitations. In most cases, we considered that it would be unreasonable to ask these students to read and amend their respective transcripts. 

Observations

We observed the members of three classes as they worked to develop their literacy or numeracy skills. Each observation was made by one of the two researchers and the research assistant. We did not take photographs during class sessions but did some sketches at these times. We also took some photographs of classroom layout and resources. During each observation, we wrote field notes and asked for and received handouts and copies of teaching materials. We also had access to the polytechnic’s online e-learning environments, including its learning management system, Moodle.

We used the polytechnic’s web pages as a briefing tool before interviews, and to enrich our descriptions as we analysed and wrote up the case study. Some of our interviewees gave us documents, papers and teaching materials, and we also used this material for illustrative purposes.

Ethical considerations

Both the University of Canterbury College of Education and the polytechnic’s ethics committees reviewed and approved the project. Interviewees all participated voluntarily after an initial approach from us. None of the tutors was asked to recruit students. Students who appeared embarrassed or very busy were not asked to participate. No staff members declined the chance to participate. All participants received a copy of information about the project.

We told the students they could withdraw from the study at any time, and we gave them consent forms to sign. We took considerable care to ensure that the students were not stressed by the research and that their identities remained anonymous. We used pseudonyms and changed any characteristics that could identify a participant. We consider that the group interviews within the adult literacy class worked particularly well because the group setting allowed the students to contribute to the discussion as and when they felt comfortable.

We produced modified versions of the staff information and consent forms for students. We also had at hand other simplified versions of these forms for those students with low literacy skills. These forms were read aloud to these students. Students and their tutors expressed their appreciation of this approach.

A.5. Analysis and reporting

Our study involved five stages of analysis. The case study and its analysis were led by the project’s principal investigator and first author. The analysis and reporting process progressed as follows.

• We began by reviewing the complete data set, initially briefly and then in depth, to create the various perspectives one by one. We used this material to draft the full report. We then analysed the data against the change models;  
• We used this material to write a first full draft report of our study and its findings. We then carefully considered this report, and also gave it to polytechnic staff in December 2008 for their feedback and corrections;
• Some months elapsed before we completed the accompanying literature review. The report was then revised again against this material, and also to improve coherence and remove items that might identify the polytechnic. We sent the revised report to the polytechnic’s Dean of Research for approval;
• On receiving and incorporating into the report the polytechnic’s feedback and further corrections, we asked external experts to carefully consider what we had written. This process resulted in further revision and edits in order to increase the contextualisation and clarity of the findings for tutors in the field;
• The outputs were reviewed by national experts, international experts and the Ministry of Education;
• Following amendment and editing to improve communication, the outputs became public.