Executive Summary

Overall, the effectiveness of computer-assisted learning (CAL) has not been conclusively demonstrated. To date, it has been shown to be less effective, on average, than other forms of intervention in education. In considering the results of evaluative research in computer assisted learning, one has to avoid confounding the medium with the method. Generally, computer-assisted learning software is under pinned by an older, neo-behaviourist theory of learning, one that has been displaced in the classroom by more social constructivist views of learning. Particularly in New Zealand primary classrooms, the approach of the software may differ considerably from widely accepted classroom pedagogy.

Computer-assisted learning programs, especially integrated learning systems, are generally costly. Their efficacy and cost effectiveness relative to alternative programs, particularly with respect to reading, is questionable. While comparative research exists with respect to effectiveness, good comparative research in relation to cost effectiveness is lacking.

Results from evaluations of integrated learning systems show highly variable results, with independent evaluations tending to be less favourable. The best results appear to be for basic maths skills; there is little evidence of gains in reading. Integrated learning systems, in their current form of neo-behaviourist, mastery learning, support the gaining of basic procedural knowledge. There is evidence that students may not be able to apply such knowledge without teacher intervention and that such knowledge may not generalise to school or system curriculum assessment tasks.

Part of the variability in outcome results stems from the different off-system assessment measures used to measure progress and part stems from the differing contexts of implementation. The latter includes characteristics of the student body and organisation for implementation including configuration of resources and deployment of personnel. Above all, this latter factor concerns integration, particularly the match between computer-assisted instruction (CAI) or the integrated learning system curriculum content and methods, and that of the school and classroom.

Research questions

This research sought to answer the following:

• What does the literature say about the effects of computer-assisted learning on student outcomes in literacy and numeracy?
• What does the literature say the effect is relative to other types of interventions?
• What does the literature say about preconditions for computer-assisted learning to be optimally effective in relation to student learning outcomes in literacy and numeracy?
• What are schools that have experienced computer-assisted learning reporting about their views of the pre-conditions necessary for effective use of computer-assisted learning again in relation to student learning outcomes in literacy and numeracy?

Overview

Part A begins with a description of the approach taken to answer the research questions. This is followed by a discussion of issues relevant to a consideration of computer-assisted learning and, in particular, its effects on learning outcomes. First, there is a discussion of definitions of terms like computer-assisted instruction (CAI), computer-assisted learning (CAL), computer-based learning (CBL) and integrated learning system (ILS). In light of this discussion, the scope of the review is clarified. Other issues relate to the extent and veracity of the available evidence. Then, the aim is to assess the evidence regarding the use of computer-assisted learning in literacy and numeracy. The major findings from the use of CAI, largely from studies employing meta-analyses are discussed. Next, to put a face to the meta-analysis research, a selection of studies of effectiveness of widely used products is presented. Studies of a longitudinal and extensive nature conclude this section. This general discussion of computer-assisted learning is followed by a more detailed consideration of integrated learning systems, which are a major focus of this report.

The second part of the report considers the relative effectiveness of CAL. At its most basic, this includes evidence from research studies that have attempted, in some way, to assess the value of CAL. Any study where cost and effectiveness are considered and any research relating the use of computers to other innovations likely to be employed in addressing the enhancement of learning outcomes in literacy and numeracy is discussed. In particular, data relating to the relative effectiveness of ILS to other innovations are presented.

The final part of the report is important in that it examines the preconditions for effective use of CAL in general and ILS in particular. This part is premised on the notion that even if CAL were designed to operate alone, using technology to improve learning is not just a matter of examining hardware and software features. Technology is a cultural object that functions in a social context where teacher and peer influences also operate. Thus this part aims to delineate the nature of influences, both positive and negative, that operate. The final section in this part discusses the data gathered from interviews with New Zealand schools regarding their views of conditions conducive to obtaining maximum benefit from an ILS.