Measuring the worth of e-Learner support systems: Developing a possible benchmarking method for evaluation effectiveness Publications
This report was commissioned by the Ministry and undertaken by the Aotearoa Tertiary Students’ Association. It tested a survey-based method for assessing learner perceptions of the effectiveness of e-Learning support systems.
The report also surveyed the beliefs of e-Learning professional staff (tutors and support personnel) about the effectiveness of those systems. The findings of the two surveys were tested for any correlation between the learners’ judgments and staff assessments.
Author(s): Jane Renwick and Stephen Owen, Aotearoa Tertiary Students' Association. Report prepared for the Ministry of Education.
Date Published: May 2005
The findings in this research (and those from overseas projects) indicate that student support services provide levels of assistance across a range of factors which contribute to the ability of e-learners to complete their study programme. Overall, the relative standard error of the quantitative findings in this report is 8.51% at a 95% confidence interval. The research confirms the factors identified in the ‘E-Learners Support Project’ are aspects of students support services which are likely to mitigate the attrition of e-learners from their study programme
Despite the fact that the population list used for this research was a year out of date at the time when the survey was undertaken, the methodology and research technique produced useful results. It is reasonable to assume that researchers could use a similar technique in the future to measure the worth of student support services, especially those provided to e-learners. It is also reasonable to assume that a similar survey to the ‘E learners Support Project’ would produce a significantly smaller standard error if it were implemented in a timely manner across common e-learning platforms to New Zealand tertiary students.
Currently, the New Zealand tertiary sector is implementing initiatives recommended in the Performance Review of the Performance Element to Tertiary Funding. As part of that review there is a proposal to undertake a survey of students’ tertiary experience. The findings of this report underline the importance of measuring the components of student support services and assessing their effectiveness. Furthermore, this research suggests that the introduction of a generic framework for a performance assessment would need to weight the responses from e-learners across components of student support services to take account of the greater reliance e-learners place on particular aspects of those services.
The findings from the research population indicate that overall the tertiary organisations are providing good student support services that approach the level of performance e-learners expect. They provide a useful bench mark that other providers could use to gauge their own performance across a range of factors which support the e-learner.
The research has highlighted areas where provision of services could be improved to meet the e-learners needs: a trial e-learning programme for potential students, easy navigation of web sites and learning platforms, improved access to generic information, for example, time management, provision of personalised communication with staff, social integration with their cohort and peers and help desk support that is readily understood. The recommendations in this report identify seven aspects of support services that providers with e-learners should measure and assess. Tertiary providers should now be encouraged to assess and monitor their performance as deliverers of those services.
Of particular interest to providers delivering these services will be the level of importance these e-learners place on the technical assistance they are given and the email communication they receive from staff. The research suggests providers are likely not to be reaching the performance level the e-learners declare optimum.
It appears from the findings that Technical Assistance is above all the most important aspect of support to e-learners. Currently tertiary providers are aware of the need to improve the delivery of this service by extending the hours that help is available.
The e-learners in this research are predominantly a female population with care giving responsibilities to a dependent person. They are clearly a population likely to do their e-learning activities outside the ‘normal’ operating hours of their provider. The recommendations of this report include a suggestion that a one place provision of technical assistance help desk support with extended operating hours is a feasible option for delivery of services to e-learners. The report suggests that the introduction of that service is advantageous as part of our national learning infrastructure.
Education Data Requests
If you have any questions about education data then please contact us at:
Email: Requests EDK
Phone: +64 4 463 8065