WickED service evaluation: Key conclusions

Young New Zealanders are increasingly embracing the online medium and it has become a source for both formal and informal learning. Both of these elements are facilitated by the WickED service via content like ‘Themes’ for teachers to integrate into lessons, as well as informal learning formats like the forums, games and quizzes.

Overall, student satisfaction with WickED is fairly high, with just under two thirds of users ‘happy’ or ‘very happy’ with the site, though close to one in four are unsure whether or not they are happy. This is reinforced by the finding that just 22% of students feel the site is more useful than other online resources used, while 24% feel it is comparable to other sites and one quarter feel it is less useful. Similarly, while a higher proportion (78%) of adult users of the site are happy or very happy with WickED, 21% are sitting on the fence.

This poses a challenge for WickED site managers as it appears that these perceptions may arise from a lack of understanding of all that is on offer on the site – among both students and teachers. Aside from games, quizzes or crosswords, or the ‘Maths Stuff’ content (used by around two in five students), there is very little use of other areas of content on WickED. For example, only 12% of students claim to have ever visited the Themes, 10% Student Gallery, 6% Forums and 4% have ever Posted on the Student Gallery. This stems from the lack of frequent and repeat use of WickED among both students and teachers.

A very large proportion of student users were found to be visiting WickED for the first time (60%) and just over one quarter of students visit the site weekly or more often. Regular use is often stimulated by lesson-based usage.

As teachers are the primary source of promotion of the site, the challenge of stimulating greater repeat visitation and exploration of all that is on offer on the site is most likely to be overcome by greater support and promotion of WickED amongst teachers. It appears that the site is not ingrained and integrated into teachers’ programmes to the extent that it could be and this may call for greater structure to some of the WickED content to provide teachers with a step by step approach to using the site. This may help in drawing them (and their students) back on the regular basis in order to complete particular programmes.

It is also notable that the area of content deemed the most appropriate for teacher-led use of WickED, ie the Themes, has achieved little claimed use by students (12%) and none of the teachers spontaneously mentioned this area of content as one which was specifically performing well. Other content areas which may place burden on resources and which are also not being used by a large number of students are the forums and ability for students to submit content. Again, greater promotion and encouragement for teachers / schools is required to stimulate better use of these tools among students as this type of ‘participatory’ usage is a major benefit of the online medium and one which can facilitate learning through building students’ communication skills and their confidence.

Quantifying WickED’s success

As mentioned above, the majority of WickED users are happy with the service. Testament to the site’s remit of promoting independent learning, students are the most satisfied with the way the site allows them to work by themselves and interestingly, students who visit WickED “to help me learn”, those who used WickED within a class session or “to get ideas for school questions or projects” are more likely to be happy or very happy with the site than those visiting for fun, to get answers for school questions or had been told to use the site by a teacher/other adult. This is a positive sign that both independent and teacher-led usage can both result in satisfaction among young users.

The site has a broad reach and reveals particular equality among students attending lower decile schools.While a slightly lower proportion of the user audience fits this profile, the gap is far narrower than among Studyit users and lower decile students are equally as satisfied with WickED as their mid and higher decile counterparts. Continued promotion of the site among teachers at lower decile schools is important to help close this gap even further and ensure all students are aware of, and make use of, the site.

WickED is also reaching students who mainly speak Maori at home and 5% of the user base does so. These students likely benefit from the Maori resources which can instill confidence for this demographic group.

While the site is predominantly reaching students who achieve good school results, those faring less well at school are also gaining value from the site – a greater proportion of this group are more positive toward WickED as a fun resource, helping them with their school work, helping them work with other students and most importantly, they are much more positive toward WickED’s facilitation of working independently. Greater promotion of the site targeted at these ‘lower achieving’ students is required to ensure a greater number of them can begin to reap the benefits of WickED.

It is also interesting to note that WickED users are largely in years 6, 7 and 8 and the target audience (7-12 year olds) is underrepresented. The survey has found that those aged 13 and above are more likely to visit the site for school work and to communicate with other students, and together with 11 and 12 year olds they are likely to use WickED in a lesson – indicating that teachers of younger students could be making more use of the site within their lessons.

It is also important to highlight that more than one quarter of students (27%) claim to have only ever used the WickED site, when asked about the range of online resources they have used for their schoolwork. This indicates a need to further promote the exploration of online tools to support student learning. Adults still rate offline resources like teachers, libraries and books above any online tools, though search engines and subject sites are well regarded to help young people learn.

To position WickED as an exciting learning resource, supporting teachers to incorporate the online medium into their lessons, greater promotion and guidance in using the site is required. As mentioned, there is also an opportunity to provide more direction and structure on the site for teachers, to encourage ongoing and regular use in the classroom. Two in five teachers noted a “lack of time for use of WickED at school” as one of the barriers to students using the site.

Expanded role

WickED’s original intent, to support learning in After School Support Centres, appears to have been expanded and now only 7% of users have ever accessed the site from such a centre. The majority of use is at school, during a class, though home-based use is growing and close to half of students have ever accessed WickED at home.

This shift reinforces the need to cater to teachers and lesson-based usage as well as facilitate content for independent use in the home or other location like the school or public library.  The large number of first time users require ‘hand holding’ through the site and potentially the inclusion of a welcome message and introduction to help them understand and explore the site’s content. As discussed, very few of the site’s content areas currently achieve good levels of usage which translates to an inefficient use of resources dedicated to developing, maintaining and hosting such content. Putting in place strategies and tactics to build awareness of the site, help teachers better incorporate the site into their lessons on a regular basis and help students explore and make use of more areas of the site independently will improve the site’s value in three ways:

• Improve the reach of the site to new audiences as well as stimulating greater use among the existing user base
• Expose students to a greater variety of content areas, tools and resources to facilitate a broader scope of learning, eg, social skills and confidence via the forums and sense of worth and achievement via submitting content and seeing it published
• Better support teachers with valuable online resources that can be used during lessons on a regular basis with minimal effort

WickED’s reach and contribution to student learning, outside formal performance measures, should continue to be tracked over time to ensure that the impact of service refinements and marketing efforts are accurately captured and the continued improvement of the site documented.