Evaluation of Student Facing Web-Based Services: WickED (Nielsen/NetRatings)

Immediate Learning For Young People

• In this section, we explore the kinds of learning that are happening through the use of the WickED site – breaking down student learning into the five core areas: Thinking, Making Meaning, Relating to others, Managing Self and Participating and Contributing.
• We also ascertain students’ motivations for using the site and their patterns of usage of the WickED service.

Patterns of usage

Frequency of visitation

A very high proportion of WickED users are visiting for the first time so the challenge is to convert a greater proportion of one-off visitors to begin using the site on an ongoing basis. At present, just over one quarter of students (26.7%) visit WickED once a week or more often. Those who have visited the site more than once are more likely to be:

• More confident in their school work (43% vs 18% of those who are having difficulty at school)
• Those who use WickED during class lessons (41% vs 13%)
• Subsequently, those who have used WickED in the classroom (66% vs 40%)

There is little difference across age and gender.

Frequency of student WickED usage

	All students N=146	Boys N=49	Girls N=97
Today was my first time	60.30%	59.20%	60.80%
Every day	6.20%	8.20%	5.20%
Few times a week	13.70%	14.30%	13.40%
Once a week	6.80%	6.10%	7.20%
Once or twice a month	8.20%	6.10%	9.30%
Less often	4.80%	6.10%	4.10%

 

Regular users of the site (ie not those visiting for the first time) visited the site on average eleven times a month and students from low decile schools visit more frequently than high decile students – 12.9 times a month, compared to 9.3 times for mid decile students and 8.9 times per month for high decile students.

Boys are slightly more likely to use WickED everyday compared to girls – and boys use the site 12.2 times a month compared to 10.3 times for girls.

Website Tenure

Among repeat users of WickED, over one in three students (36.2%) are relatively new users, having been introduced to the website within the past month. Students from low decile schools are more likely to be new users of the site, with 55% using for less than a month.

WickED Tenure 

As seen in the chart below, very few students are long term users.

Interestingly, indications are that boys are more likely to be longer term users of WickED which may correspond with their greater likelihood toward teacher-led use of WickED in the classroom.

WickED Tenure by Gender 

Locations Of Usage Of Wicked

The classroom is the primary location for WickED use, consistent with the WickED team’s initial findings about usage patterns and technical statistics indicating that the peak hours of use for the site are during school hours between 9am and 2pm, with nearly 50% of page impressions occurring during that period.

However, nearly as many students have ever used WickED at home and this correlates with the upward trends in use of the site during the after school period (between 3pm and 4pm) which were noted during 2005.

Far less popular locations of access are libraries and study centres.

As noted earlier in the report, Internet access at study centres and, subsequently, use of WickED at a study centre is very low. There is, however, little variance across school decile with 8.2% of high decile students having ever accessed WickED from a study centre compared with 4.9% of mid decile and 6.5% of low decile students.

Where have you ever used WickED 

Some key differences exist across places of access among boys and girls, and those of various age. Boys are far more likely to access the WickED site in the classroom compared to girls (67.3% vs 41.2%), probably coinciding with the large proportion of 12 year olds boys and teacher-led usage among this age group. Girls are far more likely to have accessed WickED at home (54.6% vs 30.6% of boys). This is consistent with the greater proportion of girls aged under 11 and the finding that 71% of 7-10 year olds have accessed the site from home compared with 35% of 11 and 12 year olds and 42% of those over 13.

Use of WickED in the school library are more common among those aged 11 and over and access from a study centre is greater among those 13 and over (13% vs 5% of 11-12s and 3% of 7-10 year olds).

The chart below looks at the usage of the WickED website within the classroom and school libraries by the time of day. Connection during class time dominates, though there is reasonable use at lunch time and some use in the school library after school hours.

Time of Access by Location

Looking at access at home, not surprisingly, after school or weekend use dominates though those aged 13 or more are the least likely to access the site on the weekend (50% vs 68% of 11-12s and 63% of 7-10s) but more likely to use the site after school. The chart overleaf reveals greater use at home among girls, in particular, weekend use.

Time of Home Access by Gender

Analysis of the site statistics found that visitors are less engaged (i.e. view fewer pages on average) on the weekends than during the week, illustrated in the chart below for a typical month (September 2005).  This may correlate with a different style of usage during lessons than when using the site independently at home, but reveals an opportunity to grow

Source:  AWStats September 2005

Daily engagement analysis

Motivations For Visiting WickED

The table below reveals students’ reasons for using WickED, with notable differences in motivations across the various age groups. Teenagers are more likely to visit the site for school work and to communicate with other students, and together with 11 and 12 year olds they are likely to use WickED in a lesson – indicating that teachers of younger students could be making more use of the site within their lessons.

Younger children use the site for fun but also to help them learn and as seen, there is very little ‘publishing’ to the site – 5% or less.

Reason	7-10 year olds	11-12 year olds	13+
For fun	50%	44%	44%
Help me learn	58%	49%	38%
Answers to school questions	11%	14%	24%
Ideas for school questions / projects	13%	18%	24%
Use in a class lesson	16%	22%	33%
Just curious	11%	19%	22%
Teacher / adult told me to use it	18%	32%	20%
Post schoolwork to the site	3%	5%	4%
Talk / chat with other students	3%	8%	16%

Boys are more likely to use the site for all of the above reasons with the exception of use in a class lesson, for which they are on par with girls. Boys are far more likely to post school work on the site (10% vs 1% of girls) but they are also far more likely to state that they use the site because a teacher / adult simply told them to (33% vs 21% of girls). Regardless, there are still good indications of independent thinking and ‘managing self’ across both girls and boys and those of all ages.

This is also the case among those students who feel they are having difficulty at school. They are actually more likely to use the site to help them learn (56% vs 46%), more likely to post school work to the site (11% vs 3%) and to talk / chat with other students (19% vs 7%) compared with those who are confident in their school work. This reinforces the value of the site among those faring less well at school, as a learning tool not just for curriculum-based topics but for general learning across the core competencies.

WickED Content Consumed

The chart below shows the sections of the WickED website students have ever used. “Playing a game, taking part in a quiz or crossword” is the number one activity for students at 40%, and using “Maths Stuff” comes a close second with 38.4%. This is consistent with the site behavioural statistics where results for September 2005 (chosen as an example) indicated that the areas of the site which generate the most page views are Maths Stuff (25% of page impressions in September 2005), Interactives (24.5%) and also Science Stuff (7.6%). The site tracking does not break down visitors by area of the site, so page impressions have been used as a measure of the popularity of these areas.  It is important to note that site architecture may cause these results to be skewed.

WickED Sections Used

Overall, students have used 2.9 activities on the WickED website which reveals good cross-sectional usage, particularly when taking into account the large proportion of first time visitors. Other comparable metrics include the finding that on average, Studyit visitors make use of 3.2 different subject areas on the site and across the large Australian portal, ninemsn.com.au, the average user visits 2.8 ‘channels’ of the site.

On WickED, girls are more likely to have used the fun, interactive elements of the site while maths is a key focus for boys, with “Maths Stuff” attracting the highest number of boy visitors at 42.9% and 20.4% of boys have also used the “Maths problem solving forum”. Boys are also more likely than girls to submit content to the site such as a video to WickED TV or posted to the student gallery, as seen in the chart below.

WickED Sectional Usage

Students from low decile schools are more likely to play a game or use “Math Stuff” than other students and overall, those at low decile schools are using all sections more than students from high decile schools, the exception being Multichoice Madness, completing a web hunt and submitting content to the site.

	All Students N=146	Low Decile 1-3 N=49	Mid Decile 4-7 N=41	High decile 8-10 N=49
Played a game (maths, science), did a quiz, did a crossword	40.4%	48.4%	46.3%	34.7%
Maths Stuff	38.4%	45.2%	29.3%	38.8%
Used the maths problem solving forum	16.4%	22.6%	7.3%	12.2%
Linked to online resources (in the Information station, Literacy, Maths or Technology section)	14.4%	16.1%	17.1%	8.2%
Watched a video clip on WickEDTV	13.7%	16.1%	9.8%	10.2%
Used the Maori resources	13.7%	19.4%	14.6%	10.2%
Used the literacy activities (Literacy Stuff)	12.3%	12.9%	9.8%	10.2%
Themes (e.g. The World of Flags)	12.3%	9.7%	14.6%	8.2%
Answered the weekly poll	12.3%	16.1%	4.9%	10.2%
Checked out the celebrities area	11.6%	16.1%	2.4%	10.2%
Answered questions in Multi-Choice Madness	11.6%	9.7%	7.3%	14.3%
Looked at the Student Gallery	10.3%	16.1%	7.3%	8.2%
Posted a message or read messages on the Student Forum	6.2%	12.9%	2.4%	4.1%
Completed a Web hunt	6.2%	6.5%	7.3%	2.0%
Submitted a video clip to WickEDTV	4.8%	3.2%	4.9%	4.1%
Posted something on the Student Gallery	4.1%	0.0%	7.3%	0.0%

 
It is interesting to note the relatively little use of the ‘Themes’ (now topics) given the dynamic nature of this content (ie, initiatives to create new and relevant themes each term) and that initial findings by the WickED team had found that interactives and themes were the most useful parts of the site for teaching programmes.

However, use of the themes is higher among those students who have used WickED in a class lesson (26% vs 12% generally) but it still highlights some degree of mismatch between the site managers’ dedication to this content and the levels of use it is attracting. There appears to be an opportunity to further enhance and promote this area of content to teachers to stimulate greater use among their students.

Type Of Learning Encouraged By WickED

Thinking

The games and quizzes available on the site provide opportunities for students to develop thinking skills, and they are well received by users,, particularly younger users. Expanding the sophistication of the gaming platforms will likely increase their popularity and expand usage among the target, further to the comments from the student workshops in August 2005 as well as the spontaneous suggestions for improvements obtained through the online survey.

Making Meaning

The site offers many areas where students can explore different topics either independently or with guided use. These areas include Games (previously Cool Stuff), Information Station, Homework Help and Topics (previously Themes).  These areas of content provide students with resources to support their efforts to discover meaning in ideas, and exploring below the surface of issues.

Aside from the interactive games and Maths Stuff however, there is relatively little use of other content areas and site visitors should be further encouraged to explore multiple categories of the site with further cross promotion. For example, links and information about ‘Information Station’ from within the ‘Games’ category and vice versa. At present, the site relies on users to be pro active in exploring the top navigatio options and often they can be forgotten.

Relating To Others

WickED facilitates student communication via online forums, as found under Homework Talk. This type of function can be key in helping children learn how to interact and communicate with others and Nielsen//NetRatings’ qualitative research has found that services like instant messenger are really helping young people build their social skills and social confidence.

The WickED forums can be used in a similar manner and as mentioned earlier in the report, the challenge lies in encouraging greater use of the forums – usage is low and there is some degree of dissatisfaction or indifference toward the degree to which the site helps students work with other students.

There can be a great hurdle associated with using forums, eg registration and login, and they can be daunting for new users who are unsure of the protocols and processes. While the site offers a clear link ‘How to use Homework Help’, this could be made more fun and enticing with language like ‘New to Homework Help? We’ll get you started’ or similar, in bold and colourful text. Forums can also be an excellent way to encourage repeat and frequent visitation of the site once students become engaged in their interactions and enjoy the experience.

Managing Self

The site encourages students to explore at their own pace, with no specific direction provided for ‘expected’ activities to be performed on site.  Students are thus provided with ample opportunity to take ownership of managing their own time and activities on the site and as seen in earlier results, students are visiting WickED for a variety of reasons, including independent use as well as lesson-based use. There is evidence of ‘managing self’ through pro-active use across all profiles, including those faring less well at school and most students are satisfied with the degree to which the site allows them to work by themselves.

Another broad measure of the extent to which students are using the site to self-manage is their level of engagement. Average visit depth provides an indication of the level of engagement and a good way to evaluate online engagement is to compare growth in unique visitors to growth in web pages consumed.  The average number of pages viewed per visit was 5.6 in 2005, lower than that found for 2004, indicating a slightly shallower visit depth and suggesting that visitors are exploring fewer pages on each visit overall.  As suggested in the interim report, this measure may have been depressed by a high number of visitors dipping in to view quiz results and leaving the site, as indicated by the analysis of pages receiving the highest number of page impressions.  This is likely also related to the high proportion of visits to the site (an average of 62.3% in 2005) which were under 30 seconds. 

While the average time spent by a visitor to the site in 2005 was five minutes and thirty-six seconds, the high proportion of very brief visits indicates room to improve the level of engagement with the site and the degree to which students explore and interact with the site’s content. This can help to increase the value that WickED offers users in all areas of learning – of both a formal and informal nature.

Participating And Contributing

The site provides students with the opportunity to contribute work for display in the ‘Student Gallery’ section, and students are also encouraged to develop and submit video clips.  Several students and school groups have chosen to do so, illustrated in the screen grab below:

Source: Image from student gallery

However, as a proportion of overall page impressions, both of these areas of the site generate small numbers of page impressions – the Student Gallery generated no page impressions in either September or May 2005, and WickED TV generated 1.1% of page impressions in September 2005 and 1.3% in May 2005. 

This is consistent with the survey findings – just 10% of students have visited the Student Gallery, 5% have ever submitted video to WickED and less than 5% cite ‘posting school work to the site’ as a motivation for visiting. As mentioned earlier in the report, boys are far more likely to publish work to the site than girls and efforts should be made to encourage all users of varied profiles and skill sets, to contribute work to the site.

This may begin to increase as online content sharing and ‘self publishing’ are growing areas, with the likes of sites like Flickr (photographs), YouTube (videos) and MySpace (social networking) all facilitating these activities. Barriers such as lack of awareness of the function, or technical issues are priorities to overcome for WickED site managers to ensure the site promotes the core learning areas of participation and contribution.