Evaluation of Student Facing Web-Based Services: Studyit (Nielsen/NetRatings)

Learning For Teachers And Other Adults

While the core target audience for use of the Studyit service is NCEA students; parents and teachers also play a role interacting with the resource and in influencing and guiding students’ use of the tool.

As a component of evaluating the impact of Studyit on learning outcomes, research was also conducted amongst adult users. This element of the evaluation aims to provide some insight into the impact of Studyit on school and teacher practices. It also explores how Studyit is positioned as a learning environment to students by teachers and parents.

The majority of visitation to the Studyit site is driven from student usage, however, adults comprise over one quarter of visitors, representing a significant portion of all site users. The sample composition observed is outlined in the table below and shows that a total of 131 adults responded to the survey overall - 62.6% of the adults were teachers and 25% were parents or guardians and the remainder tutors or other supervising adults.





Teachers visiting Studyit are primarily those working with students at the NCEA level; half teaching years 11-13 and only 13% are teaching year 7 and 8. On average, adult visitors are spending approximately four hours accessing the Internet each week (3.98 hours); teachers are accessing the Internet for marginally less than the average time at 3.70 hours a week and parents are using it more at 4.83 hours a week. Regardless, the time spent online among adults is far lower than that for students (7.9 hours per week, average).

Two thirds of teachers report using the Internet with students for less than four hours in a given week which is consistent with the low level of school usage reported by students. Seventeen per cent do not spend anytime online with students.

Awareness And Patterns Of Usage Of Studyit

Adults are primarily initially finding the Studyit site through Internet searches – indicating that they are proactively searching for study resources using search engines (38.5%) rather than being prompted to the service via marketing.  Approximately one quarter of adults find out about Studyit from a teacher (or another teacher) and only 15% from promotional material. Recommendation from teachers to other adults is far lower than it is between teachers and students (27% vs. 49%). There may be more opportunities to encourage teachers to “spread the word” among peers and parents.

Parents are more likely than teachers to have proactively sought out Studyit with almost half of parents finding the site via an Internet search (45.5%) compared to only one third of teachers (33.3%).

Over half of all adults using the Studyit website are new users, with just 22.9% using for longer than a year. This compares with 36% of students who have been using the site for less than a month and 26% for more than a year.

Teachers are more likely to be long term users with 31.7% using for a year or more while a very high proportion of parents (78.8%) have only started using Studyit in the past month.

With students leading the tenure of usage, followed by teachers, this indicates that while many parents are actively engaged in the search for study resources for their children, they are less likely to continue using the site for extended periods of time and typically refer their child to the site then discontinue use.

Recommendation Of Resources To Students

Adults are more likely to recommend a traditional resource such as the teacher or a book to students looking for help with school work or exam preparation.

Specialist subject websites or an Internet search engine, such as Google, are the next tier of resources recommended to students at 70.0% and 68.7% while it is interesting to note that student online forums / chat rooms receive far less recommendation (15% of adults) likely due to perceptions of online dangers associated with online contact with ‘strangers’.  This is an important finding for the way in which Studyit is positioned. Too much emphasis on the forums may have a negative impact on referral unless the value and safety of this online tool is also communicated in promotions and marketing. Education programmes for teachers and parents are also a consideration to boost their level of confidence in online forums and online interaction.





Adults were also asked to rate individual resources based on their importance in helping students locate quality information. The adults where asked to rate each resource on a scale from one to five, where a score of one indicated the resource was of not important at all and a score of five was very important. Specialist subject websites achieved the highest importance mean score from teachers (4.32), followed by books with a score of 4.26.

Not surprisingly, parents rated ‘teachers’ as the most important resource available to students with a mean importance score of 4.29.

Further charts highlighting perceived importance scores for each resource can be found in the appendices.

Perceptions Of Quality Of Service Of Studyit

The majority of adult users are satisfied with the Studyit website which is likely one of the reasons behind their strong recommendation of the site to students (refer section below). In fact, proportionately more adults are ‘very satisfied’ with the site than are students (37% vs 28%).





One in ten adults are dissatisfied with Studyit, though overall dissatisfaction is lower amongst teachers (7.3%) and slightly higher amongst parents and guardians (12.1%). While drivers of dissatisfaction have not been measured quantitatively, some of the comments among this group indicate that the minority of adults that report some level of dissatisfaction indicated qualitatively that the inclusion of past exam papers for practice purposes would improve the site and many commented on the need to “advertise” the site more widely to increase awareness.
 
The table below lists the areas in which teachers feel the Studyit website is performing well. Teachers feel the site is useful and informative, helping students with key skills and techniques. More than one in ten teachers also feel the Studyit website is a good teacher aid or resource.

	Teachers	N=
Useful site, informative	22.2%	12
Highlights key skills, techniques and achievement objectives	16.7%	9
Subject specific information	13.0%	7
Teacher aid, resource	11.1%	6
Has exam advice & examples	11.1%	6
A good site	11.1%	6
A forum for students	9.3%	5
Good links to other sites	9.3%	5
Available outside school hours	9.3%	5
A one stop shop for students	7.4%	4
Easy to navigate	7.4%	4
Good quality content	5.6%	3
NCEA related, linked	3.7%	2
Reinforces information from teacher	3.7%	2
Parents have a resource students	3.7%	2
Ability to contact experts	3.7%	2

 

On the other hand, some teachers expressed worries regarding the links to other websites; that they were not regularly updated (16.2%). 13.5% of teachers thought that there was a need to increase the awareness to all potential users. They also thought the inclusion of more examples would be useful (13.5%).

Parents and guardians thought the website was performing well in providing a forum for students (18.5%), providing students with exam examples (14.8%) and advice and many like the fact it is linked to NCEA objectives (14.8%).

As with teachers, parents felt that the Studyit website should try and increase its awareness. The inclusion of more examples, and revision tasks for students to use was also highlighted, as was the possible increase in number of subjects available on the website.

Recommendation And Positioning

Almost all adults (91.6%) have recommended the Studyit website to students, increasing to 93.9% among just teachers.

The table below shows the length of time adults have been recommending the Studyit website to students based on the length of time they have actually been using the site.

	All adults N=131	Used Studyit  for less than one month N=60	Used Studyit  for 1-6 months N=15	Used Studyit  for 6-12 months N=15	Used Studyit  for longer than 1 year N=30
Less than one month	51.7%	95.0%	6.7%	13.3%	6.7%
1-6 months	19.2%		86.7%	46.7%	3.3%
6-12 months	8.3%			40.0%	10.0%
Longer than 1 year	20.8%				80.0%

It appears that the majority of adults, who become aware of Studyit, recommend the website immediately to students, with 86.7% of 1-6 months adult users of the site recommending the site within the same time frame.

On average adults who recommend the site to students are giving them three reasons for using the site (2.86).

Studyit is commonly perceived as being a service for general learning or for study skills, study/assignment help or exam tips.  Far fewer position Studyit to students as a place to communicate with other students – an element of the site that is unique in nature but which also has negative connotations for some – in relation to safety online.

Perceived Barriers To Use

Adults felt the main barriers to students using the Studyit website was a lack of access to a computer at home (57.3%) and the speed of connection at home (45.0%). Analysis of students’ survey results has found, however, no differences in patterns of usage nor perceptions of the service between those with dial up versus broadband Internet connections at home.

The amount of time students  have to use Studyit (42.0%) and the access to a computer or the Internet at school was also felt to be a barrier among more than four out of ten adults surveyed (41.2%).

Students’ apathy with using resources such as Studyit was also listed by 40.5% of adults; teachers rated this higher than parents and guardians – 43.9% compared to 36.4%. This is a key element relating to the core skill of ‘managing self’ and greater encouragement should be given to students to make use of such resources. Particular emphasis should be placed on students attending lower decile schools to stimulate greater use and reduce the under-representation currently seen among this demographic segment. This group may correlate with lower academic achievers and those with less access to online resources, but those who would really benefit from both the formal content and informal learning that can take place on the site.