Main heading

Evaluation of Student Facing Web-Based Services: Studyit (CORE Education)

The document provides a final service report on the Studyit website as part of a larger evaluation of web-based learning services for children and young people in New Zealand. The report is complemented by similar reports relating to the AnyQuestions and WickED websites.

Author: Ann Trewern & Derek Wenmoth
Date Published: August 2008



Immediate Learning for Young People

From the evidence provided in earlier sections of this paper it is clear that Studyit provides a valued source of immediate learning for those 15 – 18 year old NCEA students who are actively involved. There is likely to be immediate learning value for those students who are registered and who use the site but do not actively participate. This group were not able to be included in this investigation. While the main focus of Studyit is to provide support for students in the Science and Mathematics curriculum areas and also in English, there is evidence of students finding support across all essential learning areas as well as study skills support and help with understanding the rules and regulations around NCEA.

The Studyit learning environment is a visually simple, no frills, resource that invites students to seek help by posting requests to the interactive forums and also to check out a wide range of static information resources. The role of the interactive forums in student learning has been the primary focus of this report. The interactive forums are based on a linear message board system. Messages are posted in the order that they are received and can only be viewed in that order. There is no opportunity for students to go back and add a comment some time later and nest the message in a previous thread. This means that threads are quickly started and completed within short timeframes of several days at the most, which maybe ideal for the age group. It also encourages students to post their request without searching for possible answers that may have already been given to the same or similar questions.

Investigation into ‘what learning is happening’ for students on the Studyit website has revealed that for maths there is a higher number of closed questions and for science in particular there is a high number of open questions (see Figure 1 and Table 3) While mentor teachers were quite adept at guiding students towards solving their problems themselves rather than providing a direct answer there appears to be some difference in the pedagogical approaches utilised between the two subject areas investigated for this report. There also appears to be a greater expectation among student users that they may need to follow up leads for inquiry rather than be given a direct answer. The data revealed that students were tending to work at higher levels of the personal inquiry continuum in Studyit (see Figure 5) compared to, for example AnyQuestions. There are a high number of messages and threads where organising, making sense, reflecting and communicating what has been learned, by suggesting a different example, can be observed.

A particularly interesting aspect of Studyit is the in-depth questioning and thick discourse observable in the forums that was analysed using the Student Observed Learning Outcomes (SOLO) taxonomy. That forty-two percent of the interactions (see Figure 4) could be classified as being at the ‘deeper level’ of understanding is indicative that the quality of learning on the Studyit website can generally be seen to be very high.

Students most value the forums because it is here they can get specific feedback related to their particular issues or learning needs. The findings indicate this with more than half of student inquiries directly related to science and maths content and nearly 20% to improving study skills. Specificity is very important to student learning and is a prime reason for the success of the site with its committed user group. Over three quarters of the messages analysed indicated a direct link to schoolwork. Nearly two thirds of threads indicated students returned to check for replies and added further questions or comment. Over three quarters of threads analysed remained completely focused on the designated topic.

Table 27: Student interview comment about the immediate value of Studyit for learning

Value of Studyit compared to the teacher(that is the value of the forums) Although you get useful verbal information from a teacher, on Studyit is written down you can access it any time and for each subject there is a list of everything you need to know for each topic in a subject. (Student interview). Studyit helped me a lot because at home I cannot ask my teacher what the answer to a question is. So its like I have a teacher I can ask questions to anytime. (Student interview).
Value of Studyit compared to books and study-notes (That is the value of the notes and information available on Studyit.) On study it teachers can answer specific questions that you need to know and if you do not understand something in the book or your study notes then you can directly ask one of the teachers in the forum. (Student interview) For my subject, English, It is as useful as the study guides. (Student interview) .  Its better - as most books only tell you the answer and not how to get there (Student interview).
Keeping students focussed in studies Study fits in with my overall studies quite well. I have developed a routine when I get stuck I either ask my teachers or the teachers on the site. Most of the time it’s the site teachers. (Student interview).
There is no cost to students Study it has the particular advantage of no cost to students as long as they have internet at home (Student interview)
Real value in Maths and physics I mostly use it for Maths and Physics as those are the subjects with specific questions that require working. With biology I can just read over notes and make the links on my own.
Value of discussion groups/forums I use this part of the site most as well as the subject content – I print that off at the start of the topic
Allows students to feel they can keep up with their studies It’s been pretty good to me. When I hit a rock in my exercises, I am delayed momentarily. It prevents me from getting too far behind in my classes.
Belief that Studyit is making a difference If I don’t understand something the teachers or other members of the site help me…Just generally study it tells you what you need to know and I believe that it really did have an effect on my results at the end of the year.

Students believed that Studyit helped them with their study in the following ways,
  • The study advice provided on the website
  • The relevancy of subject information to what students were asked to do at school
  • The value of the support students had received out of the discussion forums which kept student focussed
  • The ‘up-to date-ness’ or currency of the information available on the site
  • Helping to keep students focussed and motivated
  • A New Zealand place for students to talk and share their doubts and solutions.

The teacher mentors are highly visible on the site. Nearly one third of all threads contain messages where a student interacts with an expert teacher. However a closer analysis reveals that two thirds of all threads involve three or more people in interaction and a staggering 14% of threads involved 11 or more participants. This is indicative of a particularly high level of commitment by the core group who are using the site.

At first glance the student users do not appear use the site as a communal social space and yet a great deal of empathy and support focussing on NCEA issues and the emotional issues encountered by many exam students who have high expectations is observed to be peppered throughout the site. The researchers also noted quite a change of dialogue type once examinations were over and a quite different, almost euphoric feeling prevailed. The nature of interaction became far more fluid, reflective, meta-cognitive and included evolving conversations about, for example, approaches to study for next year.

One of the unique features of the Studyit site is the amount of support the site provides with both teacher experts and students who volunteer support for incoming requests for help. Also unique is the unusual level of equality that appears to exist between the teachers and the students on the site. There are a number of examples of students challenging the answers provided by teachers. Both students and teachers operate under aliases. Use of aliases may account for the appearance of a lack of hierarchy in the environment and may also be responsible for a unique level of openness from students in their postings.

Although teacher-mentor’s mentioned that students involved in the site varied in ability there appears to be a large number of high ability and high achieving students involved who worry a lot about excellence grades and scholarship passes. Mentor impressions of student participants are listed in Table 28.

Table 28: Mentor impressions of the core group of Studyit participants

Range of abilities… The students range of ability varies from low ability to very high ability where these high ability students do and are encouraged to help the low ability ones
… but predominantly those with higher ability concerned to do better… Many of these students seem to be the more able students. They worry about excellence grades and scholarship passes.
… or international students needing extra help. Many of the students appear to be enthusiastic about their studies. Some of the students appear to be international students needing extra help. 

Although no firm data was gathered to support this, the general impression of the research team is that those making most use of the site appear to be higher achievers (in the merit/excellence category), students who are intrinsically motivated to do well, and who accept they have a personal responsibility for their own learning. In many ways the site could be seen as catering for those who have least need of it but are most motivated to use it. The very active students are encouraged to help the less able and clearly do so.

Many of these students also appeared through the observable discourse to be highly engaged in their studies and in some of the concepts they were mentally playing with. An excellent example of this is illustrated in Table 13, on p.17 where ‘Izy’ shares a wondering with ‘Scienceteacher3’

Students indicated they visited the site daily especially around exam time, or when students did not understand something from their class work. Studyit was considered to be as much a part of the repertoire of accessible examination resources as books study notes, teachers and friends.


 

Back to top of page.