Evaluation of Student Facing Web-Based Services: Studyit (CORE Education)

Quality of Service Provision

Partnership arrangements and project management

Studyit was conceptualised and developed as a free online resource to support senior secondary students initially in mathematics and science to NCEA level. The project was funded as a Digital Opportunities project and first went live in April 2004.

The design, development and management of the Studyit website was undertaken by CWA New Media, as lead agency in a joint venture with Multiserv Educational Trust (now Cognition Consulting Limited), who initially worked with focus groups of students through the medium of video conferencing and also with visits to schools, to develop a strong and workable concept. The site was designed to reflect what students specifically wanted. This included easy access, no distractions, not too many choices and a site devoid of frills such as unnecessary animations and images. The language used on the site had evolved through careful consideration and discussion by site developers and reflects an attempt by them to avoid the explicit school/teacher/student relationship. The lead development team were keen to ensure the student voice would be heard on the site. This has been very successful.

The Ministry of Education determined the choice of subjects offered as the focus of the original Digital Opportunities projects round was a mathematics and science one. The intention was to eventually develop all areas of NCEA on a gradual basis. Since the data was gathered for this report English has been added as a subject area. Additional Digital Opportunities funding in 2005 was used to support the development of the English area of Studyit. Inclusion of English as a new area and service extension has been successfully achieved.

Teachers moderate the interactive, asynchronous forums, and are required to respond to a student request within 24 hours. Moderators were recruited by a variety of means that included advertisements in the Education Gazette, shoulder tapping and word of mouth. One teacher found the Studyit website by accident, like what she saw and contacted the Studyit coordinator to offer her help. Ministry of Education personnel recommended teachers to moderate the English forums. Quality mentors were sought and are paid on the basis of 6 hours per week. The teachers involved were passionate about the site and the benefits that participation with students in the online Studyit world had to offer.

Evidence from the findings on student online interactions earlier in this report indicates that the conceptualisation of the project around senior secondary students NCEA needs has provided a focus for student involvement that revolves tightly around study and the incentive of internal and external assessment. Studyit is clearly a successful endeavour for both developers and users but the reasons for this success are highly complex and comprise many factors that together make this learning environment work as a viable learning community. Development of the environment based on elements identified by students themselves as being important has been a novel and successful approach. The level of ease of access to the site, and ease at which materials can be located are commendable. Encouragement of student ownership through use of aliases, the transparent approaches utilised in Studyit forums, the quality of the static support content all combine to provide an excellent model for a site such as this.

Knowledge and capabilities of the online mentors

The online mentor teachers contribute significantly to the success of the Studyit project. In interviews the mentors indicated they needed to be confident, and comfortable about being online as well as very knowledgeable about subject matter. Ideally online teachers needed to be teaching NCEA classes because they were often required to be more knowledgeable about various rules, regulations and other aspects of NCEA than many teachers needed to be. The mentor teachers also noted that classroom teaching and online facilitation were very different roles and teacher mentors needed to be able to readily transfer from one environment to the other. Teachers also needed to be able to explain or describe their knowledge via an online text-based messaging which requires a range of different skills to classroom teaching. Also the philosophy underpinning support for students in Studyit was to provide guidance rather than actual answers.

Mentor teachers were experiencing, and having to deal with, a different aspect of student learning in Studyit compared to their classes. Mentors observed that many of the actively participating students tended to be high achievers and there was evidence of a number of senior students  ‘pushing themselves.’ Also it was common for actively participating students to say things about what they were taught in their respective schools but were simply not understanding or not quite ‘getting it.’ Participating students wanted to talk to a science or mathematics teacher to gain clarification or often simply to gain a different perspective on a problem. Mentors also felt it was necessary to respond to students in ways that both supported the student and did not undermine the classroom teacher.

• "I felt I had to be careful in my reply. So that the teacher was supported while the student was given a fair answer. The reason I felt the answer needed care was that I didn’t want a situation where teachers started to see Studyit as a threat to them professionally. (Mentor teacher interview)"

Mentor teachers live in various parts of the country. Although there was an opportunity to get together for a face-to-face meeting one day early on, the mentor teacher group mainly communicated with each other and with the coordinator via email. In view of the role and importance of the mentor teachers to the success of the interactive areas there appears to be some need for more support and preparation for both the online response and support role and developing familiarity with the internet resources that can be recommended to students. Several mentor teachers indicated that they were given or emailed a 6 to 7 page set of guidelines, outlining policy and procedures. Another teacher felt s/he was ‘flying blind initially.’ Another said she ‘brought herself up to speed by doing a lot of googling’. Mentor teachers indicated they would like more support especially with websites they could suggest and more feedback on performance. Another suggested a forum for the mentor teachers as a possible future addition to the site.¹

Evidence of students appreciation of the role of the mentor teachers could be observed on the website and noted scattered throughout the interviews. Students clearly and frequently expressed ‘their thank yous’ to ST (Science teacher) or MT (Maths teacher) in the threads and supported the role of online teachers in the message boards.

How students found working in the online environment

Students reported finding working with the site very easy. Few people encountered troubles with locating and finding what they needed.

The look and feel of the site was considered to be ‘pretty bland’ by one student who added, ‘but appearances aren’t important – its not going to make me visit the site any less’.

Further student comment relating to their views on using the site is listed in Table 26.

Table 26: How students found working in the online environment

General NCEA information provided	The study advice provided on the site was really useful. (Student Interview)
Value of discussion groups/forums	I use this part of the site most as well as the subject content – I print that off at the start of the topic (Student Interview)
Up to date information	its quite relevant as it’s got the achievement standards etc (Student Interview)
Ease of use of the site	The site is easy to use - user friendly and cuts the unwanted things. (Student Interview) Very easy. I have no trouble with it (Student Interview) Well I love the layout and easy to use navigation of the website. Unlike the NZQA site you can find what you want quickly and also the forum helps a lot. (Student Interview)
Responsiveness	The teachers are so fast in their replies. I am very grateful to have heard about this site. (Student Interview)

The majority of students appeared to be accessing Studyit from their home computers. There are no indications from the evidence that any students were encountering issues or problems with accessing the site or with slow download times. Students indicated they were mainly using the site as part of a repertoire of study resources that included textbooks, study guides and homework or assigned schoolwork. When they encountered a problem they would simply hop onto Studyit. It was like having a teacher handy to answer those quick and often unsolvable questions. The purpose of the site for students was strictly for just in time practical study advice, support and feedback. The site appeared to be well designed for students to achieve these aims, quickly and effectively.

One ‘gripe’ that did emerge in student interviews was the tendency to misplace requests by putting them in the wrong forum area. One student said,

• "I hate how some people just add random questions in the forum when they could have put their question in the thread already dedicated to the topic. It just clutters the discussion board. (Student interview)"

The software used for the forum areas of the site works only in linear mode, that is messages are viewed as they are sent. There appears to be a lack of ability by the forum software being used to allow for finely threaded discussions so that students can respond to a message that is several days old and place their new message close to that discussion or the ability of the teacher mentors to move messages that are wrongly placed to a better forum. Some forum softwares allow for students to make personal preferences about how they view the messages. They can change the forum-viewing mode from linear to threaded, and some offer both options as a hybrid viewing mode providing an index of the forum message list. Teachers also mentioned the need to have the latest messages appear at the top of the message list. The possibilities of linking to a social book marking site such as del.icio.us would also be an additional feature worth consideration for Studyit allowing students to connect to a broader range of interactivity that can ‘serve to sharpen and extend thinking and pique intellectual curiosity’ as well as ‘help each other to do better’.(Coogan 2006 p20).

Overall the research team presenting this report considered the tightly focussed nature of discussions in the forums to involve a greater level of in-depth and well ordered thinking than that often found elsewhere including well-focussed adult sites.

Operational characteristics

Studyit operates as an asynchronous student help site for students who have access to the internet. Both static and interactive forums are available twenty four hours a day, seven days a week. Teachers noted that support for students was generally outside of school hours. Students usually posted requests from after school until 10.30pm and then before school in the morning.

Mentor teachers generally worked within the area of subject expertise although there were some who also provided support for the ‘other’ forum offering support for study skills and NCEA information as well.

There were a far greater number of registered users who looked at the material on the site but who never actively participated in the forums. This is considered to be quite usual practice for online interactive forum sites.²

At the time of the data gathering for this report there were 7 mentors operating in the site³. Currently the Studyit site offers support for Mathematics, the Sciences and English and the number of mentors has been increased. There is a desire to expand the site so that other NCEA curriculum areas can also be offered. Extending the scope of Study raises issues for expanding the number of mentors and how they should be trained, as well as the impacts on organisation of the site that comes with expansion.