Executive Summary

This report is the latest in a series of reports focused on the NEN Trial and developed to inform the ongoing design and implementation of the Network for Learning. Data regarding the NEN Trial were collected over an 18-month period commencing in March 2011. The final surveys were implemented in July/August 2012.

The theoretical framework for this report centres on realizing the potential of digital technologies to transform the learning experiences of children; to move our schools into a 21st century, networked paradigm of operation. It is argued that the benefits of an initiative such as the Network for Learning will only be fully realised when a network of learners exists; when individuals, classes and schools are collaborating and interacting beyond the four walls of their classrooms. It is acknowledged that the extent of transformation will depend on the starting point; that what is innovative in one classroom may be business as usual in another. The high standard deviations reported in the 2011 and 2012 surveys support this.

The NEN Trial

The NEN Trial was initially implemented in 2008 with 21 Proof of Concept schools. It was then extended to 102 schools. The additional schools came from three regions: Christchurch, Nelson/Marlborough and Ashburton. The latter group of schools were the "greenfields" loop as both Christchurch and Nelson/Marlborough were already part of established regional loops. Also an established loop was the Wellington loop, whose schools were part of the Proof of Concept group.

The objectives of the NEN Trial were to better understand the policy, educational, technical and commercial issues related to the implementation of a national network for schools. Included in this was the potential transformational impact on teaching and learning practices. In this way the NEN Trial was an ideal test bed for the Network for Learning¹. This report has been written with this purpose in mind - to inform the development of the Network for Learning.

Methodology

The data reported here were gathered through online surveys conducted in July 2011 and again in July 2012. Two sets of surveys were used: a Principal Survey and a Teacher Survey. All schools participating in the NEN Trial were sent links to both surveys and asked to participate. Regional support providers were also asked to encourage schools to participate.

In addition, two of the regional support providers were interviewed to discuss the key findings from the surveys and to gather their perspective on the success - or otherwise - of the NEN Trial. Their comments have been taken into account in the discussion section of this report.

Participants

There were 44 principal respondents and 209 teacher respondents in 2012. This was less than for the 2011 surveys. Many of the 2012 respondents were different people from those who completed the survey in 2011. This meant only a very small sample of each could be used to measure changes over time. The matched sample of teachers for 2011/2012 was 53 respondents; the matched sample of principals was 26 respondents.
The demographics of the 2012 samples and the 2011/2012 matched samples were broadly representative of the NEN Trial schools.

Survey design

The surveys used in 2012 were more comprehensive than those used in 2011. The three major scales, related to teaching and learning, remained the same enabling a measure of change over time. Also the same were items related to infrastructure, expected and reported impact and teacher confidence. There were a number of additional items in the 2012 survey related specifically to the impact of the NEN Trial, the professional development provided and the attitudes and values of respondents.

The response scales, in both surveys, were mostly six-point scales. In general terms they followed the same pattern e.g. 1 = not at all, 2 = very slightly, 3 = slightly, 4 = moderately, 5 = strongly, 6 =very strongly. The major exception was the response scale used to measure the use of digital technologies in teaching and learning which was specifically designed and ranged from awareness through to regular use.

Findings

Following are the key findings from the body of this report. They are reported thematically and largely according to the different sections of the report.

Attitudes towards digital technologies

The respondents to both surveys, on average, reported that digital technologies are important priorities, even when compared to other competing priorities for limited resources and funding. The data suggest that the value and importance of digital technologies is cumulative across a range of purposes. However, the high standard deviations across all items would suggest that there is a wide variation in attitude, something that was observed in schools visited during the NEN trial evaluation.

Respondents did report that teacher access was more important than student, in terms of both the Internet and devices. This suggests there is still a teacher-centric view in these schools. However, comments made by principals with regard to Bring Your Own Device (BYOD) and wireless connectivity suggest that they are now moving to a greater focus on student access and use. The question of which is more important - teacher or student use - is perhaps a moot one. Ideally both would have access. However, increasingly the literature suggests that the potential of digital technologies will not be realised until the technology is in the hands of students².  A positive from this study is that schools do seem to be focussing on student access now. The quality of that use is still to be considered.

Further, the use of digital technologies was seen as supporting a range of outcomes rather than being used solely for a particular purpose. Though overall levels of use for different outcomes were low, it is likely that the cumulative use of technologies - in some instances- is high. The data suggest that teachers tend to either use digital technologies or not; that the level of use is reasonably consistent across a range of outcomes.

ICT Infrastructure

The levels of reported satisfaction with school infrastructure are a concern with teachers reporting that they were only slightly to moderately satisfied with the infrastructure in their school. The means ranged from 3.03 to 4.24. This represented a slight trend towards an increase in satisfaction over the reported figures in 2011. The one area where a statistically significant increase was reported in satisfaction was with student access to the Internet. This may be due to the installation of wireless networks in a number of schools, and other improvements to school networks.

The NEN Trial provided access to ultrafast broadband to the gate and upgraded school networks where necessary. However, SNUP (School Network Upgrade Programme)³ does not necessarily provide for ubiquitous and normalized student access to the Internet. Nor does either initiative provide for student access to the devices they need. Schools still need to enable student access and this is costly.  These were the areas where teachers were least satisfied. The emphasis on wireless and BYOD in the principals' comments would suggest an awareness that student access is an issue, one which is heightened in a networked paradigm.

Student devices are paid for either by schools or by parents (BYOD schemes) or a combination of both. In some instances, schools are able to access external funding grants⁴ for technology but this is not likely to be common. Others are likely to rely on the fundraising of parent organisations within their communities. It is not the place of this report to discuss this issue in detail and a description of current school funding models is beyond our brief.  Suffice to say there is anecdotal evidence that the funding of technology is an issue for schools and may well remain an issue for some time, as schools grapple with the issues of equity and responsibility related to BYOD programmes. A recent publication by Lee and Levins (2012)⁵ highlights the many variations of BYOT  and the potential for home - school collaboration to improve student access.  They suggest that BYOT is a way of pooling home and school resources but also acknowledge that schools will have to ensure those from less affluent homes are supported. It also highlights the extent to which those schools that have normalised the digital have some form of BYOD in place.

The impact of the NEN Trial

The findings from the 2012 surveys suggest that the NEN Trial, at best, has had a moderate impact on the schools that participated.  The principals reported a slightly higher level of impact than the teachers. Over both surveys, the highest reported impact was for the professional learning opportunities provided for teachers (xˉ =4.34) as reported by the principals.

For both groups, the reported impact in 2012 did not reach the expected levels reported in 2011. In a number of areas the reported achieved levels were statistically, significantly lower than what had been expected. Further, there does not appear to be a strong correlation between what was expected in 2011 and what was reported in 2012. It would seem that the respondents expected things of the NEN Trial that were not implemented. Why this is so needs to be considered. There are a number of possible explanations: their expectations were too high; they did not realise what other capabilities were needed to achieve the desired impact; the implementation of the NEN Trial was not designed to meet, or was inadequate to meet, their expectations.

The lowest areas of reported impact (slight), in both surveys, were related to interaction and collaboration outside the school community. These were also the areas where there was the largest difference between expected impact (2011) and reported impact (2012).  This is concerning if one takes the view that a National Education Network should be enabling more interaction beyond the individual school. It would appear that the schools expected something more than did occur. The NEN Trial, through the Kiwi Advanced Research and Education Network (KAREN)⁶ provided the capability for greater communication between schools and other members, but this does not appear to have occurred.

In addition to enabling changes to teaching and learning the Network for Learning is expected to allow for greater efficiencies for schools and to lower operational costs. However, the data from the NEN Trial suggest that, at least initially, this may not be the case:

• Nearly half of the teachers (49%) and 61% of the principals reported that teachers' professional workloads had increased as a result of the NEN Trial. The results were not as pronounced for administrative workloads where 37% of teachers and 24% of principals reported an increase. Very few teachers or principals reported a decrease in either administrative or professional workloads with most of those not reporting an increase reporting no change.
• Similar results were reported for the financial impact of the NEN Trial. Only 5% of principals reported the financial cost of operating their school had decreased. The rest were evenly divided between an increase and no change. Just over half of the principals (55%) reported that the cost of providing professional development to their staff had increased. Only 8% reported it had decreased.

Professional development and learning

The professional development opportunities provided through the regional support were reported, by the principals, as having the most impact on the participant schools, although it was only moderate. Further, those respondents (n=24) who completed questions related to NEN Trial professional development reported being highly satisfied overall with what they had received.

The data from the 2012 Principal Survey suggests that a range of professional development was offered across the trial. The low number of schools reporting no professional development in each of the areas suggests this was true across the three regional loops. Interestingly, there was no consistent reporting of professional development by loop in terms of those participating. It is likely that school size and location have an impact on the number of teachers able to participate.

Of note, is that less than half of the principals reported that all their teachers attended professional development related to the use of digital technologies in the classroom or to foster collaboration beyond the classroom. This means that in a number of schools the diffusion of any innovation or learning is dependent on internal collaboration and interaction networks. Of the teacher respondents, 73% reported they had received professional development related to the integration of digital technologies in their classrooms. This compares with only 35% reporting professional development related to collaboration outside the classroom and 22% reporting professional development related to their students using web 2.0 technologies.  This is likely to be at least part of the explanation for the low levels of impact of the NEN Trial. The technical features of the NEN Trial are related to greater access to the world beyond the school. The professional development appears to be focused on use within the school.

One could argue a sequenced approach to professional learning - it begins in the classroom then moves to a more networked focus. The problem with this approach is that a networked approach is about a shift in paradigm and definitions of learning and school; not about skills in using technology. Showing teachers how to use technology within their classroom will not shift their paradigm or view of schooling. Most teachers are already successful users of technology in their professional practices - it is just not as apparent in student learning⁷.

Also of note, is that the majority of the professional development these teachers report receiving has been informal. Further any workshops have tended to be led by colleagues from their school. Just over half of the teachers (56%) reported they had attended a session led by an external facilitator while 91% reported they had attended a session led by a member of their staff. While this can be seen as a positive, one needs to be cautious about the extent to which internal professional development fosters the necessary challenge and critique needed to change practice. If the internal professional learning is disseminating new knowledge, sharing and building on the learnings of others, then it can be powerful. However, too often such internal communities can reinforce and support current practice rather than supporting the creation and sharing of knowledge⁸.

Expertise and Confidence

Related to the provision of professional development is the extent to which teacher expertise and confidence is raised. In 2012 the teachers did not report particularly high levels of confidence and expertise. The area they were most confident in was their ability to learn how to use new technologies and applications (xˉ  = 4.76). The areas they were least confident in were related to the use of digital technologies to extend the learning experiences of students and to collaborate with others (xˉ  = 3.76; xˉ  =3.85 respectively). They were always less sure of their ability to facilitate student use than to use something themselves.

Interestingly, given the reported impact of the professional development there were no marked increases in reported levels of confidence or expertise across the matched sample of teachers. The statements used in the survey were very specific; they related to using digital tools to support student learning. It may be that teachers are confident in their ability to use something (skills based professional development), know how tools can be integrated into teaching and learning (pedagogical professional development), but are not sure about how to actually do these things in their classrooms in ways that will promote learning.

Teachers were also asked about the confidence and expertise of their students. Anecdotally, a number of teachers have commented in the past that students are not as expert on computers as they expect them to be and that they need a lot of support to be able to use them. This is interesting and needs further consideration given the use of digital technologies by the young in their lives outside school.

The data reported here support the anecdotal evidence in that the teachers report that it is between slightly and moderately accurate that their students are able to effectively use digital technologies. The areas where they report the lowest mean levels are for collaboration with others. Given the extent to which young people use technology to communicate and interact outside school this seems surprising.

It may be that teachers are judging their students' abilities to use technology by their view of how they should be used. That is they are focussing on the process and method of use rather than the outcome of use.  Outside school, most students are intuitive users of technology: they make the tool do want they want it to: they learn to do something when they need to and know how to get the outcome they want⁹ This may not sit easily with the more structured approach to learning in a school, or to the need of many teachers to control not only what is done but how it is done.  Nor does it sit easily with notions of there being a "right" way of doing things.

Changes in the use of digital technologies in classrooms

The data presented in this report suggest the influence of the NEN Trial on participating schools has been minimal in terms of changes to teaching and learning. While there is a statistically significant increase in student use of technology across a range of outcomes, as reported in the student outcomes scale, the overall levels of use are low.   Some possible reasons for this are discussed in the report.

Comparison between the 2011 and 2012 results show that for the other scales utilised there was a trend towards increases in use across all items but that this was not statistically significant for any of them. Further, the increase was small in all instances at both an item and component level.

In the eLearning scale (Principal survey) the Collaboration component mean remained the lowest and was statistically, significantly lower than that for either Culture or Capability. Across all the individual items in this scale the largest mean increase was only 0.37, and was related to the involvement of teachers in professional development.

Similarly, in the Teacher survey the mean level of use of digital technologies to Create artefacts remained higher than their use to Collaborate. The comparison between 2011 and 2012 for the matched sample showed an increase in the mean level of Collaboration, which approached statistical significance. The increase for the Create component was very small. The largest mean increase for any single item was still only 0.62 and was for the creation of artefacts in a collaborative/interactive environment with other teachers. None of the item increases were statistically significant. The implication is that teachers are starting to use technology in ways that extend the classroom context but only slightly.

The use of technology by students for particular outcomes, as evidenced in the student outcome scale, increased between 2011 and 2012 and in most instances this was statistically significant.  This finding suggests that there has been an increase in student use of technology within the classrooms of the respondent teachers. This does seem to contradict earlier findings and it is worth considering this difference further in future research. Perhaps the nature of learning in classrooms is changing?

However, the levels of reported use in 2012 still did not reach the levels indicated by 2011 respondents as their preferred level of use.  In both 2011 and 2012 the preferred level of use was statistically, significantly lower than the reported current level of use. This suggests teachers would like to see their students use technology more often but do not seem to be able to implement it.

This is interesting in that teachers are the major determiners of what occurs in their classroom. They largely decide the "how" of learning. Presumably, if they wanted their students to do more with digital technologies they could ensure this occurred? One explanation is that student access to technology remains a key barrier. The other explanation is that while in theory teachers say they want to do more this is not the case in practice. That is their espoused and actual theories of practice are quite different. It may be that their preferred levels are in fact what they see as "ideal" practice and not necessarily something that is possible¹⁰. While it could be argued that more time and/or greater responsiveness are needed it could also be argued that the integration of digital technologies into schools has been a policy and professional development focus for many years. It may be time to reconsider the nature of that professional learning and to genuinely reflect on the efficacy of what has occurred in the past.

Discussion

The NEN Trial evaluation was over a one-year period and as such one could argue there has been insufficient time for real change to occur. However, it needs to be remembered that many of these schools are part of well-established regional loops; they have been involved in ICTPD in the past and were presumably ready to embrace the technical capability provided by the NEN Trial.

The findings summarised above suggest that despite the access to ultra fast broadband and professional support little has changed in the schools, particularly with regard to collaboration and interaction beyond the school gates. In all sections of the report whether reporting impact, professional support or confidence, the lowest means were reported for the use of web 2.0 technologies and collaboration beyond the school. At an implementation level the NEN Trial appears to have remained focussed on the classroom. It has not been implemented within a networked paradigm.

Further, teacher use and teacher needs were always emphasised over those of students. The implication is that our education system is still intrinsically teacher-centric. The focus has been on providing the tools to the teacher first - and only now are student needs coming to the fore. This is, perhaps, understandable but it does highlight another shift for schools to make along with that from paper-based to networked. Concurrently, they need to move from being teacher-centric to student-centric; from focussing on technology for teachers to ensuring students have the technology they need. They need to also think about the world beyond the school gate and what it can offer students.

Perhaps the key finding from this report is the need for those implementing and designing the Network for Learning to showcase the benefits of being networked, of utilising the Internet to extend the learning experiences of students beyond the classroom. The real potential of digital technologies to transform teaching and learning lies in their ability to enable us to reframe our notions of learning as something that occurs in a classroom, with a teacher, to a timetable. Currently they appear to be mainly used to supplement or enhance what is being done. In many instances, they do this well, but there is a question of value for money in terms of the investment in both infrastructure and professional development. Are there sufficient benefits to warrant the cost expended?

Creating a technical network is relatively easy; creating a network of learners, a learning environment that exists beyond the school is much more difficult. This requires a philosophical shift in the thinking of all those involved in education. It requires the shift to occur before the initiative is implemented - for people at all levels of education to be thinking in a networked paradigm, to be willing to look beyond the classroom, to take risks and to look for new solutions to old problems.  Then digital technologies become part of the solution; they have a purpose and are used in ways that realise the benefits they offer.

Student access to the Internet and devices is currently problematic for most schools. It would appear that for schools wanting to maximise the potential of digital technologies and the networked world wireless and BYOD will have to be a primary focus. The findings in this report would suggest that many have already realised this. But schools need support to make the right decisions and they need help in ensuring the potential benefits are realised. The solution to this may reside in the need for schools and the government to be more explicit about whose responsibility it is to provide which aspects of the technology. If the Network for Learning can save money on content, services and access then that will free up school budgets for the necessary devices.

More problematic, arguably, is that digital technologies are largely being integrated into a 20th century notion of teaching and learning. When a school is digital it is essentially using technologies within a paper-based paradigm. They support what is happening in the classroom, they make administrative tasks easier for teachers, they make the access to information easier for both teachers and students; they enable multiple ways to present information and to synthesise what has been learned. But these uses do not transform the way students learn or what they learn. That requires a conscious decision by schools and teachers to do things differently.

Transformation of teaching and learning will not occur through the provision of technical and pedagogical capability alone. It is necessary to transform mindsets - to provide a new philosophy and paradigm within which people think. Schools and their communities need to be shown what is possible, how to utilise what is beyond the school gate and how to ensure that translates into outcomes for students.

The questions this raises are:

• How prepared are our schools and teachers to take risks? Will their communities and "the system" allow them to do so?
• In this era of accountability how do schools merge the demands of National Standards and NCEA with innovation and transformation? Particularly if the "educational experts" do not themselves understand innovation in the digital world.
• Is our education system really looking for transformation? Or does it want replacement and enhancement? Is the desired end-point the use of digital technologies in classrooms, within schools that are instantly recognisable?

If transformation of how our students learn is the desired outcome then schools need to be supported to shift their perspectives, to take risks and to share their learning: The potential benefits of doing so need to be made clear. Learning from the corporate and social worlds can help; looking beyond education for guidance. A national education network has the potential to be a powerful tool for enabling this shift.

Perhaps the words of one of the principals involved in the NEN Trial best sums up what is needed:

Three things irk me as they quickly become barriers to others engaging: How to avoid "technical train-spotting"; how to avoid people rushing off to the "next big thing"; how to avoid "brand name-blindness".  The combination of all three of these should be a diagnosable medical condition that renders you unreliable in the advising or development of others.

I am well aware that we are only part way into a Digital Revolution that is likely to change society, including lifestyle, work, and learning more so than any of the similar 'revolutions' from the past (e.g. Industrial Revolution). Knowing that we don't know how this will end nor how it will get there is also OK. We just need to factor continuous change and 'un-learning' into our thinking and approach.  I believe it is important that we equip our Teachers and Students, and those advising/training them with a wider view. So we can scaffold their learning experience to one of continuous change and development - and that's OK.

For example, "We know people learn and work better in groups. Especially when they feel confident and safe enough to share their ideas and questions. This 'software/tool/device' is one way we can currently support our learners to do this. There will be a newer version or an even better way soon enough. But in the meantime here's how we can do this, ...and how others have used it, ...and the effects that it has had..."

Most importantly we need to give our people the confidence to fail. To give something a go; have it crash on them; recover the moment and the learning; reflect on it a bit; and then at an appropriate time give it another go.

The full potential of the NEN Trial won't be realised until we learn that this is about equipping people with perspectives, as well as skills, and the confidence to fail. And through this nurture the flair to play in the future.

Footnotes

1. Network for Learning website.
2. Halverson, R. & Smith, A. (2008). How new technologies have (and have not) changed teaching and learning in schools. Journal of computing in Teacher Education, 26(2), 49-54.
   SRI International. (2011). Innovative Teaching and Learning Research 2011 Findings and Implications. Innovative Teaching and Learning (ITL) Research website.
3. For more information on this project go to School Network Upgrade Project on the Ministry of Education website.
4. This tends to be low decile schools.
5. Lee, M. & Levins, M. (2012) Bring your own technology. ACER Press, Melbourne.
6. Lee and Levins use the term BYOT (Bring your own technology) rather than BYOD (Bring your own device) as it is more inclusive of a wide range of technologies.
7. KAREN (Kiwi Advanced Research and Education Network) is a data network providing high capacity, ultra high speed connections between New Zealand's universities, polytechnics, Crown research institutes, schools, libraries, museums and archives, and out to the rest of the world.
8. This is evidenced in a wide body of literature, such as the research on the Laptops for Teachers programme. It is also evidenced in the findings in this evaluation. For more information on learning communities in a New Zealand context refer to the Network Learning Communities evaluation reports available on the Te Kete Ipurangi (TKI) website.
9. Evidence for this can be found viewing any young person on public transport or in a cafe. It can also be found in literature such as that found on the Common Sense Media website. An example is their report: Social Media, Social Life: How teens view their digital lives.published in 2012.
10. For a discussion of this see: Ward, L., Parr, J.M. and Robinson, V.M.J. (2005). Constructions of practice: Adding ICT to the swamp. Paper presented at the Hawaiian International Conference for Education, Hawaii.