Executive Summary

What are international capabilities and why measure them?

Broadly speaking, international capabilities can be described as the knowledge, skills, attitudes, and values that enable people to live, work, and learn across international and intercultural contexts. These capabilities, or aspects of them, are described by a range of terms in the literature, including international knowledge and skills, global competence, global/international citizenship, global/international mindedness, and intercultural competence.

The Ministry's background work suggests international capabilities can be seen as "the international and intercultural facet of the key competencies". Focusing on development of New Zealand students' international capabilities could, among other things:

• help make more explicit what the key competencies look like when they're applied in intercultural or international situations.
• provide a way to open a conversation with schools about internationalisation of education support New Zealand schools to better understand, analyse, and talk about th intercultural/internationalising learning activities they already do.
• open conversations about cultural diversity in New Zealand schools and communities and the opportunities this can provide for intercultural learning.
• create an opportunity for schools to revisit parts of The New Zealand Curriculum (Ministry of Education, 2007) vision, including the notion of students being "international citizens".
• encourage schools to connect with businesses and the wider community to develop learning opportunities that help students to develop innovation and entrepreneurial capabilities and connect these capabilities with intercultural and international contexts.

Measuring New Zealand students' international capabilities could help us to better understand how the schooling system helps to "increase New Zealanders' knowledge and skills to operate effectively across cultures."¹ It could feed into ongoing developments within educational policy and practice to better align curriculum, assessment, and pedagogy with the high-level goals of The New Zealand Curriculum. Looking further into the future, knowledge about how our schools support the development of students' international capabilities could assist with longer-term redesign of educational policy, curriculum, assessment, and qualifications to keep pace as demands and pressures on learning and schooling continue to change through the 21st century.

Learning from research on measuring key competencies

If we want to assess school leavers' international capabilities, how might we go about doing so? Because this question addresses a hypothetical space (as far as we have been able to establish no other nation has done exactly this) we need to draw indirect lessons from other similar assessment projects. International literature, and almost 10 years of research on the implementation and approaches to measurement of the key competencies in The New Zealand Curriculum, indicates that there is no overall "best" way to try to measure competencies, but there are different ways that are best for measuring specific expressions of competencies in specific contexts. The key competencies research indicates that assessment should include a focus on self-awareness and deliberate, strategic use of one's current capabilities if it is to translate into better lifelong learning. All of the key findings to date support the idea of a measurement approach that engages and involves students in gathering and reflecting on the evidence of their learning and growth.

Should we measure students, or the system, or both?

One key question to emerge from our study is whether this programme of measurement should find its locus within students, or at the level of the system. There are valid arguments to support both approaches. Arguments for assessing students' international competence assume that we can identify what an "internationally capable" person might know and be able to do and devise valid and reliable measures to assess these capabilities in young people. Arguments for an assessment of the system respond to challenges that it may be unreasonable to attempt to assess in students what is possibly not yet being widely taught, supported, or encouraged in schools. Furthermore, the literature establishes that from a systematic research perspective, we currently know as little
about how "internationalisation" is understood and manifested in New Zealand secondary schools' practice, as we do about students' international capabilities. Our initial contacts with schools to set up workshops indicated there is some conceptual confusion between the more familiar conception of international education as fee-paying international students in New Zealand schools and internationalisation as a process which can change aspects of the purpose, functions, or delivery of education for all learners.

International capabilities as a "nodal point"

This study reveals that the idea of international capabilities integrates several different high-level discourses around what matters for learners and learning in a 21st century globalised environment. In this sense it functions as what some theorists call a "nodal point" (Mannion, Biesta, Priestly, & Ross, 2011). Nodal points—increasingly a feature of 21st century educational policy and curriculum discourses—bring together a range of ideas and seek to fix a particular meaning that can be widely agreed on and used to catalyse educational actions. Paradoxically,
nodal points are also malleable and unstable, precisely because people will bring a range of opinions and perspectives as they make meaning from these ideas and put them into practice. On one hand, this is a good thing, because it offers people the opportunity to interpret things in a way that fits their context. On the other hand, it is likely that the means by which students' international capabilities might be developed in practice may be contested. This is because (among other reasons) people have very different ideas/visions of how New Zealand society should grow and develop and what the role of education in that process ought to be.

In terms of this exploratory study, failing to recognise these complex spaces between policy and practice has thus surfaced as a risk for ongoing work in this area. Conversely, moving forward with a clear understanding of these complexities and designing policy programmes of work with them in view has surfaced as an opportunity for further work in this area to be ground breaking, not just for New Zealand, but internationally.

The workshop sessions convened as part of this project showed that the notion of developing students' "international capabilities" interested and engaged the secondary teachers, students, and other stakeholders in our workshops in rich thinking and conversations that brought to the surface many diverse and interlinked ideas about school, learning, students, New Zealand communities, and New Zealand's place in the world.

Teacher workshops

Teachers believed their schools already supported the development of students' international capabilities through, among other things:

• a strong focus on learning languages and a school-wide focus on celebrating and recognising cultural and linguistic diversity.
• overseas trips built into learning across multiple subject areas.
• "service" programmes with an international connection.
• hosting visiting international student groups and sister-schools programmes.
• cultural fairs and festivals within the school.

Teacher workshop discussions raised interesting questions about students' equity of access to internationalising or intercultural learning opportunities, and indeed what the nature of those learning opportunities might be in one school compared with another. Although there were some hesitations about whether or how senior students' international capabilities could be assessed, the teacher workshop provided many examples of evidence on which teachers were basing their views of their schools'—and their students'—international capabilities. This suggests that schools might indeed find value in having access to assessment tools and approaches that they or their
students could use to collect and analyse data to test schools' assumptions about current practices and the learning impacts for students' international capabilities.

Student workshops

We visited two secondary schools with very culturally diverse student populations and talked to
groups of 12 and 9 senior students respectively. Encouraged to consider their own views about
what it meant to be internationally capable, or a "global citizen", both groups identified types of
knowledge and qualities that someone would need to be able to interact across cultural and
national boundaries. When asked how they thought these capabilities might be assessed or
measured, students suggested:

• interviewing students.
• looking at what international or intercultural opportunities students had been exposed to - including both what had been offered by school and through family experiences.
• keeping a record or portfolio of students' activities and experiences that contribute to international competence.
• evaluating students' knowledge of languages.
• undertaking a survey of students' interests (in other people, cultures, global issues, or events and travel).

One interesting theme to emerge from the young people's workshops was the significance of the highly multicultural social interactions and friendship groupings they experienced in their schools. Both the student and teacher workshops suggested that something interesting may be happening in parts of New Zealand where students are "growing up internationalised" in ways that their parents and teachers may not have experienced in their own youth and schooling. This suggests value in continuing to involve young people in shaping a New Zealand discourse on what it means to be internationally capable, as their lived experiences might offer insights on international or intercultural capability that differ from those of adult policymakers or teachers.

Mixed-expertise workshops

Among the 10 participants who were able to attend the mixed-expertise workshop, at least four worked in the tertiary sector, at least two worked in business development projects or programmes with an explicit focus on building business relationships between New Zealand and Asia, at least two young adults had developed their own social or commercial enterprises, and at least two had worked with refugees or in international development volunteer programmes. Compared with the teacher and student groups, the mixed-expertise group was the most easily able to engage with some of the economic policy arguments for growing New Zealanders' international capabilities. These participants were well aware of the arguments for developing New Zealand's international
business capability; indeed this was a key focus for some participants' work.

Participants in the mixed-expertise workshops were also the most likely to question presumed relationships between schooling and what contributes to becoming an internationally or interculturally capable adult. One concern arising for these participants was whether a measure of students' capabilities or international mindedness while they are still at school would genuinely indicate what those people might be, do, experience, and become over the duration of their lives, particularly when (as some pointed out) the kinds of learning experiences they felt could contribute to developing these capabilities may be less accessible to young people because of the ways school learning is commonly structured and organised.

Visit to ACER

We consulted with staff from the Australian Council for Educational Research (ACER) with a focus on understanding their work in the development of measures of ability/capability in the areas of intercultural and interpersonal competence. This was exemplified in two specific assessment programmes which were a focus of our meeting: The National Assessment Program - Civics and Citizenship (NAP-CC), and the UMAT (Undergraduate Medicine and Health Sciences Admission Test). Both tests are designed to measure ability/capabilities/knowledge of individuals (learners). However, they do not directly provide measures of students' opportunities to learn, nor do they provide information about what sorts of learning environments and contexts schools can
or should be developing in order to foster the development of students' capabilities on these measures. The periodic release of items from these tests means they can become potential resources for curriculum and teaching, and in that way they may contribute to shifts in the learning environments and opportunities that schools provide to support the development of those capabilities.

Recommendations

Overall this study suggests the wisdom of adopting a view of New Zealand senior secondary students' international capabilities as both a potentially measurable outcome for students and as a feature of the system (a process). In other words, we need to understand what opportunities students have to develop these capabilities, as well as what they actually learn from those experiences, particularly if the long-term goal is to improve or transform schooling practices to better meet 21st century learning needs.

Revisiting the overall purpose of assessing Years 12/13 students' international capabilities, and how it might be used to support better practice in schools and better learning for students, presents some different options. Previous New Zealand research identifies at least three different policy purposes for such an assessment:

Purpose 1:

For accountability at school and wider policy levels, and for reporting learning progress to any stakeholders who have a need for this information.

Purpose 2:

Improving classroom teaching and learning practices.

Purpose 3:

To empower students to become lifelong learners.

Any measurement approach will have different degrees of value in achieving each of these purposes. The options below are arranged in order from a strong focus on purpose 1 through to a strong focus on purpose 3.

1. Using an externally devised assessment framework and a national sampling approach

2. Develop formative assessment tool(s) for schools, with the understanding that some national data could be collected as a secondary benefit

3. Use the National Certificate of Educational Achievement (NCEA) to assess and record data about students' international capabilities

4. Focus on an approach that is explicitly geared towards a lifelong learning/learner empowerment

Fundamentally, this approach represents a shift away from a question like "how internationally capable are New Zealand students?" and instead centres around the more open question of "what could New Zealand students' international capabilities be?"

In outlining four feasible assessment approaches and their strengths and weaknesses, we hope we have provided a solid foundation for next-step decision making. If New Zealand work in this area aims to be genuinely innovative and world leading, we recommend beginning with a clear focus on the areas that traditional approaches to assessment have been least likely to start from in the past. These are:

• enlisting learners as key partners in shaping meaning for the construct under study (and in doing so enlisting teachers, schools, and potentially, wider communities in this process)
• utilising new technologies/networked technologies to generate new assessment possibilities.

This does not mean using technologies to do the same things in new ways, but rather, opens a space for considering what new ways we have for generating, sharing, documenting, experiencing, and utilising learning and knowledge, and what this could lead to in terms of our capabilities to know what learners are capable of.

Footnote

1. This is one of the objectives under Goal 3 of the Government's Leadership Statement for International Education (New Zealand Government, 2011).