OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes

Publication Details

In 2010 New Zealand participated in the Organisation for Economic Co-operation and Development (OECD) Review on Evaluation and Assessment Frameworks for Improving School Outcomes. The purpose of the review was to explore how systems of evaluation and assessment can be used to improve outcomes in primary and secondary schooling.

Author(s): Ministry of Education

Date Published: April 2011

Chapter 6: Student assessment

This chapter focuses on student assessment and how student achievement results are used in New Zealand to understand and improve the performance of the schooling system

6.1 Current practices

6.1.1 Overall framework for student assessment

348. The framework for assessment and evaluation in New Zealand schools is described in terms of the information needed at three different levels: student, school and system (Figure 7). While evaluation and assessment ensure accountability, the primary focus is on ensuring all actions improve student outcomes.

Figure 7: Using information to support improvement in education

Image of Figure 7: Using information to support improvement in education.

349. Assessment policy in New Zealand has consistently focused on improving learning.129 130 131 Since the introduction of self-managing schools, the following themes have been evident:

  • classroom assessment has a strong link with the quality of programmes and improvements in student learning;
  • teachers need support to consistently make sound professional judgements about student achievement and to provide responsive programmes of learning through professional learning and development programmes;
  • the importance of high quality assessment tools that support teachers to identify student achievement and recognise progressions in learning;
  • the need for schools to be able to gather, analyse and use high quality achievement information to inform decision-making and fulfil accountability and reporting requirements to their communities and the Ministry of Education.

350. A review of New Zealand’s approach to assessment commenced in 2006 and culminated in a report, Directions for Assessment in New Zealand (the DANZ report).132 This report affirms and builds on the key principles underpinning assessment policy and practice and is being used to inform the development of a Ministry of Education position paper on assessment currently in preparation.

351. Key principles that underpin the current development of assessment policy at all levels of the system include:

  • the student is at the centre of assessment practice;133
  • the curriculum underpins assessment;
  • assessment capability is crucial to improvement;
  • an assessment capable system is an accountable system;
  • multiple sources of evidence enable a more accurate response;
  • effective assessment is reliant on quality interactions and relationships.

352. Assessment policy emphasises the importance of considering rates of progress as well as levels of achievement reached. The focus is on improving the rate of progress for students, regardless of their starting point and ensuring that all students are supported to reach their full potential. This approach recognises that not all students enter school at the same starting point in their learning and that they do not necessarily progress in a steady and linear way.

National Curriculum Framework

353. The New Zealand Curriculum recognises that student assessment information contributes to assessment for learning at all levels of the system.134 The interaction between assessment, teaching and learning in informing classroom and school-wide programmes, policy and practice is also outlined in Te Marautanga o Aotearoa.135

354. The framework of the national curriculum provides the basis for the development of the progressions and standards to guide teaching and learning and enable assessment for qualification purposes.

Signposts to guide teaching and learning

355. The New Zealand Curriculum and Te Marautanga o Aotearoa contain achievement objectives. The objectives provide indicators of expected performance by curriculum level in each curriculum learning area: English, mathematics, science, social sciences; the arts; health and physical education; technology; and languages. Te reo Māori is an additional learning area in Māori-medium.

356. National Standards (English-medium in reading, writing & mathematics) and Ngā Whanaketanga Rūmaki Māori (Māori-medium in reading, writing, oral language and mathematics) are being implemented from 2010 (section 6.3).

357. National Standards (Years 1-8) together with literacy and numeracy learning progressions (Years 1-10) describe expectations of performance as students progress through schooling. The standards consist of descriptors, illustrations, and examples of student work and assessment tasks linked to school year levels.

358. Assessment in relation to these signposts occurs through teachers’ professional judgements based on a range of evidence of student learning (see Making professional judgements about progress and achievement below).

359. There are no nationally administered common assessments before Year 11.

The New Zealand Qualifications Framework

360. The New Zealand Qualifications Framework (NZQF) provides the framework for the assessment of student outcomes in secondary schooling. The NZQF has 10 levels. Levels 1-3 relate to middle to senior secondary education and basic trades training. Levels 4-6 relate to advanced trades, technical and business qualifications. Levels 7 and above are advanced qualifications of graduate and postgraduate standard.136

361. In the senior secondary school environment learners will typically progress from Level 1 to Level 3. There are no formal pre-requisites for any school subject-based standards. However, schools may require students to have achieved particular standards in a subject in a previous year before commencing study in that subject at a higher level.

362. Levels of the NZQF are not related to the age of the learner. Most commonly, assessment towards NCEA commences in Year 11 of schooling. However, it is not uncommon for students to undertake NCEA assessments earlier in their secondary schooling. In addition, multi-levelling, where students can be assessed at more than one level, for particular standards, and/or subjects, in a given year, is now a common feature in many secondary schools.

National qualifications

363. Assessment for the purpose of awarding qualifications on the NZQF is administered by the New Zealand Qualifications Authority (NZQA).

364. The rationale for the current approach to awarding qualifications is influenced by experience of previous assessment systems, legislative requirements, societal changes and changes in educational and assessment philosophy.137 Under the previous system, schools tended to teach single year courses determined by a national curriculum and syllabus. Often this was limiting for schools and students.

365. Since the inception of NCEA,138 a standards-based assessment system, schools have become better able to offer flexible senior programmes that meet the needs of students and their tertiary and workplace pathways by mixing and matching achievement and unit standards that are available in the Directory of Assessment Standards (formerly the National Qualifications Framework). As a result, schools have introduced new courses, established links with tertiary courses and increased work-related programmes.

366. Schools can still run one-year courses in traditional school subjects. However, they can also run shorter or longer courses, integrate studies, combine levels, and link with industry-based programmes. Students can work towards other national certificates (of which there are a large number) at the same time as working towards NCEA.

National Certificate of Educational Achievement (NCEA)

367. The three levels of the National Certificate of Educational Achievement (NCEA) are the most common qualifications students work towards in Years 11 to 13, the final years of secondary schooling. All State (publicly funded) schools are required to offer NCEA. Private schools can offer other qualifications and associated assessment. State schools can offer alternative qualifications as well as NCEA.

368. An NCEA is gained by accumulating credits from any part of the NZQF, regardless of whether they are curriculum-based or vocationally-based. NCEA is a multi-field qualification and allows for such flexibility of content. Credits are awarded for each standard a student achieves in their programme of study. Standards can be assessed in either English or in te reo Māori. The conditions for attaining an NCEA are recorded in Table 10, and for gaining entrance to a university programme139 in Table 11.

 
  Table 10: Attaining a National Certificate of Educational Achievement (NCEA)  

* Note: Credits can be used for more than one qualification. Some credits from a student’s previous qualification can be counted towards the next NCEA qualification. 

Qualification Definition
NCEA Level 1
(80 credits)
Students must achieve:        
  • 8 credits from approved numeracy standards               
  • 8 credits from approved literacy standards140             
  • at least 64 credits from other standards (NZQF Level 1 or above)*
NCEA Level 2
(80 credits)
Students must achieve:        
  • at least 60 credits from any standards (NZQF Level 2 or above)
  • up to 20 credits from standards at NZQF Level 1*
(At Level 2 there are no specific literacy or numeracy requirements)
NCEA Level 3
(80 credits)
Students must achieve:        
  • at least 60 credits from any standards (NZQF Level 3 or above)
  • up to 20 credits from standards at NZQF Level 2*
(At Level 3 there are no specific literary or numeracy requirements)
 
  Table 11: Gaining University Entrance 

* Note: These may be assessed in English or in te reo Māori. 

Entry Definition Level
Qualification entry into a NZ University Students require at least:        
  • 14 credits from standards in one approved subject
  • 14 credits from standards in a second approved subject
  • 14 credits from standards in up to two additional NZQF domains or two other approved subjects
(NZQF Level 3 or above)
Literacy requirements        
  • 4 credits from approved reading standards*
  • 4 credits from approved writing standards*
(NZQF Level 2 or above)
Numeracy requirements         (NZQF Level 1 or above)
Age entry into a NZ University Students over the age of 20 years old do not require any qualifications to be eligible to enter a New Zealand University Not Required


Standards used to assess learning for qualification purposes

369. Standards registered in the Directory of Assessment Standards specify learning outcomes and describe the assessment criteria; that is, what a student needs to know, or what they must be able to do, to achieve the standard. Examples can be found on NZQA’s website.141 Each successful result for a standard contributes credits toward an NCEA qualification (Level 1, 2 or 3).

370. The number of credits is dependent on the amount of time involved in meeting the requirements of the standard. One credit corresponds to approximately 10 hours of work for an average student, including instruction, practice for assessment and assessment. Credits may be accumulated from different learning institutions or workplaces towards a single national qualification.

371. There are two types of standards in the Directory of Assessment Standards: unit standards and achievement standards. These are briefly described in Table 7: Student assessment for National Qualifications.

  • Unit standards: assessed at school by teachers and workplace assessors (internal assessments). These vocationally-based standards are used widely outside schools. There are over 26,000 unit standards in the Directory of Assessment Standards, the majority of which are used in workplace training to deliver a large number of national qualifications.
  • Achievement standards: some are internally assessed by teachers at school, and others are externally assessed by national examinations (or portfolio) at the end of the year. Achievement standards are focused on the secondary school curriculum and are not used widely outside the school context. Most school curriculum subjects are divided into between four and seven achievement standards. Each standard represents a stand alone ‘topic’ idea or concept. There are approximately 850 achievement standards in the Directory of Assessment Standards. Most are used in schools.

372. Standards are organised into levels of increasing difficulty. The grades available for achievement standards are: Not Achieved, Achieved, Achieved with Merit and Achieved with Excellence. Most unit standards have two grade categories: Not Achieved and Achieved. Some unit standards have the grade Merit available, and work is under way to enable Excellence grades in unit standards where appropriate.

373. The number of credits achieved is not affected by the grade received. If the standard is worth three credits, then a student gaining an Achieved, Merit or Excellence grade will gain three credits.

Regulatory requirements related to assessment

374. The National Administration Guidelines (NAGs),142 require that schools:

  • (NAG 1b) through a range of assessment practices, gather information that is sufficiently comprehensive to enable the progress and achievement of students to be evaluated; giving priority first to student achievement in literacy and numeracy, (especially in Years 1-8) and to breadth and depth of learning related to the needs, abilities and interests of students, the nature of the school's curriculum, and the scope of the national curriculum as expressed in The New Zealand Curriculum or Te Marautanga o Aotearoa;
  • (NAG 1c) on the basis of good quality assessment information, identify students and groups of students: who are not achieving; who are at risk of not achieving; who have special education needs (including gifted and talented students); and aspects of the curriculum that require particular attention;
  • (NAG 1d) develop and implement teaching and learning strategies to address the needs of students and aspects of the curriculum identified in (c) above;
  • (NAG 2c) report to students and their parents on the achievement of individual students;
  • (NAG 2Aa) where a school has students enrolled in Years 1-8 …report to students and parents on the student’s progress and achievement in relation to National Standards …reporting to parents in plain language in writing must be at least twice a year.

375. NAG 2A is a new requirement to support the implementation of National Standards and the plain language reporting associated with this policy (Sections 2.3 and 6.3).

376. The Education Act 1989 indicates the functions of the New Zealand Qualifications Authority (NZQA) and its responsibilities in relation to assessment and examinations.

Table 12: Functions of NZQA
Section 253 of the Act outlines the functions of NZQA to:
     
  • oversee the setting of standards for qualifications in secondary schools and in post-school education and training;
  •  
  • monitor and regularly review, and advise the Minister on, the standards for qualifications in secondary schools and in post-school education and training, either generally or in relation to a particular institution or private training establishment or a particular course of study or training;
  •  
  • develop a framework for national qualifications in secondary schools and in post-school education and training in which: all qualifications have a purpose and a relationship to each other that students and the public can understand; and there is a flexible system for the gaining of qualifications, with recognition of competency already achieved.
  •  
Section 265 of the Act provides NZQA with the authority to:
     
  • set and conduct examinations and make assessments, as it considers necessary for the performance of its functions;
  •  
  • make regulations, give directions and approve examiners, assessors and moderators: grant awards to persons who reach the standards in the examinations or assessments set by the Authority.
  •  

Longitudinal dimension to student assessment

377. Most New Zealand schools (over 99 percent)143 use a Student Management System (SMS). This is a computer application designed to manage student attendance, demographic and assessment information, allowing easy reporting to parents, family and whānau, as well as the analysis of aggregated student data.

378. SMS applications are provided by several vendors and have varying functionality. Since teacher and school leader expertise in the use of the systems varies, the use of the systems is varied. Some schools make limited use of their SMS (for example, roll return purposes) while others make full use of capabilities for longitudinal tracking of individual students as well as the reporting and analysis of aggregated student data.

379. In the best cases, student data from a range of assessment resources is held within the SMS, follows the student from class to class and is used for reporting to parents, families and whānau. Teachers use aggregated student data to adapt and plan their classroom programmes, tailoring instruction according to student need. School leaders use school-wide aggregated data to investigate the effectiveness of school programmes and student learning, set targets for achievement, make resourcing decisions and determine professional development priorities.

380. Aggregated school-wide data is also used to report to the school’s Board of Trustees on progress in targeted areas according to the strategic plan. Currently, SMS from different vendors do not store data in compatible formats. A Student Record Transfer (SRT) initiative is currently under way to rectify this (Section 6.3).

381. A lifelong Record of Achievement (RoA) records a cumulative list of all NZQF registered standards and qualifications a student has achieved. Students can accumulate credits over a number of years and from many providers, including schools, post-school education and the workplace until they have completed a qualification. An RoA provides an employer or post-school education provider with a transcript of a student's achievements.

6.1.2 Student assessment procedures

382. Each school is responsible for establishing and documenting its own policies and processes for assessment within the scope of the national framework and regulatory requirements. Schools are required to gather assessment information through “a range of assessment practices” (NAG 1b). This acknowledges that no single source of information can accurately summarise a student’s achievement or progress. Schools are further required to use “good quality assessment information” (NAG 1c).

383. Advice and guidance to schools about what constitutes effective assessment practice emphasises:

  • the need to use a range of effective assessment practices to gather quality assessment evidence as an integral part of teaching and learning;
  • the need to interpret, use and respond to this information to determine next teaching and learning steps, plan classroom programmes, and support students to use assessment information to inform their own learning;
  • the need for effective quality assurance systems;
  • that this same information can be used by teachers to ‘step back’ at regular intervals and make summative professional judgements across the full range of assessment evidence;
  • these judgements are considered both in terms of broad standards and expectations appropriate to the learner;
  • the importance of including students as active participants throughout the assessment process in order to build their assessment capability;
  • that students who are involved actively in assessment are more likely to feel confident in talking about their achievement and progress with their parents, family and whānau, to take ownership of their own learning and to develop into autonomous, self-regulating learners.

Years 1-10

384. The New Zealand Curriculum and Te Marautanga o Aotearoa set out broad achievement outcomes for the levels in each curriculum learning area during schooling. National Standards (English-medium in reading, writing & mathematics) and Ngā Whanaketanga Rūmaki Māori (Māori-medium in reading, writing, oral language and mathematics), supported by literacy and numeracy progressions, establish performance expectations in Years 1-8.145

385. Schools utilise a variety of both formal and informal assessment approaches chosen to suit both the nature of the learning being assessed and the varied characteristics and experiences of the students. The mix of information gathered varies from school to school depending on context and need. The balance of different assessment practices is shown in Figure 8.

Figure 8: Frequency of different assessment practices

Image of Figure 8: Frequency of different assessment practices.

386. For Years 1-10, there are a number of assessment tools available to teachers to use as a component of their assessment programme, including tools norm-and criterion-referenced to New Zealand students.146 The Ministry of Education does not mandate the use of particular tools for Years 1-10. Mandating the use of particular tools could, over time, narrow the assessment focus and render specific tools as de facto national tests, undermining authentic teaching approaches that rely on a strong learner focus and quality professional judgement. The diversity of assessment tools also encourages innovation.

387. The progress and achievement of English language learners is monitored in relation to English Language Learning Progressions (ELLP), until students can participate in the regular classroom assessment programme involving the National Standards and/or literacy learning progressions.

388. A small number of students have very significant learning disabilities. This group of students is likely to (or expected to) learn long term within Level 1 of The New Zealand Curriculum or Te Marautanga o Aotearoa and will be receiving support through the Ongoing and Reviewable Resourcing Schemes (ORRS) or accessing Supplementary Learning Support (SLS).

389. The progress of these students is assessed in relation to the standards/progressions as part of the regular review of their learning that takes place through their Individual Education Programmes (IEPs), which are agreed in consultation with parents, families and whānau, teachers and the Ministry of Education.

Assessment towards qualifications

390. Assessment for the purposes of qualifications attainment can involve internal and/or external approaches. Students can be awarded credits towards NCEA on the basis of internal assessment by the learning institution. Other standards are assessed externally by NZQA at the end of the year in a national examination round or by portfolio for arts subjects and graphics. Assessment format is usually left to the discretion of individual assessors for internally assessed subjects.

391. The purpose of the national student examinations is to determine what students have learned against externally assessed achievement standards and to provide formal certification for this learning. The results of national examinations, and other forms of assessment, provide employers and post-school education providers with students’ educational achievement for selection processes.

392. Schools are encouraged to analyse their students’ progress for individual standards as well as subjects, since achievement is reported on the RoA for individual standards. This means that particular strengths and weaknesses can be identified and appropriate actions can be taken at a student, as well as a school level.

393. NCEA was designed around well established assessment principles. As such, it provides potential for formative assessment to be an integral part of teaching practice in preparing students for formal certification. Schools are encouraged to use opportunities for reassessment and re-submission of assessments within prescribed rules. This approach is designed to provide opportunities for formative feedback in order to maximise learner success.

394. For internally assessed standards, course work is very important. Course work involves a range of assessment activities including projects, research, essays, live and recorded presentations, portfolios and group work. This course work lends itself to formative assessment practice, as teachers provide feedback and appropriate guidance to students. A robust national external moderation system has been developed to ensure consistent assessment at the standard.

395. Students in secondary schools have the opportunity to sit examinations for New Zealand Scholarship, concurrent with NCEA examinations. Scholarship is an award to recognise top students and includes monetary reward. It does not attract credits or contribute towards qualifications but successful results do appear on the Record of Achievement.

396. Scholarship examinations enable students to be assessed against challenging standards and are demanding for the most able students in each subject. Scholarship students are expected to demonstrate high level critical thinking, abstraction and generalisation and to integrate, synthesise and apply knowledge, skills, understanding and ideas to complex situations. Approximately three percent of students undertaking each subject at NQF Level 3 are successful in the scholarship examination for that subject.

Making professional judgements about progress and achievement

397. Teachers gather evidence about student achievement and progress from multiple sources as part of regular teaching and learning. This information is used formatively to determine next teaching and learning steps, to plan classroom programmes and support students to inform their own learning.

398. This same information is used to make summative professional judgements to enable reporting and school review in the light of goals and established expectations. Established expectations include National Standards in literacy and numeracy (Years 1-8), literacy and numeracy progressions (Years 1-10), curriculum achievement levels and key competencies (Years 1-13).

399. In the National Standards context (Years 1-8), this is referred to as an overall teacher judgement (OTJ). The variety of sources available from which information can be drawn is illustrated below. The triangulation of a range of evidence supports valid and reliable progress and achievement decisions.

Figure 9: Gathering, interpreting and using assessment information

Image of Figure 9: Gathering, interpreting and using assessment information.

400. The same sort of process can be used when making summative professional judgements in Years 9 and 10 about achievement and progress in relation to curriculum achievement objectives and key competencies.

Moderation – building shared expectations in Years 1-10

401. Moderation of professional judgements in Years 7-10 is intended to improve the consistency of professional teacher judgements, rather than a formal quality assurance measure. Moderation involves groups of teachers discussing their judgements on the basis of a range of assessments and samples of student work. The intention is to build a shared understanding of the curriculum, the learning progressions and the National Standards for Years 1-8. The implementation of National Standards requires an increased emphasis on moderation in primary schooling.

402. Schools establish a moderation process as part of their effective teaching programme. The evidence-based discussions involved in moderation have been encouraged as a key component of effective assessment practice in schools for some time and many primary teachers work together to develop shared understandings of quality and progress in this way.

403. A programme of work has begun to support teachers to make consistent professional judgements. This programme includes providing professional development for teachers and school leaders in moderation processes, developing resources in moderation and providing the infrastructure to carry out moderation across a wider group of teachers (online moderation). The evaluation of the National Standards implementation in 2009-2013 will provide information about teacher judgements and moderation.

404. An ongoing programme to align common assessment tools to standards has been established. This will both assist the determination of overall teacher judgements made by individual teachers and the moderation of judgements between teachers. This alignment will increase in reliability as more research evidence becomes available.

Moderation – formal quality assurance of NCEA

405. Moderation is an essential feature of NCEA assessment. Not only does it provide feedback to teachers and help build the consistency of judgements, it also acts as an important quality assurance system. Moderation does not affect grades already issued in the assessment samples but informs teachers’ practice and provides system-wide information for future assessments and policy development.

406. The broad goal for assessors is to produce assessments for each standard that are valid and reliable and allow judgements that are consistent with the standard. An assessment activity is valid if it accurately represents the range of achievements, knowledge and skills to be assessed under the standard. An assessment activity is reliable if it gives results that are consistent and present an accurate picture of what is being measured.

407. NZQA is responsible for moderating internally assessed work to ensure it is at the nationally prescribed standard. Schools must submit 10 percent of internally assessed student work to NZQA for moderation. Assessment materials used to assess each standard must be submitted, together with the samples of student work. Moderators give feedback on the assessment materials used. NZQA also conducts checks to ensure schools have robust assessment systems.

Managing National Assessment (MNA)

408. Managing National Assessment (MNA) is designed to ensure valid, accurate and consistent internal assessments for qualification purposes. MNA has two components:

  • annual (external) moderation of assessment materials and assessor decisions (up to 20 percent of all internally assessed standards in all curriculum areas);
  • external checks of school assessment systems at least every three years.

409. In instances where the MNA process identifies problems, NZQA may:

  • require further materials to be submitted for moderation;
  • conduct supplementary systems checks;
  • investigate as potential breaches of the rules (external assessment);
  • require an action plan to address school system issues;
  • begin non-compliance procedures.

Other assessment opportunities

410. A number of New Zealand schools, at their own discretion, choose to make use of various assessment options offered by non-New Zealand agencies. For example, the International Competitions and Assessments for Schools (ICAS) are run by the University of New South Wales through Educational Assessment Australia. ICAS testing caters for students in a number of countries from Year 3 through to Year 12 and examines skills in English, mathematics, science, computers, writing and spelling. Some schools also enable students to participate in examinations such as the University of Cambridge International Examinations and the International Baccalaureate.

6.1.3 Competencies to assess students

Initial Teacher Education

411. Initial teacher education programmes in New Zealand include core content on assessment that is considered essential to beginning teaching. This content is included either as assessment-specific compulsory courses or as a component of other compulsory courses – especially those focused on curriculum, teaching and learning.147

412. However, the assessment knowledge and expertise that teacher graduates bring to the profession are variable. Provisionally registered teachers' readiness, in terms of assessment practice, has been reported as ranging from confident to inadequate.148 A case has been made for all programmes to teach assessment both in assessment-specific courses and embedded within curriculum-specific courses.149

413. Improvement of system-wide teacher assessment practice gained through initial teacher education is influenced by the school environment in which new teachers begin teaching. The lack of modelling and application of effective assessment practice in some schools has a significant impact on the effectiveness of new teachers’ future assessment practice as they are ‘socialised’ into existing practices.150

Assess to Learn

414. The Assess to Learn professional learning programme was established in 2002. Each participating school is involved in the programme for up to two years for primary and three years for secondary.

415. Assess to Learn enables teachers to understand and develop effective pedagogical strategies in a supportive environment focused on professional inquiry. The ability to choose appropriate assessment tools and to analyse and use assessment information to advance student learning are important components of this whole-school programme. The intended outcomes of the programme are:

  • improved student achievement;
  • improved student learning;
  • shifts in teachers’ assessment knowledge and practice; 
  • coherence between assessment processes, practices (including purposeful use of assessment tools) and systems in classrooms and schools so that they promote better learning;
  • strong cultures of continuous school improvement that reflect an inquiry-based approach;
  • strong professional learning communities regionally and nationally.151

The Literacy Professional Development Project (LPDP) and the Numeracy Project

416. In addition to the assessment-specific professional development provided by AtoL, other curriculum-based professional development projects contain an embedded assessment component. The Literacy Professional Development Project (LPDP) and the Numeracy Project have been shown to have significant impact on student achievement, as well as teacher practice, in terms of teaching inquiry and the use of student assessment data.152

Programmes to help teachers assess against NCEA standards

417. NZQA has 34 full-time moderators and over 200 part-time subject matter experts for moderating school-based subjects. Feedback from moderators assists in building school and teacher competency in assessing students and using assessment results. Acting on this feedback will support school improvement.

418. Regional Best Practice Workshops led by national subject moderators are provided throughout the country to maintain and develop teachers’ assessment judgements at the grade boundaries for internally assessed standards. This involves the analysis of student work and professional discussions. Exemplars of student work at grade boundaries are currently being developed for the vast majority of standards to help teachers and students to understand the criteria for Achieved, Merit and Excellence results.

419. NZQA has established a trademark for quality-assured assessment materials used for internally assessed achievement standards. The materials include the assessment activity (or task), any related resources and the assessment schedule (including sufficiency and judgement statements) used by schools to assess students against a standard.

Analysis of national external assessment results

420. NZQA routinely uses item response theory methods to analyse samples of results from the annual formal examination round. Grade distributions are monitored over time to assist in the maintenance of standards. Results of analyses assist examiners to improve the quality of examinations and maintain consistency of assessment in relation to standards.

6.1.4 Using student assessment results

421. At the individual student level, assessment results (that is, information from a broad range of assessment activity) is used to:

  • guide and improve teaching and learning on an ongoing day-to-day basis;
  • inform teacher professional judgements about achievement and progress (at specific points in time) to enable meaningful discussion with students and worthwhile reporting to parents;
  • provide evidence of learning to enable students to be awarded credits towards a qualification.

422. Student assessment results are also used to inform school self review and in the context of planning and reporting (NAG 2).

423. Achievement information is reported to both students and parents (NAG 2C). This requirement has been strengthened in the context of the implementation of National Standards. Schools are required to report to parents of Years 1-8 students on their children’s progress and achievement in relation to National Standards in plain language and in writing, at least twice a year (NAG 2A a). The intention is to ensure parents receive information that is meaningful and enables them to engage with, and support, their children’s learning.

424. Professional judgements across a range of evidence considered in light of established standards and progressions help teachers to identify students who are not achieving; who are at risk of not achieving; or who have special education needs, as required by NAG 1 (C). In the National Standards context, students may be identified as being at risk if they are achieving 'well below' a standard or are improving at a rate that is considerably less than expected. Students who are identified as ‘at risk’ may receive additional support beyond a classroom programme.

425. In the case of Years 11-13, assessment results from formal assessment activity provide evidence of learning to enable students to be awarded credits towards qualifications (NCEA) or a monetary award (scholarship). These results inform students and their families of achievement and assist students, their families and schools to plan further study or to seek employment. Most post-school study requires some pre-requisites, for example, university entrance requirements are based on achievement at Level 3, including some literacy and numeracy requirements at lower levels.

426. The NZQA public website presents data illustrating the performance of secondary students in gaining qualifications on the New Zealand Qualifications Framework (NZQF) and in New Zealand Scholarship. The website enables the creation of reports to view data on student achievement at the levels of individual schools, groups of schools and nationally and supports comparisons of data such as:

  • performance in assessments at one school or group of schools against another school or group of schools;
  • the relative performances of groups of students with different demographic characteristics;
  • longitudinal analyses of achievement data.

427. NZQA routinely monitors student results over time to consider what improvements can be made to policy and practice in relation to national examinations. Work is currently under way to enable the implementation of NCEA improvements (Section 6.3).

6.2 Implementation of student assessment

428. A major evaluation of the collection and use of assessment information was undertaken by the Education Review Office (ERO) during Terms 1 and 2, 2006. The report focused on the interaction between assessment, teaching and learning in 314 schools (253 primary schools and 61 secondary schools) and concluded that there is room for improvement in school assessment practice.153

429. NCEA results suggest that current approaches to assessment for the purpose of awarding qualifications have been successful in improving students’ outcomes, particularly for those who were not well served by the previous weighting on singular examinations. Fewer students are leaving school with no qualifications, compared with the pre-NCEA system (Figure 10) and more students are gaining a Level 2 qualification or higher (Figure 11).

Figure 10: Percentage of school leavers with little or no formal attainment

Image of Figure 10: Percentage of school leavers with little or no formal attainment.

Figure 11: Percentage of school leavers with NCEA Level 2 or above

Image of Figure 11: Percentage of school leavers with NCEA Level 2 or above.

Note: The gap in the lines on this graph is deliberate and indicates the change in the qualification measure used at Year 12. From 2003 the qualification measure used is NCEA level 2.

430. Challenges remain in the secondary sector in ensuring an appropriate balance between the formative and summative uses of assessment.

431. The ERO (2006) evaluation found that of the 42 percent of secondary schools in which there was an effective interaction between assessment and teaching and learning, this interaction was stronger in the senior school (Years 11-13) but tended to be achievement-focused and did not give an accurate picture of student progress over time. ERO’s findings appear to be supported by the work of Hume and Coll (2009)154 who concluded that teachers of Years 11-13 are implementing a narrow interpretation of formative assessment.

432. Although internal assessment is regarded as a vitally important feature of the system, it is seen by some as increasing teacher workload. External assessment is viewed by many as a necessary aspect of students’ assessment loads. However, evidence suggests that schools are increasingly favouring internal assessment; approximately two thirds of all school assessment is internal.

6.3 Policy challenges and initiatives

National Standards

433. From 2010, all schools with students at Years 1-8 are required to implement National Standards. National Standards, (English-medium in reading, writing & mathematics and Ngā Whanaketanga Rūmaki Māori (Māori-medium in reading, writing, oral language and mathematics) contribute to the assessment process because they provide the context against which evidence from a range of assessment activity can be considered.

434. The National Standards are broad descriptions of the knowledge, skills and understanding students need if they are to access the national curriculum with confidence. The intention is to build on the strong assessment for learning focus in primary schools by providing a nationally consistent means to assist teachers and students to make informed decisions about future learning needs and ensure parents have clear information to support their children’s learning.

435. The standards are being phased in over three years. Teachers are expected to assess and report to parents using National Standards in 2010, strategic planning using school-level data is expected in 2011 and Board of Trustees reporting of this data is required in 2012. The standards have been developed using evidence that includes: student achievement data from the web-based Assessment Tools for Teaching and Learning (e-asTTle) and Progress and Achievement Tests (PAT); and the extensive work undertaken to develop numeracy stages and literacy learning progressions.155

436. During 2010 further evidence was gathered against which any adjustments can be made. Evidence gathered included data from the National Standards Monitoring and Evaluation project as well as experiential evidence shared by schools and teachers implementing the standards.

Ngā Whanaketanga Rūmaki Māori

437. Such adjustments will be particularly significant in relation to Ngā Whanaketanga Rūmaki Māori. Although Ngā Whanaketanga Rūmaki Māori and related learning progressions are based on the advice of academic experts, the available evidence base is less extensive than that used in the development of the English-medium standards.

438. An extended period for consultation and trialling Ngā Whanaketanga Rūmaki Māori during 2010 will build shared ownership across the Māori-medium sector. Because each community and whānau is unique, Māori-medium National Standards must provide national consistency as well as being flexible enough to be implemented in ways that suit the spectrum of Māori-medium settings and whānau.

439. In Māori-medium education, there are assessment tool gaps in some areas of oral, reading and writing te reo Māori and Pāngarau. There are also a number of issues with existing tools. For example, some assessment tools are direct translations of English language tools and do not yet align to the new Māori-medium curriculum, Te Marautanga o Aotearoa.

440. The diversity of Māori-medium, in addition to the fact that Māori-medium settings are often small and remote, creates challenges in terms of the design of the support and professional development infrastructure to support this sector.

Stakeholder issues

441. Key concerns that have been expressed in relation to National Standards are that the standards:

  • may promote an undue focus on achievement, at the expense of progress, leading to students being labelled as failures rather than supported to further learning, as intended;
  • may compromise the implementation of the new curriculum by unduly focusing on reading, writing and mathematics;
  • are not well aligned with available tools and resources;
  • could lead to aggregated student achievement information being misused, incorrectly viewed as a proxy for school and teacher quality and unfair school comparisons made – related to this is a concern about the need to ensure consistency of teacher judgement.

442. Concerns have also been expressed about the research, theoretical and measurement underpinnings of the National Standards, the implementation timeframe and the management and resourcing of the change process.

443. The Ministry of Education has acknowledged these concerns and is monitoring them in the context of policy design and implementation. Many stakeholders are supportive of the broad aims of assessment policy and the strong focus on formative assessment, the use of teachers’ professional judgements and sharing information between teachers, students, parents, families and whānau. These emphases have been carried through into the new National Standards initiative, which should serve to strengthen the focus by providing a more consistent means for sharing information and emphasising the need for effective assessment practice and assessment capability.

444. The predominant stakeholder view is that national testing in primary schooling is inappropriate because it undermines the strong assessment for learning focus and promotes an overemphasis on the use of assessment information for accountability rather than improvement purposes.

Student record transfer

445. The use of different SMS with differing applications for data storage (Section 6.1.1 Longitudinal Dimension to Student Assessment) prompted the Student Record Transfer (SRT) initiative.

446. SRT is a software specification that vendors are now required to build into their SMS to enable interaction with a Ministry of Education server. The combined effect is that schools will be able to upload leavers’ data to a secure file server from where schools receiving the students will be able to download data, including student demographic, attendance and assessment information.

Assessment tools

447. The Ministry of Education has an ongoing programme to align the most commonly used assessment tools (tests, tasks, reading series, diagnostic interviews) to National Standards where possible. Consideration is being given to gaps in tool availability.

448. Monitoring of the use of the assessment website indicates that curriculum exemplars are one of the most used resources in our schools. The exemplars were developed in English and Māori-medium in all learning areas of the national curriculum. A process of refresh and renew is being undertaken to ensure alignment to The New Zealand Curriculum and Te Marautanga o Aotearoa being implemented from 2010. An increase in the uptake of e-asTTle has also occurred.

Professional learning

449. The Government set aside $26 million for all teachers and school leaders to access support in implementing the National Standards effectively in 2010, as a component of the National Standards policy initiative (Section 6.3).

450. The Assess to Learn Programme is an indepth, school-based programme that has operated across primary and secondary schools since 2002 (Section 6.1.3). The evaluation of this programme found that:156

  • involvement in the programme resulted in significant shifts in learning and achievement for the majority of students and shifts in professional learning and pedagogical practice for most teachers involved;
  • schools experienced improved recording and reporting systems, particularly in terms of consistency across teams or departments and more coherent teacher philosophy and practice in assessment;
  • significant gains in student learning and achievement, especially for lower-achieving students, were made and teachers and schools reported positive sustainable changes in teaching, learning and assessment processes, practices, and systems.

451. These conclusions are reinforced by data from the 2009 national evaluation of Assess to Learn.157 This data also signals significant effects in raised student achievement for all groups of students, including Māori and Pasifika students. Patterns are less clear in secondary schools, with effects varying greatly between schools.

452. Demand to participate in the Assess to Learn programme significantly exceeds supply. Currently, 155 schools are involved in the programme. Funding ($3.17m per annum) constrains the scale of delivery.

Enhancements to the National Certificate of Educational Achievement (NCEA)

453. Teacher and parent organisations have long sought a criterion-based approach to assessment and certification in national examinations. The shift from norm-referenced assessment to assessment against criteria required substantial rethinking and a fundamental change in assessment practice.158 By and large this shift has been successful and there is a growing understanding and knowledge of criteria-based assessment and the new qualifications regime.

454. Each year, NZQA carries out a statistical procedure to identify schools that have results distributions for internally assessed results, in one or more subject areas, which are significantly different to those that would be expected on the basis of their distributions of externally assessed (national examination) results. Schools with the greatest discrepancies between actual and typical distributions of results are investigated to identify the reasons for the discrepancy. Assistance with internal assessment tools and practices is then provided if the investigation indicates that such assistance is required.

455. New rules about the extent and number of further assessment opportunities and resubmissions that schools may offer students in an academic year have been put into place to ensure national consistency in further assessment.

456. NCEA certificates may be endorsed with Merit or Excellence. Students are awarded NCEA endorsed with Merit if they gain 50 credits or more from standards achieved with Merit or Excellence, and are awarded NCEA endorsed with Excellence if they achieve 50 or more credits with Excellence.

457. Course endorsement will allow school courses to be ‘Achieved with Merit’ and ‘Achieved with Excellence’. From 2011, students will be awarded a course endorsed with Merit if they achieve 14 or more credits with Merit or Excellence and will be awarded a course endorsed with Excellence if they achieve 14 or more credits with Excellence.

458. Current research shows that certificate endorsement has had a positive impact on student motivation, with levels of endorsement rising over 2007-2009. Course endorsement is expected to have a similar impact, enabling students who excel in a particular subject area to be recognised as a complement to their qualification.

459. The Standards Review programme is aligning school subject-based standards with the revised New Zealand Curriculum, and addressing issues of duplication and credit parity. This review is particularly important because it will ensure that curriculum standards have consistent associated credit values.

460. To assist with implementation of the revised standards, this programme includes the production of assessment resources for internal and external standards. These resources will help teachers and students to better understand the criteria for Achieved, Merit and Excellence grades for the revised standards. The materials will be trialled in schools to ensure they are fit for purpose. Examples of appropriate student work around each grade boundary will be identified, annotated and published along with the activities and schedules.

461. The development of new literacy and numeracy unit standards will provide an alternative pathway towards achieving NCEA Level 1 and support the initiative to raise literacy and numeracy achievement in schools.

462. Work is under way with a number of standard-setting bodies to allow for Merit and Excellence grades to be added to Unit Standards where appropriate, with the goal of improving consistency across the New Zealand Qualifications Framework. This initiative will also ensure that students are not discouraged from taking vocationally based courses or courses that combine curriculum and vocational elements.

Footnotes

  1. Ministerial Working Party on Assessment for Better Learning, (1990). Tomorrow’s Standards. Wellington: Learning Media, Ministry of Education.
  2. Ministry of Education (1994). Assessment: Policy to Practice. Wellington: Learning Media, Ministry of Education.
  3. Ministry of Education (1998). Assessment for Success in Primary Schools. Wellington: New Zealand Government, Green Paper.
  4. Absolum, M., Flockton, L., Hattie, J., Hipkins, R., & Reid, I. (2009). Directions For Assessment in New Zealand, p.5 (National Assessment Strategy review position paper, released) – can be accessed at: www.tki.org.nz/r/assessment/research/mainpage/directions/
  5. Hattie, J. A. C. (2009). Visible Learning. New York: Routledge.
  6. Ministry of Education (2007). The New Zealand Curriculum, p.40.
  7. Ministry of Education (2008). Te Marautanga o Aotearoa, (English translation) p. 13. www.minedu.govt.nz/NZEducation/EducationPolicies/MaoriEducation/Consultation/TeMarautangaOAotearoa/WhakapakehatiaOTeMarautangaOAotearoa.aspx
  8. Allen, P., Crooks, T., Hearn, S., & Irwin, K. (1997). Te Tiro Hou (Report of the Qualifications Framework Inquiry). Wellington: New Zealand Post Primary Teachers' Association.
  9. Black, P. (2000). Report to the Qualifications Development Group Ministry of Education, on the proposals for development of the NCEA. London: King’s College.
  10. NCEA Level 1 was implemented in 2002, NCEA Level 2 in 2003, and NCEA Level 3 in 2004.
  11. The requirements for University Entrance are being reviewed and are likely to change.
  12. From 2012 this will increase to 10 credits for each of literacy and numeracy. 2011 is a transition year where either 8 or 10 credits will count.
  13. www.nzqa.govt.nz/ncea/assessment/search.do?query=English&view=achievements&level=01
  14. www.minedu.govt.nz/NZEducation/EducationPolicies/Schools/PolicyAndStrategy/PlanningReportingRelevant‌LegislationNEGSAndNAGS/TheNationalAdministrationGuidelinesNAGs.aspx
  15. Ministry of Education, 1 March 2009 roll return.
  16. Ministry of Education (2009). SMS Capability Review. Wellington: Ministry of Education
  17. The purpose of the focus on literacy and numeracy skills is to ensure students are well equipped to progress in all learning areas of the curriculum. All curriculum learning areas provide contexts for the teaching and learning of literacy and numeracy.
  18. Ministry of Education website for teachers, Te Kete Ipurangi at: http://assessment.tki.org.nz
  19. Cowie, B., Jones, A., & McGee, C. (2008). Assessment Review Paper 10: Initial Teacher Education (ITE) and Assessment. Wellington: Ministry of Education.
  20. Lovett, S., & Sinclair, L. (2005). The Socialisation of Teachers into a Culture of Assessment. Wellington: New Zealand Council for Educational Research.
  21. Gilmore, A. (2008). Assessment Review Paper 8: Professional Learning in Assessment. Wellington: Ministry of Education.
  22. Lovett, S., & Sinclair, L. (2005). The Socialisation of Teachers into a Culture of Assessment. Wellington: New Zealand Council for Educational Research.
  23. Contracts for Assess to Learn Professional Development Programmes 2008-2010.
  24. Gilmore, A. (2008). Assessment Review Paper 8: Professional Learning in Assessment. Wellington: Ministry of Education.
  25. Education Review Office (2007), The Collection and Use of Assessment information in Schools. www.ero.govt.nz
  26. Hume, A., & Coll, R. K. (2009). Assessment of learning, for learning, and as learning: New Zealand case studies Assessment in Education: Principles, Policy & Practice, 16:3, 269-290.
  27. The literacy and numeracy progressions guide teaching (Years 1-10) across the curriculum.
  28. Poskitt, J., & Taylor, K. (2008). National Education Findings of Assess to Learn (AtoL) report. Palmerston North: Education Group. www.educationcounts.govt.nz/publications/schooling/27968/27984
  29. Mitchell, K., & Poskitt, J. (2009). Evaluation Assess to learn Professional Development. Auckland: The Education Group Ltd.
  30. Dobric, K. (2005). Drawing on Discourses: Policy Actors in the Debates over the National Certificate of Educational Achievement 1996-2000. New Zealand Annual Review of Education, 15, 85-109.