Indicator Framework

The indicator framework consists of:

• Criteria for selecting indicators. Eight criteria to ensure the selection of robust and enduring indicators that help focus on things that matter.
• Domains. Six domains used to organise and group indicators. The domains are based on an inputs-outputs-outcomes model that provides a logical and understandable way of relating the influences and resources that contribute to the key outcomes or achievements.
• Demographic dimensions. The different dimensions by which education sector indicators can be disaggregated to assess different sub-groups of the population, for example, gender, ethnic group.
• Education dimensions. The different education dimensions by which education sector indicators can be disaggregated to assess different groups of the population, for example, year of schooling.

Index of Indicator Framework:

Introduction
Criteria for Selecting Education Sector Indicators
Domain
Demographic Dimensions
Education Dimensions
Contextual Information

Introduction

A set of education sector indicators has been developed to help decision-makers to assess the health of the education system and to develop strategies as appropriate. In order to be more than an ad hoc collection of statistics an explicit framework for education indicators is needed.

The indicator framework provides a way of characterising and relating aspects of the education system to each other, to assist the interpretation of indicators to each other. Allowing the information provided by the portfolio of indicators to be greater than the sum of its parts.

The indicator framework consists of:

• Criteria for selecting indicators. Eight criteria to ensure the selection of robust and enduring indicators that help focus on things that matter.
• Domains. Six domains used to organise and group indicators. The domains are based on an inputs-outputs-outcomes model that provides a logical and understandable way of relating the influences and resources that contribute to the key outcomes or achievements.
• Demographic dimensions. The different demographic dimensions by which indicators can be disaggregated to assess different sub-groups of the population.
• Education Dimensions. The different education dimensions by which indicators can be disaggregated to assess sub-groups of the population.
• Contextual Information. General information that assists with the interpretation of the indicators and makes sense of trends over time and differences between sub-groups and with other countries.

Education sector indicators

Indicators are established measures used to determine how well a result has been achieved in a particular area of interest. For example, the rate of formal school qualifications helps quantify whether kids are succeeding at school. The education sector indicators focus on the outcomes of education and training, and the important influences on those outcomes.

There are three primary purposes to education sector indicators:

1. to provide a system-wide assessment - early childhood education, through schooling, to tertiary education/training - of key aspects of the education system and of education outcomes that can be monitored over time
2. to highlight national trends in various aspects of education and consider how New Zealand compares with other countries in education and skills development
3. to help identify key issues to inform the Ministry's strategic planning, policy, research and information priorities.

Education sector indicators are not designed to evaluate the performance of governments or the effectiveness of specific policies and programmes. These indicators are not targets and, on their own, do not provide enough information to diagnose problems or provide explanations. However, indicators can reveal interesting questions and stimulate hypotheses about problems or what works. In this sense, indicators effectively give an overview of a more detailed story and provide a starting point that leads the reader to ask more specific questions.

Criteria for Selecting Education Sector Indicators

The choice of indicators is as always a matter of some debate. While stability in the indicator set over time is important to assist with the identification of trends, the development and reporting on indicators will always be a work in progress.

There is always a risk of goal displacement arising from poorly selected indicators. It is therefore important to focus on measuring things that matter. The overall approach and the selection of indicators are based on:

• the broad goals of the New Zealand education system, in particularly there is a focus on those education outcomes which have both current relevance and enduring significance for New Zealand
• the best current evidence of the factors which influence learning outcomes
• international best practice in education indicators reporting.

Criteria for choosing robust and enduring indicators have been developed. The eight criteria are based on the education indicator literature, and the criteria that are employed in other sectors, such as health. Criteria for selecting indicators are:

1. Relevance to education outcomes. The indicators should measure either education outcomes, subsequent social and economic outcomes that are strongly related to education, or key influences on education outcomes.
2. National or systemic significance. The indicators should reflect progress at a national level across either the population as a whole or across significant population groups or other relevant groupings. The focus is on overall educational 'health' and progress and not on particular schools.
3. Ability to be disaggregated. The indicators should have the potential to be disaggregated along relevant dimensions to show differences between important population subgroups and other groupings. The indicators must help us understand disparities in the outcomes of education.
4. Validity, analytical rigour and accuracy. The indicators should be statistically sound and methodologically rigorous, including being derived from high quality data and providing a valid and accurate representation of the phenomenon (and changes in the phenomenon) in question.
5. Interpretability and transparency. A broad audience should easily understand the indicators and known limitations should be transparent. The indicators should have a clear and accepted normative interpretation whereby, all other things being equal, movement in one direction clearly represents an improvement in the phenomenon being measured.
6. Stability and consistency over time. The indicators should have a consistent definition (both in terms of what is being monitored and how it is being measured) and be able to be measured consistently over time. The core value of indicators is their use in monitoring trends.
7. International comparability. While the indicators we choose should reflect issues of importance within New Zealand, they need to be consistent with international indicator programmes to enable international comparisons to be made. This provides an important benchmark and standard by which to measure our progress and performance.
8. Feasibility and timeliness. The indicators should be able to be collected and analysed cost-effectively and provide current information in a timely manner.

Indicators do not always meet equally all eight of the criteria, as trade-offs between criteria are sometimes necessary. For example, lack of international comparative data has not ruled out an otherwise useful indicator. The criteria at least allow such trade-offs to be consciously made.

Domain

Part of this framework includes a series of domains. The domains are the organising structure for the key aspects of the Education System. There are six domains to the indicator framework:

• education and learning outcomes
• effective teaching
• student participation
• family and community engagement in education
• quality education providers
• resourcing.

 

The purpose of the education sector is to improve education and learning outcomes. Effective teaching, student participation, family and community engagement in education, and quality education providers are collectively the major influences on achieving that goal. Adequate resourcing underlies all these influences.

Education and learning outcomes

This domain covers the 'results' of the education system. It includes the skills, knowledge, attitudes, and values gained through the education system.

The Ministry of Education's goals in education are to raise achievement and reduce education disparity. This domain shows the extent to which the goals are being achieved. It shows if New Zealand is heading in the right direction and how well we compare to other countries in the education of our population.

The education system has to meet the wide variety of educational requirements at all ages and stages of learning. It is therefore important to consider outcomes throughout an individual's progression through the education system.

An individual's learning and experience in the education system has long term consequences and some outcomes may not be realised until much later in life. Beyond educational qualifications, labour market outcomes of education are also included in this domain.

Effective Teaching

This domain looks at teaching and educator factors associated with the quality and effectiveness of learning.

Learning is mostly organised in small group settings, where teachers are the primary agents for planning, pacing and monitoring learning. Effective teaching is a major influence over a student's learning and success. Good teaching is powerful and can offset many factors that can exert a negative influence in a student's life.

Student Participation

This domain looks at who has access to the opportunity to learn, and who is enabled to take up the opportunity.

Participating and involvement in formal education is a key influence on learning outcomes. Participation in learning has a considerable impact on a student's subsequent life; it contributes to well-being, employment, and income benefits.

Family and Community Engagement in Education

This domain covers family and community factors which influence access to education opportunities and equity of achievement and qualifications.

Supportive families and communities are powerful influences on learning outcomes. The better the formal learning environment respects and affirms the learner's home environment, and incorporates this into the learning process, the higher the level of likely achievement.

Quality Providers

This domain is concerned with the management and governance efforts that are applied to the resources in the sector to produce desired outcomes.

Quality providers play a pivotal role in achievement through their commitment to teaching excellence, support for both teachers and learners, encouraging community engagement in education, and engaging the strengths of peers and families.

Resourcing

In this domain the concern is with the quantity, adequacy and deployment of resources: financial, information and physical/material.

The quantity and deployment of financial, information, and physical resources in the education sector are critical to its performance and ability to provide quality education that leads to achievement.

Outcomes			Inputs
Education and learning outcomes: knowledge skill sense of identity values attitudes Labour market and social outcomes: employment income	Family and community engagement in education: home learning environment affordability of education	Quality providers: effective leadership teacher support community engagement	Resourcing: investment in education student allowances research and evaluation

The domains map to the conventional, and relatively well understood, indicator approach of inputs-outputs-outcomes. In reality, systems are far more complex and interactive than this approach implies. However, it can help put indicators into a context.

Demographic Dimensions

These are the different dimensions by which education indicators can be disaggregated to assess different sub-groups of the population.

Ethnic Groups

A fundamental principle of our approach to monitoring is that where an indicator is thought to be worthy of monitoring, it should be monitored by separate ethnic groups as well as the total population. The standard ethnic group breakdown used by the education sector indicators are:

• European
• Māori
• Pasifika
• Asian
• Other (includes individuals not included in the other four ethnic groups).

Current information sources have tended to limit analysis to the major ethnic groups, however, as information systems mature more detailed analysis should be possible.

Pacific peoples in New Zealand comprise at least 22 different cultures and a greater number of dialects. The seven major population groups are from Samoa, Cook Islands, Tonga, Fiji, Niue, Tokelau, and Fiji. For some areas of New Zealand there will be a benefit from undertaking analysis of indicators for these population groups, where data is available.

SocioEconomic Status

Where data allows, indicators are monitored by socioeconomic status. This is achieved by using the school-economic decile band. All schools are given a decile rating, depending on the socioeconomic rating of the area they serve.

The rating is based on Census data for the families of school age children in the areas from which each school draws students, along with school ethnicity data. The Census data include equivalent household income, parents' occupation and educational qualifications, household crowding, and income support payments received. After combining the data, all schools were ranked into deciles.

Gender

Where data allows indicators are monitored by gender.

Age

Age can be used to sub-divide the population. This occurs most often with indicators about tertiary education. However, for school based indicators, the education sector indicators usually use 'stages of education' or 'year of schooling' instead of age (see following). The stages of education and year of schooling introduce both age and curriculum into the stratification.

Region

Where technically feasible, indicators may be presented by region. It is more likely that supporting statistical information will be available by region. Information may be provided by:

• territorial authority
• local body region
• Ministry of Education district
• Ministry of Education region.

Education Dimensions

Similar to population dimensions, these are the different education dimensions by which some education indicators can be disaggregated to assess different sub-groups of the population.

Stages of Education

Education sector indicators may be specific to the different stages of education, from early childhood through to tertiary education. The stages used by the indicators are:

• special education
• early foundations; from birth to the first years of schooling
• primary schooling; the early years of schooling, Year 1 to Year 6
• middle schooling; the middle years of schooling, Year 7 to Year 10
• senior secondary schooling; Year 11 to leaving school
• tertiary education.

Special education is the provision of extra help, adapted programmes or learning environments, specialised equipment or materials to support children and young people with their learning and help them participate in education. Students with special education needs include learners with disabilities, learning difficulties, communication or behaviour difficulties, sensory or physical impairments.

Year of schooling

The stages of education that relate to schooling (primary school through to secondary school) can be subdivided further. Year of schooling measures the number of years of schooling a student has received. Many assessments of students are undertaken on a specific year of schooling. And indicators derived from these assessments will correspondingly be specific to that particular year of schooling. For indicators relating to school stages of education, year of schooling is often used instead of age.

Level of study

The National Qualifications Framework (NQF) is designed to provide:

• nationally recognised, consistent standards and qualifications
• recognition and credit for all learning of knowledge and skills.

Framework qualifications are quality assured and nationally recognised. Administered by the Qualifications Authority, the Framework is a way of structuring national qualifications.

Within the qualification framework there are ten levels of study (see below). Level 1 is the least complex and 10 the most. Levels depend on the complexity of learning. They do not equate to 'years spent learning' but reflect the content of the qualification.

The levels used by the indicators are:

• Level 1: Certificate
• Level 2: Certificate
• Level 3: Certificate
• Level 4: Certificate
• Level 5: Diploma
• Level 6-7: Bachelors level
• Level 8: Honours/Postgraduate diploma or certificate
• Level 9: Masters Degree
• Level 10: Doctorate.

Levels 1-3 are of approximately the same standard as senior secondary education and basic trades training. Levels 4-6 approximate to advanced trades, technical and business qualifications. Levels 7 and above equate with advanced qualifications of graduate and postgraduate standard.

Education provider

Education sector indicators may present information by provider type, for example, full primary schools, intermediate schools, Year 9-13 secondary schools. A Māori medium provider presentation of data could also be provided.

It is unlikely that education sector indicators will be monitored by specific provider, however, supporting statistical information may be available by provider.

Contextual Information

The education system operates within the wider demographic, economic, social and cultural context of New Zealand.

The demands for education services, the pattern of participation and education outcomes, subsequent employment outcomes, and the amount of resources available to education are all affected by a number of factors in the broader environment, for instance:

• the size of the population, especially for schooling where enrolment is compulsory between 6 and 16 years
• the general wealth of the country, which relates to its ability to invest in education
• general labour market trends can influence uptake of education; when unemployment rises participation in education also tends to increase. The relative ability of a family to meet the rising costs of education and to provide adequately for school age children depends in part on their income.

This wider information helps us to interpret the education indicators and to make sense of trends over time and differences between subgroups and with other countries.

Contextual information may include:

• trends in the size, structure and nature of the population
• trends in GDP
• labour force status of the working-age population
• income distribution.