Education Counts

Page navigation links

  • Education Counts Logo
  • Skip to Primary Navigation Menu
  • Skip to Secondary Navigation Menu
  • Skip to search
  • Skip to content

Site Search

Site Search

Site navigation menus

  • Know your Region
  • Communities of Learning
  • Find your nearest school
  • Early Learning Services
  • Directories
  • Publications
  • Indicators
  • Statistics
  • Topics
  • Data Services

Search the education counts website

Find pages with

Narrow results by:

National Standards: School Sample Monitoring and Evaluation Project Publications

Publication Details

This is the first report from the National Standards School Sample Monitoring and Evaluation Project 2010-2013, a three year project on National Standards implementation in a representative sample of schools.

This report presents results from a mid-2010 survey of 82 principals from schools in the project's sample, and an analysis of sample schools’ formats for reporting to parents in 2010 mid-year reports.

Author(s): Maths Technology Limited. Report for the Ministry of Education.

Date Published: October 2010

Summary

The School Sample Monitoring and Evaluation Project is a three year project, established to evaluate the implementation of National Standards using a large representative sample of schools. This is the first report from the project and describes results gathered using two methods: an online principal survey and an analysis of the formats schools used for mid-year reporting to parents. As this data was gathered six months after National Standards were first put in place in schools the report provides early information on the implementation of National Standards.

The project has two purposes: to describe the implementation of National Standards within schools, and to monitor and systematically evaluate the effect of National Standards on students, teachers, schools and whānau. The descriptive element of the project is focused around twelve monitoring and evaluation questions, while evaluation is based on seven intentions which describe the intended effects of National Standards. A variety of data sources are employed (see Table 1 for an overview).


Table 1 presents the project's methodology and identifies the two monitoring and evaluation questions addressed in this report:

1. How do schools use information from National Standards to report to and communicate with parents?
7. To what extent are the National Standards understood as a set of common expectations for student achievement?


Note that the relevant data sources and statements of intent for each question are also identified in Table 1.

Table 1: Methodology1,2
Antecedents
Monitoring and Evaluation Questions:
  1. To what extent are the National Standards understood as a set of common expectations for student achievement?
  2. What processes are employed by schools to maintain consistent application of the National Standards?
Data Sources:
  • Online survey: Principals and BOT representatives
  • Principal interviews
  • Schools' achievement targets and analysis of variance reports
Intentions:
  1. National Standards will provide clear information about student achievement for Boards of Trustees which can be used in decision making and resource allocation processes.
Transactions
Monitoring and Evaluation Questions
  1. In what ways do teachers use information from a variety of student assessments to make overall judgments?
  2. How dependable and consistent are teachers' overall judgments?
Data Sources:
  • Student achievement data
  • Online assessment scenarios.
  • Online surveys: teachers and principals
  • Principal interviews.
Intentions
  1. Teachers will make defensible, trustworthy judgments against the National Standards.
Outcomes
Monitoring and Evaluation Questions
  1. What changes in student achievement in reading, writing and mathematics are observed as National Standards are introduced?
  2. What changes in teachers' professional knowledge and practice are observed as National Standards are introduced?
  3. In what ways is information from National Standards used by schools to set achievement targets?
  4. In what ways is information from National Standards used by schools to describe student achievement and progress?
  5. In what ways is information from National Standards used to provide targeted teaching interventions?
  6. In what ways is information from National Standards used to identify teachers' professional development needs?
  7. How do schools use information from National Standards to report to and communicate with parents?
  8. To what extent do parents understand, value, and use National Standards information about their child?
Data Sources

Student Achievement:

  • OTJs-standardised
  • assessments

Teachers:

  • online surveys

Schools:

  • achievement targets-analysis of variance reports
  • online surveys: principals
  • individual interviews: principals
  • reporting formats

Whānau:

  • online survey: parents
  • school report
Intentions
  1. Information from National Standards assessments will be used by teachers and schools to monitor student progress and achievement against the Curriculum.
  2. As a result of using National Standards to monitor achievement and progress some students will be provided with targeted teaching interventions.
  3. Student achievement will improve.
  4. Schools will use National Standards assessment information to communicate clearly with families about their child's achievement and progress.
  5. National Standards information will be used to identify teachers' professional development needs. This will enable these to be addressed more effectively.

Sample

All English-medium, full primary, contributing and intermediate state schools were included in the project's sampling frame. A stratified sampling procedure based on three school characteristics was used, with three groups within each characteristic:

  1. School decile: one to three, four to seven, eight to ten.
  2. School type: full primary, contributing, and intermediate
  3. Region: Auckland, North Island excluding Auckland, and South Island


The sample included positions for 126 schools. Each of the 27 different combinations of school characteristics were represented proportionally to ensure the sample was representative of the larger population of schools in New Zealand.

The sample on which this report is based is smaller than the wider sample as school recruitment was still underway at the time of data collection. Included in this report are the survey responses from 82 principals and the school reporting formats of 70 schools which were received by 31 August 2010.

Table 2 shows the characteristics of the 82 schools whose principals responded to the survey. In general, schools that provided school report formats were a sub-set of schools whose principals completed the survey. All but one of the schools that provided their template for reporting to parents also completed the principal survey.

Table 2: School characteristics of respondent principals
Note:
  1. *excluding Auckland
School TypeRegionDecile
YearsSampleNationallyRegionSampleNationallyDecilesSampleNationally
1 to 8 45% (37) 45% Auckland 24% (20) 23% 1 to 3 26% (21) 27%
1 to 6 39% (32) 34% North Island* 45% (37) 48% 4 to 7 38% (31) 41%
7 to 8 16% (13) 21% South Island 30% (25) 29% 8 to 10 37% (30) 32%


In terms of the three main stratifying variables the sample is representative, although some of the combinations of characteristics are less well-represented. In particular the following schools are under-represented:

  • Low decile Year 1-8 schools in the South Island (under-represented by one school)
  • Low decile Year 7-8 schools in the South Island (under-represented by one school)
  • Low decile Year 7-8 schools in the North Island, excluding Auckland (under-represented by two schools)

Principal survey

The online principal survey contained four sections, each focused on gathering different information: principals' understandings of National Standards, principals' opinions of National Standards, communication about the National Standards, and mid-year reports to parents. Questions required a mix of closed and open responses. A copy of the survey is included as Appendix A.

Eighty-two principals were emailed a link to the survey on 27 July and asked to complete it within two weeks. An additional six principals were emailed the link to the survey as they confirmed their participation in the project; in total 88 principals were invited to complete the survey. The results of the 82 principals who completed the survey by 31 August are included in the analysis; a response rate of 93%.

Descriptive statistics were used to analyse closed responses. Common themes in open comment fields were identified. Themes identified by greater than five percent of respondents are included in the report. Due to rounding percentages in some tables do not total to 100.

School reporting formats

Eighty-eight schools were asked to provide a copy of the format they used to report to parents mid-year. As with the principal survey, requests were made to 82 of these schools on 27 July with requests sent to the remaining six schools as their participation in the project was confirmed. In total 70 schools provided reports by 31 August and these are included in this analysis; a response rate of 80%.

The document analysis was carried out by four raters. Initially the raters worked as a group to establish the characteristics and features of the report formats. Sixteen characteristics were identified and the researchers then worked independently to analyse the school report formats in terms of the presence or absence of these characteristics. Appendix B provides the details of this analysis and inter-rater reliability statistics are included on page fifteen, alongside the relevant results.

Footnotes

  1. Italics and bold type are used to indicate the two monitoring and evaluation questions which are the focus of this report. For each question the data sources used and intentions addressed are identified.
  2. The project is grounded in the analytical framework provided by Stake's countenance model. Stake, R. E. (1967). The countenance of educational evaluation. Teachers College Record, 523-540.

Navigation

  • Schooling
  • Learners
  • Education | Learning Outcomes

Downloads

  • Full Report (PDF, 3.0 MB)
  • Full Report (DOC, 6.1 MB)

Where to find out more

  • National Standards: 2011 School Sample Monitoring &...
  • National Standards: Survey of Parents & Whānau 2011
  • National Standards: 2010 School Sample Monitoring &...

Contact Us

Education Data Requests
If you have any questions about education data please contact us:
Email:   Requests EDK
Phone: +64 4 463 8065

Home Close Menu
  • Know your Region Show submenu
  • Communities of Learning Show submenu
  • Find your nearest school Show submenu
  • Early Learning Services Show submenu
  • Directories Show submenu
  • Publications Show submenu
    • Early Childhood EducationShow submenu
      • Responding to diverse cultures: Good practice in home-based early childhood servicesShow submenu
    • MāoriShow submenu
    • SchoolingShow submenu
    • PacificShow submenu
    • Tertiary EducationShow submenu
    • Learning SupportShow submenu
      • Learning Support Coordinators Evaluation: Phase 2Show submenu
    • InternationalShow submenu
    • Publication SeriesShow submenu
  • Indicators Show submenu
  • Statistics Show submenu
    • Annual Monitoring Reading RecoveryShow submenu
    • Attainment of 18-year-oldsShow submenu
    • AttendanceShow submenu
    • Attendance under COVID-19Show submenu
    • Beyond StudyShow submenu
    • Early Learning ParticipationShow submenu
    • ECE FinancesShow submenu
    • ECE ServicesShow submenu
    • ECE StaffingShow submenu
    • Early Leaving ExemptionsShow submenu
    • Entering & leaving teachingShow submenu
    • Fees Free tertiary educationShow submenu
    • Financial Support for Tertiary StudentsShow submenu
    • Funding to SchoolsShow submenu
    • HomeschoolingShow submenu
    • Initial Teacher Education StatisticsShow submenu
    • International students in NZShow submenu
    • Language use in ECEShow submenu
    • Literacy & NumeracyShow submenu
    • Māori Language in SchoolingShow submenu
    • NZ's Workplace-based LearnersShow submenu
    • Number of SchoolsShow submenu
    • Ongoing Resourcing SchemeShow submenu
    • Pacific Language in SchoolingShow submenu
    • Per Student Funding for SchoolsShow submenu
    • Post-compulsory education & trainingShow submenu
    • School BoardsShow submenu
    • School Board RepresentationShow submenu
    • School Leaver DestinationsShow submenu
    • School Leaver's AttainmentShow submenu
    • School RollsShow submenu
    • School Subject EnrolmentShow submenu
    • Stand-downs, suspensions, exclusions & expulsionsShow submenu
    • Teacher NumbersShow submenu
      • 2021Show submenu
      • 2020Show submenu
    • Teacher TurnoverShow submenu
    • Tertiary Financial PerformanceShow submenu
    • Tertiary ParticipationShow submenu
    • Tertiary Population DataShow submenu
    • Tertiary ResearchShow submenu
    • Tertiary ResourcingShow submenu
    • Tertiary Achievement & AttainmentShow submenu
    • Tertiary Summary TablesShow submenu
    • Total public expenditure on educationShow submenu
    • Transient StudentsShow submenu
    • Vocational Education & TrainingShow submenu
  • Topics Show submenu
  • Data Services Show submenu

Site information

  • Site map
  • Contact us
  • Feedback
  • About this site
  • Glossary
  • Accessibility
  • Copyright, Legal & Privacy
  • Links
  • © Education Counts 2022
  • Ministry of Education logo.
  • New Zealand Government logo.
Scroll to top of page