Summary

As part of the EER process, PTEs were assigned a Category status following the publication of an EER report by NZQA. This signalled what the overall quality rating of the PTE was, with a Category 1 representing the highest quality and a Category 4 the lowest. A Category 3 or 4 status requires improvement by the PTE and then re-evaluation by NZQA to confirm it has occurred.

An earlier Ministry study found that the Category status of PTEs had improved between 2013 and 2022. However, it did not look in depth at the pathways followed by individual PTEs and did not look at Category ratings prior to 2013.

Sufficient time has now passed so that quality ratings assigned to individual PTEs can be tracked to get more insights into the dynamics of the quality assurance system. Specifically, this new report mostly looks at cohorts of PTEs that received their first Category status in the roll out of the EER system between 2009 and 2013 and how they performed over time.

The analysis split the PTEs into two cohorts, those with a first Category status assigned between 2009 and 2011 (2009-11 cohort) and a second cohort with a first Category assigned between 2012 and 2013 (2012-13 cohort). The 2009-11 cohort comprised 331 PTEs and the 2012-13 cohort 280 PTEs.  Two cohorts were used because of the longer window of time the 2009-11 cohort had to be evaluated compared to the 2012-13 cohort.

Results of the analysis

The analysis looked to answer the following questions.

1. How did the Category of PTEs change over time? Did they improve, deteriorate, or stay the same? How did this vary between PTEs that exited the sector and those that were still operating?

To help answer the questions, the first and last Category status for PTEs with at least two statuses assigned were compared. It showed that there were different patterns of change between PTEs that had deregistered and those still registered at the end of 2024. The data showed that:

• Registered PTEs: Both cohorts had more PTEs where Category status improved than declined. This resulted in a significant improvement in the Category distribution for these PTEs with a higher percentage of Category 1 PTEs and lower percentage of Category 3 and 4 PTEs.
• Deregistered PTEs: Both cohorts had more PTEs where Category status declined rather than improved. The impact on Category distribution was that although roughly the same percentage of PTEs were Category 1 or 2, the percentage of Category 1 PTEs decreased and the percentage of Category 4 PTEs increased.

PTEs that were still registered had exhibited a significant improvement in Category status. This improvement in quality ratings would imply that the EER system was working as intended and was associated with an improvement in the quality of providers in the longer term.

The lowest Category status assigned to PTEs was then analysed to identify patterns in the track of PTEs. The data showed that:

• Over both cohorts, a significant percentage of PTEs were only ever assigned a Category 1 status, including those PTEs with three or higher statuses in a row. The figure for the 2009-11 cohort was 28 percent and for the 2012-13 cohort 23 percent.
• A significant proportion of PTEs had been assigned at least one Category 3 or 4 status. Around 40 percent of PTEs in the 2009-11 cohort and 29 percent in the 2012-13 cohort had at least one Category 3 or 4 rating.

2. What were the most common quality pathways?

This section tracked the Category status assigned to PTEs over time to identify the top 10 most common quality pathways followed. This also considered registration status when classifying pathways. Although each cohort had a slightly different order to the pathways, generally they involved PTEs displaying consistently high Category statuses over time with only a couple featuring a Category 3 status. The data showed that:

• For the 2009-11 cohort, the most common pathway was where a PTE was assigned Category 1 status four times in a row and was still registered. Category 1 and 2 statuses featured prominently in the pathways, with two instances of a Category 3 status. Seven of the top 10 pathways ended in deregistration, with three involving deregistration after the first Category status was assigned.
• For the 2012-13 cohort, the most common pathway was where a PTE was assigned a Category 2 status and then later deregistered. Category 1 and 2 statuses featured prominently in the pathways, with two instances of a Category 3 status. Deregistration also featured in six of the pathways.

3. How many times have individual PTEs been assigned a Category 3 or 4 status?

This analysis looked to quantify how common it was for lower quality ratings to recur at PTEs, including how many in a row were assigned. Generally, for PTEs assigned a Category 3 or 4 status, it proved to be a one-off event. The data showed that:

• For the 2009-11 cohort, 40 percent of PTEs were assigned a Category 3 or 4 status. The majority of those were only assigned that Category on a one-off basis. Ten percent of PTEs were assigned two Category 3 or 4 statuses, 4 percent were assigned three, and 2 percent four. The most Category 3 or 4 statuses in a row was four, which was exhibited by two PTEs.
• For the 2012-13 cohort, 29 percent of PTEs were assigned a Category 3 or 4 status. Only four percent of PTEs were assigned multiple instances of Category 3 or 4 ratings, whilst one percent had three instances.

There were a small number of cases where these low quality ratings persisted, indicating these PTEs were given multiple opportunities to improve performance which perhaps raises the question as to how many chances a PTE should be given to attain a satisfactory Category rating.

4. Did Category 3 and 4 PTEs improve in the longer term?

This analysis looked to identify how many PTEs that after initially recovering from a Category 3 or 4 status had later ‘relapsed’ and been assigned that status again. The data showed that:

*For the 2009-11 cohort, around half of PTEs who initially recovered from a Category 3 or 4 status were later assigned that level of Category again. For the 2012-13 cohort, the rate of relapse was much lower, at 16 percent.
Despite improving in the later cohort, the rate of relapse in the earlier cohort may indicate that some form of additional monitoring should be applied for a period of time even after an improvement from a low Category status.

5.  What did it take for Category 4 PTEs to improve longer term?

For those PTEs which had recovered in the longer term after being assigned a Category 4 status, the background sections of EER reports were examined to see what actions were taken prior to the turnaround in Category status.

• The most common actions were:
  • appointing new chief executives or management
  • the replacement of boards
  • seeking external assistance, whether it was for management tasks, moderation and assessment, or help in QA processes.

These actions represent substantial changes and indicates that the successful long-term recovery from Category 4 status required significant change at the PTE.

6. How is Category status associated with the likelihood of deregistration?

Earlier analysis identified that deregistration featured in many of the top 10 pathways. Given the prominence of this, the data was analysed to examine the association between lowest Category status assigned and likelihood of deregistration. As might be expected, generally the worse the Category status the higher was the likelihood of being deregistered. The data showed that:

• overall, there was a high level of ‘churn’ in the PTE sector, with 60 percent of 2009-11 cohort PTEs being deregistered and 53 percent of 2012-13 PTEs.
• in both cohorts, the likelihood of deregistering was lowest for Category 1 and 2 PTEs, with a very high likelihood of Category 4 PTEs eventually being deregistered. In terms of the latter, 86 percent of PTEs in the 2009-11 cohort and 92 percent of PTEs in the 2012-13 cohort with a Category 4 status were deregistered.

7. How do mergers and changes of ownership impact on the quality pathways of PTEs?

This section looked at how mergers and changes in ownership are associated with quality ratings. As information on mergers and ownership is not easily compiled for the hundreds of PTEs that were in operation between 2009 and 2024, in this analysis case studies of selected groupings of PTEs were used. Their history of mergers and changes in ownership and Category statuses between 2010 and 2024 were mapped on flow charts.

The analysis did not control for factors other than ownership changes and mergers which can impact on Category status. This, along with these case studies not covering all PTEs, means that the findings should be viewed with caution.

The analysis showed that:

• There were numerous changes of ownership and mergers over the 15-year period amongst the three groupings, although the number of these has generally slowed as the selected PTEs consolidated into larger entities.
• Due to the complexity of the changing environment and the unique circumstances which likely existed, it is difficult to assess whether the mergers and ownership changes impacted on the quality rating. However, there were only rare cases of a Category 3 or 4 status being assigned following a merger or ownership change and in general good quality ratings were maintained.
• There was one case where a cluster of Category 3 or 4 statuses prior to merger and change of ownership was followed, at least initially, by more Category 3 statuses. But the quality ratings of the remaining PTEs were Category 1 and 2 by the end of 2024 indicating the lower Category status did not persist long term under the new owners.