Quality ratings of PTEs: an analysis of private training establishments’ external evaluation and review reports 2009-2022 Publications
Publication Details
In this report, information from 1,701 EER reports of private training establishments (PTEs) published between 2009 and 2022 is analysed to identify patterns and trends in the quality ratings for educational performance and self-assessment.
Author(s): Dr. Warren Smart, Tertiary System Performance Analysis, Ministry of Education
Date Published: April 2024
Summary
Key points
This report analysed the quality ratings from 1,701 external evaluation and review (EER) reports for private training establishments (PTEs). The analysis looked to answer the following four questions.
1. What are the trends in quality ratings?
- Category
- Overall, the quality rating of PTEs resulting from the EER reports was generally high, with around 80 percent of the reports resulted in a Category status where there was no improvement plan required and the next EER was scheduled for the usual four years in the future.
- Within the context of a decreasing number of PTEs, the as at December data showed that the percentage of Category 1 PTEs improved over time, from 45 percent in 2013 to 58 percent in 2022. This improvement was a result of both PTEs who were registered over the period improving Category, combined with the Category of new PTEs being better than the Category of exiting PTEs.
2. What are the underlying areas driving any changes in the quality ratings?
- Statements of confidence. Although the percentage of ‘Highly confident’ ratings improved over time in both educational performance and self-assessment (using the as at December in each year data), the pattern of improvement was different. The data showed that:
- Educational performance: The percentage of ‘Highly confident’ ratings steadily increased from 45 percent in 2013 to 58 percent in 2022.
- Self-assessment: The percentage of ‘Highly confident’ ratings in 2013 (23 percent) was lower than for educational performance but improved faster up to 2018 (40 percent). From 2018 to 2021, there was little change in the percentage of ‘Highly confident’ ratings, before rising to 43 percent in 2022.
- Key evaluation questions (KEQs). The distribution of quality ratings broadly followed the patterns seen in the statements of confidence with a general improvement in the percentage of ‘Excellent’ ratings, although with the different patterns of improvement between educational performance and self-assessment. The as at December in each year data showed that:
- Overall, the percentage of ‘Excellent’ ratings was higher in educational performance than in self-assessment and in most KEQs the percentage of ‘Excellent’ ratings improved over time.
- The KEQ evaluating how well programme design and delivery matched the needs of students and other stakeholders in educational performance did not show any improvement over time, which contrasted with the general pattern of improvement.
- In 2022, the KEQ evaluating how effectively students were supported and involved in their learning had the highest percentage of ‘Excellent’ ratings in education performance (77 percent) and self-assessment (65 percent). The KEQ evaluating how well students achieved had the lowest percentage of ‘Excellent’ ratings in both educational performance (56 percent) and self-assessment (46 percent).
3. Do quality ratings vary by level of study and field of study?
- An analysis of the ratings applied to qualifications/training schemes within focus areas showed there was considerable variation in the distribution of quality ratings.
- Educational performance: The analysis of combinations of level of study and field of study showed that Level 7-10 qualifications within the field of Creative arts showed the highest percentage of ‘Excellent’ ratings (88 percent), while Level 6 qualifications in Management and commerce had the highest percentage of ‘Adequate’/’Marginal’/’Poor’ ratings (40 percent).
- Self-assessment: The combinations of level 7-10 qualifications in Education showed the highest percentage of ‘Excellent’ ratings (63 percent), while Level 6 qualifications in Management and commerce had the highest percentage of ‘Adequate’/’Marginal’/’Poor’ ratings (44 percent).
4. What characteristics of PTEs are associated with higher quality ratings?
This analysis applied statistical modelling to characteristics of private training establishments (PTEs) to see if they were associated with higher quality ratings. When comparing the characteristics with Category of PTEdescriptive statistics indicated that there was a higher percentage of Category 1 statuses if the PTE: had more experience, was larger, could not enrol international students, and was government funded. The statistics also showed that reports published in 2016-22 had a higher percentage of Category 1 statuses than reports published in 2009-15. There was not a particularly large difference in percentages between the categories in the characteristics, with the largest being seen in experience and the year the report was published.
When logistic regression was applied (which controls for other factors and tests for statistical significance), the data showed that the nature of the associations was as follows:
- Where a PTE had lesser experience, the likelihood of attaining a Category 1 status was lower for PTEs that could enrol international students compared with PTEs that could not. However, this gap disappeared for PTEs with more experience.
- Similarly, for PTEs with lesser experience, those that were government funded had a higher likelihood of attaining a Category 1 status than those who were not. Once again, this gap disappeared for PTEs with more experience.
- For PTEs that could enrol international students, larger more complex PTEs had a much lower likelihood of attaining a Category 1 status than smaller PTEs. However, for PTEs that could not enrol international students, larger more complex PTEs had a slightly higher likelihood of attaining Category 1.
- For PTEs that could enrol international students, those that were government funded had a higher likelihood of attaining a category 1 status. However, if the PTEs could not enrol international students, there was little difference in likelihood of attaining a Category 1 status whether they were government funded or not.
The same analysis was applied to the ratings for statements of confidence and showed similar associations as was seen for Category. However, in one difference, the year the EER was published was also a significant factor for self-assessment. Reports published in 2016-22 had a higher likelihood of attaining a ‘Highly confident’ rating than those published in 2009-15, even after controlling for other factors such as experience. This suggests there was improvement over and above what might be expected given the mix of PTEs.
An analysis of a subset of 324 EER reports showed that PTEs with a Low financial risk rating had a higher likelihood of attaining a Category 1 status than PTEs with a Medium or High financial risk rating. Similar results were found when looking at the likelihood of attaining a ‘Highly confident’ rating in the statements of confidence.
Navigation
Where to find out more
Contact Us
For more information about the content on this webpage, please email the: Tertiary Mailbox