Serhii Diatlenko, Local and Regional Education Expert, Polaris Programme ‘Supporting Multilevel Governance in Ukraine’

When asked how to approach school rankings based on ZNO/NMT results, most people expect a simple and straightforward answer. But it never works that way. Why?

When the State Final Examination (SFE) for school leavers in Grade 11 took the form of an External Independent Assessment (EIA or in Ukrainian – ZNO), the ranking was more accurate because all school leavers had to take the ZNO to obtain their certificate. Now, those who do not plan to enrol at a higher education institution are exempt from taking the National Multisubject Testing (NMT). Those who want to continue their studies abroad are also exempt. This means that a significant proportion of students are excluded, making it impossible to obtain an objective picture of the overall results at a particular school. It seems likely that this picture would be even worse for most schools than it is now, but that is just speculation.

In addition, like the ZNO before it, the NMT in its current form does not assess academic achievement in all subjects. Consequently, a significant proportion of compulsory educational material falls into a ‘dead zone’, which also contributes to the fragmented monitoring of school performance.

Another reason for the devaluation of school performance ratings based on ZNO/NMT results is the proportion of the result attributable to each student. In any case, this reflects the efforts of the student and their family. However, there are also other factors at play, such as access to further education (extracurricular activities, educational and cultural infrastructure), and ultimately, the involvement of tutors. In some regions, tutors are responsible for 70–80 per cent of exam preparation for students in the final year of school.

For the most part, the knowledge-based approach used in the ZNO/NMT tests does not measure a significant proportion of educational activities in schools, namely graduates’ ability to apply their knowledge to solve practical problems. The international comparative study PISA revealed differences in approaches to task formation. And I find the PISA tests to be more objective. This is the direction in which we should be working.

My final ‘complaint’ about rankings based on ZNO/NMT results is that they cannot measure the soft skills that schools form and develop in students, and which have recently been incorporated into the essence and content of education. These soft skills have a significant impact on graduates’ future success.

All of the above is important. However, even all of this together is inferior to the argument that we currently have no better mechanism for monitoring the school’s performance. This mechanism is neither complete nor perfect, but it is the only one available. If this mechanism is imperfect, it should be used with caution and the results of the ZNO/NMT, and the ratings based on them, should not be taken as absolute. In other words, there is no simple answer.

When evaluating a school’s performance, results should not be considered for just one academic year, but for several, as each year's graduates are different and circumstances can have a significant correlation. Moreover, thanks to the systematic work in this area by the Ukrainian Centre for Educational Quality Assessment and private initiatives by individual educational resources, we are able to track the trends over a sufficient number of years to draw conclusions.

If a school consistently ranks in the top thousand, that is an excellent result. Like the subsequent rankings, the ‘top thousand’ is approximate and based solely on personal experience and understanding of the situation.

For schools ranked in the second and third thousand, it is necessary to consider their individual progress in this ranking over the years.

If a school consistently ranks outside the top 3,000–4,000 educational institutions nationally, there are likely to be significant issues with the provision of quality education.

If none of the community’s schools are among the top 4,000, it suggests that there are significant problems within the community regarding its attitude towards and value of education at the levels of local authorities, educators, and local residents.

In the vast majority of cases, schools outside the top 4,000 – remember that the 4,000 threshold is not absolute – have problems with not only the quality of their graduates’ knowledge as measured by the ZNO/NMT, but also with mastery of subjects not included in the list, as well as with the value of education, educational infrastructure, graduates' ability to apply their knowledge in practice and the formation of soft skills. In other words, by highlighting the results of the ZNO/NMT, we can predict a whole range of additional problems that are not currently being measured but which could be identified through appropriate monitoring.

Those at the top of the rankings usually trust and support such ratings. They have good reason to do so, by the way. Those at the bottom use the arguments mentioned at the beginning of this article. However, they only use these arguments until they are tested externally.

And in the vast majority of cases, this assessment will corroborate the ranking results.

The column reflects the opinion of its author only. The author is responsible for ensuring the accuracy of the information. The views expressed in the column may not coincide with those of the editorial staff of the Decentralisation Portal.