José L. CERVERA, international expert of U-LEAD with Europe
This paper presents a brief description of the system for monitoring the regional and local development policy. It includes an assessment of the current situation in the form of a SWOT analysis, followed by proposals for the specific objectives and actions in a monitoring system accompanying the State Strategy for Regional Development through 2027. Strong aspects and opportunities include the possibility and interest of donors like the EU to further strengthen evidence-based policy making and to improve e-government and ICT for data transmission, storage and dissemination. Weaknesses include a complicated legal system with overlaps and discrepancies, the lack of a conceptual framework linking territorial decentralisation, amalgamation and regional development, poor coordination of the monitoring and evaluation system, and incomplete comparability with EU statistics. Unmet information needs include indicators on Smart Specialisation Strategies, on problematic territories, and on the measurement of inter- and intra-regional disparities. In addition, progress is threatened by low levels of statistical literacy and trust in official statistics among policy-makers and a shortage of staff with advanced quantitative skills.
“You can't manage what you can’t measure”
(attributed to Edward Deming, founder of quality management)
Analysis is vital to the success of policy development and the delivery of programmes, projects and operational services. It helps shape and appraise options, provides insight into how complex systems work and behave, measures system performance, and improves efficiency. However, if analysis and supporting models, data and assumptions are not fit-for-purpose, the consequences can be severe—ranging from reputational damage to financial loss and legal challenge. In the most severe cases, individual livelihoods and lives can be affected.
Monitoring and evaluation (M&E) is an essential component of any evidence-based policy-making. M&E encompasses quantitative and qualitative techniques for collecting, processing, analysing and disseminating data that are relevant to managing the policy and evaluating its impact.
Three different but inter-related areas need to be monitored during territorial development:
During the implementation of the 2014-2020 State Regional Development Strategy, several quite disconnected procedures were put in place to monitor regional development, decentralisation and the amalgamation of local communities, based on the collection and analysis of regional and local statistics. There was, however, no strategic approach to M&E but rather an ad hoc response to information needs arising at different points.
What M&E is relevant to the 2027 Strategy
Based on high-level statistical indicators, the M&E discussed here relates to the first area, territorial development. Qualitative indicators on the implementation of the State Strategy for Regional development 2021-2027 (SSRD) and the performance of regional governments, such as the coverage and purpose of legal frameworks or institutional capacity, are not. Nor are project monitoring indicators, which depend on the nature of projects—hard vs soft infrastructure projects or local vs regional—, although they may be relevant for measuring the impact of the State Fund for Regional Development (SFRD).
This paper concludes with proposals for an M&E system for the State Strategy for Regional Development 2021-2027. An evaluation of the impact is beyond its scope, as it would require a more complex approach to disentangle the proper impact of the SSRD-2027 from that of other concomitant factors, such as the international geo-political situation. Such an analysis would also require complex econometric models to explore causality or a counterfactual approach that considers what might have happened if the 2027 Strategy had not been implemented—in short, a speculative approach that needs an experimental setting, such as comparing the varied evolution of similar territories, some benefitting from a policy instrument and others—a control group—not.
A monitoring system supported by a legal foundation
The system is strongly based on legal acts, such as Cabinet decrees. On one hand, this is positive as it obliges other institutions to exchange data, but on the other, it is negative as it imposes unnecessary rigidity on the scientific aspects of monitoring, requiring flexibility in data analysis, discussed further in “Weaknesses.”
Box 1. Legal underpinnings of Regional Competitiveness and Human Development Indices
Cabinet Resolution #1029 established the methodology for the Regional Competitiveness Index (RDI) and the Regional Human Development Index (RHDI) approved in 2017. Largely structured according to EU methodology, this includes a list of basic indicators classified into three sub-index groups (Basic, Efficiency, Innovativeness) with 11 pillars indicating of sources of information, together with a format for data exchange from source institutions to MinRegion, and instructions for analysing results: disaggregation by oblast, dynamics, analytical comments, and proposals for action. Over 2018, the methodology for the RCI and RHDI was analysed in detail, leading to proposals for simplifying it, in particular with respect of the collection of data on the perceived quality and accessibility of services (education, health, local transport).
A complicated, fragmented legal basis and no conceptual framework
The existing monitoring system has a complicated legal basis and is also sub-optimal in terms of:
The analysis of regional development based on different monitoring systems is fragmented due to the co-existence of many legal requirements (Cabinet Resolutions #476 of 2009, #856 of 2015, #931 of 2015, #987 of 2016, #1029 of 2017, and more).
As an example, the Procedure and Methodology for monitoring and assessing the effectiveness of state regional policy implementation, approved by the Cabinet on October 21, 2015 in Resolution #856, lists the indicators to be used and which institutions have to provide the data, without any further role other than producing quarterly and annual ranking reports. Oblast administrations also submit analytical reports on their geographical area to MinRegion, which consolidates them. The list of national indicators overlaps and sometimes contradicts those that are supposed to be compiled by international agreements, such as the Sustainable Development Goals, or widely used for international benchmarking, such as the World Economic Forum Competitiveness Indicators.
In addition, there is no conceptual framework relating the indicators included in the monitoring systems to any theory of regional development. For instance, monitoring of the amalgamation process and decentralisation reform is currently carried out independently of the process of monitoring regional socio-economic development. The latter is based on legal acts, while the former is based on ad hoc arrangements.
Coordination mechanisms that are still not functional
Ukraine’s Law on Regional Development calls for the establishment of an inter-agency coordination mechanism, the Inter-agency Regional Development Coordinating Commission. At the same time, the Law assigns responsibility for monitoring Strategy implementation to the central executive body (CEB), meaning MinRegion. This means that the procedure for monitoring is to be defined by the Cabinet of Ministers.
Until 2018, inter-agency collaboration in monitoring regional development was focused on reporting from other agencies to MinRegion, which was, in fact, largely limited to data exchange. In 2018, an Inter-Agency Working Group was set up that included representatives of several agencies and reported to the Inter-Agency Regional Development Coordinating Commission.
An internal working group within MinRegion is to be established to ensure good oversight and support the design and implementation of an integrated information system.
Better methodology and dissemination needed for subnational data
Ukraine’s statistical system was thoroughly analysed in the Global Assessment carried out by Eurostat in 2017 and the OECD, and highlighted the most important achievements, along with the limitations of methodology and dissemination:
Other unmet information needs have been identified in specific aspects of territorial development.
Unmet Information Needs 1: Statistical evidence to support Smart Specialisation Strategies
Smart Specialisation Strategies (S3) are supposed to be integrated into the core of regional development policies. At the national and regional levels, S3 teams have been put in place with EU support through the Joint Research Centre (JRC) to identify sectors that should be targeted in S3.
Like any other development policy, the design of S3 requires statistical information:
This is necessary for the initial diagnosis of regional and local capacities, as well as for their monitoring once underway. The level of geographical detail and related geospatial profile data are important for S3, given the close link to local development policies. Industrial data exist at the oblast level, but the detailed science, technology and innovation (STI) statistics at the oblast level is very limited. Except for some labour market indicators based on registered unemployment, data at lower geographical levels is non-existent, making the preparation of Smart Specialisation strategies difficult at best.
Unmet Information Needs 2: Statistical criteria to identify problematic territories
Draft legislation was prepared to identify territories with geographic specificities, based on statistical evidence. The identification is to be made at different geographic levels: micro-districts, counties, cities, communities and combinations of these.
The use of statistical typologies based on the analytical tools of modern geoinformation technologies makes it possible to design more targeted regional and local development policies. Typologies relevant to Ukraine are those related to greying territories with low population numbers, de-industrialization and migration processes, and territories that are temporarily occupied.
In Ukraine, administrative criteria to define urban and rural territorial units are not always based on the degree of urbanisation. The definition of typologies of territories according to EU standards will require more detailed population data, at the geographical level of 1 km2 grid, to comply with the TERCET initiative. The data requirements cannot be easily met in the current situation, since statistical information at the county level and lower is scarce.
Users of territorial statistics, such as the MinRegion, oblast administrations and the new local governments of amalgamated communities need to work together with DerzhStat to improve the quality, comparability and accessibility of territorial statistics.
Unmet Information Needs 3: Measuring disparities
Back in 2013, the OECD Territorial Review of Ukraine stressed the several weaknesses in the existing monitoring systems, in particular the strong use of indicators describing levels but not measures of spread—disparities. Growing inter-regional disparities, especially in living standards and the growing concentration of people and activities within regions suggested a need to promote second-tier cities. But this required specific measurements of inequalities, concentrations, convergences, and other distributional features.
It is the Ministry of Economic Development, Trade and Agriculture (MEDTA), and not MinRegion, that prepares the report on inter-regional and intra-regional variations in regional socio-economic development. Except for a few indicators comparing ratios of maximum to minimum values across oblasts, which do not describe the distribution of values in-between, there are no other statistical measures of disparities analysed such as a coefficient of variation or Gini concentration index, or measures of intra-regional disparities across lower territorial levels—counties, cities and communities.
Ineffective use of modern ICT to integrate, generate and use regional data
Modern tools for data integration make it possible to move, analyse or visualise data from different sources, consolidate existing data with new data, and generate new information to satisfy an organisation’s needs. However, the current ICT technology used by MinRegion is anything but state-of-the-art. For one thing, it suffers from data coming from and being stored in many different, fragmented places, and formats: Excel files, PDF documents, fragmented databases, geo-referenced databases, texts on websites, and so on. The available ICT tools are used not very effectively, requiring manual links to geospatial information.
Analytical work is limited
The capacity to understand and use evidence to design better regional policies has been recognised as a factor that determines the quality of such policies. Besides data collection and descriptive analysis, staff from policy-making institutions should be able to identify relevant research papers, analyse international case studies, access acknowledged references, and apply advanced analytical methods to the analysis of territorial indicators.
There is a great shortage of capacities within the government structures to adequately addressing the challenges of using statistical and other scientific evidence to design, monitor and evaluate regional development policies.
Thanks to support from the EU and international development partners, there are significant opportunities to establish a sound regional development monitoring system over the next years by:
Ukraine’s regional statistics system is being supported to comply with EU standards
The Association Agreement (AA) between the EU and Ukraine provides for the progressive alignment of official statistics methodology with EU standards, including regional and local statistics. Art. 355 of the AA states that Ukraine’s national statistical system should adhere to the UN’s Fundamental Principles of Official Statistics, reflecting the EU acquis and including the European Statistics Code of Practice, in order to harmonise the national statistics system with European norms and standards. Eurostat has supported DerzhStat, Ukraine’s statistics agency, with technical assistance in the form of system assessments in 2012 and 2017, and training in different areas.
Box 2. European standards for territorial statistics
The Statistical Requirements Compendium is a well-established reference document for the acquis in statistics. Some interesting features of EU regional statistics include:
- Regional statistics are generated by applying common geographical breakdowns (EU NUTS Regulation) in all statistical surveys and other data collections. Data are published according to NUTS classifications, mostly at NUTS 2 level, but also at NUTS 3, in order to facilitate the analysis of intra-regional disparities and design more targeted policies.
- Sub-regional data are published by territorial typology (urban-rural, metropolitan and maritime, border areas, and so on), degree of urbanisation and city level.
- Specific legal acts are related to each domain (social statistics, economic statistics, environment statistics, and so on). The level of geographical detail depends on the domain.
Other international development partners like UNFPA and UN Women have been helping strengthen Ukraine’s statistics in their different areas of interest.
Anticipated increase in available territorial statistics
The availability of regional and local statistics that allow development and disparities among territories to be analysed constitutes an essential condition when designing and monitoring state and regional policies. The period 2021-2027 is crucial for improving territorial statistics in Ukraine and in the world:
The Government of Ukraine’s response to the challenges of making statistics available was the February 2, 2019 approval of the Programme for the Development of National Statistics through 2023 via Cabinet Decree #222. This includes several statistical operations relevant for the purpose of monitoring regional development, such as the implementation of the NUTS Regulation in 2023 to define statistical territories, the carrying out of EU-harmonised Surveys on Living Conditions and Innovation, and approximation to the European System of Accounts ESA2010.
Improved technologies for data collection and management - e-government initiatives
In addition, the decade 2020-2030 will continue to witness, thanks to ICT technologies, an increase in available data from innovative sources, such as data collected from traffic sensors, mobile phones, satellite images, energy consumption meters, and so on. This can be used to monitor transport and commuting, urbanisation processes, agriculture, the environment, and so on, providing new opportunities for generating statistics and building forecast models. Under the proper conditions, the use of geoinformation technologies, the formation of distributed databases, and the setting up of geoportals will make it possible to radically change the technology of the management system, moving from data transfer to services exchange through inter-portal connections. This, in turn will eliminate the time spent duplicating solutions for the same tasks, information will be used practically entirely online, a common information space will allow for information exchange—including among government agencies—and link various sources.
The implementation of e-Government and the Open Government policy will increase the capacity of re-using public data. Following a presidential decree, the Ministry of Digital Transformation has an ambitious plan to make all public services available online within a few years. While the main objective is the digitalisation of certain services, this suggests that the government supports the implementation of IT strategies in public administration.
The re-use of administrative data to monitor regional development can benefit from the Open Government Data of Ukraine portal called Diia, which provides a platform for disseminating files. On the Diia portal, the regional component of open data can be improved only by adequately tagging the open data files and using standard codes for territories or KOATUU codes.
Low statistical literacy among policy-makers and low trust in official statistics
Statistical literacy skills are vital for the informed use of statistics in decision-making. These can be grouped into four broad areas:
While education levels in Ukraine are high compared to other countries, policy-makers, especially at the local level, are not always sufficiently statistically literate to understand quantitative analysis. In a time of widespread disinformation, good quality statistics have to compete with poorly calculated or intentionally distorted data. The Big Data hype can also affect policy-makers’ perceptions by giving the same value to information from innovative sources like social media, as statistics that have been scientifically collected and processed.
Shortage of quantitative experts
The global labour market for specialists with skills in data processing is very dynamic, with exciting, well-paid positions in the private sector, such as the financial services, biomedicine, industry 4.0. This means that young graduates with data science skills are not likely to choose careers in the public sector.
Given its limited options for offering higher salaries, government may need to outsource regional development research projects to external actors, such as start-ups, universities and research centres. Indeed, an increasing number of NGOs is producing interactive reports based on open data. Government can also offer attractive internships to start-of-career data scientists, where they can work with real statistical, administrative, and geo-spatial data, and get useful experience.
What should be done: Monitoring the SSRD 2021-2027
Based on the author’s experience during the last years, this SWOT analysis supports the idea that both Ukraine’s Government and its international partners should focus on the system for monitoring regional and territorial development, particularly on the SSRD-2027, in three key directions: (1) improving the institutional framework for the monitoring system so that there is better inter-institutional collaboration and a better legal framework, (2) improving the quality of information used for M&E, including through the more efficient use of technology for exchanging, storing, processing and disseminating regional and local data, and (3) establishing analytical capacity to use scientific evidence at MinRegion, oblast administrations and local governments.
The following analysis of the logical framework approach identifies overall and specific objectives and expected results for the regional development monitoring system.
Overall objective of the RD Monitoring System:
Instituting a sound monitoring system to underpin regional development policy, based on a data system aligned with EU and international best practice that makes use of modern technologies.
Specific objective 1: Improve coordination and legislative framework
Specific objective 2: Adopt EU and international standards for territorial statistics
Specific objective 3: Optimize the use of modern ICT for regional development M&E
Specific objective 4: Increase analytical capacities among users of territorial statistics
Copy editor: Lidia Alexandra Wolanskyj
All terms in this article are meant to be used neutrally for men and women
José L. Cervera has participated at the International Expert Exchange, "Empowering Municipalities. Building resilient and sustainable local self-government", organized by U-LEAD with Europe Programme in December 2019. His speech delivered during one of the workshops is to a great extent depicted in this article. Despite its late publication, the article is still of significant relevance for the current discussion on decentralization reforms’ next steps in Ukraine.
In the name of the U-LEAD with Europe Programme, we would like to express our great appreciation and thanks for both inputs of Mr. Cervera. The article will be included in future online publication Compendium of Articles.
Compendium of Articles is a collection of papers prepared by policymakers, Ukrainian and international experts, and academia after International Expert Exchange 2019 and 2020, organized by U-LEAD with Europe Programme. The articles raise questions in the fields of decentralization reform and regional and local development, relevant for both the Ukrainian and the international audience. The Compendium will be published online in Ukrainian and English languages on the U-LEAD online recourses. Please, follow us on Facebook to stay informed about the project.
If you have any comments or questions about the Compendium of articles or this article in particular, please contact Yaryna Stepanyuk firstname.lastname@example.org.
This publication has been produced with the assistance of the European Union and its member states Germany, Poland, Sweden, Denmark, Estonia and Slovenia. The contents of this publication are the sole responsibility of its authors and can in no way be taken to reflect the views of the U-LEAD with Europe Programme, the government of Ukraine, the European Union and its member states Germany, Poland, Sweden, Denmark, Estonia and Slovenia.
 Please note that the article was written in early 2020 and some of observations stated may already be outdated.
 HM Treasury (2015). The Aqua Book: guidance on producing quality analysis for government. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/416478/aqua_book_final_web.pdf.
 MinRegion’s online resource provides information about project implementation: http://dfrr.minregion.gov.ua/Projects-list?PAREA=76.
 See, for instance, DG Regio (2015). “Econometric assessments of Cohesion Policy growth effects: How to make them more relevant for policy makers?” https://ec.europa.eu/regional_policy/en/information/publications/working-papers/2015/econometric-assessments-of-cohesion-policy-growth-effects-how-to-make-them-more-relevant-for-policy-makers
 Official Journal of Ukraine, 2015, #88, p. 2926
 Law of Ukraine “On the fundamentals of state regional policy,” Vіdomostі of the Verkhovna Rada (VVR), 2015, #13, Article 90.
 Global Assessment of the National Statistical System of Ukraine. https://ec.europa.eu/eurostat/documents/52535/52745/Ukraine+GA/dfaf8f9c-4d33-49ed-a9ae-57528dc56e47
 OECD (2018). Maintaining the momentum of decentralisation. OECD Multi-level Governance Studies, OECD, Paris.
 Based on Resolution #476 dated May 20, 2009 “On instituting the evaluation of inter-regional and intra-regional variations in regional socio-economic development.”
 D. Rodrigues (2013). Evidence-Based Regional Policy: Lessons and Challenges, presented in the OECD International Seminar “Política regional no contexto global: situação atual e perspectivas.”
 Statistical requirements compendium 2014 Edition. EUROSTAT, 2014.
 Nomenclature of Territorial Units for Statistics
 At the time of writing in 2019, no budget had been explicitly approved for this census.
 Established by Ukrstat Order #37 dated February 28, 2018. http://www.ukrstat.gov.ua/norm_doc/2018/37/37_2018.htm
 EPPI-Centre (2016) “The Science of Using Science: Researching the use of research evidence in decision-making.” https://www.alliance4usefulevidence.org/assets/Science-of-Using-Science-Final-Report-2016.pdf
 https://www.alliance4usefulevidence.org/. See also NESTA (2016) “Wise council insights from the cutting edge of data-driven local government.” https://www.nesta.org.uk/report/wise-council-insights-from-the-cutting-edge-of-data-driven-local-government/
Share the news:
22 September 2021
Ситуація з передачею майна районів у комунальну власність громад проявила гостру потребу у законодавчому регулюванні...
22 September 2021
On September, 22, 2021 the Committee of the Verkhovna Rada of Ukraine on the public authority organization, local...
22 September 2021
«Steps for financial experts 2021» is a seven week training cycle of seminars and workshops, starting from October,...
22 September 2021
From October, 4 until October, 8 the National Democratic Institute and the Network of South-East Europe...
22 September 2021
OSCE project coordinator in Ukraine announces the fourth wave of online-trainings for starostas, lasting from...