ETF key indicators 2012: Overview and analysis
INTRODUCTION
The European Training Foundation (ETF) supports the vision to make vocational education and training (VET) a driver for lifelong learning and sustainable development, with a special focus on competitiveness and social cohesion. Developing appropriate policies and measuring the effect of these policies requires solid evidence that covers VET and its links with the labour market, economic development, social cohesion, entrepreneurship and innovation. Evidence-based policy making has acquired considerable interest in recent years. This has paved the way for a renewed emphasis on quantitative indicators that can assist policy makers in formulating, monitoring and evaluating policies and performance. In 2010 the ETF launched a series of reviews of VET systems in all of its partner countries known as the Torino Process.
These assessments were informed by quantitative data based on a collection of relevant VET policy and system indicators. This exercise was repeated in 2012, with the quantitative data collection supported by three statistical workshops that were organised with representatives of national statistical offices and relevant ministries in March 2012. This paper is the result of the 2012 Torino Process data collection. It is intended to be a ready source of information on the state of play of VET policies and systems in ETF partner countries for national policy makers and the international community. Selected quantitative indicators are presented and analysed. Data for the EU Member States have been added to inspire policy learning and dialogue both between the EU and partner countries, and among the partner countries themselves.
A secondary aim of the paper is to raise awareness among policy makers in the partner countries of the importance of indicators in driving the policy cycle, and of the availability and sources of selected VET policy and system indicators in their countries and regions. This paper is divided into three main chapters. The first describes the data collection process of the Torino Process 2012. In the second chapter, the regional tables with selected quantitative indicators are presented and analysed. The third chapter is an introduction to international developments in VET policy and system indicators. A final chapter draws conclusions and offers suggestions for future steps. The paper can be read on its own or as a complement to the ETF’s Torino Process reports and country studies, and the ETF’s labour market reviews. All of these are available from the ETF website1. The Torino Process reports provide a more comprehensive evaluation of VET systems and trends in the labour market and in education. Download ETF key indicators 2012: Overview and analysis.

Torino Process 2012 - A Croos-Country Report Moving Skills Forward: From Common Challenges to Country-Specific Solutions.
Preface
The Torino Process is a participatory process leading to an evidence-based analysis of vocational education and training (VET) policies in a given country. It is carried out in order to build consensus on the possible ways forward for VET policy and system development, considering the contributions of VET to enhanced competitiveness, and sustainable and inclusive growth. This includes determining the state of the art and vision for VET in each country and an assessment of the progress that countries are making to achieve the desired results. More specifically, the Torino Process is a vehicle for:
- developing a common understanding of a medium/long-term vision, priorities and strategy for VET development, exploring possible options for implementing this vision and/or making further progress;
- designing and evaluating home-grown and affordable VET policies, based on evidence or knowledge and collaboration;
- updating the analyses and achievements at regular intervals;
- providing opportunities for capacity development and policy learning within and among partner countries and with the European Union (EU);
- empowering countries to better coordinate the contributions of donors to achieving agreed national priorities.
The European Training Foundation (ETF) launched the Torino Process in 2010 and the first round was concluded in May 2011 at an international conference entitled ‘The Torino Process – Learning from Evidence’. Among the outcomes of the conference was the establishment of the Torino Process as a biennial policy learning exercise founded on country ownership, participation, and a holistic, evidence-based policy analysis. The second round was launched in 2012. The Torino Process overall is open to all ETF partner countries. This report draws on the lessons learned by the ETF. Its overall objective is to present the progress that has been made in VET policy and system development, and identify constraints and future priorities for the further modernisation of VET policies and systems in ETF partner countries. It is addressed to policy makers and practitioners in the partner countries, but also to officials, researchers, experts and the donor community who are interested in learning more about the partner countries in the field of VET or related policy fields. This report was prepared by ManfredWallenborn, ETF expert, who analysed the information in the regional reports for the preparation of this document. Valuable support was provided by Doriana Monteleone, ETF statistical officer.
This report and the Torino Process are the result of a team effort. The ETF would like to take this opportunity to thank all the counterparts from the partner countries who contributed to the national reporting process in 2012, as well as the ETF country teams which facilitated the process in the countries. The ETF is also grateful to the statistical team and the internal peer reviewers who provided valuable input, comments and suggestions on the final draft of the document. Download Torino Process 2012 - A Croos-Country Report Moving Skills Forward: From Common Challenges to Country-Specific Solutions.
The other Reports.
Torino Process 2012: Central Asia; Eastern Europe; Southern and Eastern Mediterranean; Western Balkans and Turkey; Uzbekistan; former Yugoslav Republic of Macedonia; Kyrgyzstan; Azerbaijan; Armenia; Montenegro - Executive Summary; Kosovo - Executive Summary; Ukraine; Tunisie; Tajikistan; Serbia; Russia; Palestine; Moscow (Russia); Maroc; Montenegro; Moldova; Lebanon; Kosovo; Kazakhstan; Jordan; Israel; Georgia; Bosnia and Herzegovina; Belarus; Albania.

Torino Process 2012 - A Croos-Country Report Moving Skills Forward: From Common Challenges to Country-Specific Solutions.
Executive Summary
This report covers the Western Balkans and Turkey, the Southern and Eastern Mediterranean, Eastern Europe and Central Asia. The objective is to present the progress that has been made in vocational education and training (VET) policy and system development, highlight remaining obstacles in relation to the performance of VET systems, and deduce recommendations for future ETF priorities in the partner countries. In addition, the report presents the ETF’s opinion on how partner country VET systems could be further developed to increase the efficiency and effectiveness of VET policies and systems from a lifelong learning perspective.
The report presents the lessons learnt by the ETF regarding the process of reform and modernisation of VET policies and systems thanks to the high level of commitment and substantial efforts made by the partner countries during the 2012 round of the Torino Process.
This report is addressed to policy makers and practitioners of VET systems in the partner countries, but also to officials, researchers, experts and the donor community who are interested in learning more about the partner countries in the field of VET or related policy fields.
There are positive key trends and areas of progress as well as remaining issues for the further development of VET within the four regions with which the ETF cooperates. The countries are inspired by the international debate among experts on VET: VET matters in both an educational and an economic context, being an instrument for short- and long-term employability. The countries are aware of the important contributions of VET to social inclusion and to socio-economic progress in terms of growth and competitiveness. Moreover, they know from their own experience of reforming other educational sub-systems that VET reforms must be in line with the national context, that they take time, and that there is a need for persistent efforts in a number of different areas of VET systems in order to convert VET into a valid instrument for skills development.
The countries and regions analysed in the Torino Process 2012 round vary greatly. However, there are many common issues on which substantial progress has been made since the Torino Process 2010 round, such as the further development of national VET strategies that are coherent with other sector policies, the adoption and implementation of new laws and instruments, such as qualification frameworks (including new professional standards and curricula), decentralisation, greater involvement on the part of the business sector, and the beginning of rationalisation of the VET school networks. This progress is consistent with the seven priorities for further action formulated by the participants in the 2011 Torino Process Conference.
Policy makers in the partner countries are aware that VET reforms must be designed in line with the specific socio-economic and cultural contexts of the countries concerned, and that VET reforms must be linked to capacity building for implementation. However, the main conclusions of the 2012 Torino Process round highlighted five key areas for further action, which are common to all partner countries, if VET is to contribute in an effective manner to economic development, the employability of individuals and their social integration:
- creating shared, long-term visions for the development of skills from a lifelong learning perspective which effectively integrate education, training and employment with economic and social development;
- enhancing the labour market relevance of VET through a closer integration of learning and work, in learning environments that either are already available or could be created in schools, post-secondary institutions and the workplace;
- reinforcing awareness of the contribution of VET to social cohesion, through greater attention to the needs of vulnerable groups, both in initial VET (IVET) and by enhancing access to adult education and training opportunities;
- improving the quality of IVET and continuing VET (CVET), supported by improvements to elements of VET systems, in particular teacher training, teaching methodologies, qualification frameworks and the innovation of educational infrastructure/rationalisation of school networks;
- strengthening the effectiveness of public policy by sharing responsibility for VET governance and delivery between the state, the business sector and other social actors.
This represents an integrated, innovative agenda for sustainable reform in effective and efficient VET policies and systems. In this context, the term ‘integrated’ comprises a joining up of policy measures bridging VET and national and local development; it embeds VET in lifelong learning; and actively involves key stakeholders in shared multilevel governance. ‘Innovative’ brings the sense that VET policy is at the state of art – anticipating, rather than reacting to change. It suggests a VET system with the capacity to adapt and which provides the learner with the creative competences that are fundamental for long-term employability. ‘Sustainable’ brings into the concept both the dimension of green skills serving green economies and communities, and the need for long-term incremental effort to connect vision with implementation.
While ‘effectiveness’ and ‘efficiency’ remain key principles for modern, accountable public policy. According to the five building blocks of the analytical framework for the 2012 Torino Process (vision, external efficiency and demographic trends/labour market needs, external efficiency and social demand/inclusion, quality/labour market relevance and governance) the main findings from the 2012 Torino Process can be summarised as follows. The country reports show that all countries are aware that visions and VET system reforms could make a considerable contribution to societal objectives such as increased competitiveness and employability, and inclusive growth if VET reform is part of integrated and holistic country policies. Hence, the countries have developed or are currently developing strategies for VET reform. In theWestern Balkans and Turkey these strategies are closely linked to EU standards and good practice, because the region comprises five candidate countries for EU accession.
The Southern and Eastern Mediterranean shows a more disparate picture in terms of the countries’ visions for the design and performance of VET systems. As a result of demographic pressure in many countries, the most important issue is employability for young graduates. However, political instability has generated short-term priorities other than VET reforms.
The need for new visions and reforms in Eastern Europe and Central Asia are currently being discussed intensively. In the early years of transition most of the countries dedicated more of their attention to general education and higher education. Meanwhile, the countries are convinced that a well-performing VET system strongly reinforces their position in a globalised economy and also supports regional mobility. Hence, visions and strategies have been formulated more coherently with other sector strategies, and many countries in Eastern Europe aim at placing more emphasis on the design of lifelong learning strategies, as a result of their ageing societies.
VET reforms that aim at greater efficiency, economic growth and competitiveness are underpinned and justified by socio-economic developments in the regions, such as the increasing numbers of young people looking for work opportunities in precarious labour markets in many of the countries.Whereas in the EU the proportion of the total population aged 15-24 is 11.7%, it ranges from 11.7% to 19.3% in theWestern Balkans and Turkey, from 14.7% to 21.4% in the Southern and Eastern Mediterranean, from 14.0% to 20.8% in Eastern Europe, and from 18.7% to 23.5% in Central Asia.
Demographic development is one driver of high unemployment rates. In the most recent years for which data are available, the average unemployment rate in the EU was 9.7%, while the corresponding figures were between 9.0% and 44.9% in theWestern Balkans and Turkey, between 5.4% and 18.7% in the Southern and Eastern Mediterranean (however, with extremely high youth unemployment rates in Egypt 30%, Jordan 30%, Palestine 36%, Syria 32% and Tunisia 42% in 2011), between 5.5% and 19.3% in Eastern Europe and between 5.4% and 11.6% in Central Asia.
In many countries the outcomes of different education levels are not always appropriate either for current demands or for the challenges of the future, including the successful social inclusion of all learners, which is more related to the labour market relevance rather than to the level of educational offers. The countries of the former Soviet Union are still characterised by a significant amount of formal education but not always with sound levels of employability for the learners. In the EU the proportion of the total population aged 15+ that has completed at least upper secondary education amounts to 67.5%; the same proportion is between 69.2% and 93.9% in the Eastern European countries and between 71.2% and 90.2% in the countries of Central Asia. However, in the countries of theWestern Balkan and Turkey region the proportion is lower, at between 28.8% and 70.1% and in Southern and Eastern Mediterranean countries between 17.0% and 74.7%.
The situation in the Western Balkan and Turkey, and Southern and Eastern Mediterranean regions is linked in part to low levels of educational expenditure as a percentage of gross domestic product (GDP), exacerbating the problems faced by those from vulnerable groups and ethnic minorities. In the EU the percentage of GDP is 5.4%, but between 3.5% and 4.3% in the Western Balkans and Turkey and between 1.8% and 5.8% in the Southern and Eastern Mediterranean. The picture is more positive in Eastern Europe, where the percentages range from 3.2% to 9.1%, and in Central Asia, with percentages between 3.1% and 8.3%.
The quality and labour market relevance of VET programmes are key challenges for innovation and reform in all VET systems. This includes the acquisition of key competences as sound foundation skills and tools for adaption to future challenges – suitable for both short-term employment and longer-term employability. In this respect, the country reports describe policy improvements in particular in the field of entrepreneurial learning. This is seen by policy makers as having a high potential to generate a more dynamic culture among young people, business and communities and a fertile environment for job creation. The remaining problems are twofold: the need for improved employability and hence, inclusion, for learners is still a key priority, as is the need for increased competitiveness and sustainable growth. A great deal remains to be done through more effective practice-oriented modes of learning that are relevant to the labour market, and supported by better educational infrastructures in schools and enterprises, increased vocational guidance, improved training for teachers in new methodologies, updated textbooks and curricula, and more effective management on the part of school directors under the guidance of school boards with local community representation.
Such innovations in the elements of VET systems must be supported by new governance modes that involve all relevant social actors at different functional levels in future VET policy outlines. Ministries of education are also on their own transition pathway from bureaucratic administrators to intelligent moderators of social processes leading to improved VET system performance that is relevant to the labour market. A certain diversification of VET provision supports such trends: private training providers and enterprises that are working at the forefront of technology or dealing with international standards increasingly perceive VET to be an appropriate tool for business development through the updating of human capital. Traditional, publicly driven vocational schools are to some extent losing their prevailing position in human capital development in certain economic sectors, while private providers are gaining ground. Moreover, public schools have also started to cooperate with the business sector in order to better cover new learning outcomes and foster more work-based learning for increased employability.
Such processes imply the involvement of the social partners in VET policy development and implementation, and the support of additional modes of learning at the workplace in professions. These measures require capital-intensive learning environments, and are not generally available in vocational schools. A number of effective arrangements are already in place. However, the countries concerned still face challenges in installing new modes of governance according to their specific contexts.
The influence of EU policy is strongly felt across the partner countries in all education and training sub-systems. While theWestern Balkans and Turkey, and in particular the candidate countries, are engaged in Enhanced EU Cooperation in VET, in other regions the European influence on VET refers to national qualifications framework (NQF) development, quality-assurance procedures and European models of VET governance.

ETF manual on the use of indicators
1.0 GENERAL CONCEPTS
This chapter defines the concept of an indicator and explains its characteristics. The data sources that can be used to create indicators are also discussed. An indicator is only as reliable as the data it is based on, so close attention must be paid to data sources.
1.1 WHAT IS AN INDICATOR?
The Organisation for Economic Cooperation and Development (OECD, 2002a, p.25) defines an indicator as ‘a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor’. In other words, an indicator is an aggregation of raw or processed data that helps us to quantify the phenomenon under study and a tool that helps us to grasp complex realities. An indicator is not raw data, but rather uses that data to characterise or assess a particular issue. For example, the absolute number of literate adults is not a particularly useful datum until we use the statistic to create an indicator such as, for example, the adult literate population as a proportion of the total adult population in the country.
1.2 WHAT ARE THE CHARACTERISTICS OF A GOOD INDICATOR?
Several issues must be considered when creating an indicator. A good indicator should be relevant, should summarise information without distorting it, and should be coordinated, structured, comparable, accurate and reliable. Indicators need to be relevant to policy goals, and it is therefore essential to identify these goals before deciding what to measure and how to do it. For example, if the goal were to increase access to education, the relevant indicator could be the rate of participation in education. An indicator should summarise existing information without distortion. For example, if we are interested in the number of students per teacher, we need data on both the number of students and the number of teachers to obtain the student-teacher ratio. However, such data is susceptible to distortion; for example, if we include both full-time and part-time teachers, the ratios we obtain will be lower but they will not be a faithful reflection of the real situation. Thus it is important to clearly understand the nature of the data available before constructing the indicator. Indicators must also be coordinated and structured; in other words, we have to ensure that they are constructed and used in a consistent, comparable and comprehensive way. Consistency is particularly important when we are monitoring data and trends over time or comparing data between countries.
If we are to produce comparable results, the definitions and calculation methods we use must be consistent. Comparable results can only be obtained using clearly defined indicators based on identical definitions to ensure consistency even when data are collected at different times and indicators are calculated by different people. Indicators should also be comprehensive, that is, they should always encompass all relevant aspects of the phenomenon under study. Finally, indicators and the data on which they are based should be accurate and reliable, and any deficiencies in the data should be made clear. An indicator is only reliable when we can trust what it shows.
1.3 WHAT DATA SOURCES ARE AVAILABLE?
To calculate an indicator, we need data, and this can be obtained from different sources. A good data source is comprehensive in coverage, unbiased, and consistent over time. Potential data sources include surveys, censuses, administrative databases, reports, interviews and focus groups. In education, most data comes from schools in the form of statistics, such as the number of students enrolled or the number of graduates. Some of this data is aggregated at the national level by education ministries. School inspection reports can be used to assess the quality of education programmes. Surveys carried on among students provide information about student satisfaction and the effectiveness of interventions. Expert surveys can be used to assess the overall quality of VET systems. All these types of data can be used to create indicators relevant to policy goals. It is important to distinguish between primary and secondary data sources. Primary sources are original documents or data providing first-hand and direct evidence (e.g. interviews with country officials). Secondary sources include the information from primary sources that has been processed and interpreted. Other secondary sources include international organisations (e.g. World Bank (WB), the International Labour Organisation (ILO), etc), whose published data and indicators are usually based on information provided directly by countries and other primary data. Thus, when data for the calculation of indicators are available from different sources, we should expect the data from each source to produce the same results if the same definitions and calculation methods are used. Sometimes, however, national and international bodies provide disparate data; in such cases, the reasons for the differences should be identified before deciding which source to use.
1.4 QUANTITATIVE, QUALITATIVE AND PROCESS INDICATORS
Decision making procedures should be based on the systematic and regular use of evidence. Evidence is the key to an in-depth understanding of the problems that affect education and training systems and is thus a prerequisite to making informed policy choices. Consequently, having and making good use of a solid evidence base is of great importance in the fields of VET and labour market research. In VET, as in any kind of research, evidence can be divided into two main types: quantitative and qualitative. Quantitative evidence is objective information about the real world and is numerical in nature. Thus quantitative indicators are expressed as numbers, for example, the number of inhabitants in a country, or the public expenditure on VET systems as a percentage of national expenditure on education. Qualitative evidence, on the other hand, deals with the qualities of the object of study and may include subjective information, opinions or judgements about an issue. Qualitative evidence is typically expressed in the form of descriptive information, although it can also be quantified and expressed numerically. There are many sources of qualitative evidence, such as case studies, observations, reports, discussions and in-depth interviews. In this manual, we restrict ourselves to the type of qualitative evidence that can be quantified. It should be noted, however, that this is only one kind of qualitative information that can be used to analyse VET. For example, we present indicators that measure the intensity of a perception, such as the results of a survey that asks experts how much corruption they perceive in a particular country. The answers, which take the form of qualitative observations, can then be assigned a score, and the resulting numerical data can be used to quantitatively compare corruption perception and to calculate summary statistics (averages, for example). The third kind of indicator described in this manual is the process indicator. Process indicators can be used to identify problems or gaps in a particular area by measuring the actual values of the process indicators against pre-defined targets or standards. They can be based on quantitative evidence (objective information) or qualitative evidence (subjective information). In chapter 3, we provide examples of how quantitative, qualitative and process indicators are created. The indicators discussed relate to the employment and education targets established by the EU for 2020 (E&E 2020), Quality Assurance for VET (EQAVET) and the ETF Torino Process and Entrepreneurial Learning initiatives.
1.5 WHAT IS A BENCHMARK AND HOW TO CHOOSE IT?
The United Nations defines a benchmark as ‘a concrete point of reference (in the form of a value, a state, or a characteristic) that has been verified by practice (in the form of empirical evidence, experience, or observation) to lead to fulfilment of more overall objectives or visions (in isolation or together with the fulfilment of other benchmarks)’ (United Nations, 2010, p. 17). While indicators serve to quantify a phenomenon, benchmarks serve as a standard or point of reference against which the current situation may be compared. Finding appropriate standards for this purpose is not always an easy task, and context is crucial for the ETF because we need to make comparisons between different partner countries. If we want to compare countries within a single region (for example, North Africa), the results may be more instructive if we find a benchmark in that region rather than use a reference from elsewhere (an EU member state for instance), which might have higher standards but in a completely different context in terms of aspects such as labour market needs and institutions. The usefulness of the exercise is vastly increased if the context of the benchmark and that of the case under study are similar.
Download ETF manual on the use of indicators.

Good multilevel governance for vocational education and learning - Working paper
Introduction
The present working paper is based on analytical work conducted by the ETF in 2012 and broad consultation with ETF partner countries that included a pilot study (the study) on good multilevel governance in VET. The study was conducted in six partner countries, namely Azerbaijan, Croatia, Kazakhstan, Serbia, Tunisia and Ukraine, and its purpose was to map and assess the involvement of VET stakeholders across different functions of their national VET systems. The partner countries’ performance was then compared against a set of principles and indicators of good multilevel governance developed by the ETF1. The study was coordinated by the ETF Community of Practice on Governance, Partnerships and Regional Development. The results of the study, presented here, incorporate the key conclusions of the ETF corporate conference ‘Multilevel governance in education and training: challenges and opportunities’ that took place at the EU Committee of the Regions in Brussels from 31 May to 1 June 2012. During the conference the findings of the study were presented, and multilevel governance, including stakeholder participation as a means to enhance performance in VET, was discussed. Chapter 1 provides an overview of the key concepts and terminology related to governance, the rationale for ETF support of partner countries in good multilevel governance in VET, and the methodology adopted for the study. Chapter 2 presents the main findings of the pilot study on VET governance in the selected partner countries. Within the different policy areas investigated and based on cross-country analysis, the successes and gaps in both vertical and horizontal cooperation as well as the strengths and weaknesses in VET management are highlighted. In addition, this chapter introduces the key pilot results of testing ETF-proposed principles and indicators for good multilevel governance in VET. It is important to underline that this pilot study was essentially a reporting exercise with a limited scope, and that while the findings may provide a useful stimulus for further analysis and debate, they should not be seen as definitive research results. Nevertheless, the study offers useful in-depth information in a number of areas, leading to some preliminary conclusions. Chapter 3 suggests a number of areas on which policy makers and stakeholders could usefully focus, particularly the development of good multilevel governance to improve the effectiveness of VET. The conclusion summarises the key findings while identifying some trends, coordination mechanisms and ETF actions for building good multilevel governance in VET with partner countries. Finally, Annex 1 gives examples of successes and gaps in horizontal and vertical partnerships; Annex 2 introduces a roadmap for the development of good multilevel governance practice; and Annexes 3 and 4 reproduce the questionnaire used in the pilot study with a related glossary of terms used in the management of public policies in education and training. In summary, this report puts forward a number of questions and lines of action for policy makers to consider in ensuring that the governance of VET is fit for purpose.
1 For detailed information on the 2012 Torino Process, see the analytical framework for vocational education and training system reviews (ETF, 2012e).
Download Good multilevel governance for vocational education and learning.

The European Training Foundation is a decentralised agency of the European Union based in Turin, Italy. It was established by Council Regulation No. 1360 in 1990 recast No. 1339 in 2008 to contribute to the development of the education and training systems of the EU partner countries. It became operational in 1994 in its premises of Villa Gualino. The ETF currently employs approximately 130 staff and has an annual budget of about 18 million €. Madlen Serban is Director of the ETF since 1 July 2009.
Our mission is to help transition and developing countries to harness the potential of their human capital through the reform of education, training and labour market systems in the context of the Eu's external relations policy.
We base our work on the conviction that human capital development in a lifelong learning perspective can make a fundamental contribution to increasing prosperity, creating sustainable growth and encouraging social inclusion in transition and developing countries.
We recruit and deploy experts from multiple disciplines to handle complex and multidimensional topics in a team environment, in order to create new knowledge, insight and solutions.

Communication