Download Internationalisation and Quality Asssurance - Connecting European and Global Experiences.
“European Quality Labels” and Quality Assurance
Dr Achim Hopbach, President of the European Association for Quality Assurance in Higher Education (ENQA) - Managing Director of the German Accreditation Council
The development of quality assurance in European higher education reached a major milestone when ministers of higher education of the Bologna signatory countries met in Berlin in 2003 and agreed on a number of basic principles that paved the way for developing procedures and systems:
[Ministers] also stress that consistent with the principle of institutional autonomy, the primary responsibility for quality assurance in higher education lies with each institution itself and this provides the basis for real accountability of the academic system within the national quality framework.
Based on this principle, they committed themselves to developing and implementing quality assurance systems that would include:
• A definition of the responsibilities of the bodies and institutions involved.
• Evaluation of programmes or institutions, including internal assessment, external
review, participation of students and the publication of results.
• A system of accreditation, certification or comparable procedures.
• International participation, co-operation and networking. (Berlin Communiqué 2003, 3).
Based on these principles, national quality assurance systems and agencies were set up to form the basic model for assessment. When ministers officially launched the European Higher Education Area (EHEA) in 2010, external quality assurance based on this model had been implemented in all Bologna signatory countries, however in unique and diverse ways. Regarding external quality assurance procedures as such, one can say that they are by and large designed and conducted in accordance with ESG part II. (Hopbach 2012)
However, in addition to the national approach to quality assurance, initiatives at European level already existed such as accreditation schemes in the fields of economics (EQUIS) and public administration (EAPAA) and also the Institutional Evaluation Programme (IEP) of the European University Association.12 After 2003, this approach gained momentum when, with financial support of the European Commission, pan-European subject-specific initiatives for quality assurance emerged, often referred to as “European Quality Labels”, the first of which was the Eurobachelor® in chemistry which started in 2003. These approaches caught attention in political debates around 2009 when the “Report on progress in quality assurance in higher education” by the European Commission emphasized their role in fostering a stronger European dimension in quality assurance.
The seminar was dedicated to discussing the nature of these “European Quality Labels” in terms of their aims and objectives, methodological approaches and criteria used in their reviews, and particularly, to analyse their specific contribution to quality assurance in the EHEA. It also focused on “European Quality Labels” in the fields of engineering and science such as the EURO-INF Quality Label in informatics and EURO-AGES in the field of geology which were set up following the most well-known label EUR-ACE® and polifonia which started as an ERASMUS Network for Music in 2004. Recently, some of these initiatives joined together with professional accreditors in the foundation of the European Alliance for Subject Specific and Professional Accreditation and Quality Assurance (EASPA). This report highlights the main discussions and issues that arose from this seminar.
One obvious outcome of the seminar shall be mentioned right at the beginning. It seems rather inappropriate to subsume the aforementioned initiatives and approaches under the uniform heading “quality label”. On the one hand, they share the view that there is a need for a subject-specific approach to quality assurance in Europe as it is put by AEC: “assessing and accrediting institutions and programmes for higher music education must be rooted in a comprehensive understanding of the characteristics of music and the contexts and traditions in which music is created. Without such a rooting, the assessment may be preoccupied with only technical and academic aspects of musical production and ignore the innate unique and artistic characteristics of music.” (AEC 2010, p 6) On the other hand, the labels differ substantially in the implementation as far as purposes, organisational structures and activities are concerned in detail. This has to be borne in mind whenever the labels are mentioned or rather generalizing statements are made in the following.
Purpose
The principles of quality assurance in the EHEA are laid down in The European Standards and Guidelines for Quality Assurance in Higher Education (ESG) ministers of the Bologna signatory countries adopted in 2005. The ESG were “designed to be applicable to all higher education institutions and quality assurance agencies in Europe, irrespective of their structure, function and size, and the national system in which they are located.” Based on this very comprehensive claim, the ESG can be considered as the main reference point for the design of quality assurance in the emerging EHEA, be it internal or external quality assurance. (ENQA 2009)
The main purpose of these standards and guidelines was to guarantee professionally conducted quality assurance procedures on a high quality level. The ESG prefer the generic principle to the specific requirement and focus more on what should be done than how it should be achieved. Therefore, the ESG were not meant to explicitly comprise standards for quality of higher education as such (i.e. requirements for HE institutions and the design of programmes). The ESG combine two aspects in particular common standards for professionalism in terms of procedures and agencies which nowadays are not only shared within Europe and the EHEA but more and more worldwide; and the European notion of quality assurance which is in the first instance laid down in the following principles:
• HEIs bare the main responsibility for quality;
• The four stage-model applies: Internal evaluation, external evaluation by peers, publication of reports, follow-up procedure;
• External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes;
• Quality assurance processes, irrespective of the very nature and design of the chosen approach, have to serve the developmental function of quality assurance;
• Stakeholder, especially student involvement is critical in all phases, also in the development of quality assurance processes;
• And quality assurance agencies need to be independent in so far as they must have full autonomy for their procedures and decisions.
The specific meaning of the ESG lies not only in the fact that quality assurance processes are carried out throughout the EHEA based on the same standards. Even more interesting is the fact that the ESG were developed by all relevant stakeholders (EUA, EURASHE, ESU and ENQA; known as the E4 group) and, thus, make actors in the field of quality assurance share the same values and principles, fostering a common understanding. This alludes to a significant feature of the quality assurance in European higher education, which is the key role of stakeholders. (Hopbach 2012)
The most important feature of the “quality labels” refers to their respective backgrounds and purposes. In general, the emergence of many labels is to be seen in the frame of the development of the learning outcomes approach and of qualifications frameworks within the Bologna Process since 2003. Partly linked to the TUNING project, a major purpose of the labels was to add subject-specific learning outcomes or qualification frameworks to the generic approach of the Qualifications Framework of the EHEA. In particular, the Eurobachelor® project in chemistry and Polifonia can be subsumed under this heading. Whereas the quality assurance and/or accreditation function was added to the initial purpose of these labels only after some time other initiatives envisaged the set up of a subject-specific accreditation scheme at European level right from the beginning, such as EUR-ACE® and EURO-INF, both of which go further by linking subject-specific learning outcomes to standards for the design of the respective curricula.
By referring to subject-specific academic standards, the quality labels go beyond the European understanding of quality assurance which refrain from this type of standardisation, based on the principle of autonomy of HEI as stipulated in the Berlin Communiqué. The discussion about the specific purpose of the quality labels culminates
in the perhaps most relevant question which reads as follows: Who shall be responsible for defining academic standards? On the one hand, labels representatives state that this needs to be an integral part of quality assurance and thus the agency has to play a core role. On the other hand, the EHEA has set up a whole quality infrastructure which consists of qualifications frameworks, learning outcomes, and ECTS, etc., with quality assurance as only one part of it, and with different responsibilities, namely the responsibility of autonomous HEI for academic standards and of independent agencies for quality assurance procedures. Regarding common learning outcomes in the EHEA ministers, in the Communiqué of Leuven/Louvain-la-Neuve ministerial meeting in 2004 also emphasized this role of HEI:”Academics, in close cooperation with student and employer representatives, will continue to develop learning outcomes and international reference points for a growing number of subject areas“ (Leuven Louvain-la-Neuve Communiqué 2009, p. 4), without mentioning quality assurance agencies in this respect.
This leads to another important feature of some of the labels, namely the role of professional standards at the European level. The labels, for example, in the engineering sector focus on professional standards in addition to academic standards as criteria for the accreditation decision. By referring to entry qualifications for the labor market, these labels widen the focus and add to their approach elements of professional accreditation. Hence the perspective of the individual is added to the perspective of the programme. In the case for instance of EUR-ACE® this shouldn’t come as a surprise since the membership comprises also statutory bodies with responsibility for professional accreditation and thus for regulating the access to the profession. However, a discussion which is as old as the discussion about learning outcomes gains momentum through this widened focus: The discussion about a comprehensive educational mandate of HEI which goes beyond short term employability compared to a rather focused interest of professional associations which is necessarily oriented towards actual requirements in a certain professional field.
The specific nature of the purposes of the labels is closely linked with the organizational set up which, again, reveals the substantial differences between some of the labels. Mainly three groups of actors involved can be identified: HEI, professional associations/bodies and accreditation agencies. These collaborate in different combinations. Whereas polifonia is an initiative by the Association of European Conservatoires (AEC) and, thus, exclusively, run by HEIs, EUR-ACE® on the other side has only two members coming from academia, one Italian HEI and one association of faculties in Italy, and the rest representing professions or statutory bodies which regulate the professions.
This means that, in accordance with the organizational structure the academic standards might either be defined by representatives coming from academia (polifonia) or rather by professional organizations (EUR-ACE®).
It’s obvious that in the latter case the critical question about the autonomy of HEI for quality in higher education gains even more relevance.
In conclusion, one can say that the principles and purposes of the labels are broader than those of quality assurance and focus on translating the learning outcomes approach to subject specific standards at European level.
Methodology
As far as the methodology of external quality assurance in the EHEA is concerned, the principles are laid down in part II of the ESG which start with part I of the ESG that refers to the prime responsibility of the HEI for quality assurance. The other standards refer to procedural principles such as:
• Determination of aims and objectives before the process starts by including HEI (2.2);
• Application of publicly available predefined criteria in case formal decisions are made (2.3);
• Appropriate design of processes ffp (2.4);
• Publication of reports (2.5);
• Predetermined follow-up procedure (2.6);
• Periodicity of reviews (2.7);
• And system-wide analyses reports are produced describing and analysing the general findings. (2.8) (ENQA, 2009)
In general, the quality labels took these principles, as developed since the pilot projects in the mid nineties, as reference point for the design of their quality assurance and accreditation procedures, however to a different degree. They work with self-evaluation and external evaluation with a site visit by peers, and a compulsory follow-up. The Eurobachelor® label in chemistry is partly an exception since in some countries the label is awarded by the responsible committee based exclusively on a report by the HEI without any self-evaluation and also without any peer review and site visits. In other countries, the label is awarded by recognized agencies which do site visits. Also, polifonia has to be distinguished from other labels due to its wider scope. Whereas EURO-AGES, EURO-INF and EUR-ACE® are restricted to accreditation at the programme level, polifonia “offers” a more comprehensive approach with guidelines for internal quality assurance and also guidelines and criteria for external quality assurance at programme and institutional levels. Designing the guidelines in accordance with the ESG the specific contribution of polifonia is rather to be seen in “translating” the regular procedures of quality into a discipline that deviates substantially from other disciplines as far as fundamental features such as programme design, teaching, the learning environment and the whole set-up and profile of the institutions are concerned. To name but one specific feature, polifonia highlights the great variety of musical education which makes it impossible to set up prescriptive standards: “Even if objectivity can be applied to a number of aspects and concepts relating to musical skills, there are ultimately no final solutions or truths in music; there is no single method or route that will attain artistic goals.“ (AEC 2010, 14) Review panels need to be sensitive regarding this specific nature of musical education, which translates in curricular features such as private lessons, more time for self-study, etc. than in other disciplines.
The requirements for accreditation procedures for the purpose of conveying the labels of EURO-AGES, EURO-INF and EUR-ACE® are almost identical. With regard to two core aspects of quality assurance in the EHEA the three of them deviate substantially by neither requiring student involvement at all in the review panels nor foreseeing a
publication of the review reports. Both standards have to be considered obviously as core elements of external quality assurance in the EHEA. Otherwise these labels don’t provide any specific additional feature in their methodology, which would relate to subject specific questions.
One commonality of almost all labels refers to the actual implementation of reviews because they don’t necessarily conduct the reviews by themselves but also certify other bodies to do so. EUR-ACE® has authorized accreditation agencies like ASIIN and the French CTI, professional associations like the Association for Engineering Education in Russia, the Turkish Association for Evaluation and Accreditation of Engineering, and statutory bodies like the Engineering Council in UK, Engineers Ireland, and the Portugese Ordem dos Engenheiros. Eurobachelor® is working with agencies from the academic sector like ASIIN and the University Accreditation Commission from Poland, and also with professional associations like the Italian Chemical Association and the Royal Society of Chemistry for procedures in the UK and Ireland. EURO-INF is working so far with ASIIN. Polifonia does not certify other agencies but rather collaborates with national agencies in the accreditation of priogrammes.
In conclusion, one can say that in terms of methodology, the labels, different from their principles and purposes, don’t make a subject-specific contribution to quality assurance in the EHEA. The specificity of some labels consists rather of shortcomings in terms of application of the ESG.
Conclusion
The seminar and the discussions revealed that:
• European Quality Labels are not monolithic, so it is misleading to talk about “the” labels because of their substantial heterogeneity in terms of purpose, structure and procedures;
• The most important feature of the labels, and maybe the only feature they have in common, is the core role that academic standards, in some cases also professional standards, play. Some of the labels translate this role also into requirements for the design of programmes.
• In terms of methodology and design of the quality assurance procedures, no subject-specific feature applies which would add to the European approach in quality assurance. However, the labels from the engineering and informatics sectors fall short in terms of compliance with the ESG due to lack of student members on the review panels and lack of publication of reports.
• These outcomes demonstrate that most of the labels don’t make a subject-specific contribution to quality assurance as such. They should rather be called as a means to link subject-specific learning outcomes at the European level to quality assurance. This counts particularly for those labels that also apply professional standards and thus link academic accreditation of programmes to regulating access to the profession.
• The discussion revealed that the definition of subject-specific learning outcomes by agencies other than representing academia and also partly in collaboration with professional associations creates substantial tensions with “traditional” quality assurance according to the European standards which emphasize that the primary responsibility for quality rests with the individual HEI whereas standardization only applies for the level and scope of the qualifications through the Qualifications Framework of the EHEA.
References
AEC (2010), Quality Assurance and Accreditation in Higher Music Education http://www.bologna-and-music.org/home.asp?id=1704&lang=en (Accessed 13 March 2012)
Berlin Communiqué 2003, http://www.ehea.info/Uploads/Declarations/Berlin_Communique1.pdf (Accessed 13 March 2012)
ECTNA Good Practice Advice for evaluation of the Applications for the Chemistry Eurobachelor® Label http://ectn-assoc.cpe.fr/chemistry-eurolabels/doc/officials/Off_EBL101130_Eurobachelor_GPAdviceEvalAppl_201102V1.pdf (Accesser 13 March 2012)
ENQA (2009) Standards and Guidelines for Quality Assurance in the European Higher Education Area, Helsinki http://www.enqa.eu/files/ESG_3edition%20%282%29.pdf (Accessed 13 March 2012)
EUR-ACE Framework Standards for the Accreditation of Engineering Programmes (2008) http://www.enaee.eu/wp-content/uploads/2012/01/EUR-ACE_Framework-Standards_2008-11-0511.pdf (Accessed 13 March 2012)
EURO-AGES Qualifications Framewor and Accreditation Criteria for Geology Study-Progranmes in Europe (2011) http://www.euro-ages.eu/pages/final-results.php (Accessed 13 March 2012)
EURO-INF Framework Standards and Accreditation Criteria for Informatics Degree Programmes (2011) http://www.eqanie.eu/pages/quality-label.php (Accessed 13 March 2012)
Hopbach, A. (2012), External quality assurance between European consensus and national agendas, in: Curaj, A., et al. (eds.), European Higher Education at the Crossroads: Between the Bologna Process and National Reforms, Heidelberg et al.
Leuven_Louvain-la Neuve_Communiqué 2009, http://www.ehea.info/Uploads/Declarations/Leuven_Louvain-la-Neuve_Communiqu%C3%A9_April_2009.pdf (Accessed 13 March 2012)

This report is based on the ENQA workshop “Quality Assurance and Qualifications Frameworks: exchanging good practice”, which took place in Dublin, Ireland, on 9-10 February 2012 and presents articles on themes such as the state of development of qualifications frameworks, the role of agencies in the self-certification process, and the meaning of qualifications frameworks in external quality assurance.
The theme of qualifications frameworks and their relation to quality assurance is gaining urgency in the European scene as more and more countries are completing their national qualifications frameworks and quality assurance agencies need to take important decisions on how to implement them. Some of the key features of the qualifications frameworks are the specification of learning outcomes, processes for assessing learners’ attainment of the expected outcomes, their relation to the ECTS, identification of transfer and progression routes, and recognition of prior learning. Download the report Quality Assurance and Qualifications Frameworks: Exchanging Good Practice.
Conclusions
Several current reforms in higher education are having a significant impact on quality assurance and the work of quality assurance agencies. Among these, the establishment and referencing to qualifications frameworks and the adoption of assessment methods focusing on student learning outcomes pose important challenges to the methods and processes used for internal and external quality assurance.
The ENQA 2012 workshop on Quality Assurance and Qualifications Frameworks: exchanging good practice and these articles covered a wide range of issues related to qualifications frameworks, such as the state of art in their development and adoption, the role of agencies in the self-certification process, and the impact of qualifications frameworks on external quality assurance. In addition, a view from the stakeholder community was presented through an article outlining the use and importance of qualifications frameworks to the labour market and employers. Recognition of prior learning, assessment of learning outcomes, and learning outcomes in programme oriented quality assurance were also discussed in smaller working groups during the workshop.
As the first article underlines, it is clear that while several changes have already taken place, we are still at the beginning of implementing qualifications frameworks. It is important to make sure that they are developed jointly with quality assurance, learning outcomes, and other recognition tools. At the same time, the article emphasises the importance of making sure that European, but especially national qualifications frameworks become more visible and better known, so as to bring real benefits to the end users, whether individuals, higher education institutions, academic staff, or employers.
In terms of self-certification, the second author underlines that verification by selfcertification is a process in which each QA agency has a significant and obligatory role to play. The role of quality assurance is to demonstrate that programmes are based on intended learning outcomes and that qualifications are awarded on basis of achievement of these outcomes. If, however, we are to realise the objectives of the QF-EHEA in relation to transparency, mobility, and recognition, self-certification is only the first step. The Swedish case example presents a recently adopted approach to quality assurance. In line with the Bologna Process and the goal of increased employability of students, qualification descriptors were introduced and have become the tool in quality assurance of higher education in Sweden. Continuous reviewing in the coming years will help to develop best practice in implementing the new approach, and will be able to provide informed answers to questions such as: Can results of an academic study programme be measured? If yes: how can that be done? And is this European quality assurance of the 21st century?
The system used in Denmark relies on the use of external examiners for the measurement of achieved learning outcomes. Discussions on the use of learning outcomes in programme based QA in the related working group brought up a number of different methods. A conclusion of the group’s work was that if the scale of the potential small revolution brought about by learning outcomes based assessment is to be investigated seriously, the answer lies perhaps not in better measurement, but instead in finding new ways of stimulating ownership of the concept of learning outcomes itself. The working group on recognition of prior learning discussed different ways in which prior learning is used for entry into further study, achievement of credits, or for the award of an entire degree. It became clear that recognition of prior learning practices is politically sensitive and highly dependent on the national context. It was felt thus that agencies must explicitly assume this political dimension in order to define what role to play regarding recognition of prior learning processes in their own frameworks.
The working group discussing the assessment of learning outcomes concluded that there is a need to analyse the assessment of learning outcomes paying attention to the legal framework and academic context. In addition, the maturity of the higher education system as a whole, and the degree of implementation of other ‘Bologna tools’, such as qualifications frameworks, play a significant role in the process. The group felt a need to focus on the quality assurance of the assessment procedures of learning outcomes through checking the assessment practices used by programmes to assess different learning outcomes. The participants agreed that the focus of such assessment should be on the programme learning outcomes, not on the achievement of individual student.
Overall, the main conclusion of the workshop and of this publication is that there is great benefit in sharing and comparing national practices, and learning from good practice at the level of quality assurance agencies. However, the national political and legal context, as well as the degree of implementation of the Bologna reforms, has a significant impact on the way in which agencies can and should react and relate to the implementation of qualifications frameworks. Coordinating efforts in developing and implementing qualifications frameworks and other Bologna reforms is important to ensure a successful consolidation of the European Higher Education Area. In addition, all relevant actors should make efforts to ensure that information on the role and purpose of qualifications frameworks and learning outcomes-based assessment is provided to all interested parties, including employers, so that student employability can be improved, mobility facilitated, and recognition of non-formal and informal learning further developed. Download the report Quality Assurance and Qualifications Frameworks: Exchanging Good Practice.

This report is based on the ENQA workshop on the theme “Quality Assurance in Lifelong Learning” that was held from the 16-17th of May 2011 in Bonn, Germany. The workshop provided a platform for discussion and exchange of experiences among the main stakeholders in quality assurance. The workshop aimed to contribute to joint understanding of the quality assurance in lifelong learning (LLL) between stakeholders, to disseminate information on good practice of external quality assirance in LLL, and to discuss standards and procedures for external quality ssurance in LLL. Download the Report.
See also ENQA workshop on Quality Assurance and Lifelong Learning, Quality Assurance and Learning Outcomes, L’AERES, évaluée et reconnue par l’ENQA, 6th European Quality Assurance Forum.
Quality Assurance in Lifelong Learning
Endika Bengoetxea, Outi Kallioinen, Immo Schmidt-Jortzig, Richard Thorn.
Foreword
The implementation of Lifelong Learning (LLL) in European higher education institutions is one of the most important educational and carrier development oriented initiatives of this decade. Albeit an essential path in the continuous improvement of skills, competences and knowledge throughout the life of an individual, this project is also challenging, as it involves expectations not only from the educational, the social and the professional worlds, but employers and employees as well. Moreover, the fact that expectations and demands may vary nationally, regionally and locally demands understanding, transparency and coordination between lifelong learning providers. The quality assurance of implementing Lifelong Learning into European higher education institutions is currently part of ENQA’s main focus areas. ENQA is promoting debates on how to develop quality assurance processes for lifelong learning schemes. In order to contribute to joint understanding of the quality assurance in Lifelong Learning between all stakeholders, to disseminate information on good practice, and to discuss standards and procedures, ENQA organised a workshop on the theme “Quality Assurance in Lifelong Learning” that was held in May 2011 in Bonn, Germany. The workshop provided a platform for discussion and exchange of experiences among the main stakeholders in quality assurance.
This publication presents four articles based on the workshop on Lifelong Learning. The following articles will discuss the national experiences, observations and results from the perspectives of the European Commission, the Laurea University of Applied Sciences in Finland, the Foundation for International Business Administration Accreditation (FIBAA) in Germany, and the Institutes of Technology in Ireland. Achim Hopbach, President, European Association for Quality Assurance in Higher Education (ENQA)
CHAPTER 1: Quality Assurance in Higher Education Lifelong Learning: Objectives and challenges on the European Union, Endika Bengoetxea, European Commission, Belgium
1.1 Introduction
The European Union’s Europe 2020 strategy sets out a vision of Europe’s social market economy for the 21st century, with a strong focus on skills and lifelong learning. It shows how the EU can come out stronger from the crisis and how it can be turned into a smart, sustainable and inclusive economy delivering high levels of employment, productivity and social cohesion. It includes a European benchmark for raising the proportion of higher education graduates (in the age range 30–34 years) to 40% by 2020.
The strategic framework for co-operation in Education and Training for 2020 –ET20202– focuses on four key areas:
1. Making lifelong learning and mobility a reality (including a European benchmark that by 2020 at least 15% of adults (age group 25–64) should participate in lifelong learning);
2. Improving the quality and efficiency of education and training;
3. Promoting equity, social cohesion and active citizenship;
4. Enhancing creativity and innovation, including entrepreneurship.
Given the need for Europe to raise skill levels and provide high quality education and training, it is no coincidence that lifelong learning and quality assurance figure as two of these priorities. The European Commission is cooperating actively with member states and stakeholders on reforms and follow-up.
Since the Bologna process started in 1999, there has been considerable improvement in building a higher education quality assurance culture in Europe, although efforts are still required to improve cooperation at European level. At the same time, quality assurance and transparency tools may need to evolve in order to remain up to date, as for instance, the European Standards and Guidelines which are at present under review.
While much of the focus of quality assurance is on initial training courses and degrees, developing quality assurance mechanisms for continuous training is also essential. More than ever, education systems are required to offer training courses and modules that ensure the right mix of skills, and lifelong learning activities must ensure that people improve knowledge, skills and competences within a personal, civic, social and/or employment-related perspective. As part of this strategy, higher education quality assurance systems must also play their role in guaranteeing that quality assurance mechanisms are established for continuous training courses.
The need to develop flexible study paths is also a priority: the percentage of ‘nontraditional’ groups (such as part-time students) seeking training is increasing, but there are not yet sufficient initiatives to satisfy this demand. Furthermore, these mature learners express a particular concern about the quality of the educational offer, which calls for a more direct involvement of quality assurance systems in lifelong learning. Download the Report.

This report is based on the annual ENQA Internal Quality Assurance seminar on the theme of Learning from each other – using benchmarking to develop IQA that was held on 16-17 June 2011 in Helsinki, Finland.
It presents a general overview of the benchmarking theme and discusses common features and differences of the benchmarked agencies’ IQA activities in terms of the selected three themes: performance indicators, follow-up on feedback and staff competence/development. The report also puts forward the benchmarking partners’ views on strengths, weaknesses and recommendations for development of each other’s activities, as well as the good practice they have identified on the selected theme. Download the Report.
Internal Quality Assurance and Benchmarking
DOUGLAS BLACKSTOCK, NADINE BURQUEL, NÚRIA COMET, MATTI KAJASTE, SÉRGIO MACHADO DOS SANTOS, SANDRA MARCOS, MARION MOSER, HENRI PONDS, HARALD SCHEUTHLE, LUIS CARLOS VELÓN SIXTO
Introduction
The Internal Quality Assurance group of ENQA (IQA Group) has been organising a yearly seminar for its members since 2007. Staff members involved in IQA of all ENQA members can join the activities of the Group. The main objective is to share experiences concerning the internal quality assurance of work processes in the participating agencies.
The Group is coordinated by a Steering group (SG), consisting of five members. The composition of the Steering group changes gradually by election of one or two members every year.
The overarching theme of the 2011 seminar was how to use benchmarking as a tool for developing an agency’s internal quality assurance system. The seminar gathered around 45 participants in the premises of the Finnish Higher Education Evaluation Council (FINHEEC) in Helsinki on 16-17 June 2011.
“Benchmarking involves comparing different aspects of the work of a group of organisations. It can be a very flexible approach. You can compare services, products or processes; you can look at a wide range of issues or focus on areas of concern; and you can benchmark with similar organisations or take a cross-sector approach on common issues such as customer care. Benchmarking may take place as a one-off exercise or be an ongoing relationship. The benchmarking exercise should be a mutually beneficial relationship, with every organisation in the benchmarking group being able to learn and develop from the experience of others.”
The Steering group based the preparation of the benchmarking activity on this definition. Agencies which are similar to each other, i.e. in size or scope, were grouped in pairs or triplets. Each group included an agency member of the Steering group. They compared their own practice with others on a certain focus area before the seminar, between January and May 2011. In addition to good practices, the participating agencies were encouraged to openly share which processes they find challenging or ineffective in their agencies. The findings were presented in the IQA Seminar in June 2011.
The benchmarking exercise focused on the following areas:
• Benchmarking of performance indicators (with FINHEEC)
• Benchmarking of on the follow-up of feedback (with ACSUCYL)
• Benchmarking of staff competence/development (with NVAO)
The present report gathers good practise and expertise related to these three themes: follow up on feedback (chapter 2), staff development (chapter 3) and performance indicators (chapter 4). The first chapter of the report provides a general overview of the benchmarking theme and is based on the keynote speech given by Dr Nadine Burquel... Download the Report.
Conclusions
Following the comparison exercise of the two Agencies, these conclusions may be drawn: Is it possible to compare Agencies?
• The agencies work in very different contexts, using different processes although some have comparable procedures;
• The indicators used for this benchmarking exercise may seem, at first sight, to be of little value and have little meaning if considered alone. They are context sensitive;
• More data from different agencies would be needed to choose the best and most representative indicators.
Could indicators be a tool to compare the performance of the agencies?
• It is difficult and lengthy to compare agencies with this type of indicators;
• They are a good internal tools to monitor and improve the effectiveness of the quality management system;
• It is useful to know which indicators other agencies use (qualitative comparison).
FINAL THOUGHTS
Some areas to work on in the future were identified:
• The group considered that it could be more important to proceed with the exchange of practices on the use of different procedures;
• Nevertheless, when exchanging and comparing practices, agencies should also look at the way(s) in which they measure the impact of such practices (internal: resources; and external: results)
• It would be important to develop meaningful indicators to assess the impact of the agency’s work on HEIs and on the HE system as a whole. Download the Report.

La démarche AQOR s’appuie sur une réflexion de professionnels de l’orientation sur des territoires expérimentaux de toutes les régions partenaires.
Les professionnels ayant travaillé sur le sujet (2007-2009) sont issus des réseaux (CIO, BIJ, CRIJ, CIBC, CIDFF, MIFE en France) et de 6 autres régions européennes. Ce travail s’appuie sur les résultats et recommandations du projet DROA (2003-2006) pour construire une batterie d’indicateurs servant de base à une auto-évaluation, type bottom-up.
Vous pouvez télécharger ci-dessous les documents qui ont été élaborés dans le cadre de ce travail: une présentation complète et détaillée du projet AQOR, le tableau de recueil des données concernant l’auto-évaluation de la structure d’orientation, un guide des indicateurs qualité réalisé en français.
La norme « Matrix » et la démarche « Matrix revisitée »
La norme Matrix, dont se sont inspirés les CARIF OREF pour établir les critères qualité « Matrix revisité », est une norme utilisée au Royaume-Uni et expérimentée par plus de 2000 structures de terrain de l’orientation.
Le document, que vous pouvez télécharger, permet d’en savoir plus sur cette norme Matrix : définitions, critères et intérêts d’utilisation. Ce document explique la démarche des 7 CARIF OREF pour découvrir cette norme.
Synthèse et présentation de deux exemples de normes utilisées par certains CARIF / OREF et par un réseau de structures AIO
Sorti de la formation initiale pour exercer une activité professionnelle, le travailleur doit pouvoir disposer - tout au long de sa vie professionnelle - d’outils afin de construire ou reconstruire une trajectoire professionnelle. Il doit pouvoir se prémunir contre tout risque de rupture prolongée de son activité professionnelle.
La loi du 24 novembre 2009 crée un service d’information et d’orientation tout au long de la vie. Ce dernier permet de rendre plus effectif le droit à l’information et à l’orientation également créés. Pour participer à ce nouveau service public et être labellisés, les nombreux prestataires existants devront présenter certaines garanties gages de qualité.
Les responsables des CARIF OREF ont mené une réflexion sur les engagements de qualité qui pourraient s’appliquer à des prestations d’information / orientation, au travers des travaux suivants:
- Les projets européens DROA (Développement des Réseaux pour l’Orientation Active), puis AQOR (Amélioration pour la Qualité de l’ORientation) menés par le PRAO (Pôle Rhône Alpes de l’Orientation), les réseaux de l’AIO (Accueil, Information et Orientation), 3 régions françaises et 4 régions.
- Le groupe de travail "MATRIX revisité" mené par le CARIFOREF Auvergne dans le cadre de sa mission Euroguidance et ses partenaires CARIFOREF de 8 régions.
Le document que vous pouvez télécharger présente une synthèse des travaux effectués. Il permet également de comparer les grands principes et les critères retenus pour chacune de ces deux démarches.

Faciliter l’accès à l’information, assurer la cohérence de l’information sur le territoire, tels sont les objectifs de la loi du 24 novembre 2009. Cet Eclairage s’inscrit dans la continuité du numéro # 18 : synthèse sur le droit à l’information et l’orientation et du Hors-série # 1 du Point d’actualité : synthèse sur les débats concernant la construction et mise en place du service public de l’orientation. Télécharger Eclairage-23.
1.1 "Un vrai service public de l’orientation exige une solidarité complète entre tous les acteurs de la cité "
1.1.1 « Les mutations économiques font qu’on ne peut plus envisager l’orientation comme un début de parcours »
« Il n’y a pas de formation tout au long de la vie sans orientation tout au long de la vie ! »
« Les solutions actuellement mises en place par les acteurs de l’orientation sont fondées sur un couple "diagnostic-prescription", extrêmement technique, fondé sur des questionnaires et qui ne prend absolument pas en question le développement de la personne et la manière que cette dernière a d’appréhender son parcours professionnel. (…) Une telle vision de l’orientation abolit les notions de parcours et de projet. La personne orientable n’est plus qu’un individu obéissant aveuglément à des règles qui lui échappent ». Selon le vice-président du Conseil régional Rhônes-Alpes, les mutations économiques et sociologiques de la société française ont pour conséquence de ne plus envisager l’orientation comme un début de parcours, mais comme une trajectoire qui balisera chaque instant de la vie.
1.1.2 « Une orientation réussie s’effectue en trois étapes »
« L’accueil est le point le plus important. Il ne s’improvise pas et ne consiste pas qu’à recevoir des gens dans un bureau. C’est l’étape fondamentale du parcours de l’orientation ». Viennent ensuite la formalisation des besoins et, enfin, la régulation de ces derniers en vue d’offrir la formation la plus adaptée non seulement à l’individu, mais aussi au tissu économique local. « Tout est question de choix, mais dans ce cas, nous travaillons sur des paradoxes : personne ne sait si son choix sera le bon avant qu’il n’ait eu l’occasion d’effectuer ce choix et d’en assumer les conséquences.
1.1.3 La formation et l’orientation tout au long de la vie comportent des exigences relevant du politique
« La mutualisation entre l’Etat, les Régions et le tissu associatif local sont essentiels. Il nous faut développer des partenariats multipartites entre ces entités, sans qu’ils ne soient trop formels non plus car, en ce cas, ils se transforment en métastructures institutionnelles qui ne disposent que de peu de marge de manoeuvre pour appliquer les actions décidées ». Davantage que tout, le service public de l’orientation doit, selon les termes de l’élu régional, reposer sur un maillage de terrain, au plus près des territoires, de leurs ressources et de leurs besoins. « Le service public de l’orientation doit rester le tisserand de cette dentelle territoriale » a-t-il conclu.
Source : Propos de Philippe Meirieu président délégué du Pôle Rhônes-Alpes de l’orientation (Prao). Le quotidien de la formation n° 1515, 22 avril 2011.

The seminar on quality assurance and accreditation in VET and HE held at the Berlin School of Economics and Law will bring together experts from Europe and beyond and from different professional and institutional backgrounds. The seminar has two main objectives: first, to explore methods and tools for overcoming the existing dichotomy between external and internal quality assurance and, secondly, to search for synergies to support the improvement of quality in education and training provision. It is hoped that the outcomes will help to identify the core elements of a mutually reinforcing approach to quality assurance and accreditation fit for lifelong learning and applicable to both VET and HE. As a by-product, it may be possible to define crucial areas for further research to support the above-mentioned objectives. Presentations of the Expert Seminar 24-25 February 2011.
President, Berlin School of Economics and Law (BSEL), Welcome Address to the International Expert Seminar on “Quality Assurance and Accreditation in Lifelong Learning”
The international seminar that we are starting today brings together experts in quality assurance from two fields: higher education (HE) and vocational education and training (VET). HE and VET are two fields, or educational subsystems, that are institutionally distinct but are both part of a strategy of life-long learning. Life-long learning requires the permeability between VET and HE (in order to allow people, for example, to advance step-by-step from apprenticeship to an academic job)...
Regular benchmarking of internal processes of the university: The Berlin School of Economics and Law is part of an alliance of seven large and highly-reputed German Universities of Applied Sciences, known as the UAS7 alliance.
One of the major purposes of this alliance is to engage in benchmarking internal structures and processes of the seven universities. For example, we have just completed a comparative analysis of our recruiting procedures for hiring new professors. This analysis has not only compared existing procedures but has identified features of best practice in individual universities and has ended with the elaboration of common standards for recruitment processes. This has been a most stimulating experience of mutual learning. Similar benchmarking processes are currently taking place in the fields of teaching methods, the organisation of further training, and the organisation of support for research and doctoral studies.
Our expertise with the link between HE and VET: The second-largest department of the Berlin School of Economics and Law is our Department of Company-Linked Programs, where Bachelor students are recruited by firms before starting their degree programs. In these programs, the curricula provide for a close link between modules studied at the university and phases of practical training in the firms where the students are affiliated. These programs are known as “Dual studies” (Duales Studium). They constitute a success story, linking elements of HE and VET. Similar links between modules studied at the university and phases of practical training “on the ground” are also provided in some of our degree programs directed at the public sector, in particular in criminal justice and police.
Conclusion the LLL perspective
In a LLL perspective, Ranking is a powerful tool for:
Enhancing the transparency of the system
Giving more information to the customers about the quality of the Vet provision,
Rewarding the best providers (by assigning more courses or more money)
Encouraging and improving Vet providers self evaluation
But a great attention should be paid to guarantee  the equality  of the Ranking, by taking in account the different contexts and the different groups targeted by the providers.
A bad, or unfair comparison can produce perverse effects, generating lack of motivation or opportunistic behaviors.
Quality Assurance at European level: a bright future
Ten years of implementation of the Bologna and Copenhagen processes: growing importance of QA in education and training world since EU tools produced need to be quality assured and better inter-linked. The shift to learning-outcomes-based frameworks, with its related standards curricula, certification processes, assessment mechanisms and teaching methods needs credible and robust QA arrangements to support it. Strong growth in quality assurance, especially in HE with a large number of institutional players; Quality in VET less developed with fewer actors and tools available at EU level. Open issues related to how to render QA into a mechanism for promoting a quality culture within education and training institutions and for achieving permeability between education sub-sectors.
State of the Art in QA
Maturity of QA at Institution Level?
Many German HE institutions establish systematic QM as a prerequisite for system accreditation.
At present: reluctance to enter system accreditation.
Universities‘ QM systems still at initial stage (e.g. evaluation only at course level)
Have undergone accreditation at programme level.
Maturity of Universities‘ approaches is under evaluation in system accreditation.
However, so far no empirical findings available due to the small number of cases.
Challenges for QA?
Reference points for assessing quality are missing.
Against what standards are we evaluating achievements?
EQF/NQF are underutilised; could serve as reference points.
Learning outcomes are not sufficiently issued in QA procedures.
ESG accepted and in use as main source for developing QA?
Challenges for QA at Institution Level?
Closing quality circuits: what consequences (if any) are drawn from poor evaluation results (“evidence-based action“)?
Managerial power of institutions‘ leaderships?
Strategies for establishing quality culture: dialogue and rewards
Development of context-sensitive QM procedures and instruments: acceptance of QA in the academia.
Development Paths
Quality Control or Quality Development?
Accreditation or Quality Audits?
Promoting institutional autonomy, bringing evaluation back in
QA as institutional research
Providing evidence on the determinants of student learning outcomes
Evaluation research rather descriptive, analytic potentials of available data remains underutilised.
Establishment of QA units, “Chief Quality Officers“.
Benefits to the institution:
As part of the leadership (“Stabsstelle“): underlining the emphasis which is placed on QA by HE institutions‘ managements.
As part of the academia (Institutional Research Centre): more independent, high trust in the outcomes of the performed institutional research.
Open Questions
How to link QA in HE and VET?
QA in a LLL perspective:
Analysis of students‘ learning biographies as a significant contribution to establish sound evidence on study programmes‘ learning impacts.
Establishing a monitoring system on the education system‘s effectiveness.
HE and quality standards. What can be learnt from VET (and the school sector)?

ENQA workshop on Quality Assurance and Lifelong Learning, 16-17 May 2011, Bonn, Germany
ENQA, in cooperation with FIBAA, will organise a workshop on Quality Assurance and Lifelong Learning to be held on 16-17 May 2011 in Bonn, Germany.
The workshop on Quality Assurance in Lifelong Learning - Recent Developments and Future Perspectives provides a platform for discussion and exchange of experiences among the main stakeholders in quality assurance. Specifically, the workshop will be of interest to ENQA member agencies, rectors and vice-rectors responsible for QA, students and employers.
The workshop aims to contribute to joint understanding of the quality assurance in Lifelong Learning (LLL) between all stake holders, to disseminate information on good practice of external quality assurance in LLL, and to discuss standards and procedures for external a quality assurance in LLL.
This workshop will be held on 16-17 May 2011 and hosted by the Foundation for International Business Administration Accreditation (FIBAA). The event will take place in Bonn, Germany, at the University Club Bonn.
PROGRAMME
The draft programme for the workshop can be found here.
REGISTRATION
Registration for this event is open. The deadline is 15 April, 2011 or until the workshop is fully booked.
FURTHER INFORMATION
For further information about the event, please contact Maria Stalter (maria.stalter@enqa.eu) or Nathalie Costes (nathalie.costes@enqa.eu) at the ENQA Secretariat.
See also: Quality Assurance and Learning Outcomes, L’AERES, évaluée et reconnue par l’ENQA, 6th European Quality Assurance Forum.