It presents a general overview of the benchmarking theme and discusses common features and differences of the benchmarked agencies’ IQA activities in terms of the selected three themes: performance indicators, follow-up on feedback and staff competence/development. The report also puts forward the benchmarking partners’ views on strengths, weaknesses and recommendations for development of each other’s activities, as well as the good practice they have identified on the selected theme. Download the Report.
Internal Quality Assurance and Benchmarking
DOUGLAS BLACKSTOCK, NADINE BURQUEL, NÚRIA COMET, MATTI KAJASTE, SÉRGIO MACHADO DOS SANTOS, SANDRA MARCOS, MARION MOSER, HENRI PONDS, HARALD SCHEUTHLE, LUIS CARLOS VELÓN SIXTO
The Internal Quality Assurance group of ENQA (IQA Group) has been organising a yearly seminar for its members since 2007. Staff members involved in IQA of all ENQA members can join the activities of the Group. The main objective is to share experiences concerning the internal quality assurance of work processes in the participating agencies.
The Group is coordinated by a Steering group (SG), consisting of five members. The composition of the Steering group changes gradually by election of one or two members every year.
The overarching theme of the 2011 seminar was how to use benchmarking as a tool for developing an agency’s internal quality assurance system. The seminar gathered around 45 participants in the premises of the Finnish Higher Education Evaluation Council (FINHEEC) in Helsinki on 16-17 June 2011.
“Benchmarking involves comparing different aspects of the work of a group of organisations. It can be a very flexible approach. You can compare services, products or processes; you can look at a wide range of issues or focus on areas of concern; and you can benchmark with similar organisations or take a cross-sector approach on common issues such as customer care. Benchmarking may take place as a one-off exercise or be an ongoing relationship. The benchmarking exercise should be a mutually beneficial relationship, with every organisation in the benchmarking group being able to learn and develop from the experience of others.”
The Steering group based the preparation of the benchmarking activity on this definition. Agencies which are similar to each other, i.e. in size or scope, were grouped in pairs or triplets. Each group included an agency member of the Steering group. They compared their own practice with others on a certain focus area before the seminar, between January and May 2011. In addition to good practices, the participating agencies were encouraged to openly share which processes they find challenging or ineffective in their agencies. The findings were presented in the IQA Seminar in June 2011.
The benchmarking exercise focused on the following areas:
• Benchmarking of performance indicators (with FINHEEC)
• Benchmarking of on the follow-up of feedback (with ACSUCYL)
• Benchmarking of staff competence/development (with NVAO)
The present report gathers good practise and expertise related to these three themes: follow up on feedback (chapter 2), staff development (chapter 3) and performance indicators (chapter 4). The first chapter of the report provides a general overview of the benchmarking theme and is based on the keynote speech given by Dr Nadine Burquel... Download the Report.
Following the comparison exercise of the two Agencies, these conclusions may be drawn: Is it possible to compare Agencies?
• The agencies work in very different contexts, using different processes although some have comparable procedures;
• The indicators used for this benchmarking exercise may seem, at first sight, to be of little value and have little meaning if considered alone. They are context sensitive;
• More data from different agencies would be needed to choose the best and most representative indicators.
Could indicators be a tool to compare the performance of the agencies?
• It is difficult and lengthy to compare agencies with this type of indicators;
• They are a good internal tools to monitor and improve the effectiveness of the quality management system;
• It is useful to know which indicators other agencies use (qualitative comparison).
Some areas to work on in the future were identified:
• The group considered that it could be more important to proceed with the exchange of practices on the use of different procedures;
• Nevertheless, when exchanging and comparing practices, agencies should also look at the way(s) in which they measure the impact of such practices (internal: resources; and external: results)
• It would be important to develop meaningful indicators to assess the impact of the agency’s work on HEIs and on the HE system as a whole. Download the Report.