Workshop 4 Monitoring by Statistical Indicators Goal setting & formulation of operational objectives as a key challenge of monitoring systems Case 1: The use of indicators for quality development in Denmark Case 2: The use of management information for statistical monitoring in Swedish HE

Workshop 4 please delete non-discussed principles below Common Principles for Quality Assurance in E&T discussed: Focus was on indicators, their use in QD/QA, and data production The following points were touched to some extent, at least indirectly: QA to ensure accountability & improvement of E&T QA as an integral part of the internal management of E&T institutions QA systems:+ clear & measurable objectives & standards + consistent evaluation methods (self-assessm. & external review) + feedback mechanisms & procedures for improvement + widely accessible evaluation results QA as a co-operative process across levels, involving all relevant stakeholders, within countries & across Europe

Workshop 4 Main Lessons Learned/Workshop Results 1. We should put indicators in the right perspective: tools not truth; starting point not end; reasonable application; fairness related to available resources; take into account unintended effects (e.g., selectiing best pupils, give better marks) 2.To integrate indicators in decision making we need clear and operational goals and objectives (criteria, standards), and we should use a combination of “hard” (objective data with consequences) and “soft” approaches (hearing the stakeholders view and perspective). Open problem is to account for the contextual conditions. 3. Concerning data, we need publicly available individual data (instead of aggregate data) in order to calculate useful indicators, and particularly in order to make in-depth causal analyses for improving insight, and to understand contextual factors. Important issues are combination of statistics and process-produced data, quality control of the data, and (reasonable) data protection.

Workshop 4 optional Peer-learning visits for VET & HE: topichosting countrymain objective Data production as a combination of processproduced data and statistics, considering technical data reliability and data protection (maybe Sweden) To set up a comprehensive data base for the use in QD/QA Many other topics/objectives without concrete proposal: - choice of indicators, same indicators for VET and HE (maybe related to project by ENQA-VET), how do different kinds of users respond to indicators? -research oriented: how institutions behave if budget allocation is related to indicators (hard, soft ways of doing this) - how does assessment of non-formal and informal qualifications work in terms of outcomes? - how to measure competences and to include them in statistical systems?