Internal Quality Assurance from a HEI Perspective – Uppsala University 2nd Conference on Internal quality assurance at higher education institutions, 1 Dec. 2006, Bern 1 Background of the quality system at UU and the national demands 2 Main features of the quality system at UU 3 Some examples, outcomes and benefits of the system at UU 4 Some conclusions Annika.Lundmark@uadm.uu.se

Main University Building

Uppsala University – some facts Oldest university in the Nordic countries - founded in 1477 6 000 employees - including 4, 000 teachers/researchers Education and research across nine faculties Three disciplinary domains: Arts and Social Sciences Science and Technology Medicine and Pharmacy Tradition-drenched student activities Interdisciplinary campus areas More information: http://www.uu.se/english

Education 40,000 undergraduate students More than 40 programs of study 1,800 single-subject courses 2,500 graduate students Student exchange programs with 400 universities in 40 countries More information: http://www.uu.se/english

Organisation More information: http://www.uu.se/english

Background of the actual quality system at UU 1995 National Agency for Higher Education established 1993-98 1st central quality group at UU 1996 1st quality audit at UU 1999-2000 2nd central quality group & 2nd quality audit at UU 1999-2001 SAUNA – an internal quality evaluation 2001-2006 The National Agency policy changes from quality audit to quality assessment of all subjects and programs From 2001 A new quality and evaluation unit 3d central quality group From 2002 The actual quality enhancement policy

Some demands from the National Agency Quality audits at all institutions between 1995 and 2002 Quality assessments of all subjects & programs between 2001 and 2006 – ”The six year plan” Evaluations with focus on specific aspects, e.g. - students´ influence, internationalization Both quality audits and quality assessments between 2007 and 2013 A new ”six year plan” will be decided later this year Perhaps including the possibility to apply for ”honour for an excellent educational environment”

Our view of the National Agency´s six year plan between 2001 and 2006 Positive effects and conclusions: UU has been successful The NA has focused on important national problems, e.g. the situation for small subjects, lack of resources Negative effects and conclusions: Sometimes (often?) a heavy burden for departments Lack of clarity of the aims and focus The usefulness for departments and programs? the self evaluation the site visit

To discuss……….. What are your own experiences of quality audits and/or quality assessment concerning the effects on quality, quality assurance and quality enhancement? Are your experiences the same as the experiences at Uppsala university? Or are your experiences quite different? If YES: In what ways?

The concept of quality ”You know it when you see it” Quality is a concept with different potential meanings… The standard of something when it is compared to other things like it; how good or bad something is High standard a thing that is a part om a person´s character, especially something good a feature of something, especially one that makes it different from something else Oxford Advanced Learner´s Dictionary

Quality … depends ”on the eye of the beholder” develops and changes over time cannot be described in a subtle way using quantitative data can focus different aspects: prerequisites, processes, results and effects is a relative concept - fitness for purpose - value for money will have different concrete meaning depending upon what you focus, e.g. students´view of a certain course or the development of a new vocational program

The actual quality assessment system at Uppsala university * The nine faculties are responsible for their quality and quality assurance in education and research * Decisions in the University Board concerning infrastructural matters and resources * A long term policy for quality enhancement in education and research * An action plan for central quality enhancement has been decided each year * Central unit for quality and evaluation and a quality committée * 2006: - evaluation of research at UU - work on a new quality enhancement policy - quality assurance will be more integrated with the operational planning & annual reporting

Some values and conditions of quality assessment at UU Quality is created and shaped in faculties and departments … by teachers and students The central strategies and actions must reinforce a culture of quality enhancement The Vice Chancellor´s engagement is essential for a well-functioning quality system An active student union enhances work with quality issues Some external demands are good for enhancing the internal quality work

Internal resources and quality work Central level A quality committée Units for quality & evaluation, for teaching & interactive learning, for management training etc. Policies, e.g. Quality Enhancement Program Educational policy Projects, e.g. Teaching portfolios Training, e.g. teacher training ACTION ORIENTED EVALUATIONS Faculty level Different activities on quality issues, e.g. analyses of resource allocations, quality enhancement etc. Departmental Different projects level Program- & subject- course evaluations

Aims and objectives for the unit for quality and evaluation Policy work together with the quality committé, e.g. this year work on Bologna issues Implementation of the actual action plan; among others action oriented evaluations, e.g. this year reporting on an PhD-alumni evaluation Support and consultative work to faculties, departments and programs, e.g. this year - the faculty of social science – webbased course evaluations - the faculty of pharmacy – a webbased evaluation of the students views of each semester in programs - together with the Student Union: how to prevent dropouts Covering the national and international discussion

The National Agency´s Quality Assurance of Subjects and Programmes every year 2001–2006 Audit of UU 2001 2002 2003 2004 2005 2006 Unit for - Alumni evaluations of all subjects & programs Quality & Evaluation evaluation of PhD-studies PhD-alumni evaluation - Consultations & ACTION ORIENTED evaluations

The National Agency´s Quality Audit & Quality Assessments 2007-2013 Audit of UU 2007 2008 2009 2010 2011 2012 2013 Unit for - Continued Alumni evaluations Quality & and probably a second set of PhD-evaluations Evaluation Action oriented evaluations Faculty Responsible for the quality level Demands for quality enhancement in the operational plan for all faculties a certain year Reporting on quality enhancement in the annual report

Undergraduate alumni evaluations Selection of approx. 6000 students who have graduated at least two years earlier in subjects or programmes assessed by the National agency a certain year, e.g. 2006: nursing, pharmacy, law, history of art am.others Questionnaires are sent by mail Separat – more than 80 – reports to all departments of the specific subject/program A final report on the results of the different faculties

Undergraduate alumni evaluations: the Questionnaire Background questions - Sex, age, degree Position in the labour market - How long time until employment? Working position, tasks, salary etc. Evaluation of the education - Attainment of goals set for higher education in the the Higher Education Act and goals set by the Faculty Board - Attainment of skills, e.g. communication skills - What the students were especially satisfied and dissatisfied with The demands of the present employment - To what extent do the respondents use certain skills? - The usefulness of the education

Undergraduate alumni evaluations cont. Some more examples of the results: nine out of of ten students would have chosen UU again many students satisfied with teaching & learning in problemsolving & written work about six out of ten students dissatisfied with the training in oral communication Some outcomes and benefits: faculties, departments, programs and teachers get more interested and find evaluations useful quality enhancement in different areas

The value of the alumni evaluations? an interview study with responsible professors/lecturers at twelve different departments and programs the most interesting parts of the reports, e.g. the most interesting results, are - the employment situation for the alumni - how the alumni value the outcomes and their teaching and learning experiences - the appendixes in which the answers from every alumni on all open ended questions are described, e.g. current employment especially valuable in the education what was missing in the education

The value of the alumni evaluations? – cont. The usefulness of the report: very useful when - doing the self assessment - working with the Bologna process the report actualizes both new and old questions on the agenda for further discussion gives arguments for changes gives suggestions for concrete action confirmation that the quality enhancement and/or course/program planning is working well gives much information to use in study counselling

To discuss… The alumni evaluations we have implemented are ACTION ORIENTED: Which are your own experiences of the value of alumni evaluations? Can you see clear advantages and disadvantages with these centrally implemented alumni evaluations?

Evaluation of PhD-studies Questionnaires to all PhD-students * importance of and satisfaction with working & research conditions * importance of and satisfaction with supervision & dissertation work * courses * pedagogical experience & training Separate reports of the results to each department

Evaluation of PhD-studies cont. 3. Departments: self-assessment 4. Analysis of results and summary report for all faculties at institutional level 5. Faculties: analysis and suggestions for enhancement in each faculty

Evaluation of PhD-studies: some outcomes and benefits Some results: big differences between departments most PhD-students satisfied with physical aspects of their working environment some students dissatisfied with some aspects of supervision & support of their own lecturing Some outcomes and benefits: Department management and teachers are interested and find the report useful Quality enhancement in departments and faculties

Evaluation of PhD-alumni Questionnaires to all PhD-alumni who graduated between 1997 to 2001 * Current relation to Uppsala university * Labour market placement * Main working assignments * To which extent is research included in the assignment? * Usefulness of the PhD-education * Satisfaction and demands with the PhD-education in different aspects, e.g. critical scientific thinking, working in a project group (a total of 18 aspects) Separate reports of the results to each faculty

Evaluation of PhD-alumni cont. Demands and satisfaction: generic skills (Faculty of Medicine)

Some conclusions Necessary with some external demands, especially in the initial work of building an internal quality system Important that as many teachers and students as possible are involved in some elements of the quality system (incl. pedagogical training) and find it useful The University management & University Board must be involved both in discussions and decisions and find this work useful It is important to make the quality work visible Quality enhancement should be ”evidence based” (Action oriented evaluations) The quality system must be well known and it is very important to have quality in the quality work to strengthen the legitimacy of the system

Thank you for your attention www.uadm.uu.se/kvalitet Annika.Lundmark@uadm.uu.se