Presentation is loading. Please wait.

Presentation is loading. Please wait.

Professionalization of evaluation – recent activities within DeGEval Dr. Jan Ulrich Hense Department Psychologie, Empirische Pädagogik und Pädagogische.

Similar presentations


Presentation on theme: "Professionalization of evaluation – recent activities within DeGEval Dr. Jan Ulrich Hense Department Psychologie, Empirische Pädagogik und Pädagogische."— Presentation transcript:

1 Professionalization of evaluation – recent activities within DeGEval Dr. Jan Ulrich Hense Department Psychologie, Empirische Pädagogik und Pädagogische Psychologie jan.hense@psy.lmu.de Jahreskongress SEVAL 2012, Workshop 2 Freitag, 7. September 2012 Université de Fribourg

2 Dr. Jan Hense # 2 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) (1) Some comments and questions on the professionalization of evaluation

3 Dr. Jan Hense # 3 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Two uses of professional Referring to a job or designation (a professional painter) Is evaluation developing into a full-blown profession? Referring to performance on a task (a professional repair) How to carry out evaluations professionally?

4 Dr. Jan Hense # 4 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) (Mieg, 2010, p. 16) Evaluators?

5 Dr. Jan Hense # 5 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Two Classes of Professions (Mieg, 2010) Care and relationship orientation Clients or patients trust professionals expertise and integrity Readiness to work cost free in emergency situation e.g. Doctors, Lawyers, Psychotherapists Market orientation Price competition Income orientation e.g. Engineers, Consultants

6 Dr. Jan Hense # 6 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Professionalization as regulation of autonomy Definition of education and training demands Control of market entry Determination of fees and charges Monopoly of performance evaluation (Mieg, 2010)

7 Dr. Jan Hense # 7 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Conditions for the development of a new profession Combination of a problem relevant for society and knowledge on how to explain and deal with the problem Relation to central values of society (e.g. health, justice) (Mostly) academic education and training Professional societies (Mieg, 2003)

8 Dr. Jan Hense # 8 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Phases of development of a profession (Wilensky, 1964) Some are working in a new area (mostly) full time Education and training programs are developed Professional societies emerge Shaping profile by delegating lower tasks Conflict between established and graduates of new training programs Competition with neighboring professions Professional Codes und Standards Move to formal closure Rules and conditions for working in the new profession Protection of designation Declaring non licensed work illegal

9 Dr. Jan Hense # 9 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) (2) Recent DeGEval activities in the context of professionalisation 1. Competencies for evaluators 2. Didactics of Evaluation 3. Further projects and plans

10 Dr. Jan Hense # 10 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) 1.Competencies for evaluators

11 Dr. Jan Hense # 11 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Why develop "Recommendations on Education and Training in Evaluation"? Growing demand for evaluation services Lack of qualified personnel Lack of a professional profile Guidelines as contribution to the professionalization of evaluation

12 Dr. Jan Hense # 12 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) A bit of history... 2004: First German edition Authors: A. Caspari, M. Hennen, D. Scheffler, U. Schmidt & O. Schwab Published by the DeGEVal working group "Education and Training in Evaluation 2008: English translation download/order on www.degeval.de

13 Dr. Jan Hense # 13 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Goals and intended uses Standards for Training programmes –Direction for programme developers and teachers –Evaluation of existing programmes Assessing evaluators –Choice of an evaluator by clients –Self-evaluation of evaluators

14 Dr. Jan Hense # 14 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) "A competency is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context." (OECD/OCDE DeSeCo-Project) Which knowledge, skills and attitudes are needed for conducting professional evaluations?

15 Dr. Jan Hense # 15 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

16

17

18

19

20

21 Dr. Jan Hense # 21 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Development of the recommendations Theoretical approach Based on DeGEval standards of evaluation Practical procedure Expert panel Feedback cycles from DeGEval members Officially adopted by DeGEval conference

22 Dr. Jan Hense # 22 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Who uses the guidelines? Usage in advanced study programmes Berne Saarbrücken Bonn Further education courses ? Evaluation courses within domain-specific curricula?

23 Dr. Jan Hense # 23 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Soellner, R., Hapkemeyer, J. & Scheibner, N. (2010). Evaluationsausbildung in der Psychologie – eine Bestandsaufnahme. Zeitschrift für Evaluation, 9 (2), 297-308. Coverage of competencies in psychology diploma programmes

24 Dr. Jan Hense # 24 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Need for revision (to be discussed) Concept of competencies (KSA) Field-specific interpretation and differentiation Elaboration of methodological competencies Compatibility with international models of evaluator competencies, e.g. King et al. (2001; 2005); Mertens (1994) Zorzi, McGuire & Perrin (2002) Russ-Eft et al. (2008)

25 Dr. Jan Hense # 25 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) 2.Didactics of Evaluation

26 Dr. Jan Hense # 26 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) http://www.degeval.de/arbeitskreise/ aus-und-weiterbildung-in-der-evaluation/ publikationen-und-produkte

27 Dr. Jan Hense # 27 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Why didactics of Evaluation? Competency profiles Contents and learning goals of training programmes What to teach Didactics of evaluation Practical guidelines for teachers and trainers How to teach

28 Dr. Jan Hense # 28 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Goals Provide guidance for training and education programmes courses in evaluation self-learning materials (textbooks etc.) Based on current state of theories on learning and instruction Supplement existing recommendations on competency profiles

29 Dr. Jan Hense # 29 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) 0Executive Summary 1.Introduction 2.Learning contents and goals: Competence profiles for evaluators 3.Programmes 4.Types of programmes and target groups 5.Didactical foundation 6.Learning and teaching methods 7.Requirements for teachers/trainers 8.Outlook Condensed Cookbook guidelines

30 Dr. Jan Hense # 30 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Important Messages Cover whole range of competencies (KSA) Apply a mix of construction and instruction Emphasize active, self-directed learning Allow for learners own practical experiences Provide instructional support Embed all contents in authentic evaluation contexts Match method and learning goals Adapt to learners individual needs and preconditions

31 Dr. Jan Hense # 31 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Examples Chocolate cookie exercise Exercise on the logic of evaluation Reflection exercise An evaluator is like a... Discuss different evaluator roles and responsibilities Role play stakeholder resistance Awareness for possible sources of resistance to evaluation

32 Dr. Jan Hense # 32 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) 3.Further projects and plans

33 Dr. Jan Hense # 33 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Future projects Other topics and activities of the DeGEval working group Education and Training in Evaluation Overview of training courses and seminars (online-database) Textbook overview and review Evaluation capacity building Certification and credentialing of evaluators

34 Dr. Jan Hense # 34 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Future projects Overall DeGEval efforts Work group on ethics in evaluation (ethical codes) Revision of evaluation standards (until 2014)

35 Dr. Jan Hense # 35 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg) Thank you for your interest! contact: jan.hense@psy.lmu.de


Download ppt "Professionalization of evaluation – recent activities within DeGEval Dr. Jan Ulrich Hense Department Psychologie, Empirische Pädagogik und Pädagogische."

Similar presentations


Ads by Google