Presentation is loading. Please wait.

Presentation is loading. Please wait.

Professionalization of evaluation – recent activities within DeGEval

Similar presentations


Presentation on theme: "Professionalization of evaluation – recent activities within DeGEval"— Presentation transcript:

1 Professionalization of evaluation – recent activities within DeGEval
Dr. Jan Ulrich Hense Department Psychologie, Empirische Pädagogik und Pädagogische Psychologie Jahreskongress SEVAL 2012, Workshop 2 Freitag, 7. September 2012 Université de Fribourg

2 (1) Some comments and questions on the professionalization of evaluation
Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

3 Two uses of „professional“
Referring to a job or designation (“a professional painter”) Is evaluation developing into a full-blown profession? Referring to performance on a task (“a professional repair”) How to carry out evaluations professionally? Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

4 Evaluators? (Mieg, 2010, p. 16) Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

5 Two Classes of Professions (Mieg, 2010)
Care and relationship orientation Clients or patients trust professionals’ expertise and integrity Readiness to work cost free in emergency situation e.g. Doctors, Lawyers, Psychotherapists Market orientation Price competition Income orientation e.g. Engineers, Consultants Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

6 Professionalization as regulation of autonomy
Definition of education and training demands Control of market entry Determination of fees and charges Monopoly of performance evaluation (Mieg, 2010) Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

7 Conditions for the development of a new profession Combination of
a problem relevant for society and knowledge on how to explain and deal with the problem Relation to central values of society (e.g. health, justice) (Mostly) academic education and training Professional societies (Mieg, 2003) Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

8 Phases of development of a profession (Wilensky, 1964)
Some are working in a new area (mostly) full time Education and training programs are developed Professional societies emerge Shaping profile by delegating “lower” tasks Conflict between established and graduates of new training programs Competition with neighboring professions Professional Codes und Standards Move to formal closure Rules and conditions for working in the new profession Protection of designation Declaring non licensed work illegal Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

9 (2) Recent DeGEval activities in the context of professionalisation
Competencies for evaluators Didactics of Evaluation Further projects and plans Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

10 Competencies for evaluators
Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

11 Why develop "Recommendations on Education and Training in Evaluation"?
Growing demand for evaluation services Lack of qualified personnel Lack of a professional profile Guidelines as contribution to the professionalization of evaluation Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

12 A bit of history... 2004: First German edition
Authors: A. Caspari, M. Hennen, D. Scheffler, U. Schmidt & O. Schwab Published by the DeGEVal working group "Education and Training in Evaluation“ 2008: English translation download/order on Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

13 Goals and intended uses Standards for
Training programmes Direction for programme developers and teachers Evaluation of existing programmes Assessing evaluators Choice of an evaluator by clients Self-evaluation of evaluators Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

14 "A competency is more than just knowledge and skills
"A competency is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context." (OECD/OCDE DeSeCo-Project) Which knowledge, skills and attitudes are needed for conducting professional evaluations? Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

15 Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

16

17

18

19

20

21 Development of the recommendations Theoretical approach
Based on DeGEval standards of evaluation Practical procedure Expert panel Feedback cycles from DeGEval members Officially adopted by DeGEval conference Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

22 Who uses the guidelines? Usage in advanced study programmes
Berne  Saarbrücken  Bonn  Further education courses ? Evaluation courses within domain-specific curricula ? Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

23 Coverage of competencies in psychology diploma programmes
Soellner, R., Hapkemeyer, J. & Scheibner, N. (2010). Evaluationsausbildung in der Psychologie – eine Bestandsaufnahme. Zeitschrift für Evaluation, 9 (2), Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

24 Need for revision (to be discussed) Concept of competencies (KSA)
Field-specific interpretation and differentiation Elaboration of methodological competencies Compatibility with international models of evaluator competencies, e.g. King et al. (2001; 2005); Mertens (1994) Zorzi, McGuire & Perrin (2002) Russ-Eft et al. (2008) Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

25 2. Didactics of Evaluation
Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

26 aus-und-weiterbildung-in-der-evaluation/ publikationen-und-produkte Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

27 Why didactics of Evaluation? Competency profiles
Contents and learning goals of training programmes  What to teach Didactics of evaluation Practical guidelines for teachers and trainers  How to teach Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

28 Based on current state of theories on learning and instruction
Goals Provide guidance for training and education programmes courses in evaluation self-learning materials (textbooks etc.) Based on current state of theories on learning and instruction Supplement existing recommendations on competency profiles Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

29 Condensed “Cookbook” guidelines
0 Executive Summary Introduction Learning contents and goals: Competence profiles for evaluators Programmes Types of programmes and target groups Didactical foundation Learning and teaching methods Requirements for teachers/trainers Outlook Condensed “Cookbook” guidelines Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

30 Cover whole range of competencies (KSA)
Important Messages Cover whole range of competencies (KSA) Apply a mix of construction and instruction Emphasize active, self-directed learning Allow for learners’ own practical experiences Provide instructional support Embed all contents in authentic evaluation contexts Match method and learning goals Adapt to learners‘ individual needs and preconditions Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

31 Chocolate cookie exercise
Examples Chocolate cookie exercise Exercise on the logic of evaluation Reflection exercise „An evaluator is like a ...“ Discuss different evaluator roles and responsibilities Role play „stakeholder resistance“ Awareness for possible sources of resistance to evaluation Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

32 3. Further projects and plans
Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

33 Overview of training courses and seminars (online-database)
Future projects Other topics and activities of the DeGEval working group “Education and Training in Evaluation“ Overview of training courses and seminars (online-database) Textbook overview and review Evaluation capacity building Certification and credentialing of evaluators Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

34 Overall DeGEval efforts
Future projects Overall DeGEval efforts Work group on ethics in evaluation (“ethical codes”) Revision of evaluation standards (until 2014) Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)

35 Thank you for your interest!
contact: Qualitätssicherung in der Evaluation (SEVAL 2012, Universität Fribourg)


Download ppt "Professionalization of evaluation – recent activities within DeGEval"

Similar presentations


Ads by Google