1. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 1 Methodological Guidelines for Assessment of Profiles by Vocation October 2006

2. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 2 Objectives Methodological assistance For assessement of workers in the workplace Based on ‘learning outcoms’ from the national Bulgarian SER’s Linked to European VET policy

3. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 3 Definition of assessment Assessment means : the process of collecting evidence and making judgements on whether competency has been achieved to confirm that an individual can perform to the standard expected in the workplace as expressed in the relevant endorsed industry or enterprise competency standards or the learning outcomes of an accredited course.

4. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 4 Competences A competence is an integrated set of knowledge, skills and attitudes that can be employed efficiently by the candidate Knowledge Knowing facts,understand SkillsBeing able, apply AttitudesWanting, behave

5. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 5 Process of Assessment When discussing the process of assessment of vocational competences we want to focus on –Why? –What? –How? –Who? –When?

6. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 6 Why assessment ? To check whether the workers meet the professional standards To certify workers achievements in their profession To diagnose the learning needs To have feedback for future instruction To select for the next stage in work To encourage workers To evaluate AP(E)L

7. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 7 What to Assess? Classroom based assessment changes: Learning outcomes Workplace based Independent of learning pathway

8. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 8 WHAT do we assess? We assess the core competences described in the SER. A competence is an integrated set of knowledge, skills and attitudes that can be employed efficiently by the candidate Knowledge Knowing facts,understand SkillsBeing able, apply AttitudesWanting, behave

9. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 9 WHAT do we assess? We assess the learning outcomes described in the SER. Learning outcomes are the basis for an assessment program that focuses on what workers have to do to be successful in their job. Assessment identifies what action the worker should be able to perform. It also denotes the conditions under which the workers should demonstrate mastery.

10. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 10 Example: Refrigeration technician in food industry

11. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 11 WHAT do we assess? We can also assess the level of mastery Level 1 - vocations involving routine activities performed under routine conditions Level 2 - a range of complex activities performed in changing conditions Level 3 - a range of complex activities performed in changing conditions, as well as responsibilities for the performance of others Level 4 - a wide range of complex activities performed in changing conditions, as well as managerial responsibilities for the performance of others and for allocation of resources.

12. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 12 HOW do we assess? Principles of assessment –based on understanding of workers’ learning + workplace –accommodate for individual differences (flexible) –clearly explained to workers (e.g. criteria) –valid and have a reliable and fair process –allow for timely feedback –allow learners and trainers to reflect on their learning

13. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 13 HOW do we assess? Key elements of assessment: Assess in a congruent way; how the competence is implemented in the professional situation: Plan, Do, Check, Act ; Assess authenticity of products developed, of assignments or activities carried out in real professional situations; Assess on the basis of competence evidence demonstrated by the worker and which can be compared with the SER; Assess in an integrated way: theory and practice have to be assessed together. Criterion-related assessment: assessment is based on criteria taken from the SER; Assessment independent from the learning pathway: based solely on the demonstration of the competences by the worker Assessing in a specific professional context by a professional assessor

14. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 14 HOW do we assess? Assessment tools should be: –Valid, really linked to the prfile –Reliable, repeatable –Practical, usable

15. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 15 Valid assessment? To be valid, assessment tools should: –Be appropriate to the SER –match the type of skills listed in the LO –assess a broad range of the LO –have a weighting that reflects the relative importance of the range of skills in the specific LO. –have a good balance between theory and practice

16. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 16 Reliable and practical assessment? Reliable  Consistant results with different candidates  Consistant results with different assessors Practical  Takes account of available resources and equipment  Uses an acceptable amount of time

17. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 17 Assessment tools 1.Observation in real or live work situation 2.Simulation 3.A brief case (scenario) focusing on action and activity in a context 4.An extended case or scenario 5.Assignments for presentations 6.360° feedback 7.Portfolio

18. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 18 360° feedback Assessee Customer Level+1 Peer level-1suplier.Level-1 Peer Level+1

19. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 19 Assessment tools Assessment tools : –Checklists –Questions and Answers –Multiple choice questions –True/false questions –Observation in real or live work situation –Simulations –Cases –Scenarios –……

20. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 20 Important remarks on HOW to assess Assessment tools or techniques should be linked to the learning outcomes Attitudes and personal competences are difficult to assess because of subjectivity Therefore use a measuring scale Assess those personal competences that are relevant

21. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 21 Linking LO to assessment criteria Learning outcomes Methods, tools FeaturesIssuesCriteria Practical skills making and producing things manipula- ting materials and objects -Observation -Micro tasks (smaller components assessed separately and built up over time) - Checklists - Practical tests - Progress sheets - Scenarios Direct and indirect evidence - Assessment increasingly involves workplace assessors and joint - assessment decisions. - Criteria need to be discussed among assessors. -Range and scope of practical skills are specified - Simplicity or complexity of materials - Working to time scales and deadlines

22. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 22 SAM: Scale for Attitude Measurement example 1.1 initiative

23. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 23 How to assess with scenarios 1.Group the competences into families of situations with professional significance 2.Select a family of situations which can be integrated into a scenario and its components 3.Choose an integrative situation in relation with the selected family of situations 4.Decide on the criteria and the indicators 5.Decide on the weighing to be used 6.Make references to all the competences involved

24. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 24 WHEN do we assess? Assessing is a continuous process For strategic reasons we distinguish: –Diagnostic assessment: before entry of procedure –Recognition : AP(E)L –Formative assessment; continuous, more process than event; –Summative assessment; at the end of period of assessment

25. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 25 Accreditation of PRIOR ….. Accreditation of prior learning (APL) Accreditation of prior certificated learning (APCL) Accreditation of prior experiential learning (APEL) Accreditation of prior certificated and/or experiential learning (AP E/C L) Accreditation of prior learning and achievement (APL&A)

26. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 26 Importance of AP(E)L in assessment Recognition of experience Allowing people without formal qualifications to acquire qualifications Enhancing the chances of low-skilled or workers Allowing career changes Respect European principles for validation of non-formal and informal learning

27. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 27 Common European Principles for the identification and validation of non-formal and informal learning Elements of the identification and validation of non-formal and informal learning Individual entitlements Obligations of stakeholders Confidence and trust Credibility and legitimacy

28. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 28 Who is involved in assessment of competences in the workplace? the worker himself (self-assessment) his peers (peer-assessment) mentors in the company accredited assessors : experienced practitioners who are present during practical tests or to assess portfolios of workers The assessment procedure is controlled by a verifier from the ministry of labour, the ministry of education or the leading industrial body

29. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 29 Example of an assessment grid

30. PHARE 2003-BG 2003/2004 937 05.03 Magda Kirsch, Yves Beernaert & Luk Indesteege 30