1. MODEL FOR COMBINED EVALUATION AND RECOGNITION OF LEARNING OUTCOMES
Wojciech Stęchły,
Małgorzata Pomorska, Blanquefort,

2. Phase 4 – points for the agenda
What is phase 4?
Presentation of the methodology proposed
Contribution expected from the partners
Scheduling of the work planned*

3. (phase3/4) Preparing and planning experimentations
What is phase 4?
(phase3/4) Preparing and planning experimentations
Developing 1st version of the model
Conducting experimentation and testing of the model
Agreeing on the model and reporting

4. MODEL FOR COMBINED EVALUATION AND RECOGNITION OF LEARNING OUTCOMES

5. The need for a new approach
Soft skills – innovative approach to assessment needed;
Measuring AND Evidencing;
Learning outcomes approach;
Focus on the impact on learners and stakeholders satisfaction;

6. IS: Analitical framework with a set of tools;
What is the model?
IS: Analitical framework with a set of tools;
FOR: increasing quality of learning, qualifications significance and stakeholders satisfaction with learning outcomes;
AND: helps in evaluation of training based on learning outcomes.

7. Methodological background (1)
I thought „An acclaimed training evaluation model is a good starting point for developing a model for evaluation of learning outcomes”
See: Kirkpatrick's Four-Level Training Evaluation Model for Analyzing Training Effectiveness
But, it needs adjustment, to be LO oriented.

8. Training – to evaluate or not to evaluate?
Consider this reasoning:
Learning goals
(intended LO)
Training / learning
(learning provision)
Learning outcomes
(assessed LO)  +  =
Evaluating both learning outcomes and training allows us to find what can / needs to be improved.

9. Methodological background (2)
The result: Four levels of evaluation of learning outcomes and training

10. What are we going to evaluate

11. What are we going to evaluate
Assessment of learning outcomes
Achievement of new skills (learning outcomes)
Choice of assessment methods
Quality of assessment
Environment impact
Learners and employers satisfaction
Usefulness of new skills on jobs
Training provision
Choice of training methods and pedagogical tools
Organization of learning

12. How are we going to evaluate?
Assessment of learning outcomes
Surveys
Interviews +
Own analysis

13. ASSESSMENT OF LEARNING OUTCOMES

14. What is assessment of learning outcomes
a process leading to confirmation, if a person has achieved the requirements specified in a standard;
for us the standard = learning outcomes and assessment criteria;
depending on the learning outcomes, different assessment methods can be used;
Why all this? Assessment of learning for each unit increases the value of the training and certificate (we want a certificate of competency, not just attendence!).

15. Assessment methods and tools
For the purpose of having a common terminology
Method
Tool
details / specifics
For example:
Written examination
Multiple choice test
30 minutes;
70% to pass
Observation in simulated conditions
Groupwork on task
Assessor checklist; 60 minutes

16. Assessment methods – a list
Oral examination
Written examination
Presentation
Observation in simulated conditions
Observation in real conditions
Outcome/product analysis
Dossier/portfolio analysis

17. Selecting assessment methods for learning outcomes
usually for a learning outcome more than one method of assessment can be used;
assessment should show the difference between ‘knowing how to do something’ and ‘doing it’;
if the outcome is about ‘talking’, the assessment should involve talking, and if the outcome is about ‘doing’ something, the assessment should involve doing it (not just talking about how it should be done);
rule of thumb: for practical learning outcomes select practical assessment methods;

18. Assessment – things to do
planning assessment;
analyzing:
the results [e.g. pass rate];
the process [using surveys];
checking the appropriateness of the methods selected;
reporting;
improving;

19. SURVEYS AND INTERVIEWS

20. The more perspectives, the better the evaluation
before, at the moment, after;
participants, organizers, recepients;
… organization, training quality, difficulty, usefullness, attractivness, lenght, contents choice …
 Good evaluation is a hard job

21. Surveys and interviews
“Pre-training survey”;
“Post-training survey”;
“Post-assessment survey”
2 interviews:
with training participants;
with partners (employers, cooperants) [only those who had to do with trainees]

22. Chronology of actions

23. Surveys and interviews
How many?
surveys – 3 x number of participants
interviews:
with 8 participants / 2 focus groups
with 6 partners / 2 focus groups
When?
surveys (see graph);
Interviews – to be discussed, 2-3 months after the training (?)

24. Main aspects analyzed:
Name
Main aspects analyzed:
“Pre-training survey” for all participants of training
Reasons for participation;
Expected results;
Self-assessment using list of outcomes proposed
“Post-training survey” for all training participants directly after training
Realization of training goals;
Meeting expectations;
Self-assessment using list of outcomes proposed;
Choice of contents;
Possible improvements;
Organization of training.
“Post-assessment survey” for after examination
Were all outcomes tested
Were people informed about the assessment
Interviews with training participants
Usefulness of skills in workplace (examples);
Interviews with partners (employers, cooperants – those who had to do with trainees)
Assessment of trainee skills using list of outcomes proposed;
Relevance of training for needs in workplace
Key differences between two main types of qualifications in the system.

25. Surveys – things to do
Translate and adjust if needed;
Insert learning outcomes;
Conduct, collect and analyze;
Good news: if you do the surveys in google forms, you will be able to collect and analyze easier. Trainees can fill them out via mobile phones.

26. interviews – things to do:
Translate and adjust if needed;
Insert learning outcomes;
Conduct

27. Your contribution Comments on the methodology documents and questions:
 for improvement of documents, increasing transparency;
 possibly for FAQ section;
Translations and language issues:
comments;
send the name of the „Model for combined evaluation and recognition of learning outcomes” in your language and
national (or internal) definitions of what is assessment;
 for improving the international understanding of the project output;
Changes and improvements in the surveys and interviews scenarios:
e.g. additional survey for trainers, different questions because of specific context
change of 1 word is important!;
additional questions;
 improving the information collected  improving the model and LO.
Send to by ________ 2017.