1. Educational Development Centre May 2015
Assessment Planning: Why,When,What? Maristela Petrovic-Dzerdz, Instructional Designer Including Q&A Session with Dr. Anthony Marini, Senior Teaching Development Associate
Educational Development Centre May 2015
Educational Development Centre May 2015

2. Maristela Petrovic-Dzerdz, MEng, BEd, PBCID Instructional Designer
Assessment Planning: Why,When,What? Educational Development Centre May 2015
Educational Development Centre May 2015
Maristela Petrovic-Dzerdz, MEng, BEd, PBCID Instructional Designer

3. This presentation is based on: Smith P. L. & Ragan T. J
This presentation is based on: Smith P.L. & Ragan T. J. (2005) Instructional Design (3rd ed.). Hoboken, NJ: Wiley Jossey-Bass Education.

4. Post-workshop
Bring in 2-3 assessment items you use in your course (example of an essay question, multiple choice question, problem item, etc.) including the description of the assignment, for a more detailed analysis and group discussion.

5. Agenda Why to assess? When to design assessment?
What to consider when designing assessment?
When to assess?
What to assess?

6. Case study: Training on creating effective visuals

7. Case study: Learning Outcomes
Given a description of the audience characteristics, purposes of the presentation, and a source materials, the learner will be able to develop a presentation that incorporates the principles of visual design and good lettering.
Given a description of the audience characteristics, purposes of the presentation, and a handout developed for a presentation, the learner will be able to evaluate the handout based upon the principles of visual design, effective lettering, and appropriate use of the medium, and to suggest revisions.

8. Case study: One summative assessment
50-item test, open-ended and multiple-choice questions
List and discuss the eight principles of visual design
Which of the following is not a characteristic of the design element “surface”?
A. texture
B. color
C. value
D. form

9. Case study: Test results
Instructor was happy with grade distribution

10. WHY (to assess)? Stakeholders
Teacher/course designer
Students
Others (institution, employers, parents, etc.)

11. Student perspective Student behaviour is driven largely by assessment
“How can I meet the assessment requirements most effectively (in the least amount of time and with the least effort)”?

12. Course design questions
Where are we going?
How will we get there?
How do we know when we’re there?

13. WHEN (to design assessment)?
Learning outcomes
Instructional strategies

14. Course design sequence
1. Write learning objectives
2. Design assessment items (part of task analysis)
What will constitute evidence that the learners have learned?
How should students demonstrate their knowledge?
Are there skills to assess together with content?
Learning objectives
Assessment

15. Why writing assessment items immediately after writing objectives?
The intentions of the objectives will be fresh in your mind
If you cannot write an item for the objective, maybe you need to reexamine the objective
Learning objectives
Assessment

16. Robert Gagne: “Conditions of Learning” (1965).
McTighe and Wiggins: “Backward design” (1988).
John Biggs: “Constructive alignment” (1999).
Dee Fink : “Significant learning” (2003)

17. Takeaway 1
Try to think of assessment as soon as you know the course goals and learning outcomes

18. Adequate or passing performance
% of items learners must get correct
“Learners must get at least one correct out of each of the easy, medial and difficult categories”

19. Designing assessment blueprint:

20. Takeaway 2
As you start designing assessment, think about the desired level of performance. Is the required passing level ensuring the minimal acceptable performance?

21. WHAT & WHEN (to assess) A1 A2 A3

22. Types of assessment - WHEN
Entry Skill Assessments
Skills and knowledge one must have to begin instruction
Preassessment
What learners already know about the topic that they are to learn about though instruction
Can be used to gain students’ attention and to inform of the objective(s) of the lesson
Focus on the portion of the instruction not previous learned
Postassessment

23. Types of assessment – FOR WHAT PURPOSE
Formative Assessment
How is the student progressing? Combine with feedback – corrective measures. The purpose is LEARNING
Summative Assessment
What did the student learn in this course? Can student combine the knowledge and skills and demonstrate performance at the expected level? The purpose is to determine the LEVEL of COMPETENCE

24. WHAT to assess?

25. Learning to drive

26. Learning Domains based on the characteristics of the “content” that a learner must learn
Verbal information (declarative knowledge)
Intellectual skills
individual competence and ability to respond to stimuli
Cognitive strategies
capability to learn, think, and remember
Attitudes
ingrained bias towards different ideas, people, situation
Psychomotor skills
capability to learn to drive, ride a bike, draw a straight line, etc.
Robert Gagné
“Conditions of Learning” 1965.
“Most fundamental, most widely used and most useful in designing instructional materials” (Smith & Ragan, 2005.)

27. WHAT (to assess)? Verbal information (declarative knowledge)
Intellectual skills (procedural knowledge)
Cognitive strategies
Attitudes
Psychomotor skills
+ Enterprise schema and Integrated goals (With D. Merrill)

28. WHAT (to teach and assess)?
Verbal information (declarative knowledge)
Intellectual skills (procedural knowledge)
Cognitive strategies
Attitudes
Psychomotor skills

29. Declarative Knowledge: “Knowing that…”
Recall in verbatim, paraphrased, or summarized form facts, lists, names, or organized information
Recall, recognize, state in own words
Knowing declarative knowledge helps students learn higher-order, more complex objectives, such as intellectual skills

30. WHAT (to teach and assess)?
Verbal information (declarative knowledge)
Intellectual skills
Cognitive strategies
Attitudes
Psychomotor skills

31. Intellectual skills: “ Knowing how…”
The application of rules to previously unencountered examples

32. Intellectual Skills: Subcategories
Discrimination
Concepts
Principles (if-then)
Procedures
Problem Solving

33. Discrimination: learning to differentiate, to perceive that something either matches or differs from other things

34. Which of a number of figures is a rhombus?
The acquisition of concepts helps the learner to simplify the world (responds to things as members of “groups”)
Which of a number of figures is a rhombus?

35. Principles (if-then) help us predict, explain, control
“Any body completely or partially submerged in a fluid (gas or liquid) at rest is acted upon by an upward, or buoyant, force the magnitude of which is equal to the weight of the fluid displaced by the body.”

36. Procedures: In what order certain steps should be taken (and why)

37. Problem Solving: Learned capability involving selection and application of multiple rules
Proving a geometric theorem (learning how to respond to a class of problems)

38. Takeaway 3
Are you assessing “Knowing that” or “Knowing how” (or both)?
If you are assessing “knowing how”, are you PROGRESSIVELY assessing skills and knowledge in
Discrimination
Concepts
Principles
Procedures
Problem solving

39. WHAT (to assess)? Verbal information (declarative knowledge)
Intellectual skills
Cognitive strategies
Attitudes
Psychomotor skills

40. Cognitive strategies (knowing how to learn)
Rehearsal strategies (select, recall)
Elaboration strategies (tie new information to prior knowledge)
Organizational strategies (define the relationship among information)
Metacognition (knowledge about own cognitive processes and ability to control them)
Affective strategies (for focusing attention, maintaining concentration, establish and maintain motivation, manage time)

41. WHAT (to assess)? Verbal information (declarative knowledge)
Intellectual skills
Cognitive strategies
Attitudes
Psychomotor skills

42. Attitudes (“How to behave”)
Attitudes influence the choices that learners make
Instruction in attitudes often subtle and indirect
Simulation games

43. WHAT (to assess)? Verbal information (declarative knowledge)
Intellectual skills
Cognitive strategies
Attitudes
Psychomotor skills

44. Psychomotor Skills
Coordinated muscular movements that are typified by smoothness and precise timing

45. Learning to drive

46. Takeaways
Plan for assessment as soon as you know the course goals and intended learning outcomes
Consider the purpose of assessment and when in the course it should happen
Think about the expected “performance” – what is a desired level? What is a passing level?

47. Takeaway (cont’d)
Design “assessment blueprint” aligned with intended learning outcomes
There are different domains of learning and they should be assessed accordingly

48. References
Smith P.L. & Ragan T. J. (2005) Instructional Design (3rd ed.). Hoboken, NJ: Wiley Jossey-Bass Education.
Gagné, R.M. (1985). The Conditions of Learning and Theory of Instruction (4th ed.). New York: CBS College Publishing.
Bloom B.S. , Hastings, J.T., Madaus, G.F. , (1971) Handbook on formative and summative evaluation of student learning. McGraw-Hill, New York.
Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education 32:
Wiggins G. and McTighe J., (2005). Understanding by Design. Alexandria, VA : Association for Supervision and Curriculum Development.

49. Assessment Planning: Why,When,What? May 2015, EDC, Carleton University
Thank You
Assessment Planning: Why,When,What? May 2015, EDC, Carleton University
Maristela Petrovic-Dzerdz, MEng, BEd, PBCID Instructional Designer

50. Q&A Session with Dr. Anthony Marini 3M Teaching Fellow
Q&A Session with Dr. Anthony Marini 3M Teaching Fellow EDC Senior Teaching Development Associate
May 2015

51. Question 1
“I have a question about the expected performance and desired level. Today we hear students say that if you have paid your tuition, you should expect to pass the course. We had in the past cases when (and it was also in newspapers) too many students fail the course, so the question was raised if the ‘clients’ pay, yes they should expect a good course design, but should they also expect to pass the course because they paid for it?”

52. Question 2
“What to do if you have a graduate student who, you realize, does not have skills necessary at the graduate level? We know the rule that you should not get less than B- in a graduate course. A couple of times in my career I had to give a graduate student less than B- , and in one particular case I had to give student, who was about to graduate, a D- ; I was very concerned about this person going into the world with masters degree and Carleton University credentials and I wanted to send a strong message. The student complained, appealed…It is very difficult. What should be the passing level? Maybe I have not been explicit about the passing level up front? “

53. Question 3
“The ‘safe and effective’ thing resonates with me, since I teach engineering. If someone gets D- in my engineering design course, I wouldn’t trust him to design a house for my dog. From course design, from best educational practices perspective, it makes a lot of sense; but in university environment, do I have a freedom to choose what the passing level should be and not allow for a 50% to be a passing level? “

54. Question 4
“In our department we had a discussion because it seems that the grades are too high. Some instructors say that the students put so much effort, came to all the classes, all the office hours…How much can we justify grading the effort as to grading the actual level of performance in the course? “

55. Question 5
“In Europe, we have that maybe 20% of your mark is effort and participation and the 80% is actual performance. Maybe this is a solution?“