Creating Effective Outlines and Assessments New Faculty Orientation August 21, 2012 Dr. Gosha Zywno, Faculty Associate, LTO 3M National Teaching Fellow

Outlines at Ryerson must contain (see Policy 145 for more detail): Course name, course number, semester and year Prerequisites and exclusions Professors name, contact information, office hours Method of posting grades Use of instruction for faculty communication Course calendar description Course objectives and focus Topics and schedule of topics Type of teaching methods Outside class activities, if any Requirements for submission of work Plagiarism detection tools used (Academic Integrity Office has a proper statement)- deadlines for choosing not to be included in use of these tools Field placement and technology requirements Policy on use of electronic equipment in class Policy on missed work

Good course outlines consider instructional design All components of the course should consider the learning objectives This includes the course content, assessments, activities, feedback to students Outlines in different fields are slightly different (you will have a chance to view some from different fields)

Good course outlines consider instructional design The goals/objectives of the course are explicitly stated ExampleExample of learning objectives: To become familiar with the basic terms, concepts, principles, and techniques of research methods and statistics To appreciate the relations between the hypotheses being tested (question asked) and the most appropriate statistical test to use To become proficient in using SPSS software to conduct various forms of basic data analysis To feel equipped by term’s end to apply the knowledge and skills gained to better understanding the statistical methods and results in published literature and to conduct one’s own analyses To understand how to interpret the results from these analyses

Alignment What are the desired learning outcomes? “Writing Learning Outcomes,” British Columbia Institute of Technology Learning Resources Unit,

Bloom’s Taxonomy Benjamin Bloom (1956), identified three domains of educational activities: Cognitive: mental skills (Knowledge) Affective: growth in feelings or emotional areas (Attitude) Psychomotor: manual or physical skills (Skills)

Think-Pair-Share Activity Reflect individually on the topic (THINK) Turn to your neighbour (PAIR) – Logistics: introduce yourself, decide Recorder-Reporter roles Recorder – your birth date is closest to today: take notes of your discussion Reporter – you may be called on to report to the large forum – Take 5 minutes to discuss/record the topic Report back to the larger group (SHARE) – Some pairs may be called on to report results Total time: 15 minutes

Think-Pair-Share Activity Logistics: – Think (individual) – 90 sec – Pair (discuss) – 5 minutes – Share (large group) – 5 minutes Topic: – In a course you will be teaching, identify one learning objective in each of the three domains (cognitive, affective, psychomotor) Total time: 15 minutes

Decide at what cognitive level you want to accomplish your goals: Different levels of cognitive learning Bloom’s Taxonomy New Version Old Version

Bloom’s Taxonomy To measure knowledge (common terms, routine info, definitions, descriptions, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Example: ???

Bloom’s Taxonomy To measure knowledge (common terms, routine info, definitions, descriptions, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State. Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams List steps involved in titration

To measure comprehension (understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Example: ???

To measure comprehension (understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Summarize basic tenets of deconstructionism

To measure application (solving problems, applying concepts and principles to new situations, using rules to perform calculations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Example: ???

To measure application (solving problems, applying concepts and principles to new situations, using rules to perform calculations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Calculate the deflection of a beam under uniform loading

To measure analysis (recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences, breakdown of problems to see structure, determine properties, see connections), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Example: ???

To measure analysis (recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences, breakdown of problems to see structure, determine properties, see connections), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams In the speech discussed, which statements are based on facts and which are based on assumptions?

To measure synthesis (integrate learning from different areas or solve problems by creative thinking, design, form a new system or solution), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Example: ???

To measure synthesis (integrate learning from different areas or solve problems by creative thinking, design, form a new system or solution), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams How would you restructure the school day to reflect children's developmental needs?

To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Example: ???

To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support. Bloom’s Taxonomy Reference: Quizzes, Tests and Exams by Barbara Gross Davis, University of California, Berkeley, 1999 Quizzes, Tests and Exams Why is Bach's Mass in B Minor acknowledged as a classic?

Bloom’s Taxonomy Kelley, K. & McDonald, R. Information Literacy and Writing Assessment Project: Tutorial for Developing and Evaluating Assignments. University of Maryland University College.

The level of cognitive learning reached is dependent upon the type of activities assigned. Activities must be designed in order to achieve the desired level of cognitive learning. Assessment TechniquesBloom’s Cognitive Level Reached Exams & Quizzes: Multiple Choice Fill in Essay Knowledge, Comprehension, Application HomeworkKnowledge, Comprehension, Application Problem SolvingKnowledge, Comprehension, Application Case StudiesKnowledge, Comprehension, Application, Analysis, Synthesis, Evaluation JournalsKnowledge, Comprehension, Application, Analysis, Synthesis, Evaluation Research ReportsKnowledge, Comprehension, Application, Analysis, Synthesis, Evaluation PortfoliosKnowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

Alignment Design assignments (and activities) to align with your desired course outcomes; some questions to ask: What are the outcomes to be assessed? What are the capabilities/skills (implicit or explicit) in the outcomes? Does the specific assessment task match the outcomes and skills? Is the method efficient in terms of student time and staff time? What alternatives are there? What are their advantages and disadvantages? Are the marking schemes or criteria appropriate? Brown, G. (2001). Assessment: A Guide for Lecturers. The LTSN Generic Centre Assessment Series.

Effective Assessments - Summary Align with Learning Objectives & Activities Match appropriate level of Cognitive Learning Have clear expectations Have reasonable workloads Engage students with material, are meaningful Have fair and consistent grading - use rubrics

Effective Assessments - Summary Align with Learning Objectives & Activities Match appropriate level of Cognitive Learning Have clear expectations Have reasonable workloads Engage students with material, are meaningful Have fair and consistent grading - use rubrics Encourage Academic Integrity Support Information Literacy Support EAL Students Are Accessible (Universal Instructional Design)

Information Literacy An information literate student is able to: Determine the extent of information needed Access the information effectively and efficiently Evaluate information and its sources critically Use information effectively to accomplish a specific purpose Understand the economic, legal and social issues surrounding the use of information, and access and use information ethically and legally

Information Literacy Information literacy-based assessments: Encourage self-directed learning and critical thinking Favor analysis and interpretation over rote- learning Require research and evaluation on a variety of resources Provide concrete feedback on progress

Effective Assessments Information Literacy Example Follow a research trend. Select a present day issue or concern and then search the past literature at five or ten year intervals. Discuss how theories or attitudes have endured or changed. Touro College Libraries.

Avoid cheating and plagiarism: Make assignments unique and personal Vary assignments and rotate tests Request rough drafts and description of research methods, or assign work in steps Be specific about grading criteria, consistent in grading For more information, visit:

Make assignments as specific as possible: Use more obscure material Pose a more focused question Require application rather than explanation Consider a tight comparison Make assignments unique and personal, change often Discouraging Plagiarism. Center for Innovative Teaching and Learning, Indiana University Bloomington.

Effective Assessments Academic Integrity Example Rather than: Explain the basic functions of the vascular, skeletal, muscular and nervous systems Try: A cat jumps off the end of a table onto the floor. Describe how its vascular, muscular, skeletal and nervous systems contribute to this action. Discouraging Plagiarism. Center for Innovative Teaching and Learning, Indiana University Bloomington.

Break down assignments Gives students time to ask questions Encourage a simple, direct style Emphasize ‘process’ rather than ‘product’ Incorporate low-stakes writing assignments Provide models Use explicit rubrics

Effective Assessments EAL Learners Example Use familiar, high frequency words, rather than low frequency, academic or idiomatic language Rather than: “Mitotic cell division is initiated in the ______.” Try: “Mitotic cell division begins in the ______.” Teemant, A. (2010). ESL Student Perspectives on University Classroom Testing Practices. Journal of the Scholarship of Teaching and Learning, 10(3),

Accessibility Universal Instructional Design creates barrier free assignments that benefit all students #1: Design instructions with the user in mind Present information in small chunks separated by headers Put key tasks and dates in bold Use a large, clear font #2: Reduce barriers to accessing the assignment Provide access to instructions and supplemental material in multiple formats Four Ways to Incorporate Universal Design for Learning into Your Assignment Instructions

Accessibility # 3 List assignment goals and evaluation criteria from the start With focus on objectives, it is easier to be flexible without losing academic rigour # 4 Provide Examples Show different approaches to the assignment Get permission to share examples of previous students’ work Post samples in an accessible electronic format on Blackboard or class website Four Ways to Incorporate Universal Design for Learning into Your Assignment Instructions Universal Instructional Design creates barrier free assignments that benefit all students