Presentation is loading. Please wait.

Presentation is loading. Please wait.

Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives and Classroom Assessment Jeff Froyd, Texas A&M University.

Similar presentations


Presentation on theme: "Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives and Classroom Assessment Jeff Froyd, Texas A&M University."— Presentation transcript:

1 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives and Classroom Assessment Jeff Froyd, Texas A&M University

2 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Workshop Presenter Jeff Froyd, Director of Academic Development Educational Achievement Division, College of Engineering, Texas A&M University Project Director, Foundation Coalition

3 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Acknowledgement Russ Pimmel, NSF Program Officer Former Professor of Electrical Engineering at the University of Alabama Assembled much of the material for similar workshops

4 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Objectives: Participants will gain experience in Describing the rationale for preparing and using learning objectives in an individual course Preparing specifications for high quality learning objectives Writing learning objectives for a single course Preparing specifications for assessment processes/tools Generating alternative assessment processes/tools for a single course Selecting assessment processes/tools for a single course

5 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Agenda Background for learning objectives Specifications for learning objectives Writing learning objectives Background for classroom assessment Specifications for assessment processes/tools Background on alternatives for assessment processes/tools Generating alternatives for assessment processes/tools Selecting assessment processes/tools Review workshop activities

6 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Agenda Background for learning objectives Specifications for learning objectives Requirements for specifications Team exercise: develop specifications for learning objectives Workshop exercise: improve specifications for learning objectives Writing learning objectives Individual exercise: write objectives for a course or a portion of a course Team exercise: review individual objectives Workshop exercise: develop a list of suggestions for writing learning objectives Individual exercises: revise objectives Team exercise: review individual objectives Workshop exercise: reflection

7 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Agenda Background for classroom assessment Specifications for assessment processes/tools Team exercise: develop a set of specifications for assessment processes/tools Workshop exercise: improve sets of specifications for assessment processes/tools Generating alternatives for assessment processes/tools Team exercise: select some learning objectives and generate alternative assessment processes/tools Background on alternatives for assessment processes/tools Selecting assessment processes/tools Individual exercise: select a set of learning objectives and generate alternative assessment processes/tools Individual exercise: select one or more assessment processes/tools that you would use in your course Team exercise: share and review choices of assessment processes/tools Review workshop activities

8 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives Background for Learning Objectives Session Objective: At the end of the session, participants will describe themselves as more confident in their ability to hold productive conversations with their colleagues regarding the place and importance of learning objectives in the teaching-learning process

9 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Stakeholders for Learning Objectives Who are the stakeholders in conversations about preparing and applying learning objectives? Faculty Students Employers Accreditation organizations

10 Western Michigan University, 18 September 2003, Kalamazoo, Michigan EC Program Outcomes (a) an ability to apply knowledge of mathematics, science, and engineering (b) an ability to design and conduct experiments, as well as to analyze and interpret data (c) an ability to design a system, component, or process to meet desired needs (d) an ability to function on multi-disciplinary teams (e) an ability to identify, formulate, and solve engineering problems (f) an understanding of professional and ethical responsibility (g) an ability to communicate effectively (h) the broad education necessary to understand the impact of engineering solutions in a global and societal context (i) a recognition of the need for, and an ability to engage in life-long learning (j) a knowledge of contemporary issues (k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.

11 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Bloom’s Taxonomy of Cognitive Learning Knowledge: defines, recalls, matches, reproduces Comprehension: explains, gives examples Application: discovering, assessing, computing Analysis: breaking down, organizing, inferring Synthesis: creating, putting together Evaluation: appraising, judging, selecting Western Michigan University, 25 October 2002, Kalamazoo, Michigan

12 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Purpose of Learning Objectives Communicate expectations for a course Provide a context for what will be learned

13 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Objectives and Students Objectives help students Clarify their personal goals Provide framework for measuring their success. Reduce their anxiety Improve their studying effectiveness Objectives help instructors Guide preparation of classroom material Make homework assignments Aid in test design Source -

14 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Strategies For Workshop Teams Be positive, supportive, and cooperative Limit critical or negative comments At least 5 positive comments for every negative comment Be brief and concise in discussions Avoid lengthy comments, stories, or arguments Stay focused

15 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Roles Assign team roles & follow through on responsibilities Coordinator -- Coordinates discussion & develops consensus Recorder -- Writes down the ideas & reports them Gatekeeper -- Keeps the team on the subject Timer -- Makes sure the team stays on schedule With smaller teams – combine gatekeeper & timer

16 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Workshop Team Roles For first exercise Coordinator – Individual with largest class last semester Recorder/Reporter – Individual on left of coordinator Gatekeeper/timer -- Individual on left of recorder Timer -- Individual on left of gatekeeper Roles rotate clockwise on subsequent exercise

17 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Form teams of four people Time: 5 minutes Develop at least four (4) advantages and four (4) disadvantages of preparing learning objectives for a course.

18 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Advantages Communication with students expectations and what they need to learn Offers focus in preparing material to accomplish goals Evaluation tool Tie a course to program or curriculum Clarifying goals to students seeks buy-in and self-assessment ABET Assessment tool Stating goals facilitates communicates with colleagues for better coordination Written goals facilitates communications with employers and other external groups Makes course easier to transfer among faculty members because much work is already done Disadvantages Could be too narrow or focused Explicitly stating learning objectives takes (too?) much time Busy work for faculty Could turn into curriculum nightmare Poorly written objectives don’t help Could be cumbersome, ambiguous Could be more difficult to measure

19 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives Preparing Specifications for Learning Objectives Session Objectives At the end of the session, participants will  Write specifications for learning objectives  Describe themselves as more confident in their ability to describe quality learning objectives.

20 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Form of Learning Objectives Write objectives as student outcome statements Objectives should answer the questions "What must students do to prove that they have succeeded?" "What should students be able to do as a consequence of instruction?" Source -

21 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Elements of an Objective Objective must contain three basic elements: Verb describing an observable action Conditions of this action “When given x you will be able to..." Level of acceptable performance Source -

22 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Verbs for Objectives Verbs for constructing concrete objectives: analyze computeclassify collaborate compare appreciate contrast define demonstratedirectderivedesignate discussdisplayevaluateknow identifyinferintegrateinterpret justifylistunderstandorganize graspreportrespondsolicit statesynthesizenameexplain Modified from

23 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives Verbs for Categories in Bloom’s Taxonomy Knowledge Define, describe, list, reproduce, enumerate Comprehension Classify, explain, discuss, give example, summarize Application Determine, develop, compute, chart, utilize Analysis Correlate, diagram, distinguish, outline, infer Evaluation Compare & contrast, critique, justify, conclude Synthesis Adapt, combine, compare, contrast, design, generate

24 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Complex Versus Simple Objectives One complex objective versus several simple ones? High-level versus low-level objectives Example One complex objective (4 or 5 weeks of classes) “Given a verbal description of a digital module, develop an implementation using any of 7 different logic devices” 15 to 20 simpler objectives (1 or 2 per class) “Given a verbal description, draw the truth table” “Given a truth table, obtain a minimum-cost equation” … “Draw the the NAND-gate implementation for an equation”

25 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Comparison Of Complex & Simple Objectives Multiple simple objectives More manageable “chunks” for students Explicit objective(s) for each class Simple (more manageable) homework problems and test questions Single complex objective Student's attention directed to the overall process May lead to higher level learning Students must deal with complexity Students must subdivide problem on their own

26 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Advantages Of Simple Objectives Advantages of simple objectives are more important in Large classes rather than small classes Introductory courses rather than advanced courses

27 Western Michigan University, 18 September 2003, Kalamazoo, Michigan How To Deal With “Understand” In Objectives How do you write objectives when you want students to “understand” a complex concept, system, or process Identify specific tasks that indicate “understanding” Specify objectives for each task Similar comments apply to “know”, “appreciate”, “value”

28 Western Michigan University, 18 September 2003, Kalamazoo, Michigan How To Deal With “Understand” In Objectives -- Example In our computer architecture course we want students to “understand” a sample architecture made up of several modules What would students be able to do if they “understood” Objectives – Students should be able to identify: All the modules and interconnecting signals Modules involved in a given system-level operation Output values for a given input values for each module Sub-module changes given a system level change …

29 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Writing Objectives – Piecemeal Approach Writing low-level objectives for a whole course may be overwhelming Use a “piecemeal” approach Write your lectures and define the homework as usual After each class -- write down what you expect the students to be able to do These become a list of objectives Give them to the students before each exam Use them to write the exam As semester progresses -- may become comfortable writing the objectives before you prepare your lecture

30 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Evolving Objectives In a 3-credit semester course Russ Pimmel (UA, now NSF) started with over 100 objectives Four offerings later -- down to about 50 Eliminated peripheral “stuff” that was not central Broadening, informational, perspective material

31 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Objectives and Homework Assignments Homework assignments should match objectives Students need to practice and explore the skills, knowledge, and attitudes defined in objectives Frequently require supplementary homework problems In some of my courses 1/3 of homework is from textbook Rest are supplementary problems With well defined objectives Writing homework problems is straightforward Same is true for exam questions

32 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Student’s Use Of Objectives Survey in 400-level required course Did you find the objectives helpful? Yes % No % Did you read the objectives? Frequently % Occasionally % Never %

33 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise -- Guidelines For Learning Objectives Task Write 3 or 5 guidelines for good learning objectives What are the common features? What should objectives look like? Think of guidelines as specifications Methodology Brain storm individually -- 2 minutes Establish consensus as a team -- 5 minutes Report team results -- 3 minutes Revise guidelines as a team-- 2 minutes

34 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Must be testable and measurable Achievable Clearly and precisely articulated Appropriate to course and audience Relate to program objectives Linked to course outcomes to allow for assessment Simple, one sentence, common format Simple better than complex Should be specific and unambiguous Relate to topic coverage

35 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives Preparing Learning Objectives Session Objectives At the end of the session, participants will  Write learning objectives for one or more courses that they teach  Describe themselves as more confident in their ability to describe quality learning objectives.

36 Western Michigan University, 18 September 2003, Kalamazoo, Michigan EC Program Outcomes (a) an ability to apply knowledge of mathematics, science, and engineering (b) an ability to design and conduct experiments, as well as to analyze and interpret data (c) an ability to design a system, component, or process to meet desired needs (d) an ability to function on multi-disciplinary teams (e) an ability to identify, formulate, and solve engineering problems (f) an understanding of professional and ethical responsibility (g) an ability to communicate effectively (h) the broad education necessary to understand the impact of engineering solutions in a global and societal context (i) a recognition of the need for, and an ability to engage in life-long learning (j) a knowledge of contemporary issues (k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.

37 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Bloom’s Taxonomy of Cognitive Learning Knowledge: defines, recalls, matches, reproduces Comprehension: explains, gives examples Application: discovering, assessing, computing Analysis: breaking down, organizing, inferring Synthesis: creating, putting together Evaluation: appraising, judging, selecting Western Michigan University, 25 October 2002, Kalamazoo, Michigan

38 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenges in Engineering Education Challenges Challenge of lifelong learning Challenge of problem solving Challenge of engineering design Challenge of transfer

39 LASSI SCALE ENGR111 (Mean) CVEN349 (Mean) Significance* Skill Component Information Processing Test Strategies Selecting Main Ideas Will Component Anxiety Attitude Motivation Self-regulation Component Concentration Self-testing Study Aids Time Management

40 “Although the data suggest a slight upward upward trend, the trend proved not to be statistically significant based upon an analysis of variance (ANOVA). Thus the cross-sectional study did not find evidence of an increase in readiness for self- directed learning, even for students in the later semesters who are taking elective courses and their capstone courses.” Litzinger, T., Wise, J., Lee, S., and Bjorklund, S. (2003) Assessing Readiness for Self-directed Learning, Proceedings, ASEE Annual Conference Lifelong Learning at Penn State Self-Directed Learning Readiness Survey (SDLRS), Guglielmino & Associates, com/prod01.htm, March

41 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Problem Solving “Despite individual professors’ dedication and efforts to develop problem solving skill, “general problem solving skill” was not developed in the four years in our undergraduate program. Students graduated showing the same inability that they had when they started the program. Some could not create hypotheses; some misread problem statements. During the four- year undergraduate engineering program studied, , the students had worked over 3000 homework problems, they had observed about 1000 sample solutions being worked on the board by either the teacher or by peers, and they had worked many open-ended problems. In other words, they showed no improvement in problem solving skills despite the best intentions of their instructors.” Woods, D. et al (1997) “Developing Problem Solving Skills: The McMaster Problem Solving Program,” Journal of Engineering Education,

42 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Problem Solving Ineffective approach #1. give the students open-ended problems to solve; This, we now see, is ineffective because the students get little feedback about the process steps, they tend to reinforce bad habits, they do not know what processes they should be using and they resort to trying to collect sample solutions and match past memorized sample solutions to new problem situations.

43 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Problem Solving Ineffective approach # 2: Show them how you solve problems by working many problems on the board and handing out many sample solutions This, we now see, is ineffective because teachers know too much. Teachers demonstrate "exercise solving". Teachers do not make mistakes; they do not struggle to figure out what the problem really is. They work forwards; not backwards from the goal. They do not demonstrate the "problem solving" process; they demonstrate the "exercise solving" process. If they did demonstrate "problem solving" with all its mistakes and trials, the students would brand the teacher as incompetent. We know; we tried!

44 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Problem Solving Ineffective approach #3: Have students solve problems on the board Different students use different approaches to solving problems; what works for one won't work for others. When we used this method as a research tool, the students reported "we learned nothing to help us solve problems by watching Jim, Sue and Brad solve those problems!"

45 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Problem Solving Through four research projects we identified why and how these and other teaching methods failed to develop process skills and which methods were successful in developing the skills Woods, D.R., J.D. Wright, T.W. Hoffman, R.K. Swartman and I.D. Doig (1975) "Teaching Problem Solving Skills," Annals of Engineering Education, 1, 1, Woods, D.R. et al. (1979) "Major Challenges to Teaching Problem Solving" Annals of Engineering Education, 70, No. 3 p. 277 to 284, 1979 and "56 Challenges to Teaching Problem Solving" CHEM 13 News no. 155 (1985). Woods, D.R. (1993a) "Problem solving - where are we now?" J. College Science Teaching, 22, Woods, D.R. (1993b) "Problem solving - what doesn't seem to work," J. College Science Teaching, 23, Woods, D.R. (1993c) "New Approaches for developing problem solving skills," J. College Science Teaching, 23,

46 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Engineering Design The literature is filled with positive comments from students, instructors, and industrial sponsors who have participated in capstone design courses. The vast majority of participants feel that the course benefited all involved. The nature of capstone design courses, however, often leads to a purely subjective evaluation with little or no “hard evidence” of actual benefits. Born, for example, does not attempt to prove the value of senior level design courses. He simply states that he is convinced from his experiences that such courses are valuable. Other educators have similar “feelings” as to the relative costs and benefits of capstone design courses. Dutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of Literature on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of Engineering Education

47 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Transfer Researches posed this problem to people. "Suppose you are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the health tissue?"

48 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Transfer Consider the following story "A small country was ruled from a strong fortress by a dictator. The fortress was situated in the middle of the country, surrounded by farms and villages. Many roads led to the fortress through the countryside. A rebel general vowed to capture the fortress. The general knew that an attack by his entire army would capture the fortress. He gathered his army at the head of one of the roads, ready to launch a full-scale direct attack. However, the general then learned that the dictator had planted mines on each of the roads. The mines were set so that small bodies of men could pass over them safely, since the dictator need to move his troops and workers to and from the fortress. However, any large force would detonate the mines. Not only would this blow up the road, but it would also destroy many neighboring villages. It therefore seemed impossible to capture the fortress. However, the general devised a simple plan. He divided his army into small groups and dispatched each group to the head of a different road. When all was ready he gave the signal and each group marched down a different road. Each group continued down it road to the fortress at the same time. In this way, the general captured the fortress and overthrew the dictator."

49 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Challenge of Transfer After the subjects read and summarized this story, they were asked to solve the tumor problem under the guise of a separate experiment. Given the clear analogy, you might think that performance would be near ceiling. Surprisingly, only 30% of the subjects offered a convergence solution. Moreover, when these same subjects were given the suggestion that they should use the General story, 80% provided a convergence solution. This finding demonstrates that half the subjects could apply the General story to the tumor problem when they were instructed to but did not do so on their own.

50 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Focusing Activity (8 minutes) INDIVIDUALLY – use 3 minutes to write your description of learning, what it is, what it looks like, how you might recognize when it has occurred, etc. AS A PAIR – use 5 minutes to discuss descriptions with someone sitting next to you. If you have additional time, develop a consensus description of learning.

51 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Focusing Activity ??

52 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Individual Exercise -- Writing Learning Objectives Individually write a set of objectives for a topic representing a few classes Something that you recently did in class Follow your team’s guidelines Questions to consider about your objectives Do they define student behavior? Are they observable, measurable? Can you write homework & exam problems? Are they consistent with the instructor’s intent? Time: 15 minutes

53 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise -- Reviewing Learning Objectives Review each other’s objectives Questions to consider in reviewing objectives Do they follow your team’s guidelines Do they define student behavior? Are they observable, measurable? Can you write homework problems & exam questions? Are they consistent with the instructor’s intent? 15 minutes

54 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Individual Exercise -- Revising Your Learning Objectives Rewrite your learning objectives based on your team’s review Report on biggest improvement 10 minutes

55 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Biggest Improvement Put quantifiable goals that students would achieve Clarify my objectives Classify learning objectives to go from basic to specific Make a different level of objectives from whole course to chapter to each lecture Make few higher level objective to organize multiple lower level objectives to support Identify more complex goals and separate into simpler components for teaching communications Improve clarity Make more understandable to the student Use proper verbiage to clarify

56 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Minute Paper Write a one-sentence answer to the following question: What is the “muddiest point” about learning objectives? (What is the most confusing point?)

57 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Muddiest Point How to incorporate those that relate to attitude and enthusiasm for lifelong learning Why do we spend time writing things that we do and we do best? Answer to my question about the level and number of learning objectives that are appropriate for publication in my syllabus. Are there “intermediate” not published objectives? How to relate a given objective to Bloom’s Taxonomy level, and how important is it to do so? Ways of measuring each learning objective How many objectives should be used – level of detail – then how do we practically measure them? “Being brief” vs. “Being precise and accurate” Being able to link objectives to larger program outcomes or goals Organizing subject matter to achieve learning objectives

58 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise -- Reflection on Learning Objectives Assume that you are a debate team Write the single best pro and con arguments for the statement “Using learning objectives improves student leaning.”

59 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Classroom Assessment

60 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Feedback on Learning Objectives ??

61 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Agenda Background for classroom assessment Specifications for assessment processes/tools Team exercise: develop a set of specifications for assessment processes/tools Workshop exercise: improve sets of specifications for assessment processes/tools Generating alternatives for assessment processes/tools Team exercises: select some learning objectives and generate alternative assessment processes/tools Background on alternatives for assessment processes/tools Selecting assessment processes/tools Individual exercise: select a set of learning objectives and generate alternative assessment processes/tools Individual exercise: select one or more assessment processes/tools that you would use in your course Team exercise: share and review choices of assessment processes/tools Review workshop activities

62 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Classroom Assessment Background for Classroom Assessment Session Objective: At the end of the session, participants will describe themselves as more confident in their ability to hold productive conversations with their colleagues regarding the place and importance of assessment instruments and processes in the teaching-learning process

63 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Pre-Assessment Exercise Classroom Assessment Individual Write a one-sentence answer to the following question: “What can you do in the last few minutes of class to determine how well your class learned what you taught that day?”

64 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Classroom Assessment At end of session, participants will be able to: Define several classroom assessment tools Discuss the importance of using classroom tools in the teaching-learning process Write assessment tools for their classes

65 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Resources on the Web m/guidebk/teachtip/assess-1.htm m/guidebk/teachtip/assess-2.htm

66 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Types Of Assessment Assessment Used in many contexts Used for several different foci Classroom Teacher focus: concerns your performance Student focus: concerns their individual performances Program Alumni (as a group) focus: ABET Criterion 2 Graduates (as a group) focus: ABET Criterion 3 Individual graduate focus: Comprehensive exams

67 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Types Of Assessment Classroom, teacher focus Question -- How effective was a lecture, assignment, lab? Tools -- One-minute paper, student survey Classroom, student focus Question – Did a specific student achieve the learning objectives? Tools -- Exams, reports, presentation Program, graduates (as a group) focus Question -- How well did a group of students achieve a set of objectives (outcomes) in a program or course? Tools -- Standardized tests, design project report analysis

68 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Two fundamental questions: How well are learners learning? How effectively are teachers teaching? Deals with better learning and more effective teaching Provides feedback about effectiveness as teachers How students learn How they respond to particular teaching approaches. Gives students a measure of their progress as learners Classroom Assessment Modified from

69 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Incorrect Assumptions About Teaching & Learning Instructors assume students learn what they teach Tests, concept inventories, and term papers provide disappointing evidence to the contrary Students have not learned as much or as well as expected Gaps between what was taught and what was learned Sometimes considerable gaps Instructors notice gaps too late to remedy the problems Classroom assessment can uncover gaps earlier

70 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Classroom Assessment Getting Started Planning Select one, and only one, of your classes Choose a simple and quick technique Implementing Make sure the students understand the procedure Analyze student’s responses as soon as possible Responding -- “Close the feedback loop” Tell students what you learned and what you will do about it - motivates students to become actively involved

71 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Don’t use any technique that does not appeal to you Don't make it into a self-inflicted chore or burden. Try it yourself before you use it with students Allow more time than you think you will need To carry out the assessment To respond to it Make sure to "close the loop" Let students know What you learned from their feedback How you and they can use that information to improve learning Classroom Assessment – Five Suggestions committees/FacDevCom/guid ebk/teachtip/assess-1.htm

72 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Process Brain storm individually -- 1 minutes Establish consensus as a team -- 4 minutes Task: Write 2 to 4 guidelines for good classroom assessment tools (Think of guidelines as specifications) What are the common features? What should they look like? Do all guidelines have to apply to all tools?

73 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Tool should be efficient use of time for instructor and class Easy for students to use Can be used across different classes and situations Built into teaching methodology and doesn’t look like an add-on Tied to learning objectives or limited number of topics Flexible, not dependent on particular technology Done repeatedly throughout semester Easy for faculty to administer and evaluate Self-documenting, easy to show results to others Clear and precise instructions

74 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Process Brain storm individually -- 2 minutes Generate different ideas as a team -- 3 minutes Task: Generate as many possible good classroom assessment tools that might satisfy your team guidelines Generate different ideas

75 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Muddiest point Quizzes on specific objectives What is clear?/What is unclear? What did you learn today? Strengths/Improvements/Insights (SII) Self or peer assessment Who did you share X with today? Weekly feedback on homework instruction – what was clear? What was unclear? Continuous Improvement – What was beneficial? What would you change?

76 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Process Brain storm individually -- 2 minutes Generate different ideas as a team -- 3 minutes Task: Generate as many possible good classroom assessment tools where the student work product is primarily graphical and the tools might satisfy your team guidelines Generate different ideas

77 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise List topics with smiling/frowning faces Flow charts/flow sheets Process flow of your problem solving process Road map to get from A to B Bar graph of confidence related to specific objective Free body diagram Web of association to a particular term Story board of set up or explanation of particular concept Fit concept into network or tree of concepts Draw a concept

78 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Examples of Assessment Techniques Background Knowledge Probe Students respond to short-answers or multiple- choice questions General information on their level of preparation Minute Paper (most widely used) Students write brief response to "What was the most important thing you learned during this class?" “What important question remains unanswered?“

79 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Examples of Assessment Techniques (Cont.) Muddiest Point (simplest technique, remarkably efficient) Students jot down a quick response to one "What was the muddiest point in ?” A lecture, a discussion, a homework assignment One-Sentence Summary Students answer the questions "Who does what to whom, when, where, how, and why?" (WDWWWWHW) Synthesize answers into a simple, informative, grammatical sentence.

80 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Examples of Assessment Techniques (Cont.) What's the Principle? Students state the principle that best applies to a few problems

81 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Concept Map A concept map is a set of nodes that represent concepts connected by a labeled links that describe a link between concepts. Concept A Concept B Describe how concept A and concept B are related?

82 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Start with a subset of the concepts on the following page and construct a concept map that shows the concepts you have selected and how they are related. Exchange concept maps and share insights Team Exercise Building a Concept Map

83 Feedback Derivative Finite Element Analysis Integral Linear Momentum Angular Momentum Energy Interest Mass Ideal Gas Law Fick’s First Law Fick’s Second Law Vectors: Dot Product Vectors: Cross Product Ordinary Differential Equations Kirchoff’s Voltage Law Second Law of Thermodynamics Kirchoff’s Current Law Modeling Problem-Solving Force Ohm’s Law Resistance Complex Numbers Logarithmic Function Electric Flux Decision Theory Divergence Indirect Cost Capacitance Bending Moment Feedback First Law of Thermodynamics Entropy Heat Electric Field Magnetic Field Partial Differential Equations Determinants Return on Investment Phasors Brainstorming Exponential Function Conductivity Chemical Kinetics Specific Heat Elasticity Malleability Plasticity Resiliency Permittivity Current Electric Potential Curl Presentation Skills Democracy Profit Density Molecule Phase Shear Rheology Frequency Response Eigenvalue, Eigenvector Sinusoidal Functions Work Displacement Velocity Acceleration Resistivity Leadership Hess’ Law Zeroth Law of Thermodynamics Electric Potential Magnetic Flux Design Maxwell’s Equations Power Ductility Spring Constant Stress Strain Partial Derivative Permeability Charge Magnetic Potential Gradient Paragraph Rate of Return Frequency Atom Root Locus Torque Inductance Torsion Polymer Kinetic Theory of Gases

84 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Cowan’s Teaching Examples Bridge design Design and build two different bridges and grade on the lower performance design Problem-solving script Illustrate script for one type of problem, ask students to develop a script for another type of problem Cowan, J. (1998) On Becoming an Innovative University Teacher: Reflection in Action. Buckingham: SRHE and Open University Press.

85 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Some Personal Observations

86 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Relating Student Performance On Exams To Objectives Write exam using objectives Select objectives for exam questions from list Many objectives -- test questions represent a sample Exam question may involve more than one objective Use some “hard” and some “easy” questions Identify questions (& objectives) a high percentage missed Review idea in class -- give additional work Modify lecture, reading, or homework for future Change the objective

87 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Using Cooperative Learning In-class Exercises Examine students work during the in-class exercise If all have a good approach -- may be wasting time If all are lost -- may need more explanation If one-half to two-thirds have a good approach -- level and pace are right Collect and show a few solutions to in-class exercises If all have correct approach -- may be wasting time If all are wrong -- may need more explanation If one or two are correct -- level and pace are right

88 Western Michigan University, 18 September 2003, Kalamazoo, Michigan “One-Minute” Papers Common questions What one thing should be changed about ____? What one thing should not be changed about ____? What do you think about ____? What is the “muddiest” point about ____ ? Ask about Course or lecture Text or chapter Assignment or test Teaching style or class activity

89 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Some Colleagues’ Observations

90 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Informal Assessment Techniques General Guidelines Keep them anonymous Use then frequently – better feedback Close the loop Let students know results of the process David Cordes

91 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Informal Assessment Techniques - Daily Activities One-minute paper At the end of the lecture, ask students for: The most important topic that we covered today The one topic you are still confused about Single sheet of paper, no names Can read on the way back to the office Look for “common problems” Look for “did they understand my focus?” David Cordes

92 Western Michigan University, 18 September 2003, Kalamazoo, Michigan ME Plus / Delta Assessment #1 On one side of “sticky” pad Put a “+” in upper left hand corner What is something that worked well or made more sense in lab this week? On other side of “sticky” pad Put a “  ” in upper left hand corner What is something that could have been done better in lab this week? “Stick” on the door on your way out Joey Parker

93 Western Michigan University, 18 September 2003, Kalamazoo, Michigan ME st Day of Class What are a valid set of units for a mass moment of inertia? (Dynamics concept) What is the difference between a capacitor and a resistor? (Circuits concept) What is the equation of the straight line that passes through the points X=2, Y=7 and X=7, Y=2? (Math concept) Joey Parker

94 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Process Brain storm individually -- 2 minutes Establish consensus as a team -- 5 minutes Report team results -- 3 minutes Revise products as a team -- 2 minutes Task: Write 3 to 5 guidelines for good classroom assessment tools (Think of guidelines as specifications) What are the common features? What should they look like? Do all guidelines have to apply to all tools?

95 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Simple, quick, easy to administer and evaluate Provide immediate feedback to students – they need to know you’re using the results Tied to a specific topic or objective Gives usable, valuable information Identification of strengths and areas of improvement Give clear and precise instruction to the student Using graphical and textual feedback Result in action

96 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Process Brain storm individually -- 2 minutes Establish consensus as a team -- 5 minutes Report team results -- 3 minutes Revise products as a team -- 2 minutes Task: Generate as many possible good classroom assessment tools that might satisfy your team guidelines Generate different ideas that you have generated or have seen

97 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Ask students to identify an application based on concept presented in class Person A teaches Person B, and then they write what was hard to convey and what was hard to learn Identify an objective for class and rate their comfort level with number or bar graph Ask student to give example to transfer example in class to another context Students examine 2-3 methods/opinions/processes for similar purposes, ask groups to analyze similarities and differences, solicit results, and then invent their own that combines methods/opinions/processes Ask students to identify concept based on a diagram. Ask students to write problem that could be solved by concept presented in class

98 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Individual Exercise Individually write a set of assessment tools for a class or a topic representing a few classes Follow your guidelines Consider the following questions about your tool Can your students understand the task? Can your students do the task quickly? Can you analyze the results quickly? Can you summarize and report the results easily? Does it assess student learning? 5 minutes

99 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Review each other’s objectives and assessment tools Consider the following questions Does the tool follow your guidelines? Can your students understand the task? Can your students do the task quickly? Can you analyze the results quickly? Can you summarize and report the results easily? Does it assess student learning? 15 minutes

100 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Individual Exercise Rewrite your assessment tools based on your team’s review Identify the major improvement 5 minutes

101 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Minute Paper Write a one-sentence answer to the following question: What is the muddiest point about classroom assessment?

102 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Individual Exercise How can students be more involved in assessing each other, as well as the self-assessment we have largely concentrated on, and do this in reasonable time. How often is a reasonable amount of assessment? How much is too much? Going from creating an assessment tool to actually implementing the assessment. Incorporation of feedback will not be every time easy and straight forward Will students see the value of classroom assessment? Yes, with good approaches. Yes, with good feedback. Being able to design a clear, concise assessment tool that is both beneficial to students and professor

103 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Team Exercise Assume that you are a debate team Write the single best pro and con arguments for the statement “Using classroom assessment tools improves student learning.”

104 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Workshop Objectives & Action Items Recall objectives At end of session, participants will be able to define, discuss, & write Learning objectives for their courses Assessment tools for their classes Workshop provided a structure for & experience in writing Learning objectives Assessment tools Your charge – In one of your courses next semester use Learning objectives Classroom assessment tools

105 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Questions?

106 Western Michigan University, 18 September 2003, Kalamazoo, Michigan Some Personal Observations


Download ppt "Western Michigan University, 18 September 2003, Kalamazoo, Michigan Learning Objectives and Classroom Assessment Jeff Froyd, Texas A&M University."

Similar presentations


Ads by Google