1. NAUEE VIII Kingston OntarioSlide 1 Assessing Quality In Student Work Products: The Role of Customers and Expectations by Barry McNeill & Lynn Bellamy Arizona State University

2. NAUEE VIII Kingston OntarioSlide 2 Today’s Goals u Have audience achieve Knowledge Level of Learning for: –ASU Freshperson course structure –quality in a student work product –the work product assessment process –the role of the customer in assessment u Have audience achieve Knowledge and Comprehension Level of Learning for: –assessment checklists

3. NAUEE VIII Kingston OntarioSlide 3 Today’s Agenda u Introduction to ECE 100 5 min u Quality In Student Work 20 min –characteristics –customer needs –assessment process u Checklists 20 min –creation –use

4. NAUEE VIII Kingston OntarioSlide 4 Learning Objectives Tree

5. NAUEE VIII Kingston OntarioSlide 5 ASU Course Structure ECE 100 Concepts Session Laboratory Session Modeling Session 1. 120 students 2. introduction to problem solving 3. quizzes & an exam 4. lecture / discussion / work 5. work in ad hoc groups 1. 80 students 2. practice problem solving 3. work on a) artifact dissection b) mousetrap powered device 4. do assessment 5. create design notebook 6. work in semester team 7. faculty guidance 8. hands on work 1. 40 students 2. work on chapters from How to Model It 3. create models 4. start work in class & finish outside of class 5. give oral presentations of in class work 6. work at computer - two students per computer 7. faculty guidance

6. NAUEE VIII Kingston OntarioSlide 6 Quality - What Is It? u Pick a workproduct that a student submits in one of your classes u Is the quality of the workproduct something explicitly addressed in the assignment? u Write down your definition of what would constitute a quality submission (4 minutes) u Share your definition with a neighbor (5 minutes)

7. NAUEE VIII Kingston OntarioSlide 7 Quality - Some Definitions u Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy implied or stated needs (ANSI/ASQC Standard A3 - 1987) u The least expensive method of achieving quality is to deliver a design that cannot fail to satisfy the customer (Genichi Taguchi) u Quality is a characteristic of thought and statement that is recognized by a nonthinkng process. Because definitions are a product of rigid, formal thinking, quality cannot be defined. But even though Quality cannot be defined, you know, what Quality is! (Robert Pirsig)

8. NAUEE VIII Kingston OntarioSlide 8 Kano’s Customer Needs u Expected Needs (Requirements) Expected requirements are those basic attributes that the customer assumes are present in generically similar products or services u Revealed Needs (Requirements) Revealed requirements are those attributes that customers talk about when describing what would make a product or service better u Exciting Needs (Requirements) Exciting requirements are those attributes whose presence makes the customer say WoW!

9. NAUEE VIII Kingston OntarioSlide 9 Kano’s Needs vs Satisfaction

10. NAUEE VIII Kingston OntarioSlide 10 Student Grader/Faculty Expectations Do Assignment (assess work) Assess Work Meet Expect a-tions Bring Work Up to Expectations Assess Improvements YES NO One Week Needs Improvement NI Needs Improvement NI Exceeds Expectations E Meets Expectations M Exceed Expect a-tions Meet Expect a-tions NO YES Meets Expectations M Time Faculty

11. NAUEE VIII Kingston OntarioSlide 11 What Are Checklists u A method to reduce the possibility of multiple interpretations possible with written and/or oral instructions u An assessment checklist is a collection of Yes/No questions that enumerates the Expected Requirements

12. NAUEE VIII Kingston OntarioSlide 12 Why Are Checklists Hard To Make? u Since the checklists are given out before the work is started, they must be complete, nothing can be added later u A complete set of Expected requirements means nothing important is left out u A complete set cannot contain items that you might sometimes consider as not expected

13. NAUEE VIII Kingston OntarioSlide 13 How To Make Checklists u List the learning objectives for the assignment u List all the things (attributes) that have to be present in the work product if Terry Sleuth is to demonstrate that the learning objective has been accomplished u Refine the second list until it contains only definitive Yes/No questions u Select the important items from this second list

14. NAUEE VIII Kingston OntarioSlide 14 Creating A Plotting Checklist (pairs/triads - 5 minutes) u Assume the Learning Objective in assigning a plot are to master presentation of data in a chart u Create a list of attributes that would show that Terry Sleuth has mastered the learning objective

15. NAUEE VIII Kingston OntarioSlide 15 Part of Checklist II Chart Requirements (Exceeds or Meets expectations, Needs Improvement) (meeting expectations requires a YES for all of the following items) A For all of the charts in the work product, the following items must all be checked: 1. If you found the chart in the middle of the street, would you understand it? (YES, NO) 2. If the chart were reproduced, would it still be readable? (YES, NO) 3. Do both axes have descriptive titles (N.B. not a single letter) which include units? (YES, NO) 4. Are there labeled divisions (text or numbers) on the axes? (YES, NO) 5. Are the “values” for the axes at the origin of the chart clear? (YES, NO) 6. Is the dependent variable (item measured or predicted) on the vertical axis? (YES, NO) 7. Is there an appropriate, descriptive title? (YES, NO) 8. If the variables are presented in the chart’s title, is the dependent variable mentioned first? (YES, NO) 9. If there is more than one chart line, is there a descriptive legend? (YES, NO) 10. Are the data points shown? (YES, NO) B Comments if Charts Exceed Expectations or Need Improvement

16. NAUEE VIII Kingston OntarioSlide 16 Use Checklist

17. NAUEE VIII Kingston OntarioSlide 17 ECE 100 Home Page u http://www.public.asu.edu/~ece100 –Assignments, Assessments, and First Day Materials –Download Engineering Core Workbook for Active Learning, Assessment & Team Training