We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byJair Gillet
Modified over 2 years ago
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, 2011 Experiments on Education as a Service Hyoyoung Kim James R. Morrison Department of Industrial and Systems Engineering KAIST
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Contents Motivation Experiments Statistical Analysis Concluding Remarks
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Literature Review Past ResearchMy Research University Experience O [1,2,4] X Class experienceXO Kano Classification Basic Need O [1,4] X Performance Need O [1,2,4] X Excitement NeedXO QFD Approach O [1,3] X SERVQUAL Approach O [3,4] O Statistical Analysis Wilcoxon signed-rank testXO Kruskal-Wallis testXO Experiment on single courseXO  Bagchi, U.;, "Delivering student satisfaction in higher education: A QFD approach," Service Systems and Service Management (ICSSSM), th International Conference on, vol., no., pp.1-4, June 2010, doi: /ICSSSM  Mathew Joseph, Beatriz Joseph, (1997) "Service quality in education: a student perspective", Quality Assurance in Education, Vol. 5 Iss: 1, pp.15 – 21  Sangeeta Sahney, D.K. Banwet, S. Karunes, (2004) "A SERVQUAL and QFD approach to total quality education: A student perspective", International Journal of Productivity and Performance Management, Vol. 53 Iss: 2, pp.143 – 166  Kay C. Tan and Sei W. Kek, (2004) Service Quality in Higher Education Using an enhanced SERVQUAL approach, Quality in Higher Education, Vol. 10, No. 1, pp
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Motivation Course Experience Satisfaction Evaluation Teaching Better Take Home Point: Good service improves the student experience and increases satisfaction But what is good service? Service
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Motivation Identify how to increase customer satisfaction in Education Experiment 1 – GOAL: Determine Basic, Performance, and Excitement needs – Kanos model survey – Result: Identified 4 BN, 11 PN, 20 EN Experiment 2 – GOAL: Determine which services (among excitement needs) make statistically significant difference in student satisfaction – SERVQUAL survey – Result: Observed drift of service and service memory effect Numerous services improve/influence student perceptions
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Experiment 1 - Spring, 2010 GOAL: Determine Basic, Performance, and Excitement needs KAIST IE 200 Introduction to Operations Research class Class size: 186 students (from all science and engineering majors) Conducted Kanos model survey – 76 Functional and Dysfunctional questions – 18 Solution questions – 15 Overall satisfaction questions Functional Dysfunctional
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Experiment 1 - Spring, 2010 RESULTS: Basic Need (Must-be): – Lecture room density is NOT high (large room size with small number of students) – Lecture notes are uploaded at least one day before class – Lecture room is NOT disturbed by outside noise – Class time is well kept Must-be Dissatisfaction Done very wellDone very poorly Satisfaction
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Experiment 1 - Spring, 2010 RESULTS: Performance Need (One-dimensional): – Exams and/or homework are directly related to lectures – You feel you learned a lot at the end of the semester – Lecture room is close to dormitory – Professor is confident about lecture contents – Professor enjoys teaching – Professor prepares the lecture well – Professor delivers the lecture clearly – Lecture is easy to understand – Professor respects you – TAs are kind – Lecture notes are easily understandable – Lecture room is clean Satisfaction Dissatisfaction Done very wellDone very poorly One-dimensional
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Experiment 1 - Spring, 2010 RESULTS: Excitement Need: – Furniture and classroom equipment are up-to-date – Professor gives candies when answering questions – Professor and TAs provide sufficient and convenient office hours – Professor is humorous – Professor smiles while giving lectures – Professor cares about students – Professors appearance is neat – Professor calls students by name – Professor is kind – In-class activities are fun – Course material can be applied in the real world – Lectures make you think Satisfaction Dissatisfaction Done very wellDone very poorly Attractive
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Experiment 2 - Fall, 2010 KAIST IE 332 Operations Research II class, 60 students (IE majors) SERVQUAL survey to measure expectation, performance, and importance GOAL: Determine which excitement needs measurably improve student satisfaction TopicServiceDuration Chap. 2Professor is humorous2 weeks Chap. 3Candies or rewards are given when asking or answering questions2 weeks Chap. 4Fun in-class activities2 weeks Chap. 5Nominal service2 weeks Chap. 6Full service3 weeks Chap. 7Real world application + full service3 weeks
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Experiment 2 - Fall, 2010 Handling missing data: Regression Data reliability: Cronbachs alpha (α) – 0 α 1, α, reliable – Recommended to have a reliability of 0.70 or higher to be used as data – Data of six surveys: 0.89 α 0.94 can trust data Correlation – Measure of how much each pair of scores relates Statistical testing – SAS 9.1 – Wilcoxon signed-rank test (paired sample test) – Kruskal-Wallis test : little if any : low : moderate : high : very high non-parametric statistical hypothesis test for the case of two related samples a non-parametric method for testing equality of population medians among groups
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey Humor Candy In-class Application Perception 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey Humor Accept (p-value: ) Reject (p-value: ) Reject (p-value: ) Accept (p-value: ) Reject (p-value: ) Accept (p-value: ) Input – Jokes and funny/silly examples – Short funny music Output – Spillover effect – effect of humor lasted longer than other services. – H0: there is no difference or relationship between the variables of interest – If p-value 0.05, H0 is accepted no difference in measurements – Students still remembered humor and rated humor higher on the next survey 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey Well prepared Overall Service Enjoyment Kindness Outcome Correlation of : Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Input – Give candy to students when they ask or answer questions Output – Perceived candy – Drift of needs (Excitement need to basic need): 2.26/5 3.11/5 for the importance of giving candies Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey Candy : Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Input – Skits – Games – Team activity – Active learning activities Output – Correlation of between in-class activities and confidence Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey In-class Time, effort 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey Enjoyment Kindness Outcome Significant change Insignificant change 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Service HumorCandyIn-classNominalFullApplication Survey Humor Candy In-class Input – Candy – Humor – In-class activity – Enthusiasm – Neat appearance Output Statistical Analysis 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Statistical Analysis Service HumorCandyIn-classNominalFullApplication Survey Well prepared Overall Service Outcome – Positive – Negative Service HumorCandyIn-classNominalFullApplication Survey Lecture Lecture note Concentrate Significant change Insignificant change
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, Concluding Remarks Students perceived the offering of such services We observed needs drift of such services We also observed the service memory effect Needs that require further improvement: Lecture notes are easy to understand Sufficient and convenient office hours Feel you learned a lot Exams and/or homework are directly related to lectures Future Research: Service increases student learning outcome
©2011 – H. Kim, J. Morrison – ICSSSM11 – June 26, 2011 T hank You. Any Questions ?
Survey design and implementation Tampere Design, Implementation & Analysis of Innovation Surveys With a note on presenting results Anthony Arundel.
Gaining A Geological Perspective Through Active Learning in the Large Lecture Classroom Jessica Kapp, Randy Richardson, Allister Rees, Jon Patchett, Ross.
TEAM-BASED LEARNING: What Makes It: DISTINCTIVE? SO IMPORTANT? Presentation by: L. Dee Fink, Ph.D. National Consultant in Higher Education Author: Creating.
1 Lecture 7 POWER IN STATISTICAL TESTING. 2 The caffeine experiment In the Caffeine experiment, there were two groups: 1. the CAFFEINE group; 2. the PLACEBO.
Customer Assessment Office of Quality Management Office of Research Services National Institutes of Health October 2005.
Maintaining curiosity Factors that promote achievement in science June 2014.
Overview: The purpose of our study was to observe the effects of journal assignments (given for class preparation) upon student performance as well as.
A.MANN ADAPTED FROM HAMILTON AP STATS Chapter 5 Producing Data.
DSS Decision Support System Tutorial: An Instructional Tool for Using the DSS.
1 Elements of Compelling Evaluation Reporting Jon K. Price K-12 Evaluation Research Manager.
Psychology Practical (Year 2) PS2001 Correlation and other topics.
2010 Department of Automatic Control and Systems Engineering Student engagement with MATLAB and supporting independent learning J A Rossiter University.
Research Process, Research Design and Questionnaires.
Coaching: Tapping Into Your Employees Potential. 2 Objectives After this workshop you will be able to: Set the groundwork for productive coaching sessions.
First Steps in Designing Quantitative Research: Asking the Right Questions Susan R. Hutchinson, Ph.D. Department of Applied Statistics & Research Methods.
1 Measuring and Paying For Performance l Increasing organizational productivity is one of the hottest current topics in executive suites. l Managers discovered.
Hypothesis Testing Testing Statistical Significance.
Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Nonparametric Methods: Analysis of Ranked Data Chapter 18.
15.835: Entrepreneurial Marketing Session 4: Marketing Research.
1 From Research to Practice Examining Gender Differences to Increase Success for All Presentation to the American Supervision and Curriculum Development.
Business School Introduction to your Course Wednesday 27 th January, 2010 Belinda Platt.
Learner-Centered Teaching Elizabeth Normandy Teaching and Learning Center.
Noel Entwistle, Jennifer Nisbet, and Adrian Bromage Edinburgh and Coventry Workshop jointly with the Engineering Subject Centre Improving the student experience.
Copyright © 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. Lecture Slides Elementary Statistics Eleventh Edition and the Triola.
1 Psychological Practical (Year 2) PS2001 Introduction Dr. John Beech.
Business School Preparation for your Degree Wednesday 22 nd September, 2010 Lindsay Williams (Senior Lecturer) and Belinda Platt (Undergraduate Course.
1 FINANCIAL SYSTEMS. 2 Financial Systems As educators, CE practitioners are often poorly prepared to manage an operation that has many characteristics.
Teacher Evaluation in Newark: Evaluator Training August 21 or 22,
ANSF Faculty Development Course: Medical Education 1 Providing Feedback and Evaluating Students Instructor Development Course : Session # 4.
Quantitative methods Questionnaire Design. 2 Stage 1: Research aims Stage 2: Literature Stage 3: Research design Stage 4: Instrumentation Stage 5: Piloting.
© 2016 SlidePlayer.com Inc. All rights reserved.