Download presentation

Presentation is loading. Please wait.

Published byAbigail Carpenter Modified over 2 years ago

1
4/25/2014 mowais@kfupm.edu.sa Type-2 Fuzzy Logic Advisor for Evaluating Students Cooperative Training Owais Ahmed Malik King Fahd University of Petroleum & Minerals (KFUPM/HBCC) Saudi Arabia 3rd UK Workshop on AI in Education

2
mowais@kfupm.edu.sa4/25/2014 2 Overview Introduction Introduction Cooperative Training Assessment Cooperative Training Assessment Motivation for the Perception-based Assessment Motivation for the Perception-based Assessment Fuzzy Logic and Fuzzy Logic System Fuzzy Logic and Fuzzy Logic System Proposed Model for Cooperative Training Assessment Proposed Model for Cooperative Training Assessment Experiments and Discussion Experiments and Discussion Conclusions and Future Directions Conclusions and Future Directions

3
mowais@kfupm.edu.sa4/25/2014 3 Introduction Students learning performance is measured by some evaluation means. Students learning performance is measured by some evaluation means. Students Evaluation Students Evaluation Process of collecting students work Process of collecting students work Making decision based on collected information Making decision based on collected information Methods of Evaluation Methods of Evaluation Objective Objective Subjective Subjective

4
mowais@kfupm.edu.sa4/25/2014 4 Introduction Formal: Formal: Homework Homework Quiz Quiz Written Exam (Majors) Written Exam (Majors) Lab Exam Lab Exam Informal: Informal: Interview Interview Class Participation Class Participation Team Work Team Work Individual Projects Individual Projects How to evaluate a student?

5
mowais@kfupm.edu.sa4/25/2014 5 Cooperative Training Assessment Cooperative Training/Internship Cooperative Training/Internship An important tool to develop student skills An important tool to develop student skills Some real work experience in industry Some real work experience in industry A typical assessment for Coop training: A typical assessment for Coop training: Progress reports Progress reports Final report Final report Presenting the work Presenting the work External supervisor remarks External supervisor remarks Onsite visit by the internal supervisor Onsite visit by the internal supervisor

6
mowais@kfupm.edu.sa4/25/2014 6 Cooperative Training Assessment Assessment ComponentCriteria for Assessment Final Report (FR) Format and Structure Literary Quality Quality of Subject Matter Progress Report (PR) Task Description Format and Submission Final Presentation (FP) Content and Organization Speaking (Presentation) Skills Response to Questions External Evaluation (EE) Enthusiasm and Interest in Work Ability to Learn and Search for Information Relations with Co-Workers Punctuality and Delivering Work on Time

7
mowais@kfupm.edu.sa4/25/2014 7 Coop Training Assessment ExcellentGoodFairUnsatisfactory Content All information related to the coop training e.g. work place, time, location, learning etc; points are clearly presented with all necessary description of work done during training Period Sufficient information related to coop training; points are clearly presented but description of work is not thorough Incomplete information about coop training; adequate details about tasks completed during training Inadequate information about coop training; incomplete description about tasks completed during training Organization All information presented in a logical & interesting sequence; gives audience very clear picture of training; good transitions; succinct & clear Most of the information presented in logical sequence; gives audience an adequate picture of training; generally well organized; good transitions Lacks some sequence of information; difficulty in following for audience; loosely organized No sequence of information; no understanding for audience; presentation is disjointed Material (Figures/Visual Aids, Spelling / Grammar) Very effective use of visual aids; clear figures and charts; no spelling or grammatical mistakes Good use of visual aids; graphics relate to text presented; 1 or 2 spelling /grammar mistakes Occasional use of visual aids; not much related to text; few spelling/ grammar mistakes Little/no or ineffective use of visual aids; many spelling/grammar mistakes Speaking Skills Clear articulation; excellent delivery with proper volume, steady pace, good posture and eye contact; confidence Clear articulation; good delivery with good pace, usually projects voice and good eye contact Some mumbling low voice and uneven pace; little eye contact Inaudible or too loud; pace too slow or too fast; no eye contact; seems uninterested Questions/Answers Answers questions effectively and smoothly with full description; satisfy audience Answers most of the questions with little elaboration Answers only rudimentary questions; very little elaboration Can not answer most of the questions; no grasp of subject Example Rubric for Presentation Assessment: Example Rubric for Presentation Assessment:

8
mowais@kfupm.edu.sa4/25/2014 8 Motivation for Perception-based Assessment Assessment of different components of Coop training is subjective. Assessment of different components of Coop training is subjective. Communication skills during presentation Communication skills during presentation Organization of presentation/report Organization of presentation/report Literary quality of report Literary quality of report Quality of subject matter Quality of subject matter Students attitude towards work Students attitude towards work Enthusiasm and interest in work Enthusiasm and interest in work Difficult to apply the objective methods to evaluate these student activities Difficult to apply the objective methods to evaluate these student activities

9
mowais@kfupm.edu.sa4/25/2014 9 Motivation for Perception-based Assessment Assessment mostly based on perception of an evaluator Assessment mostly based on perception of an evaluator Judgment in terms of words (Excellent, Very Good, and Good etc.) Judgment in terms of words (Excellent, Very Good, and Good etc.) Conventional assessment methods usually do not consider the uncertainties in usage of words Conventional assessment methods usually do not consider the uncertainties in usage of words Motivation for type-2 fuzzy set be used to model a word Motivation for type-2 fuzzy set be used to model a word

10
mowais@kfupm.edu.sa4/25/2014 10 Fuzzy Logic (FL) Mathematical and Statistical techniques are often unsatisfactory in decision making. Mathematical and Statistical techniques are often unsatisfactory in decision making. Experts make decisions with imprecise data in an uncertain world. Experts make decisions with imprecise data in an uncertain world. They work with knowledge that is rarely defined mathematically or algorithmically but uses vague terminology with words. They work with knowledge that is rarely defined mathematically or algorithmically but uses vague terminology with words. FL designed to handle imprecision and uncertainty in the measurement process FL designed to handle imprecision and uncertainty in the measurement process Methodology of computing with words (CW) Methodology of computing with words (CW) Mimics the perception-based decision making done by humans Mimics the perception-based decision making done by humans

11
mowais@kfupm.edu.sa4/25/2014 11 Fuzzy Logic Linguistic Variable Linguistic Variable Example : Age of a person Example : Age of a person Term Set: Young, Middle-aged, Old etc. Term Set: Young, Middle-aged, Old etc. Each linguistic term is associated with a fuzzy set Each linguistic term is associated with a fuzzy set Each term has a defined membership function (MF): Each term has a defined membership function (MF): A fuzzy set A in X can be expressed as: A fuzzy set A in X can be expressed as:or

12
mowais@kfupm.edu.sa4/25/2014 12 Fuzzy Logic Example Fuzzy Set for Age:

13
mowais@kfupm.edu.sa4/25/2014 13 Fuzzy Logic Example Fuzzy Set for Literary Quality of a Report:

14
mowais@kfupm.edu.sa4/25/2014 14 Type-2 Fuzzy Set Imprecise perception-based data can be modelled by using type-2 fuzzy logic Imprecise perception-based data can be modelled by using type-2 fuzzy logic Type-2 fuzzy set is 3-dimensional representation Type-2 fuzzy set is 3-dimensional representation Type-2 fuzzy sets help us to deal with the uncertainty Type-2 fuzzy sets help us to deal with the uncertainty Footprint of Uncertainty (FOU): Footprint of Uncertainty (FOU): Bounded region in the primary membership function of a type-2 fuzzy set Bounded region in the primary membership function of a type-2 fuzzy set 2-Dimensional depiction of type-2 fuzzy sets 2-Dimensional depiction of type-2 fuzzy sets Upper and Lower Membership Functions Upper and Lower Membership Functions For more details: Mendel J. M., Uncertain Rule-Based Fuzzy Logic Systems, Prentice-Hall, Upper Saddle River, NJ 07458, (2001)

15
mowais@kfupm.edu.sa4/25/2014 15 FOUs, Upper and Lower MFs

16
mowais@kfupm.edu.sa4/25/2014 16 Type-2 Fuzzy Logic System

17
mowais@kfupm.edu.sa4/25/2014 17 Proposed Model for Cooperative Training Assessment Based on knowledge mining (knowledge engineering) methodology Based on knowledge mining (knowledge engineering) methodology Information extracted in the form of IF-THEN rules from evaluators (experts) Information extracted in the form of IF-THEN rules from evaluators (experts) Rules are modelled using fuzzy logic system Rules are modelled using fuzzy logic system Used as Fuzzy Logic Advisor (FLA) Used as Fuzzy Logic Advisor (FLA) Two-stage FLA based on interval type-2 fuzzy logic Two-stage FLA based on interval type-2 fuzzy logic Each assessment component is evaluated using an independent FLA Each assessment component is evaluated using an independent FLA Results of these FLAs are combined to calculate the final grade Results of these FLAs are combined to calculate the final grade

18
mowais@kfupm.edu.sa4/25/2014 18 Structure of Proposed Model

19
mowais@kfupm.edu.sa4/25/2014 19 Input/Output Fuzzy Sets for Proposed Model Input (criteria of assessment) and output (evaluation) attributes divided into four fuzzy sets Input (criteria of assessment) and output (evaluation) attributes divided into four fuzzy sets Type-2 fuzzy sets: Excellent, Good, Fair and Poor Type-2 fuzzy sets: Excellent, Good, Fair and Poor Survey results for labels of fuzzy sets Survey results for labels of fuzzy sets LabelMeanStd. Deviation StartEndStartEnd abσaσa σbσb Poor 04.738900.4898 Fair 4.70566.87780.49780.4295 Good 6.65568.72220.44190.3153 Excellent 8.488910.00000.32960.0000

20
mowais@kfupm.edu.sa4/25/2014 20 Membership Functions for Proposed Model FOUs for Literary Quality of a Report:

21
mowais@kfupm.edu.sa4/25/2014 21 Rules Formulation All possible combinations of antecedent fuzzy sets are employed All possible combinations of antecedent fuzzy sets are employed Consequents of rules are provided by the evaluators (experts) Consequents of rules are provided by the evaluators (experts) Each rule has a histogram of responses Each rule has a histogram of responses Number of rules depends on the number of inputs and fuzzy sets associated with them Number of rules depends on the number of inputs and fuzzy sets associated with them Example rule for Coop Evaluation FLA Example rule for Coop Evaluation FLA Rl: IF Final Report is Excellent AND Progress Report is Good AND Final Presentation is Fair AND External Evaluation is Excellent THEN GRADE is (VERY GOOD)

22
mowais@kfupm.edu.sa4/25/2014 22 Type-1 FLA (Individual FLA)

23
mowais@kfupm.edu.sa4/25/2014 23 Partial Histogram of Survey Responses for Final Report Evaluation Rule No.Antecedent 1Antecedent 2Antecedent 3 ConsequentType-1Type-2 ExcellentGoodFairPoor C avg C l avg C r avg 1Excellent 80009.1629.0779.242 2Excellent Good62008.7838.6888.874 3Excellent Fair43108.178.0618.276 4Excellent Poor05216.5336.46.666 5ExcellentGoodExcellent62008.7838.6888.874 6ExcellentGood 34107.987.8668.093 7ExcellentGoodFair05306.9436.8067.079 8ExcellentGoodPoor04316.2986.1626.435 9ExcellentFairExcellent25107.7917.6717.909 10ExcellentFairGood06207.1787.0447.311 11ExcellentFair 05306.9436.8067.079 12ExcellentFairPoor02515.8295.6865.972 13ExcellentPoorExcellent03416.0645.9246.204 14ExcellentPoorGood03416.0645.9246.204 15ExcellentPoorFair00624.954.8045.097

24
mowais@kfupm.edu.sa4/25/2014 24 Comparison for Individual and Type-1 Consensus FLAs Experiments and Discussion

25
mowais@kfupm.edu.sa4/25/2014 25 Experiments and Discussion Comparison for Individual and Type-2 Consensus FLAs (50% uncertainty)

26
mowais@kfupm.edu.sa4/25/2014 26 Experiments and Discussion Comparison for Individual and Type-2 Consensus FLAs (100% uncertainty)

27
mowais@kfupm.edu.sa4/25/2014 27 Conclusions Type-2 fuzzy sets model the perception-based evaluation Type-2 fuzzy sets model the perception-based evaluation Proposed model has the potential to capture the uncertainties in subjective evaluation Proposed model has the potential to capture the uncertainties in subjective evaluation Successful testing for small group of students Successful testing for small group of students Provides more accurate evaluation of a student as compared to existing method Provides more accurate evaluation of a student as compared to existing method

28
mowais@kfupm.edu.sa4/25/2014 28 Future Directions Testing of the system for large number of students Testing of the system for large number of students Investigating the use of the system for other courses/situations e.g. assessing group projects etc. Investigating the use of the system for other courses/situations e.g. assessing group projects etc. Type-2 fuzzy sets to be tested for representing final grades Type-2 fuzzy sets to be tested for representing final grades Deciding the optimal number of linguistic input/output variables for assessment components Deciding the optimal number of linguistic input/output variables for assessment components Working with non-singleton input from evaluators Working with non-singleton input from evaluators

29
4/25/2014 mowais@kfupm.edu.sa Thank You

30
mowais@kfupm.edu.sa4/25/2014 30 Question/Answers

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google