King Fahd University of Petroleum & Minerals (KFUPM/HBCC) Type-2 Fuzzy Logic Advisor for Evaluating Students’ Cooperative Training 3rd UK Workshop on AI in Education Owais Ahmed Malik King Fahd University of Petroleum & Minerals (KFUPM/HBCC) Saudi Arabia 3/28/2017 mowais@kfupm.edu.sa

Overview Introduction Cooperative Training Assessment Motivation for the Perception-based Assessment Fuzzy Logic and Fuzzy Logic System Proposed Model for Cooperative Training Assessment Experiments and Discussion Conclusions and Future Directions 3/28/2017 mowais@kfupm.edu.sa

Introduction Students’ learning performance is measured by some evaluation means. Students’ Evaluation Process of collecting students’ work Making decision based on collected information Methods of Evaluation Objective Subjective 3/28/2017 mowais@kfupm.edu.sa

Introduction How to evaluate a student? Formal: Informal: Homework Quiz Written Exam (Majors) Lab Exam Informal: Interview Class Participation Team Work Individual Projects 3/28/2017 mowais@kfupm.edu.sa

Cooperative Training Assessment Cooperative Training/Internship An important tool to develop student skills Some real work experience in industry A typical assessment for Coop training: Progress reports Final report Presenting the work External supervisor remarks Onsite visit by the internal supervisor 3/28/2017 mowais@kfupm.edu.sa

Cooperative Training Assessment Assessment Component Criteria for Assessment Final Report (FR) Format and Structure Literary Quality Quality of Subject Matter Progress Report (PR) Task Description Format and Submission Final Presentation (FP) Content and Organization Speaking (Presentation) Skills Response to Questions External Evaluation (EE) Enthusiasm and Interest in Work Ability to Learn and Search for Information Relations with Co-Workers Punctuality and Delivering Work on Time 3/28/2017 mowais@kfupm.edu.sa

Coop Training Assessment Example Rubric for Presentation Assessment:   Excellent Good Fair Unsatisfactory Content All information related to the coop training e.g. work place, time, location, learning etc; points are clearly presented with all necessary description of work done during training Period Sufficient information related to coop training; but description of work is not thorough Incomplete information about coop training; adequate details about tasks completed during training Inadequate information incomplete description about tasks completed during training Organization All information presented in a logical & interesting sequence; gives audience very clear picture of training; good transitions; succinct & clear Most of the information presented in logical sequence; gives audience an adequate picture of training; generally well organized; good transitions Lacks some sequence of information; difficulty in following for audience; loosely organized No sequence of information; no understanding for audience; presentation is disjointed Material (Figures/Visual Aids, Spelling / Grammar) Very effective use of visual aids; clear figures and charts; no spelling or grammatical mistakes Good use of visual aids; graphics relate to text presented; 1 or 2 spelling /grammar mistakes Occasional use of visual aids; not much related to text; few spelling/ grammar Little/no or ineffective use of visual aids; many spelling/grammar Speaking Skills Clear articulation; excellent delivery with proper volume, steady pace, good posture and eye contact; confidence Clear articulation; good delivery with good pace, usually projects voice and good eye contact Some mumbling low voice and uneven pace; little eye contact Inaudible or too loud; pace too slow or too fast; no eye contact; seems uninterested Questions/Answers Answers questions effectively and smoothly with full description; satisfy audience Answers most of the questions with little elaboration Answers only rudimentary questions; very little Can not answer most of the questions; no grasp of subject 3/28/2017 mowais@kfupm.edu.sa

Motivation for Perception-based Assessment Assessment of different components of Coop training is subjective. Communication skills during presentation Organization of presentation/report Literary quality of report Quality of subject matter Student’s attitude towards work Enthusiasm and interest in work Difficult to apply the objective methods to evaluate these student activities 3/28/2017 mowais@kfupm.edu.sa

Motivation for Perception-based Assessment Assessment mostly based on perception of an evaluator Judgment in terms of words (Excellent, Very Good, and Good etc.) Conventional assessment methods usually do not consider the uncertainties in usage of words Motivation for type-2 fuzzy set be used to model a word 3/28/2017 mowais@kfupm.edu.sa

Fuzzy Logic (FL) Mathematical and Statistical techniques are often unsatisfactory in decision making. Experts make decisions with imprecise data in an uncertain world. They work with knowledge that is rarely defined mathematically or algorithmically but uses vague terminology with words. FL designed to handle imprecision and uncertainty in the measurement process Methodology of computing with words (CW) Mimics the perception-based decision making done by humans 3/28/2017 mowais@kfupm.edu.sa

Fuzzy Logic Linguistic Variable Example : Age of a person Term Set: Young, Middle-aged, Old etc. Each linguistic term is associated with a fuzzy set Each term has a defined membership function (MF): A fuzzy set A in X can be expressed as: or 3/28/2017 mowais@kfupm.edu.sa

Fuzzy Logic Example Fuzzy Set for Age: 3/28/2017 mowais@kfupm.edu.sa

Fuzzy Logic Example Fuzzy Set for Literary Quality of a Report: 3/28/2017 mowais@kfupm.edu.sa

Type-2 Fuzzy Set Imprecise perception-based data can be modelled by using type-2 fuzzy logic Type-2 fuzzy set is 3-dimensional representation Type-2 fuzzy sets help us to deal with the uncertainty Footprint of Uncertainty (FOU): Bounded region in the primary membership function of a type-2 fuzzy set 2-Dimensional depiction of type-2 fuzzy sets Upper and Lower Membership Functions For more details: Mendel J. M., Uncertain Rule-Based Fuzzy Logic Systems, Prentice-Hall, Upper Saddle River, NJ 07458, (2001) 3/28/2017 mowais@kfupm.edu.sa

FOUs, Upper and Lower MFs 3/28/2017 mowais@kfupm.edu.sa

Type-2 Fuzzy Logic System 3/28/2017 mowais@kfupm.edu.sa

Proposed Model for Cooperative Training Assessment Based on knowledge mining (knowledge engineering) methodology Information extracted in the form of IF-THEN rules from evaluators (experts) Rules are modelled using fuzzy logic system Used as Fuzzy Logic Advisor (FLA) Two-stage FLA based on interval type-2 fuzzy logic Each assessment component is evaluated using an independent FLA Results of these FLAs are combined to calculate the final grade 3/28/2017 mowais@kfupm.edu.sa

Structure of Proposed Model 3/28/2017 mowais@kfupm.edu.sa

Input/Output Fuzzy Sets for Proposed Model Input (criteria of assessment) and output (evaluation) attributes divided into four fuzzy sets Type-2 fuzzy sets: Excellent, Good, Fair and Poor Survey results for labels of fuzzy sets Label Mean Std. Deviation   Start End a b σa σb Poor 4.7389 0.4898 Fair 4.7056 6.8778 0.4978 0.4295 Good 6.6556 8.7222 0.4419 0.3153 Excellent 8.4889 10.0000 0.3296 0.0000 3/28/2017 mowais@kfupm.edu.sa

Membership Functions for Proposed Model FOUs for Literary Quality of a Report: 3/28/2017 mowais@kfupm.edu.sa

Rules Formulation All possible combinations of antecedent fuzzy sets are employed Consequents of rules are provided by the evaluators (experts) Each rule has a histogram of responses Number of rules depends on the number of inputs and fuzzy sets associated with them Example rule for Coop Evaluation FLA Rl: IF Final Report is Excellent AND Progress Report is Good AND Final Presentation is Fair AND External Evaluation is Excellent THEN GRADE is (VERY GOOD) 3/28/2017 mowais@kfupm.edu.sa

Type-1 FLA (Individual FLA) 3/28/2017 mowais@kfupm.edu.sa

Partial Histogram of Survey Responses for Final Report Evaluation Rule No. Antecedent 1 Antecedent 2 Antecedent 3 Consequent Type-1 Type-2 Excellent Good Fair Poor Cavg Clavg Cravg 1 8 9.162 9.077 9.242 2 6 8.783 8.688 8.874 3 4 8.17 8.061 8.276 5 6.533 6.4 6.666 7.98 7.866 8.093 7 6.943 6.806 7.079 6.298 6.162 6.435 9 7.791 7.671 7.909 10 7.178 7.044 7.311 11 12 5.829 5.686 5.972 13 6.064 5.924 6.204 14 15 4.95 4.804 5.097 3/28/2017 mowais@kfupm.edu.sa

Experiments and Discussion Comparison for Individual and Type-1 Consensus FLAs 3/28/2017 mowais@kfupm.edu.sa

Experiments and Discussion Comparison for Individual and Type-2 Consensus FLAs (50% uncertainty) 3/28/2017 mowais@kfupm.edu.sa

Experiments and Discussion Comparison for Individual and Type-2 Consensus FLAs (100% uncertainty) 3/28/2017 mowais@kfupm.edu.sa

Conclusions Type-2 fuzzy sets model the perception-based evaluation Proposed model has the potential to capture the uncertainties in subjective evaluation Successful testing for small group of students Provides more accurate evaluation of a student as compared to existing method 3/28/2017 mowais@kfupm.edu.sa

Future Directions Testing of the system for large number of students Investigating the use of the system for other courses/situations e.g. assessing group projects etc. Type-2 fuzzy sets to be tested for representing final grades Deciding the optimal number of linguistic input/output variables for assessment components Working with non-singleton input from evaluators 3/28/2017 mowais@kfupm.edu.sa

Thank You 3/28/2017 mowais@kfupm.edu.sa

Question/Answers 3/28/2017 mowais@kfupm.edu.sa