1. King Fahd University of Petroleum & Minerals (KFUPM/HBCC)
Type-2 Fuzzy Logic Advisor for Evaluating Students’ Cooperative Training
3rd UK Workshop on AI in Education
Owais Ahmed Malik
King Fahd University of Petroleum & Minerals (KFUPM/HBCC)
Saudi Arabia
3/28/2017

2. Overview Introduction Cooperative Training Assessment
Motivation for the Perception-based Assessment
Fuzzy Logic and Fuzzy Logic System
Proposed Model for Cooperative Training Assessment
Experiments and Discussion
Conclusions and Future Directions
3/28/2017

3. Introduction
Students’ learning performance is measured by some evaluation means.
Students’ Evaluation
Process of collecting students’ work
Making decision based on collected information
Methods of Evaluation
Objective
Subjective
3/28/2017

4. Introduction How to evaluate a student? Formal: Informal: Homework
Quiz
Written Exam (Majors)
Lab Exam
Informal:
Interview
Class Participation
Team Work
Individual Projects
3/28/2017

5. Cooperative Training Assessment
Cooperative Training/Internship
An important tool to develop student skills
Some real work experience in industry
A typical assessment for Coop training:
Progress reports
Final report
Presenting the work
External supervisor remarks
Onsite visit by the internal supervisor
3/28/2017

6. Cooperative Training Assessment
Assessment Component
Criteria for Assessment
Final Report
(FR)
Format and Structure
Literary Quality
Quality of Subject Matter
Progress Report
(PR)
Task Description
Format and Submission
Final Presentation
(FP)
Content and Organization
Speaking (Presentation) Skills
Response to Questions
External Evaluation
(EE)
Enthusiasm and Interest in Work
Ability to Learn and Search for Information
Relations with Co-Workers
Punctuality and Delivering Work on Time
3/28/2017

7. Coop Training Assessment
Example Rubric for Presentation Assessment:
Excellent
Good
Fair
Unsatisfactory
Content
All information related to the
coop training e.g. work place,
time, location, learning etc;
points are clearly presented
with all necessary description
of work done during training
Period
Sufficient information
related to coop training;
but description of work is
not thorough
Incomplete information
about coop training;
adequate details about tasks
completed during training
Inadequate information
incomplete description
about tasks completed
during training
Organization
All information presented in a
logical & interesting sequence;
gives audience very clear
picture of training; good
transitions; succinct & clear
Most of the information
presented in logical
sequence; gives audience an
adequate picture of
training; generally well
organized; good transitions
Lacks some sequence of
information; difficulty in
following for audience;
loosely organized
No sequence of
information; no
understanding for
audience; presentation is
disjointed
Material
(Figures/Visual Aids,
Spelling / Grammar)
Very effective use of visual aids;
clear figures and charts; no
spelling or grammatical
mistakes
Good use of visual aids;
graphics relate to text
presented; 1 or 2 spelling
/grammar mistakes
Occasional use of visual
aids; not much related to
text; few spelling/ grammar
Little/no or ineffective
use of visual aids; many
spelling/grammar
Speaking Skills
Clear articulation; excellent
delivery with proper volume,
steady pace, good posture and
eye contact; confidence
Clear articulation; good
delivery with good pace,
usually projects voice and
good eye contact
Some mumbling low voice
and uneven pace; little eye
contact
Inaudible or too loud;
pace too slow or too fast;
no eye contact; seems
uninterested
Questions/Answers
Answers questions effectively
and smoothly with full
description; satisfy audience
Answers most of the
questions with little
elaboration
Answers only rudimentary
questions; very little
Can not answer most of
the questions; no grasp
of subject
3/28/2017

8. Motivation for Perception-based Assessment
Assessment of different components of Coop training is subjective.
Communication skills during presentation
Organization of presentation/report
Literary quality of report
Quality of subject matter
Student’s attitude towards work
Enthusiasm and interest in work
Difficult to apply the objective methods to evaluate these student activities
3/28/2017

9. Motivation for Perception-based Assessment
Assessment mostly based on perception of an evaluator
Judgment in terms of words (Excellent, Very Good, and Good etc.)
Conventional assessment methods usually do not consider the uncertainties in usage of words
Motivation for type-2 fuzzy set be used to model a word
3/28/2017

10. Fuzzy Logic (FL)
Mathematical and Statistical techniques are often unsatisfactory in decision making.
Experts make decisions with imprecise data in an uncertain world.
They work with knowledge that is rarely defined mathematically or algorithmically but uses vague terminology with words.
FL designed to handle imprecision and uncertainty in the measurement process
Methodology of computing with words (CW)
Mimics the perception-based decision making done by humans
3/28/2017

11. Fuzzy Logic Linguistic Variable
Example : Age of a person
Term Set: Young, Middle-aged, Old etc.
Each linguistic term is associated with a fuzzy set
Each term has a defined membership function (MF):
A fuzzy set A in X can be expressed as: or
3/28/2017

12. Fuzzy Logic
Example Fuzzy Set for Age:
3/28/2017

13. Fuzzy Logic Example Fuzzy Set for Literary Quality of a Report:
3/28/2017

14. Type-2 Fuzzy Set
Imprecise perception-based data can be modelled by using type-2 fuzzy logic
Type-2 fuzzy set is 3-dimensional representation
Type-2 fuzzy sets help us to deal with the uncertainty
Footprint of Uncertainty (FOU):
Bounded region in the primary membership function of a type-2 fuzzy set
2-Dimensional depiction of type-2 fuzzy sets
Upper and Lower Membership Functions
For more details: Mendel J. M., Uncertain Rule-Based Fuzzy Logic Systems, Prentice-Hall, Upper Saddle River, NJ 07458, (2001)
3/28/2017

15. FOUs, Upper and Lower MFs
3/28/2017

16. Type-2 Fuzzy Logic System
3/28/2017

17. Proposed Model for Cooperative Training Assessment
Based on knowledge mining (knowledge engineering) methodology
Information extracted in the form of IF-THEN rules from evaluators (experts)
Rules are modelled using fuzzy logic system
Used as Fuzzy Logic Advisor (FLA)
Two-stage FLA based on interval type-2 fuzzy logic
Each assessment component is evaluated using an independent FLA
Results of these FLAs are combined to calculate the final grade
3/28/2017

18. Structure of Proposed Model
3/28/2017

19. Input/Output Fuzzy Sets for Proposed Model
Input (criteria of assessment) and output (evaluation) attributes divided into four fuzzy sets
Type-2 fuzzy sets: Excellent, Good, Fair and Poor
Survey results for labels of fuzzy sets
Label
Mean
Std. Deviation
Start
End a b σa σb
Poor
4.7389
0.4898
Fair
4.7056
6.8778
0.4978
0.4295
Good
6.6556
8.7222
0.4419
0.3153
Excellent
8.4889
0.3296
0.0000
3/28/2017

20. Membership Functions for Proposed Model
FOUs for Literary Quality of a Report:
3/28/2017

21. Rules Formulation
All possible combinations of antecedent fuzzy sets are employed
Consequents of rules are provided by the evaluators (experts)
Each rule has a histogram of responses
Number of rules depends on the number of inputs and fuzzy sets associated with them
Example rule for Coop Evaluation FLA
Rl: IF Final Report is Excellent AND Progress Report is Good AND Final Presentation is Fair AND External Evaluation is Excellent THEN GRADE is (VERY GOOD)
3/28/2017

22. Type-1 FLA (Individual FLA)
3/28/2017

23. Partial Histogram of Survey Responses for Final Report Evaluation
Rule No.
Antecedent 1
Antecedent 2
Antecedent 3
Consequent
Type-1
Type-2
Excellent
Good
Fair
Poor
Cavg
Clavg
Cravg 1 8
9.162
9.077
9.242 2 6
8.783
8.688
8.874 3 4
8.17
8.061
8.276 5
6.533
6.4
6.666
7.98
7.866
8.093 7
6.943
6.806
7.079
6.298
6.162
6.435 9
7.791
7.671
7.909 10
7.178
7.044
7.311 11 12
5.829
5.686
5.972 13
6.064
5.924
6.204 14 15
4.95
4.804
5.097
3/28/2017

24. Experiments and Discussion
Comparison for Individual and Type-1 Consensus FLAs
3/28/2017

25. Experiments and Discussion
Comparison for Individual and Type-2 Consensus FLAs (50% uncertainty)
3/28/2017

26. Experiments and Discussion
Comparison for Individual and Type-2 Consensus FLAs (100% uncertainty)
3/28/2017

27. Conclusions Type-2 fuzzy sets model the perception-based evaluation
Proposed model has the potential to capture the uncertainties in subjective evaluation
Successful testing for small group of students
Provides more accurate evaluation of a student as compared to existing method
3/28/2017

28. Future Directions Testing of the system for large number of students
Investigating the use of the system for other courses/situations e.g. assessing group projects etc.
Type-2 fuzzy sets to be tested for representing final grades
Deciding the optimal number of linguistic input/output variables for assessment components
Working with non-singleton input from evaluators
3/28/2017

29. Thank You
3/28/2017

30. Question/Answers
3/28/2017