1 Constructing Fuzzy Signature Based on Medical Data Student: Bai Qifeng Client: Prof. Tom Gedeon

2 Proposal Explore an approach to automatic construct Fuzzy signature based on medical database It contains three questions: 1. How to identify SARS suspect patients group? 2. How to explore the relationships among symptoms? 3. How to construct fuzzy signature based on above analysis?

3 Fuzzy Logic Theory Fuzzy logic uses linguistic rules which reflect uncertainty or vagueness of concepts in natural in natural language. If 50m/h is the boundary of “slow” and “fast”, Conventional bivalent sets regards 50.1m/h as fast. What if current speed is 49.9m/h? In real world, it should be a smooth shift.

4 Fuzzy Set Now, assume there are three temperatures We can get the fuzzy sets: A fuzzy set is a set whose elements have degrees of membership SlightModerateSever e Extreme

5 Why use fuzzy sets Assume: 1. IF Fever = Slight THEN dose = Low. 2. IF Fever = Moderate THEN dose = Ave. Fuzzy value of fever is slight = 0.29 and moderate = 0.71 Value of dose will share properties of both Low and Ave range. IN OUT

6 Problem Definition A Major issue in fuzzy applications is how to create fuzzy rules the number of rules have an exponential increase with the number of inputs and terms. At least one activated rule for every input. e.g. 5 terms, 2 inputs => 25 rules 5 terms, 5 inputs => 3,125 rules

7 Sketch of Solution Three possible solutions Decrease T : Sparse Fuzzy System Decrease K: Hierarchical Fuzzy System Decrease both simultaneously : Sparse Hierarchical Fuzzy Rule Bases

8 Hierarchical Fuzzy Systems Hierarchical fuzzy systems reduce to the dimension of the sub-rule bases k by using meta – levels

9 Fuzzy Signatures Fuzzy signatures structure data into vectors of fuzzy values, each of which can be a further vector. Each signature corresponds to a nested vector structures or, equivalently, to a tree graph.

10 Fuzzy Signatures The relationship between higher and lower levels is govern by fuzzy aggregations. Fuzzy aggregation contains union, average, intersection etc. Examples: Union: AUB = max [A, B] = A or B Intersection: A∩B = min [A,B] = A and B

11 Clustering The aim of cluster analysis is to classify objects based on similarities among them. Definition of cluster is a group of objects that are more similar to one another than to members of other clusters. Clustering is unsupervised classification: no predefined classes

12 Clustering: Similarity How to evaluate the similarities of data? Cluster analysis adapts the distance between two points as the criterion of similarity. Distance-type measure has Euclidean distance and City block distance.

13 Clustering: hard/fuzzy methods

14 Clustering: Procedure

15 Clustering: Fuzzy C-Means Bezdek define objective function as : represents the deviation of data with centre. The number m governs the influence of membership grades. u ij represents the degree of membership of the data point x j belonging to v.

16 Clustering: Cluster Valid Index Xie and Beni Index The numerator calculates the compactness of data in the same cluster and the denominator computes the separateness of data in different clusters. Smaller value of numerator validity index indicates that the clusters are more compact and larger values of denominator denotes the clusters are well separated.

17 Factor Analysis Factor analyses are performed by examining the pattern of correlations between the observed measures.. X is a vector of variables, where is a vector of r<p latent variables called factors, is a (p*r) matrix of coefficients (loadings), is a vector of random errors.

18 Factor Analysis

19 Factor Analysis: Principal component analysis Principal component analysis aims to reduce the dimension of variables and these new variables can interpret most of cases.

20 Factor Analysis: Principal component analysis. x is the p dimensional variables, where U is an orthogonal matrix. 1. The loading of matrix U and vector Z( ), which correspond to the variance and vector of the principal components respectively. 2. The value represents the contribution ratio which indicates how much percentage the principal component represents of the total tendency of the variables. 3. Usually, an accumulative contribution ratio of percent can effectively represent the major variations in the original data.

21 Factor Analysis: PCA vs FA Direction is reversed: the measured responses are based on the underlying factors while in PCA the principal components are based on the measured responses

22 Factor Analysis: Factor Rotation For identify some variables having similar factor loading, we could rotate the factor coordinates in any direction without changing the relative locations of the points to each other.

23 Experiment: Scatter of Raw Data Gravities of components are deviated by the noise or outliers.

24 Experiment: Scatter After Clustering Collected data can represent the pattern of the disease more accurately.

25 Experiment: KMO and Bartlett’s Test KMO test indicates the possibility of containing underlying factors. KMO <.50, factor analysis is not useful. Bartlett's test indicate whether variables are unrelated. significance level <.05 significant relationships Kaiser-Meyer-Olkin Measure of Sampling Adequacy..608 Bartlett's Test of Sphericity Approx. Chi-Square df36 Sig..000

26 Experiment: PCA Model Initial EigenvaluesSums of Squared Loadings Total% of VarianceCumulative %Total% VarianceCumulative % Accumulative contribution ratio = 63%

27 Experiment: PCA Model It denotes that variables could be divided into 3 factors

28 Experiment: Results Factors 123 fever malaise cough dyspnea chest lymphopenia dehydrogenase aspartate kinase

29 Experiment: Results after rotation Factors 123 fever malaise cough dyspnea chest lymphopenia dehydrogenase aspartate kinase

30 Experiment: Constructed fuzzy signature Hierarchical clustering or K-means can be used to cluster each factor Weighted aggregation method in this fuzzy signature had higher performance 3 weights & 3 aggregations

31 Experiment: Possible rule bases Aggregations: Min (fever, cough, chest) Min (dyspnes, lymphopenia) Max (Min (kinase, malaise), Min(aspartate, dehydrogenase) ) Rules If a patient has fever, cough and chest. If a patient has dyspnes and lymphopenia. If patient has kinase and malaise or has aspartate and dehydrogenase

32 Experiment: Possible rule bases Further assumption: If a patient has fever, cough and chest, he/she would has 64% possibility to get SARS If he/she has kinase and malaise or has aspartate and dehydrogenase simultaneously, the possibility is increasing to 93% If he/she has dyspnes and lymphopenia, he/she can be diagnosed as a SARS Patient

33 Conclusion Advantages: 1. Fuzzy signatures are capable of improving the applicability of fuzzy systems. 2. Fuzzy signatures have the ability to cope with complex structured data and interdependent features problems. 3. With weighted aggregated, fuzzy signatures can assist experts to make decision by removing redundant information

34 Further Work Further research can be focused on evaluating underlying relationships between the structures of fuzzy signatures, aggregation functions and weights of each vector.

35 Thank you ---- Bai Qifeng

36 Appendix Demo of Fuzzy Control Sparse Fuzzy System Automatic Constructing Fuzzy Signature Fuzzy c-Means

37 Fuzzy Control Fuzzy control is the most important current application in fuzzy theory. Usually, three steps in Fuzzy control: 1. Fuzzification 2. Rule evaluation 3. Defuzzification

38 Demo of Fuzzy Control The most common one is the centre of gravity

39 Demo of Fuzzy Control Use a procedure originated by Ebrahim Mamdani as demo. The application is to balance a pole on a mobile platform that can move in only two direction, to the left or the right. The angle between the platform and the pendulum and the angular velocity of this angle are chosen as the inputs of the system. Output is corresponding to the speed of the platform.

40 Fuzzification First of all, the different levels of input and output are defined by specifying the membership functions for the fuzzy sets. For similarity, it is assumed that all membership functions are spread equally. Hence, this explains why no actual scale is included in the graphs

41 Fuzzification Input Angel Input Angel Velocity Output Speed

42 Rule Evaluation The next step is to define the fuzzy rules. The fuzzy rules are a series of if-then statements. For example: If angle is zero and angular velocity is zero then speed is also zero. If angle is zero and angular velocity is low then the speed shall be low.

43 Rule Evaluation The full set of rules are listed in table Speed Angle Neg. highNeg. LowZeroPos. LowPos. High V Neg. HighNeg. High E Neg. LowNeg. Low Zero L ZeroNeg. HighNeg. Low ZeroPos. LowPos. High O Pos. Low ZeroPos. Low C Pos. High Pos. High

44 Rule Evaluation Suppose an example has and 0.25 for zero and positive low angles and 0.6 for zero and negative low angular velocities.

45 Rule Evaluation Consider the rule "if angle is zero and angular velocity is zero, the speed is zero".

46 Rule Evaluation Consider the rule "if angle is zero and angular velocity is negative low, the speed is negative low".

47 Rule Evaluation Consider the rule "if angle is positive low and angular velocity is zero, the speed is positive low".

48 Rule Evaluation The Results overlap and are reduced to the following figure

49 Defuzzification Defuzzification is used to choose an appropriate representative value as the final output. The most common one is the centre of gravity

50 Sparse Fuzzy Systems Sparse fuzzy systems can be used in situations where full knowledge of the problem domain is not available. Problem domain experts often work with only important fuzzy rules. Self learning algorithms to tune the parameters of a fuzzy system for accuracy improvement can also lead to sparse fuzzy systems. In most cases, parameter tuning involves the reshaping of the fuzzy sets in the rule antecedents. It can happen that the shrinking of the fuzzy sets leads to gaps between neighboring fuzzy sets. Generating a sparse fuzzy system benefits from the reduced number of rules. (Chong 2004)

51 Sparse Fuzzy Systems Sparse system can reduce T. The essential idea is based on the omission of less important fuzzy rules to form sparse fuzzy systems. In sparse systems, it would be possible that inputs do not match any of the rule antecedents. Fuzzy rule interpolation is used to infer these rules for the inputs from existing fuzzy rules in the system.

52 Interpolation overview Tomato colours: back back IF colour = Red THEN it is Ripe IF colour = Green THEN it is Unripe What about a yellow tomato? Potential tomato colours:

53 Automatic Construct Fuzzy Signature Sub-Structure may be hidden in large data set. More separable the elements in subspace, the easier sub-rule base selection is. Finding suitable Π and Z 0 affect each other.

54 Sugeno and Yasukawa Approach Sugeno and Yasukawa (1991) introduced a solution for sparse rule-base generation. It clusters output data sample and induces the rules by projecting clusters of output to input domains. Cons: it only produces necessary rules for the input-output sample data

55 Projection-based Fuzzy Rule Extraction 1. Perform c-Means to cluster data along output space. The FS index of Fuzzy c-Means can be used to get a optimal number of clusters. 2. For each fuzzy output cluster, all points contained in the cluster are projected back to input dimensions. 3. The projected points in each dimension are clustered again. In this procedure, the FS index is used in conjunction with the merging index. This process will produce multiple fuzzy clusters in each dimension. 4. Each of the clusters in the input dimension is a projection of the multi-dimensional input cluster to that input dimension. Then, the clusters from the individual dimensions are combined to form the multi-dimensional input cluster. 5. For each of the multi-dimensional clusters identified, a rule can be created.

56 Fuzzy c-Means Let as a fuzzy partition C

57 Fuzzy c-Means Dunn defined a fuzzy objective function: v i is cluster center of i set Bezdek extended it to: represents the deviation of data with. The number m governs the influence of membership grades.

58 Fuzzy c-Means Limitation: it needs to know the number of clusters. How to find an optimal number of clusters. A cluster validity index proposed by Fukuyama and Sugeno (FS):

59 Finding Suitable Subspace Rules: age & experience to salary IF Age = young & Exp = LittleThen $ =Low IF Age = young & Exp = ModerateThen $ =Low IF Age = young & Exp = GoodThen $ =High IF Age = Middle & Exp = LittleThen $ =Low IF Age = Middle & Exp = ModerateThen $ =Ave IF Age = Middle & Exp = GoodThen $ =High IF Age = Older & Exp = LittleThen $ =Ave IF Age = Older & Exp = ModerateThen $ =Ave IF Age = Older & Exp = GoodThen $ =High

60 Finding Suitable Subspace Rule in a tree (Age/Exp/Con) Age Exp Y M O l M G L L H L AH AAH Age Exp Y M O G L H L AH AH Prune tree

61 Finding Suitable Subspace Rule in a tree (Exp/Age/Con) back back Age Exp Y M O l M G L L A L AA HHH Age Exp M O l M G L A L A H Prune rule tree

62 Fuzzy Signatures in SARS Diagnosis The following scheme is of some daily symptom signatures of patients:

63 Fuzzy Signatures in SARS Diagnosis Two examples with linguistic values and fuzzy signatures.

64 Fuzzy Signatures in SARS Diagnosis An aggregation method can compare components regardless of the different numbers of sub- components.

65 Fuzzy Signatures in SARS Diagnosis Aggregation methods for different symptoms here are different with that of signatures of same symptoms. Here, we define weights are: fever = 1, Cough = 0.9, Nausea = 0.4, Sore = 0.25 back