1. Computational Intelligence
Winter Term 2017/18
Prof. Dr. Günter Rudolph
Lehrstuhl für Algorithm Engineering (LS 11)
Fakultät für Informatik
TU Dortmund

2. Plan for Today Fuzzy Clustering
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 2
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 2

3. Cluster Formation and Analysis
Introductory Example: Textile Industry
→ production of T-shirts (for men)
best for producer : one size
best for consumer: made-to-measure
vs.
 compromize: S, M, L, XL, 2XL
5 sizes
→ OK, but which lengths for which size?
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 3
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 3

4. Cluster Formation and Analysis
idea:
 select, say, 2000 men at random and measure their “body lengths“
arm‘s length,
collar size,
chest girth, …
 arrange these 2000 men into five disjoint groups
such that
– deviations from mean of group as small as possible
– differences between group means as large as possible
in general:
arrange objects into groups / clusters
such that
– elements within a cluster are as homogeneous as possible
– elements across clusters are as heterogeneous as possible
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 4
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 4

5. Cluster Formation and Analysis
numerical example: points uniformly sampled in [0,1] x [0,1]  form 5 cluster
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 5
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 5

6. Hard / Crisp Clustering
given data points x1, x2, …, xN
objective: group data points into cluster
such that
- points within cluster are as homogeneous as possible
- points across clusters are as heterogeneous as possible
 crisp clustering is just a partitioning of data set { x1, x2, …, xN }, i.e.,
{ x1, x2, …, xN } and
where is Cluster and denotes the number of clusters.
Constraint:
hence
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 6
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 6

7. Hard / Crisp Clustering
Complexity: How many choices to assign N objects into clusters?
more precisely:
→ objects are distinguishable / labelled
→ clusters are nondistinguishable / unlabelled and nonempty
 Stirling number of 2nd kind
 enumeration hopeless!  iterative improvement procedure required!
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 7
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 7

8. Hard / Crisp Clustering
idea: define objective function
that measures compactness of clusters and quality of partition
→ elements in cluster Cj should be as homogeneous as possible!
→ sum of squared distances to unknown center y should be as small as possible →
(Euclidean norm)
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 8
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 8

9. Hard / Crisp Clustering
→ elements in each cluster Cj should be as homogeneous as possible! →
Definition
Theorem
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 9
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 9

10. Crisp K-Means Clustering
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 10
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 10

11. From Crisp to Fuzzy Clustering
objective for crisp clustering:
→ rewrite objective:
expresses membership
objective for fuzzy clustering:
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 11
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 11

12. Fuzzy K-Means Clustering
where
subject to
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 12
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 12

13. Fuzzy K-Means Clustering
two questions:
ad a) 
→ weighted mean! 
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 13
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 13

14. Fuzzy K-Means Clustering
ad b)
apply Lagrange multiplier method:  
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 14
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 14

15. Fuzzy K-Means Clustering
after insertion:
problems:
- choice of K
calculate quality measure
for each #cluster; then choose best
- choice of m try some values; typical: m=2; use interval → fuzzy type-2
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 15
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 15

16. Example: Special Case |Ji| > 1
black dot is center of
- red cluster
- blue cluster
- yellow cluster in case of equal weights
uij = 1 / |Ji| for j  Ji appears plausible
but: different values algorithmically better
→ cluster centers more likely to separate again (→ tiny randomization?)
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 16
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 16

17. Measures for Cluster Quality
 Partition Coefficient
( “larger is better“ )
→ crisp partition
→ entirely fuzzy
 Partition Entropy
( “smaller is better“ )
→ entirely fuzzy
→ crisp partition
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 17
G. Rudolph: Computational Intelligence ▪ Winter Term 2017/18 17