Presentation is loading. Please wait.

Presentation is loading. Please wait.

Machine Learning - Conceptual Clustering - 4/11/2015Page 1 Conceptual Clustering Unsupervised, spontaneous - categorizes or postulates concepts without.

Similar presentations


Presentation on theme: "Machine Learning - Conceptual Clustering - 4/11/2015Page 1 Conceptual Clustering Unsupervised, spontaneous - categorizes or postulates concepts without."— Presentation transcript:

1 Machine Learning - Conceptual Clustering - 4/11/2015Page 1 Conceptual Clustering Unsupervised, spontaneous - categorizes or postulates concepts without a teacher Conceptual clustering forms a classification tree - all initial observations in root - create new children using single attribute (not good), attribute combinations (all), information metrics, etc. - Each node is a class Should decide quality of class partition and significance (noise) Many models use search to discover hierarchies which fulfill some heuristic within and/or between clusters - similarity, cohesiveness, etc.

2 Machine Learning - Conceptual Clustering - 4/11/2015Page 2 Cobweb Cobweb is an incremental hill-climbing strategy with bidirectional operators - not backtrack, but could return in theory Starts empty. Creates a full concept hierarchy (classification tree) with each leaf representing a single instance/object. You can choose how deep in the tree hierarchy you want to go for the specific application at hand Objects described as nominal attribute-value pairs Each created node is a probabilistic concept (a class) which stores probability of being matched (count/total), and for each attribute, probability of being on, P(a=v|C), only counts need be stored. Arcs in tree are just connections - nodes store info across all attributes (unlike ID3, etc.)

3 Machine Learning - Conceptual Clustering - 4/11/2015Page 3 Category Utility: Heuristic Measure Tradeoff between intra-class similarity and inter-class dissimilarity - sums measures from each individual attribute Intra-class similarity a function of P(A i = V ij |C k ), Predictability of C given V - Larger P means if class is C, A likely to be V. Objects within a class should have similar attributes. Inter-class dissimilarity a function of P(C k |A i = V ij ), Predictiveness of C given V - Larger P means A=V suggests instance is member of class C rather than some other class. A is a stronger predictor of class C.

4 Machine Learning - Conceptual Clustering - 4/11/2015Page 4 Category Utility Intuition Both should be high over all (most) attributes for a good class breakdown –Predictability: P(V|C) could be high for multiple classes, giving a relatively low P(C|V), thus not good for discrimination –Predictiveness: P(C|V) could be high for a class, while P(V|C) is relatively low, due to V occurring rarely, thus good for discrimination, but not intra- class similarity –When both are high, get best categorization balance between discrimination and intra-class similarity

5 Machine Learning - Conceptual Clustering - 4/11/2015Page 5 Category Utility For each category sum predictability times predictiveness for each attribute weighted by P(A i = V ij ), with k proposed categories, i attributes, j values/attribute The expected number of attribute values one could guess given C

6 Machine Learning - Conceptual Clustering - 4/11/2015Page 6 Category Utility Category Utility is the increase in expected attributes that could be guessed, given a partitioning of categories - leaf nodes. CU({C 1, C 2,... C k }) = K normalizes CU for different numbers of categories in the candidate partition Since incremental, there is a limited number of possible categorization partitions If A i = V ij is independent (irrelevant) of class membership, CU = 0

7 Machine Learning - Conceptual Clustering - 4/11/2015Page 7 Cobweb Learning Algorithm 1. Incrementally add a new training example 2. Recurse down the at root until new node with just this example is added. Update appropriate probabilities at each level. 3. At each level of the tree calculate the scores for all valid modifications using category utility (CU) 4. Depending on which of the following gives the best score: –Classify into an existing class - then recurse –Create a new class node – done, can get next example –Combine two classes into a single class (Merging) - then recurse –Divide a class into multiple classes (Splitting) - then recurse

8 Machine Learning - Conceptual Clustering - 4/11/2015Page 8 Cobweb Learning Mechanisms Classifying (Matching) - calculate overall CU for each case of putting the example in a node at current level New Class - calculate overall CU for putting example into a single new class- Note gradient descent (greedy) nature. Does not go back and try all possible new partitions. –If created from internal node, simply add –If created from leaf node, split into two, one for new and old These alone are quite order dependent - splitting and merging allow bi-directionality - ability to undo

9 Machine Learning - Conceptual Clustering - 4/11/2015Page 9 Cobweb Learning Mechanisms Merging - For best matching node (the one that would be chosen for classification) and the second best matching node at that level, calculate CU when both are merged into one node, with two children Splitting - For best matching node, calculate CU if that node were deleted and it’s children added to the current level. Both schemes could be extended to test other nodes, at the cost of increased computational complexity Can overcome initial “misconceptions”

10 Machine Learning - Conceptual Clustering - 4/11/2015Page 10 Cobweb Comments Generalization done by just executing recursive classification step Could use different variations on CU and search strategy Complexity: O(AVB 2 logK) for each example, where B is branching factor, A (attributes), V (average number of values), K (classes) Empirically, B usually between 2 and 5 Does not directly handle noise - no defined significance mechanism Tends to make “bushy” trees, however high levels should be most important class categories (because of merge/split causing best breaks to float up, though no optimal guarantee), and one could just use nodes highest in the tree for classification Does not support continuous values

11 Machine Learning - Conceptual Clustering - 4/11/2015Page 11 Extensions - Classit Cannot store probability counts for continuous data Classit uses a scheme similar to Cobweb, but assumes normal distribution around an attribute and thus can just store a mean and variance - not always a reasonable assumption Also uses a formal cut-off (significance) mechanism to better support generalization and noise handling (a class node can then include outliers) More work needed


Download ppt "Machine Learning - Conceptual Clustering - 4/11/2015Page 1 Conceptual Clustering Unsupervised, spontaneous - categorizes or postulates concepts without."

Similar presentations


Ads by Google