Semi-supervised Affinity Propagation Inmar Givoni, Brendan Frey, Delbert Dueck PSI group University of Toronto

Affinity Propagation Clustering algorithm that works by finding a set of exemplars (prototypes) in the data and assigning other data points to the exemplars [Frey07] Input: pair-wise similarities (negative squared error), data point preferences (larger = more likely to be an exemplar) Approximate maximization of the sum of similarities to exemplars Mechanism – message passing in a factor graph

Semi-supervised Learning Large amounts of unlabeled training data Some limited amounts of side information Partial labels Equivalence constraints Half moon data

Some Motivating examples

AP with partial labels All points sharing the same label should be in the same cluster. Points with different labels should not be in the same cluster. Imposing constraints Via the similarity matrix Explicit function nodes

Same label constraints Set similarity among all similarly labeled data to be maximal. Propagate to other points (teleportation) Without teleportation, local neighborhoods do not ‘move closer’. e.g. Klein02] S(x1,x2)=0 x1 x2 y2 y1

Different labels Can still do a similar trick and set similarity among all pair-wise differently labeled data to be minimal. But no equivalent notion of anti-teleportation. x1 x2

Adding explicit constraints to account for side-information

Adding explicit constraints to account for side-information

Problems Let’s call all the labeled points portals They induce the ability to teleport… At test time, if we want to determine a label for some new point we need to evaluate its closest exemplar, possibly via all pairs of portals - expensive. Pair-wise not-in-class nodes for each pair of differently labeled points is expensive. Introducing…

Meta-Portals An alternative way of propagating neighborhood information. Meta-portals are ‘dummy’ points, constructed using the similarities of all portals of a certain label. We add N new entries to the similarity matrix, where N is the number of unique labels.

Meta-portals mtp’s can be exemplars. Unlike regular exemplars, mtp’s can be exemplars for other points but choose a different exemplars themselves

These function nodes force the MTP’s to choose other data points as their exemplars. Similarities alone are not enough, since both MTP can choose same exemplars and still have –inf similarities.

Some toy data results

Future work Investigate interplay between modifying similarities and incorporating explicit constraints. Possible tool for user-guided labeling