Word classes Chris Brew The Ohio State University

Word senses  The problem is that many words have several distinct meanings. Bank “the ground bordering a river” Bank “an establishment for custody of money”  In word-sense disambiguation we try to find out which meaning goes with which instance  Perhaps surprisingly, this idea of word sense is seriously problematic.

A more realistic case  Slide (n): the slide of a trombone a childrens’ toy The act of sliding A landslide (and its metaphorical uses) A loss in stock value A kind of musical grace note A specific kind of shot in Curling A head-first slide into third base A fracture in a lode resulting in the dislocation or displacement of a portion of it.

Dictionary definitions  One could just say, each dictionary definition is a sense, but if we also want intuitions, we may have to compromise  Even if we do this, which dictionary to use?

Word classes  Two potentially conflicting notions Use word classes to predict next word Use word classes to capture semantic commonalities  If we use distributional statistics to build classes, what will they be like?

Distributional clustering Define the properties of a word that one cares about, and give them numerical values. Pull them together into a vector Viewing the vector as a point in space, cluster the words to form classes

Dimensions of variation  What goes into the vector The most important influence  How one measures distance between vectors Options include cosine, KL-divergence, information radius  Which algorithm to use Exhaustive enumeration of all potential clusters is way too costly, heuristics are needed.

Things to cluster by  Next word (Brown et al)  Syntactic relations (Pereira,Tishby)  Parallel corpora (Brown et al, Gale et al)  Words in window

Distance measures  Euclidean distance  Cosine distance (avoids over-dependence on length)

Algorithms  Top-down tree construction (McMahon and Smith)  Bottom-up tree construction (Brown et al.) Guided by loss of MI  Classical clustering algorithms K-means Hierarchical clustering Ward’s method

Where to learn more  M&S ch 7 (v. good on background  Charniak ch 9 and 10 (v. good on algorithms)  Manual for the R statistics system, especially the mva module  Schulte im Walde’s thesis, our joint papers