1. Finite-state automata and Morphology

2. Outline Morphology What is it? Why do we need it? How do we model it?
Computational Model: Finite-state transducer

3. Structure of Words What are words?
Orthographic tokens separated by white space.
In some languages the distinction between words and sentences is less clear.
Chinese, Japanese: no white space between words
nowhitespace  no white space/no whites pace/now hit esp ace
Turkish: words could represent a complete “sentence”
Eg: uygarlastiramadiklarimizdanmissinizcasina
“(behaving) as if you are among those whom we could not civilize”
Morphology: the structure of words
Basic elements: morphemes
Morphological Rules: how to combine morphemes.
Syntax: the structure of sentences
Rules for ordering words in a sentence

4. Morphology and Syntax Interplay between syntax and morphology
How much information does a language allow to be packed in a word, and how easy is it to unpack.
More information  less rigid syntax  more free word order
Eg: Hindi: John likes Mary – all six orders are possible, due to rich morphological information.

5. Why Study Morphology? Morphology provides
systematic rules for forming new words in a language.
can be used to verify if a word is legitimate in a language.
efficient storage methods.
improving lexical coverage of a system.
group words into classes.
Applications
Improving recall in search applications
Try “fish” as a query in a search engine
Text-to-speech synthesis
category of a word determines its pronunciation
Parsing
Morphological information eliminates spurious parses for a sentence

6. Structure of Words
Morphology: The study of how words are composed from smaller, meaning- bearing units (morphemes)
Stems: children, undoubtedly,
Affixes (prefixes, suffixes, circumfixes, infixes)
Immaterial, Trying, Gesagt, Absobl**dylutely
Different ways of creating words
Concatenative
Adding strings to the stems (examples above)
Non-concatenative
Infixation (e.g. Tagalog: hingi (order)  humingi)
Templatic (e.g. Arabic root-and-pattern) morphological systems
Triconsonant CCC expands into words using templates that represent more semantic information.
Hebrew: lmd (study)  lamad (he studied), limed (he taught), lumad (he was taught)
Stacking of affixes in English is limited (three or four); but in Turkish nine or ten affixes can be stacked.
Agglutinative languages

7. Different Classifications
Sets of words display (almost) regular patterns.
Open class words: Noun, Verb, Adverb, Adjectives
Closed class words: Pronoun, Preposition, Conjunction, Determiners.
Kinds of morphology
Inflectional
Word stem + grammatical morpheme results in a word of the same class.
Morpheme serves a syntactic function: grammatical agreement, case marker
-s for plural on nouns, -ed for past tense on verbs
Only Nouns and Verbs inflect in English for grammatical reasons
In French, adjectives agree (match on gender and number) with nouns they modify.
Derivational
Word stem + grammatical morpheme results in a different class of word.
Nominalization: -ation suffix makes verbs into nouns (e.g. Computerize  computerization
-ly suffix makes adjectives into adverbs (e.g. beautiful  beautifully)
Give example for each part-of-speech.

8. Inflectional Morphology
Word stem + grammatical morpheme
Morpheme serves a syntactic function: grammatical agreement, case marker
-s for plural on nouns, -ed for past tense on verbs
Results in word of the same class as the stem
(bat  bats; man  man’s; jump  jumps, jumped, jumping)
Noun morphology:
Regular nouns: bat  bats, fish  fishes
Irregular nouns: spelling changes (mouse  mice; ox  oxen)
Verb morphology:
Regular verbs: jump  jumps (-s form), jumped (past/ -ed participle), jumping (- ing participle)
Irregular verbs: eat  eats (-s form), ate (past), eaten (-ed participle), eating (- ing participle)
Irregular forms have to be stored.
Language learners typically make errors on irregular forms.
hit  hitted
Ask class for other examples of regular and irregular nouns and verbs

9. Inflectional Morphology
Nominal morphology
Plural forms
s or es
Irregular forms (goose/geese)
Mass vs. count nouns (fish/fish, or s?)
Possessives (cat’s, cats’)
Verbal inflection
Main verbs (sleep, like, fear) verbs relatively regular
-s, ing, ed
And productive: ed, instant-messaged, faxed, homered
But some are not regular: eat/ate/eaten, catch/caught/caught
Primary (be, have, do) and modal verbs (can, will, must) often irregular and not productive
Be: am/is/are/were/was/been/being
Irregular verbs few (~250) but frequently occurring
Irregularity a consequence of frequency?
So….English inflectional morphology is fairly easy to model….with some special cases...

10. (English) Derivational Morphology
Word stem + grammatical morpheme
Usually produces word of different class
More complicated than inflectional
E.g. verbs --> nouns
-ize verbs  -ation nouns
generalize, realize  generalization, realization
E.g.: verbs, nouns  adjectives
embrace, pity embraceable, pitiable
care, wit  careless, witless
E.g.: adjective  adverb
happy  happily
But “rules” have many exceptions
Less productive: *evidence-less, *concern-less, *go-able, *sleep-able
Meanings of derived terms harder to predict by rule
clueless, careless, nerveless

11. How do humans represent words?
Hypotheses:
Full listing hypothesis: words listed
Minimum redundancy hypothesis: morphemes listed
Words are formed using rules.
Experimental evidence:
Priming experiments (Does seeing/hearing one word facilitate recognition of another?) suggest neither
Regularly inflected forms prime stem but not derived forms
But spoken derived words can prime stems if they are semantically close (e.g. government/govern but not department/depart)
Speech errors suggest affixes must be represented separately in the mental lexicon
easy enoughly

12. Finite-State Morphological Parsing
takes : root=take, category=V, person=3, number=sg
cities : root=city, category=N, number=plural
ships: a. root=ship, category=N, number=plural
b. root=ship, category=V, person=3, number=sg
What does a parser need?
Lexicon: root forms of the words in a language
Morphological Rules: morpheme ordering rules
Orthographic Rules: spelling change rules
We will represent each of these components as finite-state automata.
Parsing versus Recognition
Recognition: yes/no
Parsing: derivation structure
indecipherable  [in [[de [cipher]] able]]

13. Lexicon as an FSA
Words can be represented as paths in a FSA with characters as transition symbols.
Also called a “trie” structure
List of words: bat, cat, mat, sat c a t b m s

14. Morphological Recognition
Check if a word is a valid word in the language.
For nouns: cats, geese, goose
For adjectives: Derivational morphology
Adj-root1: clear, happy, real (clearly)
Adj-root2: big, red (~bigly)
reg-n
plural (-s) q0 q1 q2
irreg-pl-n
irreg-sg-n
adj-root1
un-
-er, -ly, -est q0 q1 q2
adj-root1 q5 q3 q4
-er, -est 
adj-root2

15. Morphotactic Rules
An automaton that encodes valid English word forming rules.
Koskenniemi’s two-level morphology
Lexical string: stem and morphemes
Surface string: spelling of the derived word
Pair up lexical string with the surface string.
(N,bat,pl; bats)
(V,bat,3sg; bats)
Align the strings at the character level
N:e b:b a:a t:t pl:s
V:e b:b a:a t:t 3sg:s
Relation that maps characters to other characters.

16. Finite-State Transducers
Finite State Acceptors represent regular sets.
Finite State Transducers represent regular relations.
Relation
If A and B are two regular sets; relation R ⊆ A x B
Example: {(x,y) | x ∊ a*, y ∊ b*}
FSTs can be considered as
Translators (Hello:Ciao)
Parser/generators (Hello:How may I help you?)
As well as Kimmo-style morphological parsing
Examples of fsts on board

17. Finite State Transducers – formally speaking
FST is a 5-tuple consisting of
Q: set of states {q0,q1,q2,q3,q4}
: an alphabet of complex symbols, each an i:o pair s.t. i  I (an input alphabet) and o  O (an output alphabet) and  ⊆ I x O
q0: a start state
F: a set of final states in Q {q4}
(q,i:o): a transition function mapping Q x  to Q q0 q4 q1 q2 q3
b:m
a:o
!:?

18. More FST examples FSTs are functions,
Compose (L1,L2) = {(x,y) | (x,z) ∊ L1 and (z,y) ∊ L2}
3((q1,q2),(x,y)) = (q3,q4) if ∃z 1(q1,(x,z)) = q3 and 2(q2,(z,y)) = q4
Inverse(L) = {(x,y) | (y,x) ∊ L}
FirstProjection(L) = {x | (x,y) ∊ L}
SecondProjection(L) = {y | (x,y) ∊ L}
Speech Recognition as FST composition
A: Acoustic  phones
Lex: phones  Words
Gram: Words  Words
Speech Recognition: A ● Lex ● Gram
Machine Translation as FST composition
LexTrans: SourceWords  TargetWords
LexReorder: UnOrderedTargetWords  OrderedTargetWords
Weighted FSTs allow for ranking of the generated outputs.
Bilingual dictionary
FST/FSM tools

19. FST for a 2-level Lexicon
E.g.
Nominal Morphology
c:c
a:a
t:t q0 q1 q2 q3
g:g q4 q5 q6 q7
e:e
e:o
e:o
s:s
+PL:^s# q4
reg-n
+SG:#
+N:
irreg-n-sg q0 q2 q5 q7
+SG:#
irreg-n-pl q6
+PL:#

20. Orthographic Rules Define additional FSTs to implement spelling rules
Gemination: consonant doubling (beg  begging),
Elision: ‘e’ deletion (make  making),
Epenthesis: ‘e’ insertion (fox  foxes), etc.
Y replacement: ‘y’ changes to ‘ie’ before ‘ed’ (trytries)
I spelling: ‘I’ changes to ‘y’ before a vowel (lielying)
Rewrite rule notation: a  b / p _ d
Examples: Rule: a  b/a _ b
aaabaab  aabbabb (Sequential; there are other control strategies: parallel, feedback).
Spelling rules: e  e / x ^ _ s (eg. fox^s  foxes)
Multiple spelling rules might apply for a given word.
Eg: spy^s  spies (yi, ee)
All spelling rules execute in parallel.
Rewrite rules can be compiled into FSTs.

21. FST-based Morphological Parsing
Remember FSTs can be composed.
Lexicon : List of stems
Morphotactics: Rules for adding morphemes
Orthographic Rules: Spelling Rules
Morphology =
Lexicon ● Morphotactics ● Orhtographics
All morphological analyses generated for a given word,
…but in a context a preference ordering on the analysis is needed.
Weighted FSTs to the rescue.
Fox +N +3pl fox^sfoxes

22. Porter Stemmer
Lexicon free stemmer; heuristic rules for deriving the stem.
Rules to rewrite the suffix of a word.
“ational”  “ate” (eg. relational  relate)
“-ing”  e (eg. motoring  motor)
Details of the rules in Appendix B in the book.
Purported to improve recall in IR engines.

23. Role of Morphology in Machine Translation
Every MT system contains a bilingual lexicon
Bilingual lexicon: a table mapping the source language token to target language token(s).
Two options:
1. Full-form lexicon
every word form of the source token is paired with the target token
large table if the vocabulary is large for morphologically rich languages
2. Root-form lexicon
pairing of stems from the two languages
Reduces the size of the lexicon
requires morphological analysis for source language
bats  (bat, V, 3sg) (bat, N, pl)
morphological generation for target language
(bat, V, 3sg)  bats
Unknown words: words not covered in the bilingual lexicon
- with morphology, one can guess the syntactic function
Compounding of words:
English: simple juxtaposition (“car seat”), some times (“seaweed”)
German: fusion is more common
(“Dampfschiffahrtsgesellschaft”  steamship company

24. Readings For this class HS: pages 15-16, 81-85
JM: chapter on Morphology
For next class
Machine Translation divergences (Dorr 94)
Example based Machine Translation (Somers 99)