1. Semantic Role Labeling: English PropBank
Computational Corpus Linguistics
Martha Palmer

2. Ask Jeeves – A Q/A, IR ex. What do you call a successful movie?
Tips on Being a Successful Movie Vampire ... I shall call the police.
Successful Casting Call & Shoot for ``Clash of Empires'' ... thank everyone for their participation in the making of yesterday's movie.
Demme's casting is also highly entertaining, although I wouldn't go so far as to call it successful. This movie's resemblance to its predecessor is pretty vague...
VHS Movies: Successful Cold Call Selling: Over 100 New Ideas, Scripts, and Examples from the Nation's Foremost Sales Trainer.
Blockbuster
LING 5200, 2006

3. Ask Jeeves – filtering w/ POS tag
What do you call a successful movie?
Tips on Being a Successful Movie Vampire ... I shall call the police.
Successful Casting Call & Shoot for ``Clash of Empires'' ... thank everyone for their participation in the making of yesterday's movie.
Demme's casting is also highly entertaining, although I wouldn't go so far as to call it successful. This movie's resemblance to its predecessor is pretty vague...
VHS Movies: Successful Cold Call Selling: Over 100 New Ideas, Scripts, and Examples from the Nation's Foremost Sales Trainer.
LING 5200, 2006

4. Filtering out “call the police”
Different senses,
- different syntax,
- different kinds of participants,
- different types of propositions.
call(you,movie,what) ≠ call(you,police)
you movie what you police
LING 5200, 2006

5. WordNet – Princeton (Miller 1985, Fellbaum 1998)
On-line lexical reference (dictionary)
Nouns, verbs, adjectives, and adverbs grouped into synonym sets
Other relations include hypernyms (ISA), antonyms, meronyms
Typical top nodes - 5 out of 25
(act, action, activity)
(animal, fauna)
(artifact)
(attribute, property)
(body, corpus)
LING 5200, 2006

6. Cornerstone: English lexical resource
That provides sets of possible syntactic frames for verbs.
And provides clear, replicable sense distinctions.
AskJeeves: Who do you call for a good electronic lexical database for English?
LING 5200, 2006

7. WordNet – Princeton (Miller 1985, Fellbaum 1998)
Limitations as a computational lexicon
Contains little syntactic information
Comlex has syntax but no sense distinctions
No explicit lists of participants
Sense distinctions very fine-grained,
Definitions often vague
Causes problems with creating training data for supervised Machine Learning – SENSEVAL2
Verbs > 16 senses (including call)
Inter-annotator Agreement ITA 71%,
Automatic Word Sense Disambiguation, WSD 63%
First big problem. I’m going to tell you how I’m solving this
LING 5200, 2006
Dang & Palmer, SIGLEX02

8. WordNet – call, 28 senses
1. name, call -- (assign a specified, proper name to;
"They named their son David"; …)
-> LABEL
2. call, telephone, call up, phone, ring -- (get or try to get into communication (with someone) by telephone;
"I tried to call you all night"; …)
->TELECOMMUNICATE
3. call -- (ascribe a quality to or give a name of a common noun that reflects a quality;
"He called me a bastard"; …)
4. call, send for -- (order, request, or command to come;
"She was called into the director's office"; "Call the police!") -> ORDER
LING 5200, 2006

9. WordNet: - call, 28 senses WN2 , WN13,WN28 WN15 WN26
WN3 WN WN4 WN 7 WN8 WN9
WN1 WN22
WN WN25
WN18 WN27
WN5 WN 16 WN6 WN23
WN12
WN17 , WN WN10, WN14, WN21, WN24
Most depressed I’ve ever been professionally was when I realized how many senses English verbs have - how
Overloaded.
LING 5200, 2006

10. WordNet: - call, 28 senses, Senseval2 groups, ITA 82%, WSD 70%
WN5, WN16,WN WN15 WN26
WN3 WN WN4 WN 7 WN8 WN9
WN1 WN22
WN WN25
WN18 WN27
WN2 WN 13 WN6 WN23
WN28
WN17 , WN WN10, WN14, WN21, WN24,
Loud cry
Bird or animal cry
Request
Label
Call a loan/bond
Challenge
Visit
Phone/radio
Bid
LING 5200, 2006

11. Filtering out “call the police”
Different senses,
- different syntax,
- different kinds of participants,
- different types of propositions.
call(you,movie,what) ≠ call(you,police)
you movie what you police
LING 5200, 2006

12. Proposition Bank: From Sentences to Propositions (Predicates!)
Powell met Zhu Rongji
debate
consult
join
wrestle
battle
Proposition: meet(Powell, Zhu Rongji)
Finding the relations between entities is the primary goal. These are often conveyed by verbs. One verb can convey the same information using many different sentence forms. Knowing what type of verb it is allows you to predict the sentence forms it can occur in. Then all of these sentence forms can be mapped onto the same underlying representation. We call this underlying representation the proposition. Fortunately, language reuses the same sets of sentence forms for entire families of verbs. We can generalize our mapping from a set of forms to the underlying proposition for one verb to other verbs in the family. These can be in the same semantic class, they can also be in different semantic classes. Determining membership in a family will determine the associated sentence forms for that verb, and vice versa, i.e. finding a verb occurring in particular sentence forms will determine its class membership.
Powell and Zhu Rongji met
Powell met with Zhu Rongji
Powell and Zhu Rongji had a meeting
meet(Somebody1, Somebody2)
. . .
When Powell met Zhu Rongji on Thursday they discussed the return of the spy plane.
meet(Powell, Zhu) discuss([Powell, Zhu], return(X, plane))
LING 5200, 2006

13. Semantic role labels: break (agent(Marie), patient(LCD-projector))
Marie broke the LCD projector.
break (agent(Marie), patient(LCD-projector))
cause(agent(Marie),
change-of-state(LCD-projector))
(broken(LCD-projector))
Filmore, 68
Jackendoff, 72
agent(A) -> intentional(A), sentient(A),
causer(A), affector(A)
patient(P) -> affected(P), change(P),…
Dowty, 91
LING 5200, 2006

14. Capturing semantic roles*
SUBJ
Richard broke [ ARG1 the laser pointer.]
[ARG1 The windows] were broken by the hurricane.
[ARG1 The vase] broke into pieces when it toppled over.
SUBJ
SUBJ
*See also Framenet,
LING 5200, 2006

15. Frame File example: give –
Roles:
Arg0: giver
Arg1: thing given
Arg2: entity given to
Example: double object
The executives gave the chefs a standing ovation.
Arg0: The executives
REL: gave
Arg2: the chefs
Arg1: a standing ovation
LING 5200, 2006

16. Annotation procedure
PTB II - Extraction of all sentences with given verb
Create Frame File for that verb Paul Kingsbury
(3100+ lemmas, 4400 framesets,120K predicates)
Over 300 created automatically via VerbNet
First pass: Automatic tagging (Joseph Rosenzweig)
Second pass: Double blind hand correction
84% ITA, 91% Kappa Paul Kingsbury
Tagging tool highlights discrepancies Scott Cotton
Third pass: Solomonization (adjudication)
Betsy Klipple, Olga Babko-Malaya
LING 5200, 2006

17. NomBank Frame File example: gift (nominalizations, noun predicates, partitives, etc.
Roles:
Arg0: giver
Arg1: thing given
Arg2: entity given to
Example: double object
Nancy’s gift from her cousin was a complete surprise.
Arg0: her cousin
REL: gave
Arg2: Nancy
Arg1: gift
LING 5200, 2006

18. Trends in Argument Numbering
Arg0 = proto-typical agent (Dowty)
Arg1 = proto-typical patient
Arg2 = indirect object / benefactive / instrument / attribute / end state
Arg3 = start point / benefactive / instrument / attribute
Arg4 = end point
LING 5200, 2006

19. Additional tags - (arguments o adjuncts?)
Variety of ArgM’s (Arg#>4):
TMP - when?
LOC - where at?
DIR - where to?
MNR - how?
PRP -why?
REC - himself, themselves, each other
PRD -this argument refers to or modifies another
ADV –others
LING 5200, 2006

20. Inflection, etc. Verbs also marked for tense/aspect
Passive/Active
Perfect/Progressive
Third singular (is has does was)
Present/Past/Future
Infinitives/Participles/Gerunds/Finites
Modals and negations marked as ArgMs for convenience
LING 5200, 2006

21. Word Senses in PropBank
Orders to ignore word sense not feasible for 700+ verbs
Mary left the room
Mary left her daughter-in-law her pearls in her will
Frameset leave.01 "move away from":
Arg0: entity leaving
Arg1: place left
Frameset leave.02 "give":
Arg0: giver
Arg1: thing given
Arg2: beneficiary
How do these relate to traditional word senses in WordNet?
LING 5200, 2006

22. WordNet: - call, 28 senses, groups
WN5, WN16,WN WN15 WN26
WN3 WN WN4 WN 7 WN8 WN9
WN1 WN22
WN WN25
WN18 WN27
WN2 WN 13 WN6 WN23
WN28
WN17 , WN WN10, WN14, WN21, WN24,
Loud cry
Bird or animal cry
Request
Label
Call a loan/bond
Challenge
Visit
Phone/radio
Bid
LING 5200, 2006

23. Overlap with PropBank Framesets
WN5, WN16,WN WN15 WN26
WN3 WN WN4 WN 7 WN8 WN9
WN1 WN22
WN WN25
WN18 WN27
WN2 WN 13 WN6 WN23
WN28
WN17 , WN WN10, WN14, WN21, WN24,
Loud cry
Bird or animal cry
Request
Label
Call a loan/bond
Challenge
Visit
Phone/radio
Bid
LING 5200, 2006

24. Overlap between Senseval2 Groups and Framesets – 95%
WN1 WN WN3 WN4
WN6 WN7 WN WN5 WN 9 WN10
WN11 WN12 WN WN 14
WN WN20
develop
LING 5200, 2006

25. Sense Hierarchy (Palmer, et al, SNLU04 - NAACL04)
PropBank Framesets – ITA >90%
coarse grained distinctions
20 Senseval2 verbs w/ > 1 Frameset
Maxent WSD system, 73.5% baseline, 90% accuracy
Sense Groups (Senseval-2) - ITA 82% (up to 90% ITA)
Intermediate level – 71% -> 74%
WordNet – ITA 71%
fine grained distinctions, 60.2% -> 66%
We have been investigating whether or not the sense groups developed for Senseval-2 can provide an intermediate level of hierarchy in between the PropBank Rolesets and the WN 1.7 senses. Our preliminary results show that 95% of the verb instances map directly from sense groups to Rolesets, with each Roleset typically corresponding to two or more sense groups.
LING 5200, 2006

26. Limitations to PropBank
Args2-4 seriously overloaded, poor performance
VerbNet and FrameNet both provide more fine-grained role labels
WSJ too domain specific, too financial, need broader coverage genres for more general annotation
Additional Brown corpus annotation, also GALE data
FrameNet has selected instances from BNC
LING 5200, 2006

27. Improving generalization
More data?
Can we merge FrameNet and PropBank data?,
What about new words and new usages of old words?
General purpose class-based lexicons for unseen words and new usages?
VerbNet, but limitations of VerbNet
Semantic classes for backoff?
WordNet hypernyms; WSD example
lexical sets (Patrick Hanks)
verb dependencies - DIRT, (Dekang Lin), very noisy
We’re still a long way from events, inference, etc.
LING 5200, 2006

28. FrameNet: Telling.inform
Time
In 2002,
Speaker
the U.S. State Department
Target
INFORMED
Addressee
North Korea
Message
that the U.S. was aware of this program , and regards it as a violation of Pyongyang's nonproliferation commitments
LING 5200, 2006

29. FrameNet/PropBank:Telling.inform
Time
ArgM-TMP
In 2002,
Speaker –
Arg0
(Informer)
the U.S. State Department
Target –
REL
INFORMED
Addressee –
Arg1 (informed)
North Korea
Message –
Arg2
(information)
that the U.S. was aware of this program , and regards it as a violation of Pyongyang's nonproliferation commitments
LING 5200, 2006

30. Frames File: give w/ VerbNet PropBank instances mapped to VerbNet
Roles:
Arg0: giver
Arg1: thing given
Arg2: entity given to
Example: double object
The executives gave the chefs a standing ovation.
Arg0: Agent The executives
REL: gave
Arg2: Recipient the chefs
Arg1: Theme a standing ovation
LING 5200, 2006

31. OntoNote Additions The founder of Pakistan’s nuclear department
Arg0:
Arg1:
The
founder of
Pakistan’s
nuclear department
Abdul Qadeer Khan
has
admitted he
transferred
nuclear technology to
Iran,
Libya,
and
North Korea
Founder
Nation
Agency
Person
Acknowledge
Transfer
Know-how
Department
Arg1: NP PP VP S
SBAR
Admit
Arg0:
Arg1:
Transfer
Arg2:
PropBank adds shallow semantics to the parse trees. The only semantics captured in PropBank is the verbs and their arguments.
For every verb (“admitted” and “transferred”), its arguments are marked (as strings, not as terms in a formal system, such as an ontology).
NYU will add argument structure for nouns (e.g., “founder”), but that is not yet integrated with PropBank.
Note that there is
no entity structure nor types.
No co-reference
No formal terms, but just strings
Technology
Arg1:
OntoBank adds
Co-reference
Word Sense Resolution into Predicates
Entity types and predicate frames connected to nodes in ontology
LING 5200, 2006