Representation of Actions as an Interlingua Karin Kipper & Martha Palmer Presented in CIS630 by Sriram Venkatapathy

Framework Command Animation In Performed by Natural Language a Human Agent Command In Natural Language (“open a door”)

Primary Goal Command Animation In Performed by Natural Language a Human Agent Command In Natural Language (“open a door”) Bridge the Gap

Parametrized Action Representation (PAR) Used to bridge the gap between the command and the animation performed by the virtual human agent. PARs make explicit many details that are underspecified in the human language.

PARs as an Interlingua This detailed representation is well suited for an interlingua for MT applications because Animations of Actions – and therefore the PARs that control them – will be equivalent for the same actions described in different languages.

PARs as an interlingua – contd. Spanish instruction English instruction PARs Animation

PAR Action’s participants (agent and objects) Kinematic Properties like its path, manner and duration.

PAR – contd. Traditional state-space properties of actions such as applicability conditions and preparatory actions. Termination and Post assertions which determine when an action is concluded.

PAR ( for the action “contact”)

Hierarchy of actions Verbs can be represented in a lattice that allows semantically similar verbs, such as motion verbs or verbs of contact, to be closely associated. A common parent captures the properties that these verbs all share.

Verbs related to “contact”

Representation of “hammer” Inherits PAR of hit , and ultimately the PAR for contact. with “forceful” manner. and “hammer” as an instrument.

PAR – contd. PAR is intended to provide slots for information that is typically conveyed in modifiers or adjuncts. “John hit the ball” “John hit the ball with a bat” “John swung mightily and his bat hit the ball with a resounding sound”. All map to the same PAR schema.

Generating animations

Deriving the PAR schema Synchronous Tree-adjoining grammar is used for parsing natural language instructions and obtaining the predicate-argument dependencies. Assemble the corresponding schemas, and fill in the participants and modifiers to output the PAR schema.

Planner These schemas might be underspecified for actions like “enter”, “put” and hence don’t have enough information for the animation. So, a planner is used to Select the way (activity) “enter by walking/swimming” Determine preparatory actions “open the door” Decompose complex actions “put the glass on the table”

PAR as Interlingua PAR representation is a general template. It includes properties of the action that can occur linguistically either as the main verb or as adjuncts to the main verb phrase. Captures divergences such as, for verb-framed versus satellite-framed languages.

Verb-Framed vs Satellite-Framed Satellite-Framed (English) Manner -> Main verb. Motion (path or path + ground location) -> Satellite Ex. The bottle floated out. Verb-Framed (Spanish) Motion -> Main verb Manner -> Satellite Ex. La bottella salio flotanda. (the bottle exited floating)

Verb-Framed vs Satellite-Framed For the sentences, The bottle floated into the cave. (English) La bottella entro flotanda a la cueva (Spanish) (The bottled entered floating the cave)

Verb-Framed vs Satellite-Framed The PAR schemes don’t distinguish the representation for these sentences, because there is a single schema that includes both the manner and the path without specifying how they are realised linguistically. EN float/[par:motion,activity:float] into/[term:in(AG,OBJ] SP entrar/[par:motion,term:in(AG,OBJ) flotar/[activity:float]

Schema for the above sentences

Comparison with other work Considerably different from the approach outlined in Palmer et al. (1998) which discusses the use of Feature-based Tree-adjoining grammars. It was a transfer-based mechanism expressed in Synchronous TAG to capture divergences in VFL and SFL through the use of semantic features and links between the grammars.

Comparison with other work Similar to Lexical Conceptual Structures (LCS) approach. LCS allows the separation of the concepts of motion, direction, and manner of motion in the sentence “John swam across the lake”. (represented as GO,PATH,MANNER). This approach allows for a similar representation and the end result is the same.

Conclusions This work discusses a parameterized representation of actions grounded by needs of animations in a simulated environment. Generalizations based on action classes provide the basis for an interlingua approach that captures the semantics without committing to any language-dependent specification. The PAR schema incorporates in a single template both VFL and SFL languages.

Thank you.