1. Intelligent MultiMedia Storytelling System (IMSS) - Automatic Generation of Animation From Natural Language Input
By Eunice Ma
Supervisor: Prof. Paul Mc Kevitt
School of Computing and Intelligent Systems
Faculty of Informatics
University of Ulster, Magee

2. Project Goals
To interpret natural language story and movie (drama) script input and to extract concepts from the input
To generate 3D animation and virtual world automatically, with speech and non-speech audio
To integrate the above components to form an intelligent multimedia storytelling system for presenting multimodal stories

3. Previous Research Schank’s Conceptual Dependency (CD) Theory
Primitives
SAM
Automatic Text-to-Graphics Systems
WIP/COMET/APT
WordsEye
‘Micons’ and CD-based language animation
O Nuallain and Smith’s Spoken Image (SI) system

4. Previous Research (continued)
Multimedia Storytelling
Larsen’s Interactive Storytelling
AesopWorld
Multimedia Interactive Systems
AIMI
AlFresco
KidsRoom
Animated Intelligent Agents
Cassell’s REA and SAM

5. Software Analysis Natural language processing tools 3D Animation Tools
Gate
CPK NLP Suite
WordNet
3D Animation Tools
Poser 4.0
3D Studio Max
VRML 2.0/97
Microsoft DirectX
The Actors – Using Conversational Agents
COLLAGEN
Microsoft Agent
BEAT

6. Context Diagram IMSS Story in natural language Speech (dialogue)
Movie/drama script
IMSS
animation
Tailored menu for script input
non-speech audio
I/O Media of IMSS

7. General Architecture Natural language story Script writer
Script parser
Prefabricated objects
Basic vocabulary
Natural Language Parser
Text To Speech
Sound effects
Concept database
mapping
3D authoring tools
Animation generation
Primitives library
Code combination
3D world with audio

8. Data Flow Diagram Primitives library Natural language parser
labelled script
Animation director
VRML without sound nodes
Scene&Actor descriptions
Media coordination
TTS
Synthesized animation
Script parser
dialogues
script
Non-speech audio
Sound effect driver
script
Script writer
story
Music library

9. DFD of Animation Director
Primitives library
Actor managers
marked script
Script distributor
World builder
VRML without sound nodes
Scene generator
VRML codes
Props provider
coordinator

10. Project Schedule

11. Conclusions
The objectives of IMSS meet the challenging problems in language animation.
Mapping language primitives with visual primitives
Sophisticated spatial relations between spatial cognition and prepositions in English
Visualization requires a gigantic ‘common senses’ knowledge base
Representing stories by temporal multimedia requires high coordination
Prospective practical applications
Children education
Multimedia presentation
Movie/drama production
Script writing
Computer games, Virtual Reality