Cursive A novel interaction technique for controlling expressive avatar gesture Francesca Barrientos and John Canny UC Berkeley UIST 12 November 2001, Orlando Florida

Cursive212 November 2001 Avatar worlds Verbal communication –Chat –Spoken Nonverbal communication –Mechanisms Change image Play animation –Gestures have continuous qualities

Cursive312 November 2001 Current techniques Specialized widgets Vlnet expression panel 1.Select 2.Modulate 3.Trigger 3 step process ComicChat EmotionWheel Select and modulate Hunt and click diverts attention Avatar agency BodyChat Driven by chat text Lacks direct control

Cursive412 November 2001 Expressive and Controllable Want gesture to be expressive –Range of gestures –Vary quality of gesture Single step - modeless Enable co-occuring verbal and nonverbal communication Solution: pen gesture input

Cursive512 November 2001 Mapping pen to avatar gesture Pen gesture carries symbolic and qualitative data Human/avatar gesture can be symbolic and qualitative Pen gesture simultaneously selects avatar gesture and modulates multiple expressive qualities DataPenAvatar SymbolicLetter of alphabetGesture type Qualitative SizeMagnitude DurationSpeed PressureEffort

Cursive612 November 2001 Conceptual model Library Generated offline Input User writes letter in GUI Generation Gesture selection and modulation Animation Gesture performed GUI library Gesture generator Avatar animator w- s- r-

Cursive712 November 2001 Example: modulating size Writing letter ‘s’ triggers animation Single gesture Modulation along single parameter Writing size mapped to gesture’s size

Cursive812 November 2001 Extraction Character recognizer Feature extractor w- s- r- Speed.5 Size.3 Inputs modulate gesture Style parameters Gesture Type w- s- r- Type library GUI

Cursive912 November 2001 Modulation through multilinear interpolation Joint trajectory I –Rotation angles over time Gesture type β –Semantic category –Set of 2 n prototypes - G –n style parameters –Prototypes represent extremal trajectories Gesture instance I β (u,v ) –Vector of joint trajectories –Multilinear interpolation on type produces instance Speed u Size v k l GβGβ I β (0,1) I β (0,0) I β (1,1) I β (1,0) Iβ(l,k)Iβ(l,k) I β (0,k)I β (1,k)

Cursive1012 November 2001 Speed parameter special case Interpolation along curves of different durations Time dilation step –Determine duration for interpolated curve –Choose sample rate on interpolated curve –Compress slower curve - sample at proportionately slower rate –Sample faster trajectory at proportionately faster rate Angle θ t

Cursive1112 November 2001 Architecture Gesture producer Animator Animation commands VRML script Gesture commands - w- s- r-

Cursive1212 November 2001 Communication User logs into Vworld server Other visitors receive notification Other visitors request and download avatar copy Animator opens socket connection to producer Generator sends gesture commands via socket Web server VWorld server Cursive communicates with any copies of user’s avatar user hostvisitor host login notification request avatar open socket sends commands Download avatar

Cursive1312 November 2001 Summary Current techniques lack expressive control of avatar gesture Apply pen gesture to avatar gesture Modulate recorded gestures using multilinear interpolation Augmented VRML avatars can drop into multi-user virtual worlds

Cursive1412 November 2001 Conclusion Expressive avatar gesture Single action invokes and modulates gesture Multidimensional modulation Enables co-occuring verbal and non-verbal communication Pen gestures control Thank Erin.