Presentation is loading. Please wait.

Presentation is loading. Please wait.

So nification of the facial expressio ns New Interaction Techniques Department of Computer Science University of Tampere 27 February 2003 Guo Zhiguo Ioulia.

Published by Modified over 8 years ago

Similar presentations

Presentation on theme: "So nification of the facial expressio ns New Interaction Techniques Department of Computer Science University of Tampere 27 February 2003 Guo Zhiguo Ioulia."— Presentation transcript:

1

2 So nification of the facial expressio ns New Interaction Techniques Department of Computer Science University of Tampere 27 February 2003 Guo Zhiguo Ioulia Guizatdinova Zhiguo.Guo@uta.fi ioulia_gizatdinova@pochtamt.ruZhiguo.Guo@uta.fiioulia_gizatdinova@pochtamt.ru

3 Contents Introduction to the problem Background & related research Objectives Methods Resources & experimental setup Preliminary results Prospectives p 02_23 27.02.03 Sonification of the facial expressions

4 Sonification is the use of non-speech audio to convey information more specifically sonification is the transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation [1] p 03_23 27.02.03 Sonification of the facial expressions Introduction to the problem

5 The main question for sonification of the facial expression is: How to convey the emotional expressions/facial features/ of some person presented like any visual imaging into the sounds (e.g. earcons, music chords) or whatever ? Introduction to the problem p 04_23 27.02.03 Sonification of the facial expressions

6 The sonification system for facial expressions should allow the transformation of the visual information (attributes specifically describing emotional expression/state) into sound patterns This kind of sounds (“eARmoticons”) should evoke similar emotional feelings of the listener that is, to provide an emotional access through “audio snapshot” when visual perception is blocked or imaging is hidden for observer p 05_23 27.02.03 Sonification of the facial expressions Introduction to the problem

7 Some facial expressions can hardly be transformed by sound (e.g., congenital or non-classified features) Blind people have a wide experience with real environmental sounds, while the congenital blinded can have not any association to visual facial imaging; only tactile memory of some facial features can be insufficient or non-adequate p 06_23 27.02.03 Sonification of the facial expressions Background & Related Research

8 There is not signs’ system/auditory or tactile/specifically & briefly describing facial features/expressions. Only some general emotional states are presented like artificial set of static visual/tactile symbols [2, 11]. p 07_23 27.02.03 Sonification of the facial expressions Background & Related Research Happiness : ) Sadness : ( Frustration : />:-( Madness

9 There have been a lot of approaches to data sonification of like math functions, simple graphics, diagrams, … p 08_23 27.02.03 Sonification of the facial expressions http://www.visualprosthesis.com/javoice.htm The vOICe Java Applet Background & Related Research

10 p 09_23 27.02.03 Sonification of the facial expressions Relevantly, there are a lot of earcons used in MS Windows and other software, sound signaling for special equipment and application… sound Background & Related Research

11 Objectives The goal of this project is designing and usability investigation of the sonification system for attributes specifically describing facial expression to provide an emotional access through eARmoticons when visual perception is blocked or imaging is hidden for observer p 10_23 27.02.03 Sonification of the facial expressions

12 Schedule Project’s activityWeeksExpected results Overview on the topic Definition of the interaction scenarios & research methods 1 Sonification & visual attributes Scenarios Design of software prototype Design & collection of sound files 1 Software prototype Sound database Classification of the facial expressions by sound Revision of software 1 Sonification System for facial attributes Revised software Usability exploration of the new interaction techniques 2Data Data analysis Report writing 1212 Project paper p 11_23 27.02.03 Sonification of the facial expressions

13 Research Methods constructive research to design and simulate the new techniques, software tools and doing usability evaluation of novel sound objects – eARmoticons empirical research is based on crossmodal information transfer and perception of intermodal events p 12_23 27.02.03 Sonification of the facial expressions

14 Preliminary Modelling p 13_23 27.02.03 Sonification of the facial expressions Facial expression (image, picture, video, etc) sounds Video processing Facial expression-to-sound mapping transformation system Recognition of the sound patterns to facial expression As a whole Facial landmarks Sound Database

15 Resources & Experimental Setup p 14_23 27.02.03 Sonification of the facial expressions Subjects (people with normal vision) Stimuli – eARmoticons – earcons specifically & briefly describing facial features/expressions FACS – Facial Action Coding System [ 7, 8] a prototype of the sonification system for facial expression transformation

16 Resources & Experimental Setup p 15_23 27.02.03 Sonification of the facial expressions Experimental procedure

17 Resources & Experimental Setup p 16_23 27.02.03 Sonification of the facial expressions Data analysis: reaction time accuracy = error number

18 Preliminary results p 17_23 27.02.03 Sonification of the facial expressions Classification of the emotions by Ekman & Friesen (1975) http://www.paulekman.com/ surprisehappiness anger disgust sadnessfear NEUTRAL POSITIVE NEGATIVE

19 p 18_23 27.02.03 Sonification of the facial expressions Inner brow raiser Outer brow raiser Brow lowerer Upper lid raiser Cheek raiser Lid tightener Lid droop Slit Eyes closed Squint Facial landmarks.Eye & eyebrow Preliminary results

20 p 19_23 27.02.03 Sonification of the facial expressions Lip pressor Lip corner puller Lips part Jaw drop Mouth stretch Lip funneler Lip suck Cheek puffer Dimpler Lip corner depressor Lower lip depressor Lip puckerer Lip stretcher Lip tightener Chin raiser Facial landmark. Mouth Preliminary results

21 p 20_23 27.02.03 Sonification of the facial expressions Nose wrinkler Upperupper lip raiser Nasolabial deepener Facial landmark. Nose Preliminary results

22 The sonification system of facial expressions which includes new sound mapping techniques to display facial features and expressions Prospectives p 21_23 27.02.03 Sonification of the facial expressions

23 1.http://www.icad.org/websiteV2.0/Resources/glossary.html#Shttp://www.icad.org/websiteV2.0/Resources/glossary.html#S 2.http://www.tsbvi.edu/Education/vmi/tactile_symbols.htm#EMOTIONShttp://www.tsbvi.edu/Education/vmi/tactile_symbols.htm#EMOTIONS 3.http://www.mcsdtechcenter.org/multimedia/sounds-samp.htmhttp://www.mcsdtechcenter.org/multimedia/sounds-samp.htm 4.http://www.matthawkins.co.uk/html/sound_fx_index.htmhttp://www.matthawkins.co.uk/html/sound_fx_index.htm 5.http://kin-dza-dza.etel.ru/ http://www.soundobject.org/http://kin-dza-dza.etel.ru/http://www.soundobject.org/ 6.http://viswiz.gmd.de/~eckel/publications/eckel98a/eckel98a.htmlhttp://viswiz.gmd.de/~eckel/publications/eckel98a/eckel98a.html 7.http://www-2.cs.cmu.edu/afs/cs/project/face/www/facs.htmhttp://www-2.cs.cmu.edu/afs/cs/project/face/www/facs.htm 8.http://www.paulekman.com/http://www.paulekman.com/ 9.http://www.ifeelpixel.com/download/whatsnew.htmhttp://www.ifeelpixel.com/download/whatsnew.htm 10.http://www.symbols.net/emoticons.htmhttp://www.symbols.net/emoticons.htm 11.http://www.fingertek.com/smilies.htmhttp://www.fingertek.com/smilies.htm 12.http://www-unix.mcs.anl.gov/~kaper/Sonification/http://www-unix.mcs.anl.gov/~kaper/Sonification/ 13.http://www-pablo.cs.uiuc.edu/Project/Pablo/PabloSonification.htmhttp://www-pablo.cs.uiuc.edu/Project/Pablo/PabloSonification.htm references p 22_23 27.02.03 Sonification of the facial expressions

24 27.02.03 Sonification of the facial expressions Thank yo u for yo ur attentio n!

Download ppt "So nification of the facial expressio ns New Interaction Techniques Department of Computer Science University of Tampere 27 February 2003 Guo Zhiguo Ioulia."

Similar presentations

A New Pan-Cultural Facial Expression of Emotion

A New Pan-Cultural Facial Expression of Emotion
Instructors Edition. Psychology in Action, 9 th ed. By Dr. Karen Huffman Facial Characteristics Jim Matiya Psychology in Action 9 th Edition Karen Huffman.

Instructors Edition. Psychology in Action, 9 th ed. By Dr. Karen Huffman Facial Characteristics Jim Matiya Psychology in Action 9 th Edition Karen Huffman.
Created By: Lauren Snyder, Juliana Gerard, Dom Williams, and Ryan Holsopple.

Created By: Lauren Snyder, Juliana Gerard, Dom Williams, and Ryan Holsopple.
UNIVERSAL DESIGN FOR LEARNING N ANCI L EE. Introduction to Universal Design Universal Design was a term coined by architect Ronald Mace. Design Principles.

UNIVERSAL DESIGN FOR LEARNING N ANCI L EE. Introduction to Universal Design Universal Design was a term coined by architect Ronald Mace. Design Principles.
Module 2: Assessment in Creative Arts © 2006 Curriculum K-12 Directorate, NSW Department of Education and Training.

Module 2: Assessment in Creative Arts © 2006 Curriculum K-12 Directorate, NSW Department of Education and Training.
Facial expression as an input annotation modality for affective speech-to-speech translation Éva Székely, Zeeshan Ahmed, Ingmar Steiner, Julie Carson-Berndsen.

Facial expression as an input annotation modality for affective speech-to-speech translation Éva Székely, Zeeshan Ahmed, Ingmar Steiner, Julie Carson-Berndsen.
Adviser：Ming-Yuan Shieh Student：shun-te chuang SN：M

Adviser：Ming-Yuan Shieh Student：shun-te chuang SN：M
As members of a community we will: Think about our environment and landmarks through comparing and contrasting them with New Zealand. We will be investigating.

As members of a community we will: Think about our environment and landmarks through comparing and contrasting them with New Zealand. We will be investigating.
 INTRODUCTION  STEPS OF GESTURE RECOGNITION  TRACKING TECHNOLOGIES  SPEECH WITH GESTURE  APPLICATIONS.

 INTRODUCTION  STEPS OF GESTURE RECOGNITION  TRACKING TECHNOLOGIES  SPEECH WITH GESTURE  APPLICATIONS.
Introduction To System Analysis and Design

Introduction To System Analysis and Design
1 Affective Learning with an EEG Approach Xiaowei Li School of Information Science and Engineering, Lanzhou University, Lanzhou, China

1 Affective Learning with an EEG Approach Xiaowei Li School of Information Science and Engineering, Lanzhou University, Lanzhou, China
UPM, Faculty of Computer Science & IT, 2009 1 A robust automated attendance system using face recognition techniques PhD proposal; May 2009 Gawed Nagi.

UPM, Faculty of Computer Science & IT, A robust automated attendance system using face recognition techniques PhD proposal; May 2009 Gawed Nagi.
Mark Frank - RU HMS Symposium 2003 Deception, digitization, & counter-terror. Mark G. Frank School of Communication, Information, & Library Studies UCSF.

Mark Frank - RU HMS Symposium 2003 Deception, digitization, & counter-terror. Mark G. Frank School of Communication, Information, & Library Studies UCSF.
Cyclic input of characters through a single digital button without visual feedback Yang Xiaoqing New Interaction Techniques Dept.

Cyclic input of characters through a single digital button without visual feedback Yang Xiaoqing New Interaction Techniques Dept.
Automatic Pose Estimation of 3D Facial Models Yi Sun and Lijun Yin Department of Computer Science State University of New York at Binghamton Binghamton,

Automatic Pose Estimation of 3D Facial Models Yi Sun and Lijun Yin Department of Computer Science State University of New York at Binghamton Binghamton,
Emotion, Intuition and “Hot” Thinking I.Psychological Perspectives on Emotion  the role of thinking in emotion II.Studying Emotion III.Blink: Thin Slicing.

Emotion, Intuition and “Hot” Thinking I.Psychological Perspectives on Emotion  the role of thinking in emotion II.Studying Emotion III.Blink: Thin Slicing.
Evaluating Non-Visual Feedback Cues for Touch Input Device Selina Sharmin Project for the course New Interaction Techniques Spring.

Evaluating Non-Visual Feedback Cues for Touch Input Device Selina Sharmin Project for the course New Interaction Techniques Spring.
TAUCHI – Tampere Unit for Computer-Human Interaction Automated recognition of facial expressi ns and identity 2003 UCIT Progress Report Ioulia Guizatdinova.

TAUCHI – Tampere Unit for Computer-Human Interaction Automated recognition of facial expressi ns and identity 2003 UCIT Progress Report Ioulia Guizatdinova.
25 February 20031 New Interaction Techniques Target Selection Under Time Pressure Conditions New Interaction Techniques Department of Computer and Information.

25 February New Interaction Techniques Target Selection Under Time Pressure Conditions New Interaction Techniques Department of Computer and Information.
FACE RECOGNITION BY: TEAM 1 BILL BAKER NADINE BROWN RICK HENNINGS SHOBHANA MISRA SAURABH PETHE.

FACE RECOGNITION BY: TEAM 1 BILL BAKER NADINE BROWN RICK HENNINGS SHOBHANA MISRA SAURABH PETHE.

Similar presentations

Ads by Google