Presentation on theme: "The Extended Cohn-Kanade Dataset(CK+):A complete dataset for action unit and emotion-specified expression Author：Patrick Lucey, Jeffrey F. Cohn, Takeo."— Presentation transcript:
1The Extended Cohn-Kanade Dataset(CK+):A complete dataset for action unit and emotion-specified expressionAuthor：Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara AmbadarConference on Computer Vision and Pattern Recognition 2010Speaker：Liu, Yi-Hsien
2Outline Introduction The CK+ Dataset Emotion Labels Baseline System ExperimentsConclusion
3Introduction In 2000, the Cohn-Kanade (CK) database was released Automatically detecting facial expressions has become an increasingly important research area1.目的是為了促進對於自動偵測個人臉部表情的研究，自發表之後CK資料庫就成為對於演算法的發展與評估最被廣泛使用的資料庫之一2.自動偵測臉部表情涉及了computer vision, machine learning and behavioral sciences 而且可被用在像是 security, human-computer-interaction, driver safety, and health-care 等等應用上
4Introduction(Cont.)The CK database contains 486 sequences across 97 subjects.Each of the sequences contains images from onset (neutral frame) to peak expression (last frame).The peak frame was reliably FACS(Facial Action Coding System ) coded for facial action units (AUs).Facial Action Coding System (FACS)
5Introduction(Cont.)Facial Action Coding System (FACS) is a system to taxonomize human facial movements by their appearance on the face臉部行為編碼系統，是藉由對臉部的外觀進行人類臉部動作分類的系統
6Introduction(Cont.)While AU codes are well validated, emotion labels are notThe lack of a common performance metric against which to evaluate new algorithmsStandard protocols for common databases have not emerged在CK發表後的十年間，CK資料庫常常被拿來使用，但也因此出現了三個問題所以emotion labels常常被用來指定成某個另外的情緒，而不是他原本代表的情緒缺少跟新演算法評估的基準
7The CK+ DatasetParticipants were 18 to 50 years of age, 69% female, 81% Euro-American, 13% Afro-American, and 6% other groupsImage sequences for frontal views and 30-degree views were digitized into either 640x490 or 640x480 pixel arrays with 8- bit gray-scale or 24-bit color values.
8The CK+ Dataset(Cont.)For the CK+ distribution, they have augmented the dataset further to include 593 sequences from 123 subjects (an additional 107 (22%) sequences and 26 (27%) subjects).For the 593 posed sequences, full FACS coding of peak frames is provided.
9Emotion LabelsThey included all image data from the pool of 593 sequences that had a nominal emotion label based on the subject’s impression of each of the 7 basic emotion categories: Anger, Contempt, Disgust, Fear, Happy, Sadness and Surprise.如果就這樣把這些標籤當成ground truth是很不可靠的 ，直接這樣進行訓練之類的動作可能會發生錯誤
10Emotion Labels(Cont.)Compared the FACS codes with the Emotion Prediction Table from the FACSAfter the first pass, a more loose comparison was performedThe third step involved perceptual judgment of whether or not the expression resembled the target emotion category.所以，他們根據FACS來為這些情緒標記，實行步驟就是這三個步驟情緒預測表格列出除了輕視(Contempt)以外每種情緒的原型和主要變化的臉部配置，也就是AU的組合，如果某片段滿足某情緒原型和主要變化，就暫時將這片段分類到那情緒如果有個片段包含某個不屬於此情緒原型和變型的AU，就要判斷這片段到底屬不屬於這情緒，判斷標準就像這表格，下面的圖是例子3. 需要第三個步驟是因為FACS碼只敘述最後一張的表情而沒有顧及到在形成最後一張的表情時所需要的臉部變化，也就是說，要決定某個表情是否為某情緒的表現是需要直接從頭看到尾
14Baseline System(Cont.) Active Appearance Models (AAMs)The shape s of an AAM is described by a 2D triangulated mesh.In particular, the coordinates of the mesh vertices define the shape s = [x1; y1; x2; y2; …. ; xn; yn]
15Baseline System(Cont.) SPTS：The similarity normalized shape, refers to the 68 vertex points for both the x- and y- coordinates, resulting in a raw 136 dimensional feature vectorCAPP： The canonical normalized appearance, refers to where all the shape variation has been normalized with respect to the base shape上面是SPTS下面是CAPP
16Baseline System(Cont.) SVMs(Support Vector Machines) attempt to find the hyper plane that maximizes the margin between positive and negative observations for a specified class.實線Support Hyper-planes，這篇論文使用的是二分法，例如在分類時分為Angry 跟 not Angry，或Happy 跟 Not Happy
17Experiments Emotion detection. To maximize the amount of training and testing data, they believe the use of a leave-one-subject-out cross-validation configuration should be used.
21ConclusionIn this paper, they try to address those three issues by presenting the Extended Cohn-Kanade (CK+) databaseAdded another 107 sequences as well as another 26 subjects.The peak expression for each sequence is fully FACS coded and emotion labels have been revised and validated
22Conclusion(Cont.)Propose the use of a leave-one-out subject cross-validation strategy for evaluating performancePresent baseline results on this using our Active Appearance Model (AAM)/support vector machine (SVM) system.