1. Micro expression Detection using Strain Patterns - Sridhar Godavarthy Based on V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42

2. Microexpressions What are microexpressions? –Subtle movements of the human body –Quick enough to be completed within the blink of an eye –Not large enough to be called a micro expression

3. Microexpressions Contd… Examples: Raising an eyebrow Shrugging of shoulders Pout of lips Fast blinking of eye Non Examples: Talking Smiling Laughing Anger

4. How do we go about it? Majority of the work “referenced” from : V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42 However, to keep things clear, we discuss each step in detail.

5. Feature Based –Need to identify features – Difficult! –Features may be ill defined( when camouflaged) –Usually requires manual intervention –Produces a sparse motion field –Produce Good correspondence in large motion Optical Flow based –Fully automated –Dense Motion field. –Requires constant illumination First Step – Obtaining Motion Field

6. 3D Strain –Ideal –No high speed equipment available to capture range images 2D Strain –Well – not much of a choice –Authors could use existing data. Second Step – Strain Computation Type

7. Finite Element Method –Forward modeling when Dirichlet condition is satisfied –Good at handling irregular shapes –Computationally expensive –This method is an approximation to the solution Finite Difference Method –Strain, a tensor, can be expressed derivatives of the displacement vector –This can be approximated by a Finite Difference Method. –Very efficient when carried out on a regular grid. –This method is an approximation to the differential equation Strain Computation - methods

8. Finite Strain tensor Cauchy tensor Finite Difference Method

9. Video Video is a collection of individual images also known as frames In reality: spatial and temporal compression using properties of the scene. Any video can be decoded into a series of frames. 24/30 frames per second of video.

10. Video Coding The science of encoding a video in a manner such that –Minimum number of bits are used –Motion compensated prediction can be performed from a previous frame.

11. Motion Estimation Key technique in image sequence compression

12. Optical Flow Pattern of apparent motion of objects/surfaces/edges caused by relative motion between the observer(camera) and the scene

13. Optical Flow Pattern of apparent motion of objects/surfaces/edges caused by relative motion between the observer(camera) and the scene

14. Elasticity Different materials have different elasticity Elasticity can be modeled Known Calculate

15. Optical Strain Variation of displacement values obtained from optical flow –Calculated by taking the derivative of each pixel  Sobel operator (central difference)

16. Facial Strain What is Facial Strain? –Strain on soft tissue when expressions are made. –Anatomical method –Uses a pair of frames to measure deformation

17. Strain Measurement Finite Difference Method Compute spatial derivatives from discrete points. –Forward Difference Method –Central Difference Method –Richardson extrapolation

18. The Process Start Divide videos into Training and Testing Sequences Read Training Video Read Neutral Micro Exp Macro Exp Frames Decode Video Calculate OF & OS for Micro & Macro frames

19. The Process Contd… Read Test Video Read Neutral Frames Decode Video Calculate Optical Flow between Neutral frame and each frame OF within desired range? Reject Frame Calculate Optical Strain between Neutral frame and this frame OS within desired range? Reject Frame MICRO EXPRESSION Stop No Yes For Every other Frame

20. Program Output…

21. Program Output Contd…

22. RESULTS – Training & Test Frame: Neutral Frame: Micro Optical FlowNormalized optical flow

23. RESULTS - Test Frame: Neutral Frame: Micro Optical FlowNormalized optical flow

24. RESULTS - Test Frame: Neutral Frame: Macro Optical FlowNormalized optical flow FALSE POSITIVE

25. RESULTS - Test Frame: Neutral Frame: XYZ Optical FlowNormalized optical flow NOT DETECTED

26. Decisions and Consequences Use OF to reject highly improbable frames –Improves performance –Rejects frames with both Micro and Macro Exp. –Segment image into regions? Threshold for classification was set to be ⅔T.O.S < M.O.S < 5/3 T.O.S T.O.S = Training Optical Strain. M.O.S = Measured Optical Strain

27. Possible Options Use only Optical Flow Use only Optical Strain Vary the thresholds Use an alternative for Max(OF/OS)

28. Future work Do not compare all frames( Skip frames immediately after a positive) Identify only one positive in a sequential list of positives Segment images to get separate Micro and Macro expressions

29. What I could not do Display strain as an image. Segment image into regions and have Regions of Interest

30. V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42 Vasant Manohar, Matthew Shreve, Dmitry Goldgof and Sudeep Sarkar, "Finite Element Modeling of Facial Deformation in Videos for Computing Strain Pattern", International Conference on Pattern Recognition, Dec. 2008 Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R. Sullins, and Rupali Patil, "Imaging And Characterization Of Facial Strain In Long Video Sequences",xxxx Malcolm Gladwell,” Blink: The Power of Thinking Without Thinking”, Back Bay Books (April 3, 2007) References

31. Thank You! Sridhar Godavarthy Dept. Of Computer Science and Engineering University of South Florida sgodavar@cse.usf.edu About the Template Fire was considered the most fearful of weapons in Greek Mythology. This template – “Fireball” is a depiction of the might and power of fire, punching through ignorance and enlightening the audience

32. http://www.youtube.com/watch?v=mUdDhWfpqxg