Presentation is loading. Please wait.

Presentation is loading. Please wait.

Video Analysis Tool Box for Digital Video Forensics By Susinda Perera Department of Computer Science and Engineering, University of Moratuwa, Supervised.

Similar presentations


Presentation on theme: "Video Analysis Tool Box for Digital Video Forensics By Susinda Perera Department of Computer Science and Engineering, University of Moratuwa, Supervised."— Presentation transcript:

1 Video Analysis Tool Box for Digital Video Forensics By Susinda Perera Department of Computer Science and Engineering, University of Moratuwa, Supervised by Dr. Chathura De Silva PhD (NUS-Singapore), MEng (NTU-Singapore), BSc Eng.(Hons) (Moratuwa) Senior Lecturer Department of Computer Science and Engineering, University of Moratuwa,

2 Agenda Project Intro Implementation – Video Stream Analyzer – Video Stabilizer – Compressed domain video analysis Demo – Video Stream Analyzer – Video Stabilizer

3 Problem Statement Can we trust digital videos? – Are they real, computer generated or tampered Extract some wanted Information from video – Difficult due to unclearness of video

4 Can we trust digital videos? Figure 1 ‑ 1 : A still from controversial video aired on Channel 4 Source – YouTube

5 Can we trust digital videos? Figure 1 ‑ 2 : A still from eagle catching kid Source –

6 Unclear Videos

7 Solution Video Stream analysis tool – To ensure the authenticity and integrity Video Enhancement tool – To improve the visual quality

8 Market Analysis Video Stream Analyzing Tools – Elecard StreamEye Elecard StreamEye – Tektronix MPEG Software Tools Tektronix MPEG Software Tools – MPEG-2 Transport Stream packet analyser MPEG-2 Transport Stream packet analyser – TSReader TSReader – MTS4EA Elementary Stream Analyzer MTS4EA Elementary Stream Analyzer

9 Screenshots from Elecard StreamEye

10 Some Features Of Video Stream Analyzing Tools Navigation and display of media stream picture-by-picture (I, P, B). Display of the current frame. Display of the time, type, size and number of a current frame in a stream, decoding order and offset from the file beginning. Display of the bit rate (declared in the sequence header) and a calculated bit rate. Display of detailed information about macroblocks in MPEG-1 (ISO/IEC ), MPEG 2 (ISO/IEC ), MPEG-4 (ISO/IEC ) and AVC/H.264 (ISO/IEC ) video streams. Information about motion vectors Frame-accurate positioning. Display of the stream and gathering of statistics relating to the entire file.

11 What is missing in existing tools All high end tools are commercial ones, Expensive Almost all tools are designed for DVB purposes Lack of detailed analyzing features for video forensic Lack of textual representation

12 Video Forecsic /Enhancement Software Cognitech Ocean Systems dTective Salient Stills VideoFOCUS StarWitness Avid Technology, Inc. Intergraph Video Analyst TREC, Inc. Forevid MotionDSP Ikena Amped FIVE Kinesense

13

14 Features Of Video Enhancement Tools Video Stabilization Denoising Deblur Filters Detection Filters Enhancement Histogram Editor Segmentation Tracking Transform Zoom Velocity Reconstruction

15 Implementation of Video Stream Analyzer

16 Literature Review Market Analysis – Most of tools are for DVB, not for forensic purposes – Lack of textual output Possible resources [31- 35] – Video Player libraries / source codes – Decoder Libraries/Source Codes – MPEG2Event[33]is a possible candidate

17 Video Stream Analyzer Implementation – Based on MPEG2Event[33] library – Mpeg demuxer – ffmpeg for stream information – A simple thread sync model to get motion vectors – Quartztype library for video player implementation – C# inbuilt xml libraries for xml manipulation

18 What is MPEG stream MPEG Bit-Stream Structure (Source [1] )

19 GOP Structure and Display order GOP Structure and Transmission Order. (Source: [2])

20 Video Stream Analyzer https://code.google.com/p/mymscproject/source/browse/tr unk/Experiments/Mpeg2Event/Downloads/VideoAnalyzerSe tup.msi?r=174

21 Video Stream Analyzer Features – Free and open source – Textual output (xml) of analysis reports For further analysis – All most all the features in slide 10 – Motion vector display – Frame navigation – Bit stream index/ Display index view – User data display – Motion vector summary – xml output – Full video summary – xml output

22 Video Stream Analyzer -outputs

23 Motion Vectors Summary I 0 false true I false true I false true I

24 MacroBlocks Summary

25 Implementation of Video Enhancement tool

26 Video Enhancement Tool Objective – Enhance the visual quality – Video Stabilization – Noise removal – Color Correction

27 Video Stabilization Removing annoying shaky motion from videos helpful in identifying people, number plates, etc. from low-quality video cameras Three aspects – Inter frame motion estimation – Motion smoothing and compensation – Filling up the missing image areas. Main references [3], [4], [5]

28 Video Stabilization Inter Frame Motion Estimation Result of video stabilization. Top row: Original input sequence, middle row: stabilized sequence which still has missing image areas, and bottom row: stabilized and completed sequence. The grid is overlaid for better visualization. (Source [3])

29 a) Natural scene, b) when camera is moved, c) when image is shifted to compensate the motion. Handling camera motion

30 Motion Model

31 Motion Estimation

32 Stabilization

33 Motion Smoothing A stabilized motion path is obtained by removing undesired motion fluctuation. Assumed that the intentional motion in the video is usually slow and smooth Uses Gaussian kernel in most literatures – Applies Gaussian kernel to neighboring N frames Gaussian kernel + curve fitting methods

34 Motion Smoothing Let N t = {j|t-k<=j<=t+k} be the neighboring frames And I t is the frame at the origin Calculate the position of each neighboring frame I s, relative to frame I t using transform matrixes defined above ( lets say T s t ) Find the correcting transformation S from the original frame I t to the motion-compensated frame I’ t according to Where G is a Gaussian kernel of size k

35 Motion Smoothing The global transformation chain T defined over the original video frames I i, and the transformation from the original path to the smoothed path S. (Source [3])

36 Filling up missing image areas Not addressed here – Due to time limitations and complexity Some techniques used in research literature – Motion Inpainting [3] the local motion data in the known image areas is propagated into the missing image areas. The propagation starts at pixels on the boundary of the missing image area. Using motion values of neighboring known pixels – Use of dynamic programming [3],[6]

37 Noise Removal (Denoising Filters) Denoising Filter Categories [7] – Nonmotion compensated spatiotemporal – Motion compensated spatiotemporal – Nonmotion compensated temporal – Motion compensated temporal filters What is first? motion compensation or denoising? – Each have their procs and cons – Depends on the video References – [7], [8], [9], [10], [11]

38 Implementation of Video Enhancement Tool

39 Video Stabilization – Based on the algorithm described in [5] – + Improvements to reduce error propagation Makes a additional pass (slow) – + Improvements to support color video Noise Removal and color Correction – Based on openCV filters

40 A Compressed Domain Approach for Video Tamper Detection

41 Common Modes of Tampering Classification – Spatial, Temporal or Both (Spatio-temporal) Temporal – Frame insertion, deletion, modification and reordering Spatial – Similar, but operates on objects within a frame – Based on MPEG's layer structure ([12]) GOP level, frame level and single- or multi-block level GOP jittering, inter-GOP frame and intra-GOP frame jittering, or their combinations

42 Tamper detection techniques Video Authentication Pasive Pixel Domain Pixel Statistics Based Detecting resampling Double MPEG compresion Inconsistant Noise patterns Duplication detection Physics Based Motion Analysis Shadow Analysis Compressed Domain Active Watermarking Digital Signature References – [12], [13], [14], [15], [16], [17]

43 Compressed Domain approaches Many researches can be found on detecting alterations by image processing means. But very few initiatives in Compressed Domain – Due to the lack of techniques to grab the frame data from compressed video Possibilities – Analyze stream structure headers w.r.t. its contents – Analyze userdata – Analyze inconsistencies in frame sizes, block sizes – Analyze motion vectors

44 Our Approach Based on motion vectors Assumption – For an unaltered video the motions present in the video need to be smooth and therefore should not contain sudden changes – Tampering disturbs the motion flow in a video Extract the motion information from video and compare them with the above assumption. Motion information is derived from the motion vectors [17], [18]

45 Concept of Motion Vector MPEG achieves it its high compression rate by the use of motion estimation and compensation Instead of encoding each block in the current frame, matching block is search in the past frame and if suitable block is found, the difference between the two macroblocks is encoded and transmitted The displacement between the two macroblocks is represented using motion vector

46 Concept Of Motion Vector cont.. Suppose that the macroblock “x” is the macroblock we wish to encode and macroblock “y” is the counterpart in the past frame. A search is done around “y” (search area) to find the best match for “x”. The displacement between the two macroblocks gives the motion vector associated with “x”.

47 Motion Vectors vs MacroBlocks As a part of MPEG encoding process motion vectors are calculated and inserted to the bitstream. In mpeg bitstream motion vectors are associated with macroblocks. But not every macroblock will contain a motion vector There are four basic types of macroblock types – Intra – Skipped – Forward predicted – Backward predicted

48 Picture Type Possible Macroblock TypesMotion Vectors IIntraNo motion vectors present PSkipped, Forward predicted Only forward motion vectors may present BSkipped, Forward predicted, Backward predicted Both Forward and backward motion vectors may present Forward and Backward predictions Summary of motion vectors and macroblocks

49 Deriving Motion Field from Motion Vectors Motion vectors represent the motions,But not directly (therefore we need to derive) – Steps Reorder the bitstream into display order – A simple algorithm Generate Motion felid from motion vectors – [18] defines set of rules. – Simplified mechanism and rule set is described in the [19]. Filter out the outliers – Median filter (3*3 as in [18])

50 Simplified rule set for deriving motion field 1.Macroblocks with no motion vector have the same movement as in the previous image. – This is required since there is a small percentage of intra-coded blocks in B- and P-frames that would otherwise have unknown motion. Also, I-frames have no motion at all so this rule will provide a smooth motion field for them. The justification for this is step is that consecutive images are strongly correlated therefore there is a high probability that the motion field has not changed much. 2.When a macroblock has two motion vectors, the one pointing back is reversed and added to the one pointing forward. – This step accomplishes both noise reduction (due to the averaging effect) and normalizes vector magnitude (so they span three images in B-frames which is what happens in P- frames). 3.Motion vector magnitude is normalized. – If a macroblock in a B-frame only has one motion vector then it has to be multiplied by a factor so its magnitude corresponds to a vector spanning three images. 4.Skipped macroblocks in I-frames have no movement, while in P-frames, they have movement similar to the previous block. – This is a rule obtained directly from the MPEG-2 standard.

51 Implementation of rules TransitionMotion Vectors Present I to B P to B P to P motion = mbF2.FwdMV B to B B to P If only forward vectors present: motion = mbF2.FwdMV – mbF1.FwdMV If only backward vectors present: motion = mbF2.BwdMV –mbF1.BwdMV If both forward and backward vectors are present: motion = ((mbF2.FwdMV – mbF1.FwdMV) + (mbF2.BwdMV –mbF1.BwdMV) )/ 2 P to INot defined Let’s consider a transition from frame F1 to F2, Lets define mbF1 and mbF2 be the corresponding macroblocks in F1 and F2.

52 Motion field The derived motion field is considered as the motion for the frame to frame transition. Now the derived motion field contains the motion information for every macroblock Motion vectors of the left image

53 Derived Motion field for the above motion vector field and next frame

54 Smoothed motion field of the above

55 Detecting Motion Inconsistencies If we consider a typical video, most of pixel data in two adjacent frames are same. Differences are present due to moving objects of the scene, but these differences are small in magnitude. Similar assumption is made in [13] In simple terms, for three adjacent frames, motion between second and third frame is very much similar to motion between first and second frames

56 Mathematical definition of the assumption

57 How to detect tampering Get motion vectors for every frame Derive motion fields for each transition Check whether motion fields satisfies the above two equations, for every transition (i.e every motion field with its next) Based on our assumption if it does not satisfy above it is more likely to be a manipulated video.

58 Results

59 Summary Video Stream Analysing tool – Existing tools are for DVB and not for forensic purposes – Implemented using open stack of tools and libraries – Supports detailed view of bit-stream as well as textual outputs fro further processing Video enhancement tool – Two major issues in video footprints are unclearness and shakes – This tool helps to stabilize a video and reduce noise – Order of stabilize and denoise, has their own pros and cons, and also may depends on video Compressed domain approach for tamper detection – Deriving motion fields from motion vectors – Rule set – Consistency in motion Assumption and mathematical derivation – Detecting motion inconsistencies Showed that video bit-stream data (compressed domain) gathered by stream analyzing tool can be use as an aid for video forensic analysis

60 Future Work Video Stabilization algorithm can be improved to Fill the missing image areas. Video stream analyzer need to support new formats of video. Compressed domain tamper detection mechanisms need to be provided as tools and can be integrate with stream analyzer tool. Motion object segmentation methods can be used in combination with proposed motion field method – detect motions of individual objects and then apply our assumption – which would give better results since we apply our assumption on individual motion fields

61 References [1] MPEG-2 White Paper. [Online]. oducts/dc1000/WhitePapers/DC1000-DVD1000MPEG2whitepaper.pdf oducts/dc1000/WhitePapers/DC1000-DVD1000MPEG2whitepaper.pdf [2] MPEG Encoding basics, 1st ed.: Snell & Wilcox, 2007 [Online] matters.net/docs/resources/Digital%20Files/MPEG/MPEG%20Encoding%2 0Basics.pdf matters.net/docs/resources/Digital%20Files/MPEG/MPEG%20Encoding%2 0Basics.pdf [3] Rongjie Shi, I-fan Shen, Wenbin Chen Rong Hu, "Video Stabilization Using Scale-Invariant Features," in Proc. of the 11th International Conference Information Visualization, 2007, pp [4] Eyal Ofek, Weina Ge, Xiaoou Tang, Heung-Yeung Shum Y. Matsushita, "Full-Frame Video Stabilization with Motion Inpainting," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp , July [5] Jeffrey B. Woodward Hany Farid, "Video Stabilization and Enhancement," Department of Computer Science, Dartmouth College, TR , 1997.

62 [6] E. Shechtman, and M. Irani Y.Wexler, "Space-time video completion," in IEEE Conf. on Computer Vision and Pattern, vol. 1, 2004, pp [7] R. P. Kleihorst, S. Efsratiadis, A. K. Katsaggelos, and R. L. Lagendijk J. C. Brailean, "Noise reduction filters for dynamic image sequences: A review," Proceedings of the IEEE, vol. 83, pp , [8] B. Coll, J. M. Morel A. Buades, "Denoising image sequences does not require motion estimation," Dpt. Matematiquesi Informatica, Universitat Illes Balears. [9] William T. Freeman 2 Ce Liu 1, "A High-Quality Video Denoising Algorithm based on Motion Estimation," 1Microsoft Research New England, 2 Massachusetts Institute of Technology [10] M.I.Sezan, A.M. Tekalp M.K. Ozkan, "Adaptive motion compensated filtering of noisy image sequences," IEEE Trans, circuits, vol. CSVT-3, pp , Aug [11] B. Coll, J. M. Morel A. Buades, "A non-local algorithm for image denoising," IEEE International Conference on Computer Vision and Pattern Recognition, 2005

63 [12] Hong Heather Yu Peng yj, "Classification Of Video Tampering Methods and Countermeasures Using Digital Watermarking," SPIE, Multimedia Systems and Applications, vol. IV, pp [13] Wang Weihong, "Digital Video Forensics," DARTMOUTH COLLEGE, Hanover, New Hampshire, Thesis [14] Farid H Wang W, "Exposing Digital Forgeries in Video by Detecting Double MPEG Compression," ACM Multimedia and security workshop, [15] S.K. Singh S. Upadhyay, "Video Authentication: Issues and Challenges," IJCSI International Journal of Computer Science Issues, Vol. 9, Issue 1, No 3, January [16] Farid H Wang W, "Exposing Digital Forgeries in Interlaced and De-Interlaced Video," EEE Transactions on Information Forensics and Security, pp , [17] Farid H Wang W, "Exposing digital forgeries in video by detecting duplication," ACM Multimedia and security workshop, 2007.

64 [18] M.T.Coimbra and M. Davies, "Approximating optical flow within the MPEG-2 Compressed Domain," IEEE Trans. Circuit Syst. Video Tech., vol. 15, no. 1, pp , Jan [19] Joseph Gilvarry, "Extraction of Motion Vectors from an MPEG Stream," School of Electronic Engineering, Dublin City University, [20] MPEG2Event Class Documentation. [Online]. [21] mpegdemux. [Online]. [22] Information technology – Generic coding of moving pictures and associated audio information, ITU-T, Standard Recommendation ITU-T H.222.0, [23] Video coding for low bit rate communication, ITU-T, Standard ITU-T Recommendation H.263, [24] SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS - Advanced video coding for generic audiovisual services," ITU-T, Recommendation ITU-T H.264, 2013.

65 [25] SalientStills VFPro 4. [Online]. [26] Tektronix. [Online]. softwarehttp://www.tek.com/mpeg-test-video/mpeg- software [27] Elecard Streameye Studio. [Online]. ysis/streameye-studio.html ysis/streameye-studio.html [28] MediaInfo. [Online]. Sliq Media’s WMSnoop. [Online]. [29] MPEG-2 Transport Stream packet analyzer. [Online]. [30] StreamXpert TS Analysis Software. [Online]. [31] Libmpeg2. [Online]. [32] FFMpeg. [Online]. [33] MPEG2Event. [Online]. [34] ooMPEG. [Online]. [35] MPEG-1/2 DirectShow Decoder Filter. [Online].


Download ppt "Video Analysis Tool Box for Digital Video Forensics By Susinda Perera Department of Computer Science and Engineering, University of Moratuwa, Supervised."

Similar presentations


Ads by Google