1. 1 Week 6 Fatemeh Yazdiananari

2. 2 Feature Extraction 75 Validation Videos Were sent to the cluster for DTF feature extraction The extracted features are large in size Max size:152GB Min size: 540MB Total size: 1.6TB Programing Since the files are large Textscan, fgetl, dlmread Ran each feature and saved all the information into a mat file Used the mat files to run the rest of the codes.

3. 3 Histogram code For each video Obtain 4 feature matrix Per feature find the closest codebook term and save the index Using the saved indices run a histogram function Save : video name, first 10 elements of features, Indices of Tr, Hof, Hog, and Mbh, and histogram of Tr, Hof, Hog, and Mbh.

4. 4 OverView 15 Validation Videos Holistic Histograms of 15 videos Binary SVM Sliding Windows (10sec long windows) Histograms of the Sliding window Binary SVM

5. 5 Histograms 15 validation videos  Normalized histograms HOF histogram for one videoHOG histogram for one video

6. 6 Histograms (cont.) MBH histogram for one videoTR histogram for one video

7. 7 Action recognition steps Binary SVM Trained using the UCF101 train splits (split 1) obtained models Tested using the 15 validation videos

8. 8 SVM & Results Added two UCF101 features  Class 1 and class 2 Ran Binary SVM Classification results (right)  Accurately predicted class 1 and 2 of UCF101 (11.76%)

9. 9 Sliding window steps Obtain the frame rate of the video (videoreader) 10secs worth of frames: multiply to frame rate and get the frame number (window size) Load the histograms of the 15 videos Using the frame number we read in the histograms and save them into a structure After all the videos are divided into their windows and saved we load them run normalization binary SVM Trained on UCF101 split 1 Tested on all the windows

10. 10 Sliding Window Histogram Hof histogram for one sliding window Hog histogram for one sliding window 15 Validation Videos Divided into sliding windows per 10secs Normalized histograms

11. 11 Sliding Window Histogram (cont.) Mbh histogram of one sliding window Tr histogram of one sliding window

12. 12 SVM & Results Added two UCF101 features  Class 1 and class 2 Ran Binary SVM Classification results  Classified class 1 and 2 of UCF101 accurately  The sliding windows were misclassified as class 38

13. 13 Problems Certain rows take on the same value across the 101 classes (horizontally)  Causing the code to crash Columns have the same value in them across the sliding windows (vertically)  For this reason the SVM predicts the same class for each sliding window.

14. 14 Conclusion Action recognition using the existing methods on temporally untrimmed videos were done. The videos are long YouTube videos (THUMOS’14) The approach based on one Bag-of-words histogram per videos (state-of-the-art) obviously failed as expected. The Sliding Window approach failed since: The histograms extracted from the untrimmed clips may not include any particular action; such histograms happen to be similar to typical histograms of Class 38  misclassification as class 38. The effect of not having the exact boundaries of the shots/clips have a nontrivial impact on the formed histograms.

15. 15 Next Steps Visual comparison of the class 38 histograms with the untrimmed histograms  a visual similarities should be observed. Extracting the histograms from UCF101 videos and classifying them using the same code used for the untrimmed videos  sanity check. Manually identifying the boundaries of the action in the untrimmed videos and selecting the windows accordingly  investigating the impact of unknown boundaries and partial windows.