Video Mining Learning Patterns of Behaviour via an Intelligent Image Analysis System

Introduction All large archives of video are now available in repositories, there is hidden much potentially useful knowledge Latent, Useful, Interesting Information Data mining Techniques Large archives of videos Jang Hee Kyun

Introduction Video mining is more challenging, due to lack of explicit structure in the raw data in video archives Ex) Surveillance, Analysis of expert’s activities Studying animal behaviour, Emulate robot behaviour Each applications there are often patterns of activity They can be classified in order to gain more general insights into agent and object movements and behaviour Jang Hee Kyun

Video Mining Interesting in that the raw data in videos is expressed in a way that is not directly amenable to the use of conventional mining techniques There is a lot of variation in the details of the patterns and it is important to abstract out the key features of a behaviour Unwanted or irrelevant details and noise can be filtered out Jang Hee Kyun

Video Mining Video mining of patterns of behaviour and their inter- play, has to deal with very dynamic situations Techniques that have been developed for the extraction of temporal rules from collections of time series data We now want to identify patterns of data that are unusual, and discover inter-relationships between the patterns of different agents and objects Jang Hee Kyun

Discover Rules First Stream Detect ‘abnormal ‘ or ‘interesting ’ behaviour The ability to learn what is ‘abnormal ’ or ‘ interesting ’ Ultimate goal is that they will use only innate knowledge Second Stream Summarisation for behavioural pattern detection/ matching in the second stream using AI (Artificial Intelligence) and DM (Data Mining) algorithms for time series analysis Jang Hee Kyun 6

Discover Rules It will be able to operate without the direct intervention of a user, and be able to control its own focus of attention to some extent This will in turn influence how it operates in related situations in the future Jang Hee Kyun 7

Background and approach We use our own system, ModTrack, for vehicle detection and tracking “Independent Moving Object Detection Using a Colour Background Model” by F. Campbell – West, P. Miller, H. Wang DM (Data Mining) aspects Tracking system Identify abnormal behaviour Infer unusual pattern of activity AI (Artificial Intelligence) aspects Learning how another agent learns and making use of the results Jang Hee Kyun 8

Method of representation and analysis Our objective is to reverse engineer what we observe in the real world by using a vision or imaging system We need to emulate the behaviour of the real world actors How does a robot adjust its knowledge about the behaviour of a light using the adaptive learning paradigm? Jang Hee Kyun 9

Method of representation and analysis We use the robot’s intention not only as a consideration for our decision making, but also as a guide for our accumulation of observations We make a qualitative assessment by distinguishing suggestion and confirmation Jang Hee Kyun 10

Example Jang Hee Kyun 11

Method of representation and analysis 1. As it detects sequences of such atomic movements the system records them 2. Behaviour pattern detection Classify these behavioural seuqences Classifier is important requirement 3. Represent activities rule set Behaviour matching and prediction Jang Hee Kyun 12

Conclusions and Summary ModTrack was used to obtain the behavioural traces of the robot/agent Using representation we can build a new representation of what is happening with the raw data We have shown how detailed behaviours from video can be coarsened and mined to obtain useful knowledge Jang Hee Kyun 13