ETISEO Benoît GEORIS and François BREMOND ORION Team, INRIA Sophia Antipolis, France Lille, December th 2005

2 Fair Evaluation Unbiased and transparent evaluation protocol Large participation Meaningful evaluation

3 Fair Evaluation Meaningful evaluation  Etiseo video sequences: dedicated to specific problems, with several levels of difficulties  Definition of specific ground truth, reference data and metrics taking advantage of contextual information  For each partner, presentation of results for problems (one after one) that will be studied by ETISEO with variations from easy to hard  For a specific problem, global comparison of partner performances (e.g., sensitivity of tracking wrt occlusion)

4 First Evaluation Cycle Analysis Evaluation of the participation:  Questionnaire & video clips  Number of participants and focus From simple (common) conditions: no occlusion, middle view, well contrasted, no artifact, no light change, color, mono-camera To complex conditions:  Detection  Classification  Tracking  Multiple cameras  Events What can be modified for the 2nd evaluation cycle?  Video clips & ground truth (now)  Reference data, metrics & comparison (later)