1. INTRODUCTION Heesoo Myeong, Ju Yong Chang, and Kyoung Mu Lee Department of EECS, ASRI, Seoul National University, Seoul, Korea http://cv.snu.ac.kr Learning Object Relationships via Graph-based Context Model Why Context for Scene Understanding? PROPOSED METHOD Results on SIFT Flow Dataset Quantitative Results on Standard Datasets   Jain et al. dataset (Jain et al., ECCV10): 250 training images, 100 test images, 19 labels   SIFT Flow dataset (Liu et al., CVPR09): 2,488 training images, 200 test images, 33 labels Table 1: Per-pixel classification rates and (average per-class rates)   For a test image, retrieve T most similar training images using global features EXPERIMENTS Inference   Use Fully connected Markov Random Field (MRF) model: Previous Works & Limitations Our Contributions Prefer frequently appeared objects Not invariant to the number of pixels/regions   A novel context link view of contextual knowledge   Applying label propagation to context link prediction   Nonparametric context model scalable to large datasets   Incorporating contextual information is crucial for scene understanding   Recently, object-object relationships have shown better performance than scene- object relationships [Rabinovich et al., ICCV07] Our Approach   The similarity graph naturally reflects visual similarity Context link on the similarity graph Semi-supervised context link prediction   A novel view for representing object relationships   Object relationships are usually formulated as co-occurrence statistics or spatial relation [Rabinovich et al., ICCV07, Gould et al., IJCV08, Jain et al., ECCV10]   No appearance info. has been taken into account during context modeling process Building Building Building Building ?? Road Query image Conventional context model Building Crosswalk Our context model Building Retrieved training image (Exemplar) Crosswalk   Graph-based and Exemplar-based context model   Utilize object relationships adaptively according to the visual appearance of objects   Learn object-object relationships between all pairs of regions across whole object class pairs Retrieved training images (Exemplars) Test image (building,car)-link Similarity graph ? ? car predicted (building,car)-link Building Similarity edge Similarity edge   K-nearest neighbor similarity graph is constructed among regions from both the query image and the corresponding retrieved image set   Decompose context link prediction problem into two independent label propagation subproblems [Lu and Ip, ECCV10] Label propagation sky building car road crosswalk building car road car sidewalk building sky car plant Input imageGround truthSuperParsingOursBaseline classifier 53.986.090.7 60.766.776.3 85.485.287.5 Input image Ground truthSuperParsingOursBaseline classifier tree sky bison field 40.542.9 56.4 74.7 74.699.8 sky water sand Results on Jain et al. Dataset   We have proposed a novel framework for modeling image-dependent contextual relationships using graph-based context model   Experimental results demonstrate that the proposed context model overcome the limitation of conventional context models relying on object label agreement and gives richer appearance-based context information Baseline MRF Our approach (b) Jain et al. dataset (a) SIFT flow dataset Conclusion Per-class recognition rate