Object-based Image Representation Dr. B.S. Manjunath Sitaram Bhagavathy Shawn Newsam Baris Sumengen Vision Research Lab University of California, Santa Barbara

Object-based Image Representation2 Outline Context and Objective Introduction Object Extraction and Description Time series object coding Future research ideas Conclusion

Object-based Image Representation3 Context Large-scale Image Database user Query Retrieved images Query example: “Give me all images similar to image X.”

Object-based Image Representation4 Objective To develop an object-based image representation scheme in order to facilitate the following: Faster access of data in the context of object-based querying Reducing required storage space for images Relating maps and geographical aerial images Study of spatio-temporal relationships in/among aerial images Note: Although our dataset consists of aerial images, we expect the scheme to be useful for other image datasets as well.

Object-based Image Representation5 The Object-based Approach Assumptions: Useful information (for searching) in images is concentrated in smaller regions termed objects. Objects are mostly homogeneous in color and texture and can be characterized thus. Most queries on image databases are in terms of objects; e.g. “Give me all images having a brown field.”

Objects from the image An aerial image Examples of objects

Object-based Image Representation7 Why Object-based approach? Uses semantic information for querying (user friendly) Efficient description of images –We ignore portions of images that would not be used for querying Potential reduction of storage space –Store images as collections of objects –Redundancy removal in time-series of objects

Object-based Image Representation8 Past Research Object Extraction: Identify and extract semantic objects from aerial images. –So far, we have done this manually –Working on automatic segmentation using semantic models (Sumengen) Object Description: Find efficient descriptors for the objects (Bhagavathy) –Shape: binary alpha plane –Dominant Colors (Deng, Manjunath) –Dominant textures

9 RGB to LUV conversion Take means of clusters K-means clustering Convert means to RGB object Dominant color feature 24 Gabor filters K-means clustering Take means and percentages object Dominant texture feature 24-dim outputs

10 Time-series object coding Objective: To apply object-based video coding (based on MPEG-4) techniques for coding time-series of objects.

Object-based Image Representation11 Future Research Possibilities Storage space issues –describe image as a collection of objects –reconstruct image from its objects Relation between maps and aerial images –maps have information, images have data Spatial and temporal relationships among objects –variation of objects with time –spatial querying (Newsam) Application in wireless networks

Object-based Image Representation12 Conclusion The object-based approach enables semantic querying which is more user-friendly Time-series compression of objects reduces required storage space for large images Potential to reduce required bandwidth for wireless transmission of image information Enables the study of temporal change in images Enables spatial querying