IBM QBIC: Query by Image and Video Content Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC 28223

Slides:

Advertisements

Similar presentations

Relevance Feedback and User Interaction for CBIR Hai Le Supervisor: Dr. Sid Ray.

Advertisements

Image Retrieval: Current Techniques, Promising Directions, and Open Issues Yong Rui, Thomas Huang and Shih-Fu Chang Published in the Journal of Visual.

Content-Based Image Retrieval

Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC.

Automated Shot Boundary Detection in VIRS DJ Park Computer Science Department The University of Iowa.

Chapter 11 Beyond Bag of Words. Question Answering n Providing answers instead of ranked lists of documents n Older QA systems generated answers n Current.

1 Content Based Image Retrieval Using MPEG-7 Dominant Color Descriptor Student: Mr. Ka-Man Wong Supervisor: Dr. Lai-Man Po MPhil Examination Department.

UMC – HCI seminar series 1 Human Computer Interaction Query by Sketch Chi-Ren Shyu Department of Computer Engineering and Computer Science University of.

Image Search Presented by: Samantha Mahindrakar Diti Gandhi.

ADVISE: Advanced Digital Video Information Segmentation Engine

Issues for Multimedia Privacy & Security ---- Video Content Privacy Protection, Copyright Protection & Database Access Control Jianping Fan Dept of Computer.

SWE 423: Multimedia Systems

Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,

CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.

Multimedia Search and Retrieval Presented by: Reza Aghaee For Multimedia Course(CMPT820) Simon Fraser University March.2005 Shih-Fu Chang, Qian Huang,

ISP 433/633 Week 5 Multimedia IR. Goals –Increase access to media content –Decrease effort in media handling and reuse –Improve usefulness of media content.

Stockman MSU CSE1 Image Database Access  Find images from personal collections  Find images on the web  Find images from medical cases  Find images.

Traditional Database Indexing Techniques for Video Database Indexing Jianping Fan Department of Computer Science University of North Carolina at Charlotte.

T.Sharon 1 Internet Resources Discovery (IRD) Introduction to MMIR.

Visual Information Retrieval Chapter 1 Introduction Alberto Del Bimbo Dipartimento di Sistemi e Informatica Universita di Firenze Firenze, Italy.

A fuzzy video content representation for video summarization and content-based retrieval Anastasios D. Doulamis, Nikolaos D. Doulamis, Stefanos D. Kollias.

DVMM Lab, Columbia UniversityVideo Event Recognition Video Event Recognition: Multilevel Pyramid Matching Dong Xu and Shih-Fu Chang Digital Video and Multimedia.

Video Streaming via Transcoding Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC 28223

Content-Based Video Retrieval System Presented by: Edmund Liang CSE 8337: Information Retrieval.

A Brief Overview of Computer Vision Jinxiang Chai.

Multimedia and Time-series Data

Wavelet-Based Multiresolution Matching for Content-Based Image Retrieval Presented by Tienwei Tsai Department of Computer Science and Engineering Tatung.

Multimedia Databases (MMDB)

Image and Video Retrieval INST 734 Doug Oard Module 13.

Multimedia Information Retrieval

Image Retrieval Part I (Introduction). 2 Image Understanding Functions Image indexing similarity matching image retrieval (content-based method)

Content-Based Image Retrieval

Computer Vision – Overview Hanyang University Jong-Il Park.

IMAGE DATABASES Prof. Hyoung-Joo Kim OOPSLA Lab. Computer Engineering Seoul National University.

Understanding The Semantics of Media Chapter 8 Camilo A. Celis.

Content-Based Image Retrieval Using Fuzzy Cognition Concepts Presented by Tienwei Tsai Department of Computer Science and Engineering Tatung University.

March 31, 1998NSF IDM 98, Group F1 Group F Multi-modal Issues, Systems and Applications.

What do you understand about how each system works to index-retrieve images? Manually Index Expensive but effective.

Image Classification for Automatic Annotation

Bachelor of Engineering In Image Processing Techniques For Video Content Extraction Submitted to the faculty of Engineering North Maharashtra University,

MMDB-9 J. Teuhola Standardization: MPEG-7 “Multimedia Content Description Interface” Standard for describing multimedia content (metadata).

Content-Based Image Retrieval QBIC Homepage The State Hermitage Museum db2www/qbicSearch.mac/qbic?selLang=English.

Semantic Extraction and Semantics-Based Annotation and Retrieval for Video Databases Authors: Yan Liu & Fei Li Department of Computer Science Columbia.

Content Based Color Image Retrieval vi Wavelet Transformations Information Retrieval Class Presentation May 2, 2012 Author: Mrs. Y.M. Latha Presenter:

Video Databases What are it uses? –Sports –Surveillance How do we query it? –Mosaic-based Query Language.

Query by Image and Video Content: The QBIC System M. Flickner et al. IEEE Computer Special Issue on Content-Based Retrieval Vol. 28, No. 9, September 1995.

VISUAL INFORMATION RETRIEVAL Presented by Dipti Vaidya.

MULTIMEDIA DATA MODELS AND AUTHORING

Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC Relevance Feedback for Image Retrieval.

3D Motion Classification Partial Image Retrieval and Download Multimedia Project Multimedia and Network Lab, Department of Computer Science.

Course Project Lists for ITCS6157 Jianping Fan. Project Implementation Lists Automatic Image Clustering You can download 1,000,000 images from You can.

Content-Based Image Retrieval Readings: Chapter 8:

Digital Video Library - Jacky Ma.

Visual Information Retrieval

Automatic Video Shot Detection from MPEG Bit Stream

ITCS 6157/8157: Visual Database

Multimedia Content-Based Retrieval

Semantic Video Classification

Personalized Social Image Recommendation

Security Issues for Visual Data: Copyright and Access Control

Color-Texture Analysis for Content-Based Image Retrieval

Self-Organizing Maps for Content-Based Image Database Retrieval

Project Implementation for ITCS4122

Image Segmentation Techniques

Multimedia Information Retrieval

Ying Dai Faculty of software and information science,

Ying Dai Faculty of software and information science,

Midterm Exam Closed book, notes, computer Similar to test 1 in format:

Color Image Retrieval based on Primitives of Color Moments

Midterm Exam Closed book, notes, computer Similar to test 1 in format:

Presentation transcript:

IBM QBIC: Query by Image and Video Content Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC

How can I access video in database over networks? 1. How to access video ? 2. How to represent video ? 3. How to index large-scale videos ? 4. How to access videos in database ? 5. How to transmit query results over IP ? Networks 6. How to control user’s access ?

1. System Architecture How to access video in database? ImagesVideos Shot extraction 1. Representative Frame 2. Motion-based objects 3. shots Object Identification R-frames scene sketch Positional color/texture object Location/color User-defined Color/texture video Object motion Camera motion Feature extraction sceneobjectsshots Motion objects

1. System Architecture How to access video in database? user Query interface Color Texture Shape Multi-object Sketch Location Text Positional color/texture object motion camera motion user defined existing image Match engine Database Indexing Returned via similarity order

2. How to access image or video in QBIC? a. Video shots b. Video objects or sketches & drawings c. Representative frames Access approaches a. Example images b. user-constructed sketches or drawings c. Selected color and texture patterns d. Camera & object motion e. Other graphical information d. Motion types e. Other information

2. How to obtain accessing units in QBIC? Shot Detection

2. How to obtain accessing units in QBIC? a. Difference Calculations b. Automatic Decision MakingVia Pre-Defined Thresholds.

How we do the shot detection?

2. How to obtain accessing units in QBIC? Object extraction

2. How to obtain accessing units in QBIC? Object extraction

How we can do the object extraction?

3. How to represent these video units? Images Global color Global texture Positional color Positional texture Sketch, shape User-defined color/texture

3. How to represent these video units? Videos Global color Global texture Positional color Positional texture Sketch, shape User-defined color/texture Camera motion/object motion

Color HSV color histogram, dominant color, … Texture Edge histogram, wavelet coefficients, Tamura features, … Motion Directional motion histogram, Camera motion, … Other features Video Sequence Shot 1 Shot i Shot n How we do the shot representation?

4. How to index images/videos in QBIC? feature space Videos in Database

4. How to index images/videos in QBIC? High-dimensional visual features K-L Transform to reduce dimensions Low-dimensional R*-tree indexing

4. How to index images/videos in QBIC? Overlap on two Dimensions!

4. How to index images/videos in QBIC? Karhunen-Loeve Transformation New Eigenvectors M is the matrix for videos! S is the KL transform matrix!

5. How to realize query in QBIC?

How we can do the video query?

5. How to realize query in QBIC?

How we can do the mosaic?

Why we use mosaic for video representation?

6. What lost by QBIC? a. Mapping from visual features to semantic concepts It is hard, but we have to do this. Why? Visual Features Semantic Clusters Video Contents in Database Weighted mapping? How to do this mapping?

6. What lost by QBIC? b. High-dimensional visual indexing It is a basic problem in database area, but only database people cannot solve this challenging problem for visual indexing! Video in Database Cluster 1Cluster iCluster n Subcluster 11Subcluster 1j Subcluster n1 Subcluster nl Subregion 111 Subregion 11k Subregion nl1Subregion nlm object1111 object nlm1 Disk for Cluster 1 Disk for Cluster i Disk for Cluster n

6. What lost by QBIC? c. User input in the query procedure: QBIC can permit user to select something.

6. What lost by QBIC? d. How to integrate keywords with visual features?

6. What lost by QBIC? e. How to provide user-intensive browsing?

7. What happen now on QBIC? You can find the current version of QBIC system on: Homework: What kind of technique we have discussed used in QBIC?

8. Other Projects a.Chabot at UC Berkeley b. Viper at Europe c. Virage

8. Other Projects d. PicHunter at NEC e. Ifind at Microsoft f. Photobook at MIT