Presentation on theme: "SWE 423: Multimedia Systems"— Presentation transcript:
1 SWE 423: Multimedia Systems Chapter 4: Graphics and Images (3)
2 Outline Image Processing Basics Textbook Section 4.3 Image FeaturesImage SegmentationTextbook Section 4.3Additional Reference: Wasfi Al-Khatib, Y. Francis Day, Arif Ghafoor, and P. Bruce Berra. Semantic modeling and knowledge representation in multimedia databases. IEEE Transactions on Knowledge and Data Engineering, 11(1):64-80, 1999.
3 Image ProcessingImage processing involves the analysis of scenes or the reconstruction of models from images representing 2D or 3D objects.Image AnalysisIdentifying Image Properties (Image Features)Image SegmentationImage RecognitionWe will look at image processing from a database perspective.Objective: Design of robust image processing and recognition techniques to support semantic modeling, knowledge representation, and querying of images.
4 Semantic Modeling and Knowledge Representation in Image Databases Feature Extraction.Salient Object Identification.Content-Based Indexing and Retrieval.Query Formulation and Processing.
6 Feature Extraction Layer Image features: Colors, Textures, Shapes, Edges, ...etc.Features are mapped into a multi-dimensional feature space allowing similarity-based retrieval.Features can be classified into two types: Global and Local.
7 Global FeaturesGenerally emphasize coarse-grained pattern matching techniques.Transform the whole image into a functional representation.Finer details within individual parts of the image are ignored.Examples: Color histograms and coherence vectors, Texture, Fast Fourier Transform, Hough Transform, and Eigenvalues.What are some of the example queries?
8 Color Histogram How many pixels of the image take a specific color In order to control the number of colors, the domain is discretizedE.g. consider the value of the two leftmost bits in each color channel (RGB).In this case , the number of different colors is equal to __________How can we determine whether two images are similar using the color histogram?
9 Color Coherence Vector Based on the color histogramEach pixel is checked as to whether it is within a sufficiently large one-color environment or not.i.e. in a region related by a path of pixels of the same colorIf so, the pixel is called coherent, otherwise incoherentFor each color j, compute the number of coherent and incoherent pixels (j , j), j = 1, ..., JWhen comparing two images with color coherence vectors (j , j) and (j , j), j = 1, ..., J, we may use the expression
10 Texture Texture is a small surface structure Examples include Natural or artificialRegular or irregularExamples includeWood barksKnitting patternsThe surface of a sponge
12 Texture Two basic approaches to study texture Structural analysis searches for small basic components and an arrangement ruleStatistical analysis describes the texture as a whole based on specific attributes (local gray-level variance, regularity, coarseness, orientation, and contrast.Either done in the spatial domain or the spatial frequency domain
13 Global Features Advantages: Disadvantages: Simple. Low computational complexity.Disadvantages:Low accuracy
14 Local FeaturesImages are segmented into a collection of smaller regions, with each region representing a potential object of interest (fine-grained).An object of interest may represent a simple semantic object (e.g. a round object).Choice of features is domain specific:X-ray imaging, GIS, ...etc require spatial features (e.g. shapes [may be calculated through edges] and dimensions.)Paintings, MMR imaging, ...etc may use color features in specific regions of the image.
15 Edge DetectionA given input image E is used to gradually compute a (zero-initialized) output image A.A convolution mask runs across E pixel by pixel and links the entries in the mask at each position that M occupies in E with the gray value of the underlying image dots.The result of the linkage (and the subsequent sum across all products from the mask entry and the gray value of the underlying image pixel) is written to the output image A.
16 ConvolutionConvolution is a simple mathematical operation which is fundamental to many common image processing operators.Convolution provides a way of `multiplying together' two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality.This can be used in image processing to implement operators whose output pixel values are simple linear combinations of certain input pixel values.The convolution is performed by sliding the kernel over the image, generally starting at the top left corner, so as to move the kernel through all the positions where the kernel fits entirely within the boundaries of the image.
17 Convolution Computation If the image E has M rows and N columns, and the kernel K has m rows and n columns, then the size of the output image A will have M - m + 1 rows, and N - n + 1 columns and is given by:Example page 60.
19 Prototype Systems QBIC (http://www.hermitagemuseum.org) Uses color, shape, and texture featuresAllows queries by sketching features and providing color informationChabot (Cypress)Uses color and textual annotation.Improved performance due to textual annotation (Concept Query)KMeDUses shapes and contours as features.Features are extracted automatically in some cases and manually in other cases.