Intelligent Robotics Today: Vision & Time & Space Complexity

Color Perception in Human Two kinds of light receptors: rods and cones Rods register the relative light and dark Cones perceive color –Three kinds of cones: red, green, blue

Computer Vision Uses the electromagnetic spectrum to produce an image. –Visible light, x-rays, thermal, infrared

Representation Image: picture like format where there is a direct physical correspondence to the scene being viewed. –There are multiple readings in a grid Pixels: “picture element” –Measure depends on the type of spectra being used –Image function: converts a signal to a pixel value

CCD Cameras Charged-Couple Device: detects visible light –Light fall on an array of metal-oxide semiconductor capacitor (MOS) –Line transfer or frame transfer –Frame grabber A/D conversion –A/D conversion Slower than CCD Frame buffers

Representations Grayscale –8-bit – 256 discrete gray values –0 black, 255 white

Representations Different method for representing color –RGB space: red, green, blue –HSV space: hue, saturation, and value

RGB Color Space –24 bit color (8 bits per color)

RGB Representations Interleaved –RGB values stored together Red = image[row][col][0] Green = image[row][col][1] Blue = image[row][col][2] Separate –RGB values stored as separate 2D arrays Red = image_red[row][col] Green = image_green[row][col] Blue = image_Blue[row][col]

Region Segmentation

Identifying a region in an image with a particular color –Thresholding –Binary image

HSV Representations –HSV space: hue, saturation, and value Hue is the dominant wavelength Saturation is the lack of whiteness Value is intensity or brightness the light –Linear transformation between RGB and HSV

HSV Color Space

Comparison of Region Segmentation Original Image RGB segmentation HSV Segmentation

IC Color Vision System Color Space Finding color blobs in an image Making a color model Tuning the camera

Color Selection Plane Hue=0 Hue=360 Sat=0 Val=224 Sat=224 Val=224 Sat=224 Val=0 Note: 224 is the range of values the camera pixels put out in each of R, G & B

Color Blobs For color tracking, a rectangular piece of the color selection plane is selected. All of the pixels in the image whose color falls within that piece are selected. Selected pixels that are contiguous are combined as blobs Each blob has a size, position, number of pixels, major and minor axis, etc. The blobs correspond to objects seen in the image that are the desired color (as given by the specified piece of the of the color selection plane).

Color Models The XBC can segment the image using three different pieces of the color selection plane (each is called a color model) simultaneously It can track a number of blobs from each color model It can display the video in any one of three ways –Raw (live video) –Processed (pixels not selected by a color model are dark) –Combined (processed combined with raw show you can see where the blobs are on the live video)

More on Color Models A Color Model-HSV specifies a bounding box in the color selection plane Moving either edge towards the center line constrains the range of accepted color values to only include more vivid colors. If everything you want is being accepted but so is a lot of other junk you don't want, move the corners closer to the center.

Onscreen Instructions for setting color models

Onscreen Instructions for setting color models

Move Mode Mode Designator Press the SELECT button to view the onscreen instructions Corner move mode is selected by the L or R button and is indicated by a highlighted corner Symmetric Move or Resize mode (M or R) is selected by the START button

Processed Video The A button cycles through live, processed, and combined video

XBC Camera Check 1.Create a color model for channel 0 that sees a red screw driver 2.Load xbctest.ic onto your XBC 3.Run the program 4.Select the vision test 5.Select the correct channel/model to see red 6.Follow onscreen directions to get data on the blobs 7.If you like your model, save it to flash 8.For more info, see XBC Camera in IC Help

Which Color Model? The XBC stores 3 channels of color (0, 1, 2) There are 3 models maintained for each channel: 1.The currently active model (the one you see in the modify model menu) 2.The model in flash (you can save the currently active model and camera settings using the save to flash menu item or make them active by loading from flash) 3.The default model and camera settings which are made active by selecting restore to default

Tuning the Camera By default cameras turn on Auto White Balance (AWB) and dynamically adjust their color temperature to keep the amounts of red and blue in the field of view roughly balanced. This is fine if the content of the field of view is roughly balanced between red and blue and if the goal is to look good to humans in changing, arbitrary lighting situations. However, when trying to do color tracking, dynamically changing color temperature is generally counter productive. Instead, you should use the Vision/Camera Config menu and press the Start button to interactively calibrate the white balance while pointing the camera at a white sheet of paper. This will adjust the Red and Blue components of the color temperature until the amount of each in the scene balances, then turn off AWB to lock those values in. After this procedure, it's a good idea to go to the Vision/Flash Memory menu, select "Setting: ", and then "Save to Flash".

Tuning the Camera Focus the un-focus the lens –Depending on what you are looking for it can help to have the image a little out of focus