1. CMUcam Tom Kneeland

2. CMUcam – What is it? Digital Camera with an onboard microcontroller Track a blob based on programmable color thresholds Calculate centroid Calculate mean color and variance RS-232 or TTL ~17fps

3. Components – CMOS Camera Omnivision OV6620 Color Camera 17 MHz clock 8 bit RGB or YCrCb* 8 or 16 bit bus Max Resolution: 356 x 292 60 fps

4. Components - Microcontroller Ubicom SX28 RISC Architecture 28 pin DIP 75 MIPS @ 75 MHz 2048 words of ROM 136 bytes of SRAM

5. How Does It Work – Firmware Written in C 2035 of the 2048 words of ROM 135 of the 136 bytes of SRAM Color Blob Tracking Frame Dump Line Mode Color Statistics Two serial IO protocols

6. How Does It Work - Configuration Camera Resolution: 176 x 144 8-bit bus RAM Allocation –80 bytes for a pixel buffer (one row) –56 bytes for processing Effective Resolution: 80 x 144

7. How Does It Work – Pixel Format 8 bit RGB or YCrCb –4:2:2 RGBG –8-bits per component –Macropixel = 2 image pixels –Red and Blue are shared –16 bpp

8. How It Works – Pixel Format 16 bpp x 80 pixels = 160 bytes! Convert to a 2:1:1 format Skip every other pixel Share Red and Blue Values Sacrifices resolution and color accuracy

9. Color Blob Tracking Compare each pixel to min and max values Determine if they fall within bounding box Confidence Measure: –#pixels/box_area * 256 Centroid –Sum x,y of matched pixels –Divide by num detected pixels

10. Frame Dump and Line Mode Frame Dump –Limited by serial I/O –One or Two Columns at a time Line Mode –Line by Line binary image of detected pixels –Available during blob tracking

11. ASCII Protocol cr – program camera registers sw – set window (bounding box) coords gm – get mean value –S R mean G mean B mean R deviation G deviation B deviation tc – track color function –M x cen y cen x 1 y 1 x 2 y 2 #pixels confidence

12. ASCII Protocol Example CMUcam v1.2 :cr 18 44 17 2 19 32 ACK :sw 30 60 50 80 ACK :pm 1 ACK :gm ACK S 150 20 30 5 2 6 :pm 0 ACK :sw 0 0 80 143 ACK :tc 145 18 24 155 22 36 ACK M 50 80 38 82 53 128 35 98 M 52 81 38 82 53 128 35 98 M 51 80 38 84 53 128 35 98