EE 113D Final Project: Colorization Spring, 2006 Group Members: Johnny Cheng Brian Cheung Austin Wong Professor: R. Jain TA: Rick Lan

Motivation Enhances Visual Appeal of the Image –Photoshop Application –Colorize old photographs Perceptually Enhance the Information Content of the Image –Eg. X-ray imaging, scientific illustration Minimize Human Labor –No longer need to choose colors from a palette to color individual components

Visual Idea Representation +=+=

Method 1. Convert 3 dimensional RGB pixel values to 2 dimensional YCbCr color space. 2. Luminance comparison between source (colored) and destination (grayscale) image. 3. Transfer of CbCr chromaticity values. 4. Convert YCbCr color space back into RGB color space.

YCbCrRGB R´ = 1.164(Y - 16) (Cr - 128) G´ = 1.164(Y - 16) (Cr - 128) (Cb - 128) B´ = 1.164(Y - 16) (Cb - 128)

Procedures: Preparing the Images We took 2 similar 162 x 210 images, one in RGB format and the other in grayscale, and decreased the resolutions to two 8x10 images using Photoshop. Although the pictures are not discernable, colorization can still be seen

Procedures: Preparing the Images Using Matlab, we used the imread command to read in the coefficients of each image The RGB image created an 8x10x3 matrix of coefficients and the grayscale image created an 8x10 matrix of coefficients. We used the Matlab program to perform a luminance comparison between the two image files thus creating a new YCbCr image The RGB image, for consistent dimensions, was converted into a YCbCr format with dimensions 8x10.

Procedures: Assembly Programming The equations that we implemented in assembly language are: R´ = 116(Y - 16) + 160(Cr - 128) G´ = 116(Y - 16) - 81(Cr - 128) - 39(Cb - 128) B´ = 116(Y - 16) + 202(Cb - 128). We used the MACP function from the DSP chip, which multiplied two numbers and added it to the accumulator Syntax for a specific block: A = #0 B = *AR1+ B = B - #16 AR7 = #operand1 *AR7 = B MACP(*AR7,const1,A)

Procedures: Assembly Programming loop1;R´ = 1.164(Y - 16) (Cr - 128) and store it into output1 A = #0;Clears the accumulator B = *AR1+;This section will perform the expression, (Y-16) B = B - #16 AR7 = #operand1;AR7 will hold the address location of operand1 nop *AR7 = B nop MACP(*AR7,const1,A);This will multiply B to const1 (116) and add it the value ;stored in the accumulator B = *AR3+;This will increment the Cr conversion constant matrix B = B - #128;This section will perform the expression, 1.596(Cr - 128) AR7 = #operand2;AR7 will hold the address location of operand2 nop *AR7 = B nop MACP(*AR7,const2,A);This will multiply B to const2 (160) and add it the value ;stored in the accumulator *AR4+ = A;Set accumulator A to output1and then increment AR4 if (*AR0- != 0) goto loop1 ;If the count is not at 0, go back to loop1

Procedures: Output Once we obtained the coefficient values inside the memory, we exported the data onto a dat file, where the values were in hexadecimal format The results would be compared numerically and visually The values were then copied onto an excel spreadsheet and were converted back into decimal form. PhotoshopYCbCr and luminance comparisonConvert to RGB

Understanding the Matlab Code Processing time and memory constraints Choosing the suitable Assembly Code Writing the Assembly Code

Trying to figure out what lines of code meant what and trying to figure out why the author chose to write that particular code was a challenge for us. For example, we needed to understand which lines of code converted the images into its 3-dimensional RGB form and which lines of code did the luminance comparison. Once we got a handle on the matlab code, we realized it was impossible for us to complete the whole process given the amount of time. We decided to cut down on our specifications and only chose a part of the Matlab Code that we were going to imitate on the DSP chip. We decided to focus on changing the image back into its 3- dimensional RGB form from the newly colored YCbCr grayscale image.

Since the amount of time it took to execute the Matlab code on our images exceeded one minute, we didn’t know how long it would take the DSP to perform similar operations. We decided to shrink the image to a more workable size from the 162x210 to a 8x10. It greatly reduced the Matlab processing time to about 10 seconds, which would correlate to a more reasonable job for the DSP chip to carry out.

We had a difficult time finding out which Assembly code to base our project on. We realized that we were basically going to have to write our own code using the similar format and structure as the assembly code from Experiment A (page 78)

Not being fully adept in Assembly, we had a hard time writing some of our code. For instance, we needed to perform a subtraction in some of our RGB conversion equations. There were no arithmetic instructions for doing subtraction, multiplication, and then addition.

After solving our syntax issues, the DSP chip still wasn’t performing any of the algebraic operations correctly. The values obtained from the RGB equations were doubled what was expected. We had initialization problems that occurred with the.data and.output. We copied over the wrong coefficients from Matlab, and thus we were processing the wrong values on the DSP.

Results - Red Scaled DSP HEX Values Scaled DSP Dec Values Un-scaled DSP Dec Values Un-Scaled Matlab Values Scaled Matlab valuesDifferencerelative difference% relative difference CB F A BFC A BB EDC B

Results - Green Scaled DSP HEX Values Scaled DSP Dec Values Un-scaled DSP Dec Values Un-Scaled Matlab Values Scaled Matlab valuesDifferencerelative difference% relative difference 1DDC A E C E D A CA

Results - Blue Scaled DSP HEX Values Scaled DSP Dec Values Un-scaled DSP Dec Values Un-Scaled Matlab Values Scaled Matlab valuesDifferencerelative difference% relative difference CB F A BFC A BB EDC B

Total Average Error: % From Matlab:

A Closer Look Matlab: DSP:

Future Developments 1. Convert 3 dimensional RGB pixel values to 2 dimensional YCbCr color space. 2. Luminance comparison between source (colored) and destination (grayscale) image. 3. Transfer of CbCr chromaticity values. 4. Convert YCbCr color space back into RGB color space.