1. Introduction to Media Computation
Barb Ericson
Georgia Institute of Technology
June 2005
Georgia Institute of Technology

2. Georgia Institute of Technology
Learning Goals
Understand at a conceptual level
What is media computation?
How do digital pictures work?
How do digital sounds work?
Georgia Institute of Technology

3. What is Media Computation?
Processing
picture elements
sound fragments
movie frames
Text files and HTML pages
The speed and storage capacity of modern computers makes this possible
Even for beginning students just learning to program
Georgia Institute of Technology

4. How Does Color Vision Work?
Our eyes and brain work together to make sense of what we see
The cones in our eyes are what allow us to see in color
The rods allow us to see black, white, and shades of gray
Our cones are sensitive to red, green, and blue light
All other colors are combinations of these
See and for an explanation of vision.
See for a picture of rods and cones.
See and for some optical illusions.
See For
Georgia Institute of Technology

5. Red, Green and Blue Light
White light is a combination of red, green, and blue
Full intensity red, green, and blue combined
Black is the absence of all light
No red, green or blue light
All other colors are combinations
Of red, green, and blue
Of different intensities
See for an applet that lets you mix colors.
See for an explanation of why paint primary colors are different that light primary colors.
Georgia Institute of Technology

6. Georgia Institute of Technology
Color Exercise
Start DrJava
In the interactions pane type
ColorChooser.pickAColor();
Click on the RGB tab and move the sliders to change the intensity of red, green, and blue
Make white, black, red, blue, green, yellow, violet, and orange
Can you figure out why the colors range from 0 to 255? If I tell you that each color is stored in 8 bits does that help?
Hold up a CD and change the angle of it with the light. What colors do you see?
Do you think pickAColor is an object method or a class method? How can you tell?
Georgia Institute of Technology

7. How do Digital Cameras Work?
There are red, green, and blue filters that capture the amount of each color at a position
A part of a grid
There are many positions
picture element or pixel
640 x 480 is low resolution
1600 x 1200 is high resolution
The more pixels the better the picture
Can enlarge it without it looking grainy
For more information on how digital camera work see
Georgia Institute of Technology

8. How do Computer Displays Work?
A display has pixels (picture elements)
Each pixel has a red, green, and blue component
Combinations of red, green, and blue give the resulting color
Black is 0 red, 0 green and 0 blue
White is 255 red, 255 green, 255 blue
See for more information on how computer monitors work.
See for information on CRT devices.
See for information on LCD devices.
Colors are often digitized as 24 bits per pixel, 8 bits for red, 8 for green, and 8 for blue. Thus the intensity of the red can be from 0 to 255 since 8 bits are used.
Georgia Institute of Technology

9. Pictures are made up of Pixels
Digital cameras record light at pixels
Monitors display pictures using pixels
Our limited vision acuity helps us to see the discrete pixels as a smooth picture
If we blow up the picture we can see the pixels
Georgia Institute of Technology

10. Georgia Institute of Technology
Digital Pictures
Capture the intensity of the red, green, and blue colors at each pixel
Stored as a bunch of numbers
8 bits for red, 8 bits for green, 8 bits for blue
Need nearly 1 million bytes to store a 640 x 480 picture
Need 3 million bytes to store an image from a 1 megapixel (million pixel) camera
Displayed as red, green, and blue colors on the computer display
Lots of them close together
Our brain sees a smooth color image
Georgia Institute of Technology

11. Georgia Institute of Technology
Picking a File Name
Use the class method pickAFile() on our FileChooser class to pick a file
> System.out.println(FileChooser.pickAFile())
Look for a file that holds a digital picture
.jpg or .gif
Look for a file that holds a digital sound
.wav or .aif or .aiff or .au
The result might look like
C:\intro-prog-java\mediasources\cat.jpg
The files hold values that are an encoding of a sound or a picture.
Georgia Institute of Technology

12. Georgia Institute of Technology
Parts of a File Name
Path separators ‘\’
Path – where on the hard disk the file is
C:\intro-prog-java\mediasources\
Base file name
cat.jpg
File extension
.jpg
Georgia Institute of Technology

13. Georgia Institute of Technology
Creating Objects
Remember that a class is a factory that creates objects of that class
We ask a class to create an object by using the keyword: new ClassName
We also ask the class to initialize the object
And pass data to help initialize it
Georgia Institute of Technology

14. Creating a Picture Object
A picture needs data to use to create the picture
One way to create it is to give it the full path name of the file to read the data from
We know how to get the full path name
Using FileChooser.pickAFile()
> System.out.println(new Picture(FileChooser.pickAFile()))
Picture, filename C:\intro-prog-java\mediasources\partFlagSmall.jpg height 217 width 139
Georgia Institute of Technology

15. Georgia Institute of Technology
Where is the Picture?
When you printed the result of
new Picture(FileChooser.pickAFile())
You got the file name, width, and height
So it looks like a picture object was created
But, where is the picture?
To get it to display send the picture object the message “show()”
new Picture(FileChooser.pickAFile()).show()
Georgia Institute of Technology

16. Georgia Institute of Technology
How did that work?
The expression inside the parentheses is evaluated first
FileChooser.pickAFile()
This returns the full path name of a file as a String object
“C:\intro-prog-java\mediasources\cat.jpg”
Next the Picture class was asked to create and initialize a new object
using the full path name of the file as the place to read the picture data from
new Picture(“C:\intro-prog-java\mediasources\cat.jpg”)
Then we asked the new picture object to show itself
pictureObject.show()
Georgia Institute of Technology

17. Georgia Institute of Technology
Naming the Results
Another way to do the same thing is to name each of the results
Gives us a way to refer to the result again
Makes it easier to understand
In Java we must give a type when we create a new name
This is called declaring a variable
Type name; or
Type name = something;
In Java the “=“ doesn’t mean equal but assign or “become the name for”.
Georgia Institute of Technology

18. Georgia Institute of Technology
Assignment Exercise
Try some integer calculations using the assignment operator (‘=‘)
int x = 3;
int y =4;
int z;
z = x + y; // assign z to be x + y
System.out.println(z);
System.out.println(x);
System.out.println(y);
Georgia Institute of Technology

19. Working with Variables
You must declare a variable before you can use it
Declare a variable once and only once
You will get an error if you declare it more than once
You can reuse variables
That you have declared
Georgia Institute of Technology

20. Variables are Temporary
Hold values until
Reassigned
int x = 2;
x = 3;
DrJava is exited
The interactions pane is reset
You can force this by clicking the “Reset” button
Or by right click and choose “Reset Interactions”
Happens automatically when you compile
Georgia Institute of Technology

21. Georgia Institute of Technology
Allowed Types in Java
The type for a variable declaration can be
A primitive type: int, double, char, or boolean
The name of a Class
in the Java language
String, JFrame, BufferedImage, etc
or the name of a class created by you or someone else
Picture, Sound, FileChooser
Note that primitive types are all lowercase. Class names start with an upper case letter.
We created the classes Picture, Sound, and FileChooser. These are not part of the Java language.
The class JFrame is a main window in Swing.
The class BufferedImage is what really holds the picture data.
Georgia Institute of Technology

22. Object and Primitive Variables
Primitive variables cause space allocation for their size
The contents of the space is set to the variable’s value
Object type variables cause space allocation for an address
The contents of the space is the address of the referred to object
Or null if it isn’t referring to an object yet.
int a = 3 a 3
String str =“Hi”;
str
reference
See for a discussion of the sizes of primitives and the differences between primitive variables and object variables.
“Hi”
Georgia Institute of Technology

23. Georgia Institute of Technology
Memory Exercise
Draw the memory used for the following:
int x = 2;
int y = 7;
int z = x + y;
String fullName;
String firstName = “James”;
String lastName = “Clark”;
fullName = firstName + lastName;
Georgia Institute of Technology

24. Georgia Institute of Technology
Naming each Piece
First let’s pick the file name and save a reference to the resulting String object
String fileName = FileChooser.pickAFile();
Next, let’s create the Picture object and save a reference to it.
Picture pictureObj = new Picture(fileName);
Now send the show() message to the picture object
pictureObj.show();
Notice that we don’t need to declare a variable each time we use it. We only need to give the type the first time we mention the name. In fact, the code won’t compile if you try to declare two variables with the same name.
Georgia Institute of Technology

25. Georgia Institute of Technology

26. Georgia Institute of Technology
Naming Each Part
This is showing the picture tammy.jpg. Tammy is a Computer Science student at Georgia Tech.
Georgia Institute of Technology

27. Georgia Institute of Technology
Show Picture Exercise
Try both ways of creating a picture object and showing it.
new Picture(FileChooser.pickAFile()).show()
And do each piece one step at a time naming the result of each method
String fileName = FileChooser.pickAFile();
System.out.println(fileName);
Picture picture = new Picture(fileName);
System.out.println(picture);
picture.show();
Use System.out.println(expression) to see the value of the thing referred to by the variable.
Georgia Institute of Technology

28. Substitution and Evaluation
In programming you can
Use a literal
Use a variable
Use an expression
Use the result of a method invocation
Values get substituted for variable names when expressions are evaluated
Georgia Institute of Technology

29. Georgia Institute of Technology
What is a Sound?
Any object that produces vibrations in matter will make a sound
The vibrations produce a sound wave
The pitch is based on the frequency of the cycles in the sound wave
The loudness is based on the amplitude (height) of the sound wave
See for more information on sound.
Georgia Institute of Technology

30. Georgia Institute of Technology
How does Hearing Work?
The outer ear “catches” sounds
The eardrum vibrates
The inner ear translates the vibrations to nerve impulses for the brain to interpret
See for more information on how hearing works.
See for how hearing works.
Georgia Institute of Technology

31. How does Recorded Sound Work?
Phonograph recordings capture sound continuously, as an analog signal
CDs and DVDs sample sounds and record numbers that represent the sound at the time of the sample
44,100 samples per second
See for more information on analog and digital sound recording.
See for how an analog signal is recorded.
See for information on how a CD is recorded.
Georgia Institute of Technology

32. Georgia Institute of Technology
Why Digitize Sound?
High fidelity
Reproduced sound is very similar to the original
Perfect reproduction
Sounds the same every time
Easy to transmit
Download as data
Easier to manipulate on a computer
Even though there are billions of bits
Georgia Institute of Technology

33. Georgia Institute of Technology
Playing a Sound
We can create a Sound object just as we created a Picture object
Get a file name
String fileName = FileChooser.pickAFile();
Create the sound object by asking the class to create a new Sound object and initialize it by reading data from the given file name
Sound sound1 = new Sound(fileName);
Play the Sound
sound1.play();
Be sure to pick a file that holds a digital sound. These will have the extension .wav.
Georgia Institute of Technology

34. Georgia Institute of Technology
Play Sound Exercise
Try creating a Sound object and playing it by
Specifying it all at once
Specifying it in steps
How would you play the same sound twice?
Georgia Institute of Technology

35. Georgia Institute of Technology
Summary
Computers can do math and make logical comparisons
Computers can execute billions of instructions per second
Computers keep getting faster, smaller, and cheaper
Pictures, sounds, text and movies can be digitized
Media computation can mean processing millions to billions of bytes
The speed of modern computers makes media computation possible even for beginners
Georgia Institute of Technology