1. Alice and Media Computation: Goals for this session Show interesting things you can do combining Alice and Media Computation. Provides a story-telling motivation to explore image and sound manipulation To see how one integrates Media Computation images and sounds into Alice. In slide set: A brief Java example of Chromakey A set of Python utilities to make simple image and sound manipulation easier. To focus more on Alice and less on textual programming, to start.

2. “Can’t I do everything in Alice?” Why do programming outside of Alice? Sometimes you want special images or sounds in your Alice story. You can make them by programming those images and sounds, then bringing them into Alice. Sometimes you want to add in special movie effects to your Alice movie. Overall, programming outside of Alice is the special effects studio for Alice.

3. Alice and Media Computation Intro 3 Sound Frequency Project Create an Alice world with 3 frogs Resize one to be bigger and one to be smaller Using croak.wav Create a higher sound Take every other sample Create a lower sound Take every sample twice Import the sound files and have the frogs play their sounds and tap their foot

4. Alice and Media Computation Intro 4 Using Chromakey Create objects in Alice and position them as desired Delete the ground Set the color of the background to green in the world properties Use a conditional to replace the green color You may also need to scale one of the pictures

5. Alice and Media Computation Intro 5 Drawing Projects Draw something on an Alice picture Like a hat, glasses, or a scarf Fireworks Create images in Media Computation that you use as textures in Alice like a gradient for the ground Clip an Alice picture To look like a porthole Create a poster advertisement for your Alice movie. With your name on it Create a comic panel with Alice pictures

6. Alice and Media Computation Intro 6 Comic Strip Project Create several scenes in Alice and take pictures of the scenes Use Media Computation to create a Comic Panel and/or Comic Strips Draw different types of text balloons Speech balloon, thought balloon, scream balloon

7. Alice and Media Computation Intro 7 Add Yourself to an Alice Movie

8. Alice and Media Computation Intro 8 Add Alice Characters to a Real Movie Split out frames from a movie of people Capture frames from an Alice movie using MovieCapturePanel Use chromakey to copy the Alice items to the real movie

9. How to use image manipulation with Alice A. Create an image to appear as a Billboard in an Alice world. B. Create images in Alice that you can manipulate as snapshots. Like with chromakey C. Make movies in Alice that you can then manipulate as video.

10. A. Loading images as billboards in Alice Step 1: Make a cool image that you want to appear as an object in your Alice world. Step 2: Import the image as a Billboard.

11. Step 3: Move it around and use it like an object

12. B. Manipulating an Alice world snapshot Step 1: Set up an Alice world the way you want it. If you want to chromakey, drag the “Ground” into the trash make the “Atmosphere Color” of the world blue (default) or green.

13. Creating the snapshot Step 2: Program your objects to move where you want them to go. Step 3: Play your world, and at the right time, hit “Pause” and “Take Picture”

14. Snapshot from Tutorial, Moved to the Moon >>> capturedbirds = makePicture( getMediaPath("capture00.jpeg")) >>> capturedbirds Picture, filename W:\mediasources\capture00.jpeg height 360 width 666 >>> birds = scale(capturedbirds, 0.75) >>> birds Picture, filename None height 271 width 500 >>> chromakeyBlue(birds,moon)

15. C. Manipulating an Alice video from Python Step 1. Tell Alice not to remove frames after making a video! (Change Preferences.) Only have to do this once.

16. Record your video Step 2: From File menu, choose “Export Video.” Click “Record” then “Play” When done, “Stop Recording” and “Export Video.”

17. Your frames are now saved

18. Doing Chromakey to a Set of Frames import os def chromakeySet(frames,background): newdir = "C:/temp/" for file in os.listdir(frames): if file.endswith(".jpg"): printNow(file) frame = makePicture(frames+file) chromakeyBlue(frame,background) writePictureTo(frame,newdir+file)

19. Using Chromakey to put Penguin in the jungle >>> jungle = makePicture( getMediaPath("jungle.jpg")) >>> chromakeySet( "W:/mediasources/frames/", jungle)

20. Using newly created sounds in Alice

21. Making your characters play your sound

22. Choosing to import a sound

23. All set!

24. A Java-based example Doing Chromakey with an Alice-generated image, from Java, to demonstrate the analagous methods.

25. Get a picture from Alice for Chromakey from Java Click on Edit and then on Preferences and then on the Screen Grab tab Change image format to png (has loseless compression) Set up the scene in Alice Add the objects and position them as desired ChromakeyAlice 25

26. Get a picture from Alice for Chromakey Right click on ground in the object tree And then select delete If you need to change the sky (atmosphere) color Click on world in the object tree Then click on properties in the detail window Change the atmosphereColor to green ChromakeyAlice 26

27. Save the picture to a file Click on the Play button in Alice Then click on the Pause button to stop the action when you want to take a picture Click on the Take Picture button to save a picture Usually on the desktop as captureXX.png ChromakeyAlice 27

28. Chromakey Method /** * Method to do chromakey using the passed background * color and the distance to the color * @param newBg the new background image to use to replace * @param color the background color to compare to * @param dist the distance that limits the chromakey * it will happen if the distance is less than or equal * to this value */ public void chromakey(Picture newBg, Color color, double dist) { Pixel currPixel = null; Pixel newPixel = null; ChromakeyAlice 28

29. Chromakey Method - cont // loop through the columns for (int x=0; x<getWidth(); x++) { // loop through the rows for (int y=0; y<getHeight(); y++) { // get the current pixel currPixel = this.getPixel(x,y); /* if the color at the current pixel is within the passed distance of the * chromakey color then change the background */ double currDist = currPixel.colorDistance(color); ChromakeyAlice 29

30. Chromkey Method - cont if (currDist <= dist) { newPixel = newBg.getPixel(x,y); currPixel.setColor(newPixel.getColor()); } ChromakeyAlice 30

31. Chromakey onto current picture /** * Method to do chromakey by copying the pixels that have a larger distance * to the passed source color than the passed distance. * @param sourcePict the picture with the background color * @param sourceColor the background color to compare to * @param dist the distance from the color to use * @param xSourceStart the starting X for the source picture * @param ySourceStart the starting Y for the source picture * @param xSourceEnd the ending X for the source picture * @param ySourceEnd the ending Y for the source picture * @param xTarget where to copy to in target (this) * @param yTarget where to copy to in target (this) */ ChromakeyAlice 31

32. Chromakey Method public void chromakey(Picture sourcePict, Color sourceColor, double dist, int xSourceStart, int ySourceStart, int xSourceEnd, int ySourceEnd, int xTarget, int yTarget) { Pixel currPixel = null; Pixel newPixel = null; // loop through the columns and rows for (int x=xSourceStart, tX = xTarget; x<xSourceEnd; x++, tX++) { for (int y=ySourceStart, tY = yTarget; y<ySourceEnd; y++, tY++) { ChromakeyAlice 32

33. Chromakey - cont // get the current pixel currPixel = sourcePict.getPixel(x,y); /* if the color at the current pixel is greater than the * passed distance from the source color then copy it */ if (currPixel.colorDistance(sourceColor) > dist && tX < this.getWidth() && tY < this.getHeight()) { newPixel = this.getPixel(tX,tY); newPixel.setColor(currPixel.getColor()); } ChromakeyAlice 33

34. Prepare for chromakey Get the color that the background is Blue = (73,124,204) Figure out the boundary of what you want to copy 0,0, width-5,height Figure out where to copy to 155,170 ChromakeyAlice 34

35. Chromakey Result Picture dino = new Picture(FileChooser.pickAFile()) Picture beach = new Picture("beach.jpg"); import java.awt.Color; beach.chromakey(dino,new Color(73, 124, 204),10,0,0, dino.getWidth()-5,dino.getHeight(),155,170) beach.explore(); ChromakeyAlice 35

36. Challenge Try to use the chromakey method on an Alice picture You can use one you have created Or use one from the AlicePicts folder ChromakeyAlice 36

37. Introducing Image Manipulation for Alice We’ll start with Python, introducing some very simple code for making it easy to do the image and sound manipulations for Alice. Available in the python-book folder. We’ll do one example in Java.

38. What color shirt? Where is it? Y=210 X=220 X=400 >>> shirt = makePicture(pickAFile()) >>> explore(shirt) >>> print shirt Picture, filename C:\mediasources\IMG_0808.JP G height 480 width 640

39. distance() function tells how close colors are >>> print white color r=255 g=255 b=255 >>> print black color r=0 g=0 b=0 >>> print distance(white,black) 441.6729559300637 >>> print red color r=255 g=0 b=0 >>> print pink color r=255 g=175 b=175 >>> print distance(red,pink) 247.48737341529164

40. Changing the whole shirt def changeAllShirt(picture): myGreen = makeColor(132,240,204) for pixel in getPixels(picture): color = getColor(pixel) if distance(color,myGreen) < 200: setColor(pixel,pink)

41. Results

42. Just doing part of the picture, and changing fewer pixels def changeShirt(picture): myGreen = makeColor(132,240,204) for x in range(220,400): for y in range(210,480): pixel = getPixel(picture,x,y) color = getColor(pixel) if distance(color,myGreen) < 50: setColor(pixel,pink)

43. Results

44. Coloring the shirt from a different picture >>> beach = makePicture(pickAFile()) >>> shirt = makePicture(pickAFile())

45. Swap a new background for a color def newBackgroundShirt(picture,background): myGreen = makeColor(132,240,204) for x in range(220,400): for y in range(210,480): pixel = getPixel(picture,x,y) color = getColor(pixel) if distance(color,myGreen) < 100: backgroundPixel = getPixel(background,x,y) backgroundColor = getColor(backgroundPixel) setColor(pixel,backgroundColor)

46. Results

47. How do we change the size of a picture? To copy a picture, get the color values from pixels in one picture, and set those color values in the pixels in the other picture. def copyPicture(picture): returnPicture = makeEmptyPicture(getWidth(picture),getHeight(picture)) for pixel in getPixels(picture): color = getColor(pixel) setColor(getPixel(returnPicture,getX(pixel),getY(pixel)),returnPicture) return returnPicture

48. To scale the picture smaller = scale(picture,0.5) How does it work? To get it to be only ½ the size (in both horizontal and vertical) directions, we need to lose some pixels. Easiest way: Skip a few. Every other pixel gets copied. Can generalize this for any size scaling.

49. Code to scale def scale(picture,factor): newHeight = int(factor*getHeight(picture))+1 newWidth = int(factor*getWidth(picture))+1 returnPic = makeEmptyPicture(int(newWidth),int(newHeight)) sx = 0 for tx in range(0,newWidth): sy = 0 for ty in range(0,newHeight): if (int(sx) < getWidth(picture)) and (int(sy) < getHeight(picture)): sp = getPixel(picture,int(sx),int(sy)) tp = getPixel(returnPic,tx,ty) setColor(tp,getColor(sp)) sy = sy + (1/factor) sx = sx + (1/factor) return returnPic

50. How do we grab part of a picture? guys = makePicture(getMediaPath("IMG_0805.JPG")) james = copyPartPicture(guys,352,217,618,475) copyPartPicture(picture,startX,startY,endX,endY) Gives you a new picture inside the box defined by upper-left corner and lower-right corner.

51. Code to grab part of a picture def copyPartPicture(picture,x1,y1,x2,y2): returnPicture = makeEmptyPicture(x2-x1,y2-y1) targetx = 0 for srcx in range(x1,x2): targety = 0 for srcy in range(y1,y2): srcpixel = getPixel(picture,srcx,srcy) targetPixel = getPixel(returnPicture,targetx,targety) setColor(targetPixel,getColor(srcpixel)) targety = targety + 1 targetx = targetx + 1 return returnPicture

52. How do we combine parts of a picture? grayscale(betsy) clearRed(james) pastePicture(betsy,scale(james,0.35),0,0) pastePicture(canvas,partial,startX,startY) Pasting “partial” picture onto the “canvas” starting from (startX,startY)

53. Code to paste a picture def pastePicture(canvas,source,startx,starty): targetx = startx for x in range(0,getWidth(source)): targety = starty for y in range(0,getHeight(source)): srcpixel = getPixel(source,x,y) if (targetx < getWidth(canvas)) and (targety < getHeight(canvas)): targetPixel = getPixel(canvas,targetx,targety) setColor(targetPixel,getColor(srcpixel)) targety = targety + 1 targetx = targetx + 1

54. How do use programming to replace re-typing code? Building a collage of pictures: def makeACollage(): betsy = makePicture(getMediaPath("IMG_0802.JPG")) hunter = makePicture(getMediaPath("IMG_0808.JPG")) guys = makePicture(getMediaPath("IMG_0805.JPG")) james = copyPartPicture(guys,352,217,618,475) grayscale(betsy) clearRed(james) pastePicture(betsy,scale(james,0.35),0,0) maxBlue(guys) pastePicture(betsy,scale(guys,0.35),450,340) negative(hunter) pastePicture(betsy,scale(hunter,0.35),0,340) writePictureTo(betsy,"C:/collage.jpg")

55. Result:

56. How do we replace the background of a picture with something else? Take a picture with a background that is easy to test and isn’t in the foreground. Two examples. Which one do you think will work?

57. Code to chromakey for any background color def chromakey(picture,background,color,threshold=100.0): for pixel in getPixels(picture): pixelColor = getColor(pixel) if distance(color,pixelColor) < threshold: newColor = getColor(getPixel(background,getX(pixel),getY(pixel))) setColor(pixel,newColor)

58. Trying to make the yellow background work >>> chromakey(yellowBarb, moon, makeColor(167,159,110)) >>> writePictureTo(yellowBarb, "C:/yellowBarb-th100.jpg") >>> chromakey( yellowBarb, moon, makeColor(167,159,110),50) >>> writePictureTo( yellowBarb, "C:/yellowBarb-th50.jpg") Yellow is too close to Barb’s skin tone to work.

59. Using the general chromakey form for a blue background >>> chromakey(blueJenny,moon,makeColor(36,62,95))

60. Doing a specific version of Chromakey Coming up with a test for being “truly blue” def chromakeyBlue(picture,background): for pixel in getPixels(picture): pixelColor = getColor(pixel) if getRed(pixel)<getBlue(pixel) and getGreen(pixel)<getBlue(pixel): newColor = getColor(getPixel(background,getX(pixel),getY(pixel))) setColor(pixel,newColor)

61. Nicely chromakeyed