1. A Media Computation Cookbook Manipulating Images and Sounds for Use in Alice Part 1: Image Manipulations Part 2: Advanced Image Manipulations, e.g., changing colors in an area Part 3: Chromakey for digital video effects Part 4: Manipulated Alice images Part 5: Sound manipulations Part 6: Manipulated sounds in Alice

2. How sound works: Acoustics, the physics of sound Sounds are waves of air pressure ◦ Sound comes in cycles ◦ The frequency of a wave is the number of cycles per second (cps), or Hertz  Complex sounds have more than one frequency in them. ◦ The amplitude is the maximum height of the wave The amplitude of the sound determines the volume. The frequency of the sound determines the

3. Live demos here! See real sounds, and real musical instruments

4. Digitizing Sound: How do we get that into numbers? We take the height of the wave at regular intervals. ◦ That’s called the Sampling Rate. ◦ Each sample is between -32768 and +32767 ◦ How many samples do we need? Nyquist Theorem: ◦ We need twice as many samples as the maximum frequency in the sound in order to represent (and recreate, later) the original sound. Phones capture 8,000 samples per second (8 Kilohertz, Khz). Highest sound humans can hear is 22 Khz. CD quality sound is 44,100 samples per second 500, 1000, 1250, 1312, 1475,…

5. Loading sounds from WAV files >>> sound = makeSound(pickAFile()) >>> explore(sound)

6. Where to get sounds? Any.WAV file from anywhere on the Internet. Speech.zip on http://coweb.cc.gatech.edu/mediaComp- teach/43 http://coweb.cc.gatech.edu/mediaComp- teach/43 ◦ Expand it and put it in your media folder. Musicsounds.zip on same page.

7. How do you increase volume? >>> increaseVolume(sound) >>> explore(sound) def increaseVolume(sound): for s in getSamples(sound): value = getSampleValue(s) setSampleValue(s, ???)

8. Code for sounds makeSound(file) – makes a sound explore(sound) – look at and plaly sounds getSamples(sound) – returns a list of samples. getSampleValue(sample)/ setSampleValue(sample, – gets and changes samples getSampleValueAt(sound,inde x)/setSampleValueAt(sound,in dex,value) getLength(sound) – returns number of samples in sound. makeEmptySound(len) – makes a new sound with # of samples. writeSoundTo(sound, filename) – writes out the sound (end filename with “.wav”)

9. What if you set all the samples as big as you can? def maximize(sound): for s in getSamples(sound): value = getSampleValue(s) if value >= 0: setSampleValue(s,32768) if value < 0: setSampleValue(s,-32767) Can you make out what sound this is?!? >>> maximize(sound) >>> explore(sound)

10. Splicing sounds together >>> setMediaPath() 'D:\\WorkingDocs-Copy\\Glitch-MediaComp\\musicsounds\\' >>> guzdial = makeSound(getMediaPath("guzdial.wav")) >>> issnd = makeSound(getMediaPath("is.wav")) >>> inside = makeSound(getMediaPath("inside.wav")) >>> guzdialis = makeSplice(guzdial, issnd) >>> play(guzdialis) >>> guzdialisinside = makeSplice(guzdialis,inside) >>> explore(guzdialisinside)

11. Code for makeSplice def makeSplice(sound1,sound2): newSound = makeEmptySound(getLength(sound1)+getLength(sound2)) index = 0 for s in getSamples(sound1): setSampleValueAt(newSound,index,getSampleValue(s)) index = index + 1 for s in getSamples(sound2): setSampleValueAt(newSound,index,getSampleValue(s)) index = index + 1 return newSound

12. Mixing sounds to make a chord def makeChord(): notec = makeSound(getMediaPath("brass-c4.wav")) notee = makeSound(getMediaPath("brass-e4.wav")) noteg = makeSound(getMediaPath("brass-g4.wav")) mixce = makeMix(notec,notee) play(mixce) mixceg = makeMix(mixce,noteg) play(mixceg) return mixceg >>> chord=makeChord() >>> explore(chord)

13. Mixing sounds in different amounts def makeChord(): notec = makeSound(getMediaPath("brass-c4.wav")) notee = makeSound(getMediaPath("brass-e4.wav")) noteg = makeSound(getMediaPath("brass-g4.wav")) mixce = makeMix(notec,notee) play(mixce) mixceg = makeMix(mixce,noteg,0.75,0.25) play(mixceg) return(mixceg) >>> chord=makeChord() >>> explore(chord)

14. Making an Echo >>> always = makeSound(getMediaPath(“always.wav")) >>> explore(always) >>> echo = makeEcho(always) >>> explore(echo)

15. How the echo code works def makeEcho(sound,spacing=10000,repeat=5): newSound = makeEmptySound(getLength(sound)+spacing*repeat) #Put the initial sound in index = 0 for s in getSamples(sound): setSampleValueAt(newSound,index,getSampleValue(s)) index = index + 1 #Now put the echoes in for echoes in range(1,repeat): echoSound = makeEmptySound(getLength(sound)+spacing*repeat) index = echoes * spacing for s in getSamples(sound): setSampleValueAt(echoSound,index,getSampleValue(s)) index = index + 1 newSound = makeMix(newSound,echoSound,0.9,0.1) return newSound

16. Grabbing part of a sound >>> this = makeSound(getMediaPath("thisisatest.wav")) >>> explore(this) >>> testSound = makePartSound(this,47000,58100) >>> explore(testSound)

17. Code for grabbing part of a sound def makePartSound(sound,start,finish): newSound = makeEmptySound(finish+start) index = 0 for src in range(start,finish): value = getSampleValueAt(sound,src) setSampleValueAt(newSound,index,value) index = index + 1 return newSound

18. Making sounds lower or higher >>> shorter = makeScaleSound(this,0.5) >>> explore(shorter) >>> longer = makeScaleSound(this,2.0) >>> explore(longer) >>> same = makeScaleSound(this,1.0)

19. How we make sounds higher or lower def makeScaleSound(sound,factor): newLength = int(getLength(sound)*factor) newSound = makeEmptySound(newLength) src = 0 for index in range(0,newLength): value = getSampleValueAt(sound,int(src)) setSampleValueAt(newSound,index,value) src = src + 1.0/factor return(newSound)

20. Scaling is always the same def makeScaleSound(sound,factor): newLength = int(getLength(sound)*factor) newSound = makeEmptySound(newLength) src = 0 for index in range(0,newLength): value = getSampleValueAt(sound,int(src)) setSampleValueAt(newSound,index,value) src = src + 1.0/factor return(newSound) def scale(picture,factor): newHeight = int(factor*getHeight(picture))+1 newWidth = int(factor*getWidth(picture))+1 returnPic = makeEmptyPicture(int(newWidth),int(new Height)) sx = 0 for tx in range(0,newWidth): sy = 0 for ty in range(0,newHeight): if (int(sx) < getWidth(picture)) and (int(sy) < getHeight(picture)): sp = getPixel(picture,int(sx),int(sy)) tp = getPixel(returnPic,tx,ty) setColor(tp,getColor(sp)) sy = sy + (1/factor) sx = sx + (1/factor) return returnPic

21. Try it! Make a function to decrease volume Make a sentence by splicing words together, or make a song by splicing notes in a series. Make a recording of yourself saying something, then make it ECHO…