1. R Programming & Digital Audio Donald Byrd rev. 14 Jan. 2008 Copyright © 2006-08, Donald Byrd

2. rev. 7 Feb. 20072 Elements of Digital Audio (1) Requirements of discrete time sampling –Pohlmann’s “video of ride over bumpy road” analogy Sampling rate determines maximum frequency –Human hearing goes up to ca. 15-20 KHz –Sampling Theorem (Nyquist, Shannon, etc.): need 2 samples per cycle Less than 2 samples/cycle => aliasing Visual equivalent: wheels “going backwards” –Video & movies have low sampling (frame) rates Practically, need more than 2 CD sampling rate = 44,100 = 20 KHz * 2.205

3. rev. 6 Feb. 20073 Elements of Digital Audio (2) Sample depth (bits per sample) determines freedom from quantization noise –Also called bit depth, sample width, or bit width –SQNR (Signal-to-Quantization Noise Ratio) = about 6 dB per bit –For digital audio, almost always 8, 16, or 24 bits –Usually (CDs, etc.) 16 bits => ca. 96 dB SQNR Exceed maximum sample => clipping –A nasty type of distortion –Very different from overdriving analog media

4. rev. 3 Feb. 20074 Elements of Digital Audio (3) A simple example –Input: sound waves => microphone => Analog to Digital Converter (ADC) => computer, etc. –ADC needs low-pass filter to avoid aliasing –Output: computer, etc. => Digital to Analog Converter (DAC) => loudspeaker => sound waves –DAC needs low-pass filter to avoid imaging (related to & often confused w/ aliasing) NB: theoretically, should apply “sinc” function instead of low-pass filter, but that’s impractical (equiv. to ideal low-pass filter?) Process introduces noise & distortion

5. 15 Sep. 20065 Audio in R: the tuneR Package tuneR is an “add-on” library for R Adds functions to create, work with, & analyze Wave (.wav audio) files Installation: type “install.packages()”, or (with the R GUI) use menu command Packages>Install Packages For more information, see “tuneR” under Packages at http://www.r-project.org/

6. rev. 20 Jan. 20076 Structure of the tuneR Wave Object left: vector containing samples for left channel right: vector containing samples for right channel (NULL if mono) stereo: a boolean to say if stereo or mono samp.rate: sampling rate (e.g., 44,100 = 44,100 samples per sec. for CD quality) bit: sample depth, in bits: controls quantization (usually 16, e.g., for CDs; can be 8 for low quality) left samp.rate right stereo bit An object in R has slots. The Wave object has 5 slots.

7. rev. 3 Apr. 077 Creating a Wave Object from Scratch install.packages() # do only once after installing R on a computer library(tuneR) # do every time you run R & need tuneR setWavPlayer("/Library/Audio/playRWave") # do every time you run R & need to play sounds (OS X only) # # Create & play 2.5-sec. sine wave with pitch about middle C (262 Hz) wavs <- sine(262, duration=2.5, samp.rate=16000, bit=16, xunit="time") play(wavs)

8. rev. 22 Oct. 078 Creating a Wave Object from a File install.packages() # do only once after installing R on a computer library(tuneR) # do every time you need tuneR setWavPlayer("/Library/Audio/playRWave") # do every time you run R & need to play sounds (OS X only) # # Set the working directory to the correct path for your computer. setwd("work") # Read Wave (samples plus sampling rate, depth, etc.) from file; display and play it. wav <- readWave("Piano.mf1st5secMono.A4.wav") print(wav) #plot(wav@left)# slow if it's a long sound! play(wav)

9. rev. 27 Jan. 20079 What Do We Have? play(wav) –Uses whole Wave object plot(wav, nr=1000) –Uses whole Wave object plot(wav@left) ---------> plot(wav@left[1:5000]) –Uses just vector of samples wav –Shows the Wave’s 5 slots: Wave Object Number of Samples: 198562 Duration (seconds): 4.5 Sampling rate (Hertz): 44100 Channels (Mono/Stereo): Mono Bit (8/16): 16

10. rev. 22 Oct. 0710 Wave Manipulation Example #1 # Assumes “Creating a Wave Object” already done samData <- wav@left samData1 <- samData*3 plot(samData1) wav1 <- Wave (left=samData1, samp.rate=wav@samp.rate, bit=wav@bit) play(wav1)

11. rev. 3 Apr. 0711 Wave Manipulation Example #2 R code –# Assumes “Creating a Wave Object” already done –samData <- wav@left –samRate <- wav@samp.rate –samDepth <- wav@bit –newSamRate <- samRate/2^(6/12) –wav2 <- Wave(left=samData, samp.rate=newSamRate, bit= samDepth) –play(wav2) Effect: pitch is 6 semitones = tritone lower

12. rev. 3 Apr. 0712 Wave Manipulation: More Techniques in R Not Wave-specific, just standard R –See “An Introduction to R” (R-intro.pdf) Under Manuals, at http://www.r-project.org/ 1. Extract every nth element –samData3 <- samData[seq(3, len, by=3)] 2. Make two sounds overlap – # Append 0’s to samData3, or there would be NA which causes error later –samData3[len:round(0.5*samRate)] <- 0 –samData3 <- samData3+samData

13. 27 Nov. 0713 Programming in General (1) Details are often vital (& errors are costly) –A great many details really are. Commonly: Quote marks, including single vs. double Capitalization –“Wav” & “wav” are different –TIP: “steal” as much as possible! Via Copy & Paste is ideal: avoids typos Programs tend to be very hard to understand –TIP: include useful, readable comments –TIP: choose variable names for clarity “wavdata” isn’t good; how about “samples”? –TIP: consistency helps clarity and correctness Don’t mix “v = expr”, “v v” Use the same variable name for something in every prog. Program defensively

14. rev. 3 Apr. 0714 Programming in General (2) Comments –Classic example of a bad comment x <- x+1# add 1 to x –Doesn’t explain anything! Good commenting style (thanks to Ed Wolf) # Using the Add Sines Demo, create and play a wave at G3, # then do the same for a wave at 5/4 this frequency. Finally, # normalize the sum of the two waves and listen to result. … # create and play first sound wave swave1 <- sine(f, duration=secs, samp.rate=sr, bit=16, xunit="time") play(swave1) …

15. 3 April 200715 Programming in General (3) Block comments (w/ overall description) more important than comments on single stmts Ideal: say just the right things: Not too much or too little –Basic principle of all human communication –…including this slide show & music notations (CMN, tablature, etc.) –…and comments in a program Other aspects of formatting & style –Variable names Choose variable names for clarity camelCase is helpful –Space around operators –“v <- f(expr)”, not “v<-f(expr)”

16. rev. 20 Jan. 200716 Programming in R (1) R offers to save workspace when you quit –Are you sure it’s what you want? –TIP: Just say no. Can restore original with ‘load( ".Rdata " )’ (?) –TIP: Use a text editor & files to save work If real text editor (not word processor) file, can run with R “source” command Regardless, can Copy & Paste, even just part of file setwd() to correct path for your computer –Depends on where you have files –Can be tricky, esp. in Windows Typical Windows ex.: setwd("C:/Documents and Settings/donbyrd.ADS/Teaching/N560") Easier: use R GUI “Change Working Directory” menu command!

17. rev. 14 Jan. 0817 Programming in R (2) Many useful built-in functions –Many of them handle vectors (no loop needed) diff(v): vector of consecutive differences sum(v): sum of vector elements –Random numbers with various distributions: runif (uniform), rnorm (normal), etc. –read.table, table (and related functions) –fft –tuneR adds sine, square, noise, bind, mono, etc. R (and tuneR) have excellent on-line help –Type either ‘ help(sine) ’ (e.g.) or ‘ ?sine ’ …but NB: sometimes need ‘ help("sine") ’ –TIP: Copy & Paste from help window! –Caveat: terminology is statistics oriented

18. rev. 3 Apr. 0718 Programming in R (3) Introducing loops –Loops are hard for many beginners –A very simple (though pointless) example mnnV <- 1:6# make mnV a 6-place vector mnnV# see what mnnV is before loop for (n in 1:6) { mnnV[n] <- n+59 } mnnV#...and after –Instead of “in 1:6”, can use any vector! –C, Perl, etc. users can put the vector in the “for” for (n in seq(1, 6)) { –Loop is a type of control statement

19. 27 Nov. 0719 Software Engineering & Debugging (1) Experience: all complex programs have bugs –Judge in Florida e-voting case: claim that voting machine software was buggy is speculation –True, but… ! Disclaimer: I don’t know any hard evidence Expect bugs & program defensively True stories –The program that failed only on Wednesdays! Why? Hint: “Wednesday” has 9 characters –Weeks of debugging to find a “1” that should have been “i”

20. 27 Nov. 0720 Software Engineering & Debugging (2) Good engineering (design, coding, comments, etc.) => less debugging & more robust (reliable & flexible) programs Don’t underengineer …but don’t overengineer, either! Underengineering is much bigger danger for inexperienced programmers Main factors –Complexity of problem –Is program or code it includes likely to be used for very long? –If so, how expert future programmers are likely to be

21. 27 Nov. 0721 Software Engineering & Debugging (3) Standard technique: zero in on problem code Debug on short/simple cases, not long/complex ones –Makes it practical to look at results of several print statements –Reduces or eliminates long delays to see results –“short/simple” often means simply not much data –Can easily reduce days of debugging to hours Usually easy to do by turning lots of data into a little data –Real situation: nThemes <- 3500, or 20 sec. audio file –For testing: use nThemes <- 4 (say), or 1 sec. audio –Caveat! the “little” data may not show the bug –…and if bug results from a conceptual problem, fixing it may be very hard

22. 8 Sept. 0722 Debugging in R Basic technique: zero in on bug with print (or cat) & plot –E.g., before & after doing something questionable print(c("max before scaling=", max(wNotes@left))) wNotes <- wNotes*2.5 cat("max after scaling=", max(wNotes@left), “\n”) –cat merges its arguments, gets rid of the extra parens –…but doesn’t end the line => do it yourself with “\n” –If you use “source” (& inside loops?), just naming variable doesn’t work; must use print or cat A good debugger can be very helpful –Especially w/ complex programs, or… –Learning a new language –R has a debugger; one student tried & liked it

23. 3 Sep. 200723 Dangers of R (1) More danger of nasty bugs in R than many programming languages & environments –No explicit types => doesn’t warn of questionable usage –No variable declarations => doesn’t catch typos (only a problem in old versions of R?) –Both above like Perl (e.g.), but Java (e.g.) is great on both => Java programmers likely to be careless! Defensive programming –E.g., add “sanity checks” as you work, use conventions for variable names, etc. –always important: a subtle bug can waste a huge amount of time Weeks of debugging to find a “1” that should have been “i” –…but especially in dangerous environments like R

24. rev. 14 Dec. 0824 Dangers of R & tuneR (2) “Gotchas” in R (all from real life) –Operator precedence, esp. in “for” statement In sets, need parentheses to get addition before “:” E.g., “start:(start+5)”, not “start:start+5” ! –Undocumented requirements (tuneR): some functions fail (with no error message) if frequency isn’t an integer –“;” is usually ignored, but not always –Line break sometimes starts a new statement, not always Other real-life examples from Don’s classes –Undeclared variable: “allNotes” vs. “allnotes” (only a problem in old versions of R?) –Call function that returns a value but ignore value –Nonexistent named params. sometimes give error, not always Danger much worse because R often gives lousy feedback for errors or likely errors –Apparent exception: play w/ unnormalized values –…but that’s tuneR, not R

25. rev. 27 Jan. 200725 Programming in R with tuneR On OS X (and LINUX): play() problem –Must say what program to use to play Waves Either setWavPlayer once, or add 2nd param. to each play() –OS X can use QuickTime Player It’s on every OS X machine, & it works, but… –Usually gives scary error messages; must hit the escape key to get R to continue; leaves open more & more QuickTime Players –OS X alternative: playRWave Works fine, but… Not pre-installed; you must get & install it. Instructions at: –http://www.informatics.indiana.edu/donbyrd/tuneRPlaybac kOnMacs.txt