1. Interdisciplinary Courses and Assignments in Digital Sound Production Jennifer Burg Department of Computer Science Wake Forest University

2. Points of View, Levels of Abstraction sciencearts application theory

3. In what ways is it helpful, in the teaching of computer science, to move students from theory to practice – which can be, essentially, moving them through levels of abstraction? What teaching tools are available to help us do this? My examples are from the realm of digital sound, presented as a computer science course, sometimes with an interdisciplinary flavor.

4. We can tell students: sin(  ) = a/b where a is the length of the side opposite angle  and b is the length of the side adjacent to angle . and This is a sine wave: But a more dynamic presentation can make this clearer. [Interactive tutorial “Visualizing Waveforms in MATLAB”*]Interactive tutorial “Visualizing Waveforms in MATLAB” *Exercises and tutorials in this talk are from The Science of Digital Media by Jennifer Burg, to be published by Prentice-Hall in August 2008.

5. We can show students how to apply a frequency filter to sound in Audition, Music Creator, or Logic: [Screen capture of applying a high shelf filter in Cakewalk Music Creator] Allowing the students to implement their own filters gives them understanding at a lower level of abstraction. [Screen capture of generating a 440 Hz tone in Audition] [MATLAB exercise on FIR and IIR filters] [MATLAB exercise on writing a program for a transfer function to create a filter] [Exercise on creating an FIR filter with the windowing method]MATLAB exercise on FIR and IIR filtersMATLAB exercise on writing a program for a transfer function to create a filterExercise on creating an FIR filter with the windowing method

6. We explain the Fourier transform to students with words and equations:

7. But an interactive tutorial can help to make the concepts clearer. Note: The following links are web addresses. [Interactive tutorial on Fourier transform]Interactive tutorial on Fourier transform [Interactive tutorial on the difference between the DCT and the DFT]Interactive tutorial on the difference between the DCT and the DFT [Interactive tutorial on windowing functions for the Fourier transform]Interactive tutorial on windowing functions for the Fourier transform And an assignment that has you apply the Fourier transform for filtering can be even more revealing, as we’ll see in a moment with an assignment using MAX/MSP. Or we can have students implement the Fourier transform with their own program: [Fourier programming assignment]Fourier programming assignment

8. So what is my point? I’m just trying to reinforce, with examples, something that you already know: Not all students learn the same way. Some think in terms of mathematical abstractions, some algorithmically, some visually, some verbally. Even if they think, habitually, a certain way, it’s good for students to see the concepts presented from different points of view and at different levels of abstraction.

9. Computer learning environments can help us to place students at different levels of abstraction. Collaborative work in these learning environments can provide alternative perspectives from students of different disciplines.

10. Levels of Abstraction Digital Audio and MIDI AuditionMusic Creator MATLAB MAX/MSP C Programs

11. Levels of abstraction, moving from theory to application, represent one axis. The other axis moves from science to the arts. What’s the pedagogic usefulness of moving along this other axis, between disciplines?

12. artists (musicians or digital sound designers) digital audio and MIDI processing Points of View computer scientists When and why compress the dynamic range? Does it sound thin? Should we add reverb? How does compression of dynamic range work? Does it affect frequencies? What filtering tools exist? Should I make my own?

13. My collaborator, Jason Romney, Digital Sound Designer from North Carolina School of the Arts. NSF CCLI grant: “Linking Science,Art, and Practice with Digital Sound” Jennifer Burg, PI Jason Romney, co-PI

14. The experiment: Three graduate students from North Carolina School of the Arts’ School of Design and Production joined seven Digital Media students to try some sound exercises in MATLAB and MAX/MSP.

15. The assignment: Try to create a single-frequency note in digital audio within MSP. See if you can generate a sine wave at the frequency you want. The sound should be in digital audio form. Try to apply a filter to the sound to determine if its frequency component or components are what you expect them to be. If you can get the frequency out of the filter as a number, see if you can then map that number to a MIDI note of the same frequency. That is, send a MIDI output to a MIDI device to see if you can play that same note through the MIDI device.

16. MAX/MSP patch

17. The experiment: Music and theatre students join computer science students in a ½ semester course entitled “Digital Sound for Music and Theatre.” How much will they be able to learn about the science and practice of digital sound production? Will they be able to produce a song or a theatre scene? How will they interact and collaborate?

18. The assignment: A computer science student and theatre student are paired to do the sound for the opening storm scene of Shakespeare’s The Tempest.

19. Storm scene from “The Tempest”

20. To get to the point, is there something to be gained by placing students of different disciplines in collaborative learning situations? … for example, computer science students and students of digital sound design for theatre.

21. It makes the work more INTERESTING, RELEVANT, and EXCITING. The computer science students learn what tools are really useful to practitioners and artists, and how these tools are used. The music and theatre students learn that they can have greater power over their tools by discovering how to do things at a lower level of abstraction.

22. Ideas generated: How to solve a sound design problem for the play Floyd Collins. How to solve a sound design problem for The Lion King. Things that matter to sound designers and things that don’t matter Training your ears.

23. This summer: An 8-week music production workshop with 3 computer science students and 3 music students. We’ll see what happens!