03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 1 ENEE408G: Capstone Design Project: Multimedia Signal Processing Design Project 4: Digital Audio Processing

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 2 Outline of Design Project 4 Part I : Perceptual Coding and MP3 Part II : Digital Audio Watermarking Part III: Synthetic Audio (I): MIDI Part IV: Essays: IP rights on Digital Media Part V: Mobile Computing and Pocket PC Programming Bonus part I: Synthetic Audio (II): MPEG4-SA

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 3 Part I. Perceptual Coding & MP3 (1) Anatomy of Human Ear

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 4 Part I. Perceptual Coding & MP3 (2) Absolute Threshold of Hearing

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 5 Part I. Perceptual Coding & MP3 (3) Simultaneous Masking

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 6 Part I. Perceptual Coding & MP3 (4) Audio Extraction by GoldWave Compare spectrum for original audio and reconstructed MP3 audio difference

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 7 Pat II. Digital Audio Watermarking (1) Watermark Embedding, Detection, and Attack Use AudioMark to embed, detect, and attack Digital Audio Watermarking Design Digital Watermarking in Time Domain Digital Watermarking in Frequency Domain Pseudo-Noise Sequence (Optional)

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 8 Part II. Digital Audio Watermarking (2) Pseudo-Noise Sequence For more reference: : Simon Haykin: Communication Systems, 4 th edition, Wiley, 2000

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 9 Part II. Digital Audio Watermarking (3) Time Domain Convert message into bits Put the bits into LSBS of the audio samples Repeatedly embed each message bit in a number of audio samples for a more reliable detection

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 10 Part II. Digital Audio Watermarking (4) DCT Domain

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 11 Part III. Synthetic Audio (I) Musical Instrument Digital Interface (MIDI)

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 12 Part IV. Essay and Ethics Assignment IP Rights in Digital Media in Engineering Ethics Intellectual Property and copyright play important roles in contemporary society. Discuss the issues of IP rights protection for digital multimedia among your team, and interview your friends/families to see what they think about this. Write an essay to report and summarize your friends and opinions Ethics essay: follow Part IV instruction (due in two weeks) Ethics discussion and assignment is on Wednesday 04/06 (due date will be announced later)

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 13 Part V. Pocket PC Programming From the above three parts, we have learned the fundamentals of digital audio processing. Now design a simple Pocket PC application related to digital audio processing using the Microsoft eMbedded Tools. Ideas: Demo Project 4(1): Piano Demo Project 4(2): TCP/Playsound Demo Project 4(3): MP3/MIDI player

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 14 Bonus Part I. Synthetic Audio (II) Structured Audio: One “component” in the MPEG audio standard. MPEG 4 audiosystemvideo SA Natural codingSynthetic coding AACT/FCELPParametric TTS ISO/IEC sec5 From John Wawrzynek's talk

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 15 Bonus Part I. Synthetic Audio (II) Idea: Musical performance have temporal structure that changes over several timescales MPEG-4 Structured-Audio Structured Audio Orchestra Language (SAOL) Structured Audio Score Language (SASL) Sample-by-sample 10’s of usec Amplitude & timbre envelopes: 10’s of msec Note-by-note: 100’s of msec From John Wawrzynek's talk For more reference: John Wawrzynek's talk

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 16 Bonus Part I. Synthetic Audio (II) SPlay SNet

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 17 Announcement All files related to this project can be found in the AudioZip.zip and the bonus part can also be found in Bonus.zip. The grade for this project weights 12% of the final grade: 8% for technical tasks and 4% for the ethics essay and assignment. Matlab tasks: 2 (Part II. 2), Pocket PC task: 1

03/18/2005ENEE408G Spring 2005 Multimedia Signal Processing 18 Announcement Project reports and Matlab files need to be improved in their structures and descriptions. TA evaluation.