2. Penn ESE250 Fall2013 -- DeHon 1 ESE250: Digital Audio Basics Day 1: January 15, 2013 Overview Note lecture feedback form Handout: Preclass/Inclass http://www.startrek.com/watch_episode/lRRlMleoSJ9m 10 minutes in

3. Penn ESE250 Fall2013 -- DeHon 2 Star Trek Technology Forward looking Science Fiction Envisioned many wonderful things –Warp Drive –Transporter –Phasor –…and a hand-held communicator http://en.wikipedia.org/wiki/File:20090704-1971_StarTrekTOSCommunicatorReplica.jpg

4. Penn ESE250 Fall2013 -- DeHon 3 This was 1966 Star Trek debut 20lb. Cell phone 1965 –Motorola 1973 Long distance was expensive Computers owned by companies and universities –20 years after ENIAC, 10 years before Apple Internet was conceived (but 3 yrs to first node) Cameras used film Beatles release “Yesterday and Today” on LP –And you had to go to a record store to buy it One year after Gordon Moore hypothesized his famous “Law” http://en.wikipedia.org/wiki/File:2007Computex_e21Forum-MartinCooper.jpg

5. Penn ESE250 Fall2013 -- DeHon 4 Things were brewing… Moore’s Law

6. Penn ESE250 Fall2013 -- DeHon 5 Things were brewing… Moore’s Law Internet Grew

7. Penn ESE250 Fall2013 -- DeHon 6 Things were brewing… Moore’s Law Internet Grew Personal Stereo – Sony Walkman 1979 http://en.wikipedia.org/wiki/File:Sony_Walkman_WM-2.jpg

8. Penn ESE250 Fall2013 -- DeHon 7 Things were brewing… Moore’s Law Internet Grew Personal Stereo – Sony Walkman 1979 Compact Discs 1982 –700MB, 80 minutes music http://en.wikipedia.org/wiki/File:Compact_disc.svg

9. Penn ESE250 Fall2013 -- DeHon 8 Things were brewing… Moore’s Law Internet Grew Personal Stereo – Sony Walkman 1979 Compact Discs 1982 –700MB, 80 minutes music IBM PC/XT 1983 –10MB Hard Disk http://en.wikipedia.org/wiki/File:Ibm_px_xt_color.jpg

10. Poll How many have –Cell phone ? –MP3 player? –Digital camera? Use GPS? How obtain music? How communicate with friends outside of school? –Voice phone, e-mail, text message, facebook, skype? Where do you go to find answers? Penn ESE250 Fall2013 -- DeHon 9

11. In your lifetime… GPS open to public 1993, handheld 1997 Google 1998 First commercial MP3 player 1998, iPod 2001 Wikipedia launched 2001 iTunes launched 2003 Skype initial release 2003 Facebook launched 2004 Twitter launched 2006 Kindle 2007 Siri 2011 Penn ESE250 Fall2013 -- DeHon 10

12. Penn ESE250 Fall2013 -- DeHon 11 Cool Stuff 2013 Today’s Cool Stuff is: computerized, networked, and based on digital media Cell phones (smaller than ST Communicator) MP3 players (make walkman bulky) Internet enabled Digital cameras and video recorders Realistic Video Games Integrated (e.g. iPhone, iPad) DVRs (e.g. TiVo) E-book readers (e.g. Kindle) 3D printers (e.g. Makerbot) Augmented Reality (e.g. Google Glasses) What else add to this list?

13. Penn ESE250 Fall2013 -- DeHon 12 Changing World: Small World Skype and cheap long distance Ubiquitous Internet –(even our grandparents) Ubiquitous SMS texting Facebook Instantly in touch …compare old movies…

14. Penn ESE250 Fall2013 -- DeHon 13 Changing World: Easy Sharing Easy Instant sharing and storage Photos, videos, writing Web, Facebook, Youtube, Blogs Backed up, Cloud Accessible anywhere in the world Indexed and searchable Can carry it with you

15. Penn ESE250 Fall2013 -- DeHon 14 Changing World: Instant Gratification Web Search iTunes Amazon.com Netflix

16. Penn ESE250 Fall2013 -- DeHon 15 Changing World: New Wealth, New Players Microsoft founded 1975 –World’s richest man Apple founded 1976 –Highest valued company Oracle 1977 CISCO 1984 NVIDIA 1993 E-Bay 1995 Google 1998 Facebook 2004 Twitter 2006

17. Penn ESE250 Fall2013 -- DeHon 16 Convergence Big Ideas and Advanced Technology 1.Digitize Everything 2.Cheap Digital Processing 3.Cheap Storage 4.Cheap Digital Bandwidth

18. Penn ESE250 Fall2013 -- DeHon 17 Enabled by Visionary Engineers Hard work, inspiration, and competition –…would not have just happened Most inconceivable just prior –Compare how archaic the “future” looks in most movies just 20 years old What’s next? How can we harness to make the world better?

19. Penn ESE250 Fall2013 -- DeHon 18 Outline Motivational Introduction (completed) Short version: what this class is about Class story: 1 slide / week This Class: operationally Wrapup

20. Penn ESE250 Fall2013 -- DeHon 19 This Class Understand how your smartphone works –Cell phone + MP3 player Use as focal point to understand these –Ideas, trends, technologies Use as context for computer engineering

21. Virtualization of the World Can represent things as bits –Sound, pictures, movies –Location, situation, … –shapes, circuits, drugs, DNA Cheap/powerful ways to automatically manipulate –…and reproduce Penn ESE250 Fall2013 -- DeHon 20 TRON 1982

22. Penn ESE250 Fall2013 -- DeHon 21 Class Story: One Slide Sound can be converted to/from bits –And compressed –Without loss of information More information can be discarded without humans noticing  fewer bits Process this information with inexpensive machines Store it for retrieval Send it between machines –Even if not directly connected

23. Sound in Bits How many bits in a Byte? Bytes in a Megabyte? Bytes in a Gigabyte? How many bits to store a typical song? Penn ESE250 Fall2013 -- DeHon 22

24. Penn ESE250 Fall2013 -- DeHon 23 Course Map Numbers correspond to course weeks

25. Penn ESE250 Fall2013 -- DeHon 24 Longer Story

26. Penn ESE250 Fall2013 -- DeHon 25 Sound Sound is a pressure wave http://www.archive.org/details/SoundWavesAn

27. Sound Waves Penn ESE250 Fall2013 -- DeHon 26 Source: http://www.mediacollege.com/audio/01/sound-waves.html Hz = 1/s 1kHz = 1000 cycles/s

28. Penn ESE250 Fall2013 -- DeHon 27 Pressure to Voltage Microphones convert pressure to voltage –(speakers/headphones voltage to pressure) Parallel plate capacitor with pressure activated plate –Over short time scale Q conserved If change d, –What happens to C? –What happens to V? V d

29. Penn ESE250 Fall2013 -- DeHon 28 Course Map Numbers correspond to course weeks

30. Penn ESE250 Fall2013 -- DeHon 29 Week 2: Discrete Sampling Voltages can be sampled discretely –Both in time and amplitude Can turn sound wave into sequence of bits –0111 1001 1011 1100 1101 1110 1110 1111 1111 …. What precision do we need? Compact Disks –16bits at 44KHz –How many bits is a typical 3- minute song? http://en.wikipedia.org/wiki/File:Pcm.svg http://en.wikipedia.org/wiki/File:Compact_disc.svg

31. Penn ESE250 Fall2013 -- DeHon 30 Course Map Numbers correspond to course weeks

32. Penn ESE250 Fall2013 -- DeHon 31 Digital Storage (week 10) Record bits to non-volatile memory –Store and reproduce sound Media: CD, Hard Disk, Flash http://en.wikipedia.org/wiki/File:Hard_disk_platters_and_head.jpg http://en.wikipedia.org/wiki/File:Compact_disc.svg http://en.wikipedia.org/wiki/File:DSCN0411.JPG

33. Capacity How much memory does your MP3 player have? Penn ESE250 Fall2013 -- DeHon 32

34. Penn ESE250 Fall2013 -- DeHon 33 Course Map Numbers correspond to course weeks

35. Penn ESE250 Fall2013 -- DeHon 34 Week 3: Lossless Compression Statistics of data allow compression If all symbols (characters, voltages) aren’t equally likely, –Can assign shorter bit sequences to most common cases – and reduce bits required Morse Code: THE ̶ ● ● ● ● ● = 6 symbol Not log 2 (26)=5 / letter  15 http://en.wikipedia.org/wiki/File:English-slf2.PNG Enables: Kindle to store 1500 books in 2GB, SMS text

36. Penn ESE250 Fall2013 -- DeHon 35 Course Map Numbers correspond to course weeks

37. Penn ESE250 Fall2013 -- DeHon 36 Week 4: Time-Frequency There are other ways to represent 4 quarter notes 1 whole note Cheat: A5, B5, G5, G4, D5

38. Penn ESE250 Fall2013 -- DeHon 37 Time-Frequency Conversion There are other ways to represent How does the musical staff representation of sound differ from time samples?

39. Penn ESE250 Fall2013 -- DeHon 38 Time-Frequency Conversion There are other ways to represent –Frequency representation particularly efficient 880988784392587 Frequencies in Hertz

40. Penn ESE250 Fall2013 -- DeHon 39 Time-Frequency Conversion 1s / quarter note  8s of sound –44K Hz x 16b/sample x 8s = 5632K =5Mbits –How many keys on piano? –(7b/note+8b/duration) x 5 note = 75 bits?

41. Penn ESE250 Fall2013 -- DeHon 40 Course Map Numbers correspond to course weeks

42. Penn ESE250 Fall2013 -- DeHon 41 Week 5: Nyquist-Shannon We only need to sample twice the maximum frequency component of a signal http://en.wikipedia.org/wiki/File:Pcm.svg

43. Penn ESE250 Fall2013 -- DeHon 42 Course Map Numbers correspond to course weeks

44. Penn ESE250 Fall2013 -- DeHon 43 Week 5: Psychoacoustics Human input apparatus is limited. If we only care about human perception …that reduces the information needed. Limited maximum frequency The Theory behind MP3 http://www.mp3-tech.org/programmer/docs/mp3_theory.pdf http://en.wikipedia.org/wiki/File:Pcm.svg

45. Penn ESE250 Fall2013 -- DeHon 44 Course Map Numbers correspond to course weeks

46. Penn ESE250 Fall2013 -- DeHon 45 Week 7: Psychoacoustic Compression MP3 based on putting these together –Convert to frequencies –Determine frequencies that don’t matter –Quantize (fewer than 16b) for less important Significantly smaller size than raw, sampled bits Bands Lossy Coding

47. Penn ESE250 Fall2013 -- DeHon 46 Course Map Numbers correspond to course weeks

48. Week 8: Hardware To perform decompression –For audio playback Need to perform –a few million operations Operations addition, multiplication –Calculate cosines –Scale values –Add waveforms –per second of audio Penn ESE250 Fall2013 -- DeHon 47

49. How fast does your laptop run? –Cell phone? Modern chips run 100MHz to GHz –  only need one multiplier, adder –Reuse hardware it in time Penn ESE250 Fall2013 -- DeHon 48

50. Early based on PortalPlayer series –Two ARM7TDMI cores –80MHz each Current use ARM7 or ARM8 iPod Processor Penn ESE250 Fall2013 -- DeHon 49

51. Penn ESE250 Fall2013 -- DeHon 50 Week 9: Operating System This hardware can be virtualized and shared among tasks Angry Birds

52. Penn ESE250 Fall2013 -- DeHon 51 Course Map Numbers correspond to course weeks

53. Penn ESE250 Fall2013 -- DeHon 52 Week 10: File Systems The data can be stored persistently, and organized so it can be found again. [Source: http://www.tldp.org/LDP/tlk/fs/filesystem.html] (12) indirect blocks

54. Penn ESE250 Fall2013 -- DeHon 53 Course Map Numbers correspond to course weeks

55. Penn ESE250 Fall2013 -- DeHon 54 Week 12: Networking Bits can be transported between machines –Giving rise to near speed-of-light “travel” and connectivity –If v=c/2, how long to travel to LA? c=3x10 8 m/s; PHL->LA~=4500km

56. Penn ESE250 Fall2013 -- DeHon 55 Course Map Numbers correspond to course weeks

57. Penn ESE250 Fall2013 -- DeHon 56 Week 12: Intellectual Property Who own’s the bits? What is the law? Why is the law? Why should you care (as engineers)? How is the world changing?

58. Penn ESE250 Fall2013 -- DeHon 57 Course Map Numbers correspond to course weeks

59. Penn ESE250 Fall2013 -- DeHon 58 Week 13: User Interfaces These capabilities can be harnessed by all people –Not just engineers …but we must design for people –For the non-engineers

60. Penn ESE250 Fall2013 -- DeHon 59 Course Map Numbers correspond to course weeks

61. Penn ESE250 Fall2013 -- DeHon 60 This Class: Operational

62. Penn ESE250 Fall2013 -- DeHon 61 Class Goals Provide digital audio background for ESE350 –Where might build digital audio platform Context and motivation for CMPE major Appreciate how CMPE, EE, CIS impact today’s world Help you start thinking like engineers

63. Penn ESE250 Fall2013 -- DeHon 62 Outcomes apply knowledge of math, science, and engineering (transforms, sound modeling) design and conduct experiments (psychoaccoustics experiments, hardware bottlenecks, software behavior) design a system to meet needs (design file system) understand professional and ethical responsibility (intellectual property, user interfaces) understand the impact of engineering solutions in global, economic, environmental, and societal context (security and enabled capabilities) knowledge of contemporary issues (grounding in contemporary performance of components, security, user interfaces) use the modern engineering tools (oscilloscopes, graphical programming for signal processing, spreadsheets and programming, benchmarking, and profiling)

64. Penn ESE250 Fall2013 -- DeHon 63 Course Staff Instructors: –Edin Kadric –Coryan Wilson-Shah Supervisor –André DeHon

65. Penn ESE250 Fall2013 -- DeHon 64 Course Organization Lecture once a week -- Tuesday Lab once a week -- Thursday Materials: –Slides, big idea, further reading

66. Penn ESE250 Fall2013 -- DeHon 65 Grading Based on lab work Teams of 2 in lab –We assign; different every week Individual writeups Drop lowest score on attempted labs Read admin handout on policies –Collaboration, lateness, credit adjustment

67. Penn ESE250 Fall2013 -- DeHon 66 Your Action Read: Big Idea Download Lab 1 Prepare for lab –Further instructions on lab assignment Lab 1 on Thursday in Detkin Lab –Moore 101

68. Penn ESE250 Fall2013 -- DeHon 67 Changing World Automated computation changed world –Faster than we imagined World being digitized and refitted for computerized control and mediation –People-to-people, people-to-machines –Infrastructure from bricks/concrete/steel to networking/computers/software Enabling new engineering –Computerization at center Exciting and dangerous Computer Engineering at center

69. Penn ESE250 Fall2013 -- DeHon 68 Parting Thought From 1 st computer to PCs in 30 years –Eniac 1946  Apple 1976 From first PCs to iPhone next 30 years –Apple 1976  iPhone 2007 What will next 30 years hold? –Beginning of your career What will you imagine, create, enable?

70. Penn ESE250 Fall2013 -- DeHon 69 Please Return Lecture Feedback