Presentation is loading. Please wait.

Presentation is loading. Please wait.

Audio Information Retrieval and Audio Search By Chris Mc Coy.

Similar presentations


Presentation on theme: "Audio Information Retrieval and Audio Search By Chris Mc Coy."— Presentation transcript:

1 Audio Information Retrieval and Audio Search By Chris Mc Coy

2 Brief Overview The basics of audio information retrieval and audio search The basics of audio information retrieval and audio search What are some audio IR mechanisms? What are some audio IR mechanisms? Briefly: Compression Briefly: Compression Searching by audio query Searching by audio query How it works How it works The future of audio IR and audio search The future of audio IR and audio search

3 The Basics of Audio Search and Audio Information Retrieval (cont.) What is it? What is it? Audio information retrieval is the process of retrieving audio information from the available resources Audio information retrieval is the process of retrieving audio information from the available resources Text based searching for audio information is most common Text based searching for audio information is most common Audio search (content-based retrieval) is a method of retrieving information by using a piece of audio information (ex: a melody of a song) Audio search (content-based retrieval) is a method of retrieving information by using a piece of audio information (ex: a melody of a song)

4 Why is Audio IR important? Audio information in the form of music is one of the dominant forms of entertainment Audio information in the form of music is one of the dominant forms of entertainment News reporting News reporting Comedy audio segments Comedy audio segments Online radio and sports broadcasts Online radio and sports broadcasts Audio in presentations leads to a more interesting and interactive product Audio in presentations leads to a more interesting and interactive product Research and homework Research and homework

5 What are some Audio IR Mechanisms? Text based Text based Peer-to-peer file sharing software Peer-to-peer file sharing software FTP FTP Streaming audio Streaming audio Websites Websites Online network drives Online network drives Clip Art Clip Art Audio search devices and software Audio search devices and software

6 Peer-to-Peer File Sharing Software Peer-to-peer software connects one computer to another directly without a central point of management Peer-to-peer software connects one computer to another directly without a central point of management Information can be queried and results transferred from one computer to another computer Information can be queried and results transferred from one computer to another computer

7 Peer-to-Peer File Sharing Software (cont.) Some common software examples Some common software examples Bearshare Bearshare KaZaA KaZaA WinMX WinMX Others: eDonkey2000, Furi, Blubster, Grokster, Madster, etc. Others: eDonkey2000, Furi, Blubster, Grokster, Madster, etc.

8 Free download version and $19.95 pay version (6 months) with extra features Free download version and $19.95 pay version (6 months) with extra features Retrieves the following audio information Retrieves the following audio information Movie and television sound clips Movie and television sound clips Music songs (.mp3 most common form) Music songs (.mp3 most common form) Historic news reports Historic news reports Famous speeches Famous speeches Other types of media: Other types of media: Videos, pictures, text documents, etc. Videos, pictures, text documents, etc.

9

10

11 Peer-to-Peer File Sharing Software (cont.) Problems Problems Spyware Spyware Programs that collect information about the user and usage of the computer Programs that collect information about the user and usage of the computer Virus transmittal Virus transmittal Trojan horse Trojan horse Ex: “Rolling Stones – Ruby Tuesday.exe” Ex: “Rolling Stones – Ruby Tuesday.exe” Too many unrelated results returned Too many unrelated results returned Query for the music band “Love” Query for the music band “Love” Thousands of songs with “love” in the title are returned Thousands of songs with “love” in the title are returned Mismatched names and identification info Mismatched names and identification info

12 Compression Audio Compression is a method used to decrease the size of an audio file to conserve disk space Audio Compression is a method used to decrease the size of an audio file to conserve disk space MP3 is one of the most common forms of compressed audio MP3 is one of the most common forms of compressed audio MP3 is a lossy format MP3 is a lossy format Depending on quality: 1/5 or 1/10 size of.wav Depending on quality: 1/5 or 1/10 size of.wav Lossy format – tradeoff: sound quality and file size Lossy format – tradeoff: sound quality and file size Other common types of compressed audio: Other common types of compressed audio: Real Audio (lossy format) Real Audio (lossy format).SHN (Shorten) (loseless format).SHN (Shorten) (loseless format) Controversy exists over the sharing of audio information in compressed format Controversy exists over the sharing of audio information in compressed format

13 Joke

14 FTP Common storage area for audio information Common storage area for audio information Many FTP cater to particular areas of audio Many FTP cater to particular areas of audio FTP sites for trading rare music from a particular band FTP sites for trading rare music from a particular band Old archived radio or historic audio files Old archived radio or historic audio files To find some FTP sites do a search on a search engine (ex: “Beatles FTP”) To find some FTP sites do a search on a search engine (ex: “Beatles FTP”)

15 FTP (cont.) Information needed Information needed IP (ex: ) IP (ex: ) Port (ex: 21) Port (ex: 21) Login (ex: music) Login (ex: music) Password (ex: mp3) Password (ex: mp3)

16

17 Streaming Audio Used on many commercial websites to provide sound samples of music Used on many commercial websites to provide sound samples of music Ex: Ex: Allows for quick audio information retrieval Allows for quick audio information retrieval No permanent download needed No permanent download needed Used for many news and radio broadcasts Used for many news and radio broadcasts Ex: (live radio broadcasts) Ex: (live radio broadcasts)www.nfl.com Real Audio and Windows Media Player are the most common players Real Audio and Windows Media Player are the most common players

18

19

20 Websites Some websites have audio information available for download free of cost Some websites have audio information available for download free of cost Usually stored on their own personal storage space Usually stored on their own personal storage space

21 Online network drives Good for sharing audio information between people on the same network Good for sharing audio information between people on the same network Students on a residence hall network Students on a residence hall network Employees at work on the same network Employees at work on the same network Positives Positives Fast transfer between computers Fast transfer between computers Negatives Negatives No transfers if network is down No transfers if network is down Virus transmittal Virus transmittal

22 Clip Art Good for finding audio information for presentations or laughs Good for finding audio information for presentations or laughs Power Point has a built in sound clip organizer which you can query by text Power Point has a built in sound clip organizer which you can query by text

23 Audio Search: Content-Based Retrieval New developments and technologies allow querying IR mechanisms by audio New developments and technologies allow querying IR mechanisms by audio New audio mining (aka audio indexing) tools allow both speech processing and search technology all in one package New audio mining (aka audio indexing) tools allow both speech processing and search technology all in one package Data can be time stamped and queried later by speech or by text Data can be time stamped and queried later by speech or by text Good for referencing logged business calls Good for referencing logged business calls

24 Audio Search: Content-Based Retrieval (cont.) New device by Philips Electronics in the Netherlands (hope to hit consumer market 2004) New device by Philips Electronics in the Netherlands (hope to hit consumer market 2004) Microphone device captures your voice Microphone device captures your voice “Audio fingerprints” are determined “Audio fingerprints” are determined Melody query then is sent to a database Melody query then is sent to a database Results are returned Results are returned Good for finding a song you don’t Good for finding a song you don’t know by name but know by tune know by name but know by tune

25 Audio Search: Content-Based Retrieval (cont.) Attributes of an audio signal used to index Attributes of an audio signal used to index Amplitude - the maximum amount of displacement of a particle on the medium from its rest position Amplitude - the maximum amount of displacement of a particle on the medium from its rest position Frequency - how often the particles of the medium vibrate when a wave passes through the medium Frequency - how often the particles of the medium vibrate when a wave passes through the medium

26 Audio Search: Content-Based Retrieval (cont.) Other attributes used to index audio information Other attributes used to index audio information Average energy: loudness of audio signals Average energy: loudness of audio signals Bandwidth: frequency range of a sound Bandwidth: frequency range of a sound Brightness: Midpoint of the energy distribution of a sound Brightness: Midpoint of the energy distribution of a sound Harmony: In harmonic sound the spectral components are mostly whole number multiples of the lowest, and most often, the loudest frequency. The lowest frequency is called fundamental frequency Harmony: In harmonic sound the spectral components are mostly whole number multiples of the lowest, and most often, the loudest frequency. The lowest frequency is called fundamental frequency Pitch: how high a sound is; use fundamental frequency as an approximation Pitch: how high a sound is; use fundamental frequency as an approximation

27 Audio Search: Content-Based Retrieval (cont.) Positives Positives Quick and easy searching of databases Quick and easy searching of databases Less problems with text labeling Less problems with text labeling Negatives Negatives Sometimes there are difficulties with speech and sound recognition Sometimes there are difficulties with speech and sound recognition

28 The Future of Audio IR Unanswered Questions Unanswered Questions Will faster computers = faster audio IR and search mechanisms? Will faster computers = faster audio IR and search mechanisms? What direction will the new audio IR systems head towards? Content-based retrieval or text based retrieval? What direction will the new audio IR systems head towards? Content-based retrieval or text based retrieval? How will new file storage mediums and new compression methods affect audio IR? How will new file storage mediums and new compression methods affect audio IR? What will the impact of querying by audio be once the software hits the commercial market? What will the impact of querying by audio be once the software hits the commercial market?

29 References After Napster: The Beat Goes On. Retrieved November 25 th from After Napster: The Beat Goes On. Retrieved November 25 th from Anonymous (2002, November). Name that tune. Technology Review, Cambridge. Volume 105, issue 9, page 18. Anonymous (2002, November). Name that tune. Technology Review, Cambridge. Volume 105, issue 9, page 18. Anonymous (2002). The Phsyics Classroom. Retrieved December 4 th from Anonymous (2002). The Phsyics Classroom. Retrieved December 4 th from Data Compression. Stanford University. Retrieved November 24 th from Data Compression. Stanford University. Retrieved November 24 th from Gerard, Mike (2002). Security Risks of Peer-to-Peer Software across the Internet. Retrieved November 23 rd from Gerard, Mike (2002). Security Risks of Peer-to-Peer Software across the Internet. Retrieved November 23 rd from Mitchell, Robert L (2002, August). Search engines break the sound barrier. Computerworld. Volume 36, issue 32, page 34. Mitchell, Robert L (2002, August). Search engines break the sound barrier. Computerworld. Volume 36, issue 32, page 34. Napster by all the top cartoonists. Retrieved November 17 th from Napster by all the top cartoonists. Retrieved November 17 th from Shankland, Stephen (2001, April). Sun to show peer-to-peer software. CNET News.com. Retrieved November 25 th from Shankland, Stephen (2001, April). Sun to show peer-to-peer software. CNET News.com. Retrieved November 25 th from SHN FAQ. Retrieved November 24 th from SHN FAQ. Retrieved November 24 th from Wang, Wanshuang. Indexing and Retrieval of Multimedia Data. Retrieved December 4 th from Wang, Wanshuang. Indexing and Retrieval of Multimedia Data. Retrieved December 4 th from


Download ppt "Audio Information Retrieval and Audio Search By Chris Mc Coy."

Similar presentations


Ads by Google