Hear&There An Augmented Reality System of Linked Audio Joey Rozier Karrie Karahalios and Judith Donath Sociable Media Group MIT Media Lab ICAD 2000 Conference - April 3, 2000

Overview Project Purpose and Goals Brief Scenario Basic Augmentation Details Hardware Authoring and Playback Security and Privacy

Purpose and Goals Augment Environment with Audio –Augmented Reality (or Mixed Reality) Develop Social Augmentations –Oral Histories –Focus on Authoring Activation Networks - Linking Self-Contained and Portable Limit Use of Visual Cues

Example Scenario Augmented Boston Common Can hear audio “left behind” (maybe history) Guided towards related audio (moo...) Author can add his/her own sound Others are unaware

Augmentation Details The SoundSpot –Basic unit of augmentation –Any type of audio (voice, music, whatever) –“Placed” at a specific location (longitude/latitude) –Has a specified size –Other audio properties (braids, etc.) –User walking “into” SoundSpot hears it

Augmentation Details Example

Augmentation Details Must Determine User’s Location GPS –Typical Accuracy: Order of 10m –Accuracy of Ours: sub-meter –Refresh Rate: 10Hz –Drawback: Bulk –Result: Media Lab Courtyard

Hardware GPS Compass PalmPilot Laptop Headphones Microphone

Authoring and Playback Focus on user authoring –Most other systems do not –Simple! Playback is still very important –New author options lead to new playback issues

Authoring Author creates SoundSpots –In-Field –Using GUI

Playback: Basics User dons system Very simple start-up Moves through environment Hears sound tied to location

Playback: Spatialized Sound Each sound “appears” to be coming from center of a SoundSpot Java 3D API Compass attached to headphones Do not deal with environment effects (enclosed spaces, etc.) Future work will improve spatialized sound

Playback: Two Modes Explorer –All SoundSpots activated –Can be very cluttered –Ultimate control with listener Activation Networks

Playback: Activation Networks For Linking and Organizing Sounds Author creates Link Categories –e.g. My Family, About Media Lab Author associates Link Categories with a SoundSpot User can activate Link Category when standing in SoundSpot StartSpots

Playback: Activation Networks Example StartSpot in Boston’s Public Garden –Link Categories SMG: SMG SoundSpots Weeping Willow: interesting SoundSpots about weeping willow trees My Favorites: my favorite SoundSpots –When entering StartSpot, activate “Weeping Willow”; now those “exist” –Weeping Willow spots may have their own LinkCategories

Playback: Directing the User Palm Interface –Easy, intuitive –But uses visual information! Audio Beacons?

Security and Privacy Authentication for Authors Open System for Users Wireless Connection to Server?

Future Work Finish Implementation! More Portable System Temporal Information Other forms of audio augmentation

Contact and Contributors Primary Contributors –Judith Donath –Karrie Karahalios –Birendro Roy and Tuan Phan