Presentation is loading. Please wait.

Presentation is loading. Please wait.

Positional Audio and Window Highlighting for the Access Grid Robert Putnam Boston University Scientific Computing and Visualization.

Similar presentations


Presentation on theme: "Positional Audio and Window Highlighting for the Access Grid Robert Putnam Boston University Scientific Computing and Visualization."— Presentation transcript:

1 Positional Audio and Window Highlighting for the Access Grid Robert Putnam Boston University Scientific Computing and Visualization

2 Boston University Scientific Computing and Visualization Group 2 Agenda The problem Prototype positional audio Prototype window highlighting Distribution/installation Discussion

3 Boston University Scientific Computing and Visualization Group 3 Who's talking now? In some AG contexts, it is not always clear which video window contains the current speaker. The problem is exacerbated by: –Multiple, concurrent speakers –Users unfamiliar with the AG –Users unfamiliar with each other Remedy: look for familiar face or moving lips.

4 Boston University Scientific Computing and Visualization Group 4 What’s that noise? In some AG contexts, it is not always clear who is transmitting echo or noise. Remedies: –Broadcast appeal –Operator mutes RAT streams one at a time until offender is found.

5 Boston University Scientific Computing and Visualization Group 5 Solution #1: Positional audio Use location of site's video window(s) to determine its audio position in stereo field.

6 Boston University Scientific Computing and Visualization Group 6 Solution #1: Positional audio Preliminary system design –Components (Minimally modified) vic and RAT “Agent” which communicates with vic and RAT –Sequence of events Operator moves window Agent gets report of window movement from vic Agent sends audio panning command to RAT –Need: communications & audio localization layers. agent RAT ? vic ? ?

7 Boston University Scientific Computing and Visualization Group 7 Positional audio RAT has built-in support for “3D audio”* RAT’s localization methods: –Interaural Intensity Difference (IID) Difference in sound intensity between near and far ear. –Interaural Time Difference (ITD) Difference in time of arrival between near and far ear –Head Related Transfer Function (HRTF) 3D-location-specific filtering effect of ears, head, and torso. *Actually “1D,” since there’s only a single degree of freedom.

8 Boston University Scientific Computing and Visualization Group 8 Positional audio Revised system design: What about communications? Next slide… agent RAT 3D audio vic ? ?

9 Boston University Scientific Computing and Visualization Group 9 Message Bus RAT’s control, user interface and engine already communicate via the Message Bus (“mbus*”): So… rat-uirat-engine mbus *see rat-control

10 Boston University Scientific Computing and Visualization Group 10 Positional audio Revised system design: Next: what mbus message is used to enable 3D audio? agent RAT 3D audio vic mbus

11 Boston University Scientific Computing and Visualization Group 11 3D audio mbus message 3D audio mode is enabled for a given RAT source via an audio.3d.user.settings message: rat-uirat-engine mbustcl *see audio.3d.user.settings

12 Boston University Scientific Computing and Visualization Group 12 Talking to RAT So, agent sends audio.3d.user.settings message to rat-engine: –ssrc = “Synchronization source” = RTCP stream id –azimuth = horizontal angular distance of listener from sound source. (90 = panned left; -90 = panned right; 0 = panned center) agentrat-engine mbus audio.3d.user.settings ssrc azimuth vic mbus ?

13 Boston University Scientific Computing and Visualization Group 13 Listening to vic Next, create new mbus message –window.xy Normalized xpos (0 – 1.0) Normalized ypos (0 – 1.0) Name of video window (e.g., “LANL(Video)”) Whenever video window is moved, resized, or renamed, send window.xy message to agent. agentrat-engine mbus audio.3d.user.settings vic window.xy mbus

14 Boston University Scientific Computing and Visualization Group 14 Associating vic and RAT streams Agent responsible for determining which video windows correspond to which RAT streams (ssrc’s): As of AGtk 2.4, it is assumed that vic and RAT streams have the same name –I.e., RAT “LANL” corresponds to vic “LANL(Video)” agentrat-engine 04b6910a “LANL” rtp.source.name vic window.xy “LANL(Video)”

15 Boston University Scientific Computing and Visualization Group 15 Solution #2: Window highlighting System shown at May 2003 AG retreat Feature request: visual indication of current speaker –But, danger of distraction/fatigue (i.e. too much visual clutter or “flashing”).

16 Boston University Scientific Computing and Visualization Group 16 Window highlighting Change border color of site’s video windows when its audio rises above a threshold. Two new mbus messages: –RAT: rat.source.amplitude(source_id amplitude) Report each stream’s peak amplitude 2-3X / second –vic: highlight.window(window_id on_off) agentrat-engine rat.source.amplitude audio.3d.user.settings vic window.xy highlight.window

17 Boston University Scientific Computing and Visualization Group 17 Window highlighting

18 Boston University Scientific Computing and Visualization Group 18 Window highlighting Window stays highlighted during brief pauses in audio unless another site begins transmitting. Each site’s local video windows also highlight when RAT transmitting (“On Air” alert). Window border highlight color and width is configurable in AG service.

19 Boston University Scientific Computing and Visualization Group 19 Setting up the system Install stereo speakers/amplifiers Install stereo wiring to and from echo canceller (aka “the Gentner”): RAT pc Gentneramplifier line out l r

20 Boston University Scientific Computing and Visualization Group 20 Setting up the system Create left+right echo cancellation reference:

21 Boston University Scientific Computing and Visualization Group 21 Setting up the system Remove or disable original video consumer and audio services and add two new ones: –HighlightVideoConsumerService.zip –PositionalAudioService.zip

22 Boston University Scientific Computing and Visualization Group 22 Setting up the system Configure HighlightVideoConsumerService: Windows Desktop Venue Client

23 Boston University Scientific Computing and Visualization Group 23 What next? Ready for beta testers! Get it here: –http://scv.bu.edu/~putnam/ag-audio/services-2.4.zip –Zip file contains services for Windows / Linux, PIG / multi-computer configurations. Request features / report bugs here:


Download ppt "Positional Audio and Window Highlighting for the Access Grid Robert Putnam Boston University Scientific Computing and Visualization."

Similar presentations


Ads by Google