1. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Experiments with the Sonic Browser Eoin Brazil, Mikael Fernström Interaction Design Centre University of Limerick, IRELAND www.idc.ul.ie Laura Ottaviani Università di Verona, Dipartimento Scientico e Tecnologico, ITALY

2. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Outline IntroductionIntroduction The Sonic BrowserThe Sonic Browser Audio Similarity Rating ToolAudio Similarity Rating Tool Sound Designer ToolSound Designer Tool ConclussionsConclussions

3. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Introduction 1 A new approach to psychophysical experiments for scaling audio stimuliA new approach to psychophysical experiments for scaling audio stimuli Browsing & Sorting of a stimuli setBrowsing & Sorting of a stimuli set Interactive 2-dimensional interface for creating a similarity space for audioInteractive 2-dimensional interface for creating a similarity space for audio Method can be used to supplement pairwise comparisonMethod can be used to supplement pairwise comparison

4. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Introduction 2 Premix environment via the use of an AuraPremix environment via the use of an Aura Browsing & Searching of a sound collectionBrowsing & Searching of a sound collection Dynamic property filtering mechanismsDynamic property filtering mechanisms Tight CouplingTight Coupling –Objects under the Aura are high-lighted and played simultaneously –Changes in one visualisation are reflected in others

5. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Sonic Browser Screenscape and SoundscapeScreenscape and Soundscape Multiple coupled visualisationsMultiple coupled visualisations Direct sound representationDirect sound representation Data loggingData logging Dynamic property filter using text filtering or AlphaSlidersDynamic property filter using text filtering or AlphaSliders

6. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Sonic Browser - Visual Visual Representation: Star-field, TreeMap, HyperTree, TouchGraph visualisationsStar-field, TreeMap, HyperTree, TouchGraph visualisations Arbitrary mapping, controlled by the userArbitrary mapping, controlled by the user – shape, colour, size, horizontal and vertical location Sounds playing are high-lighted.Sounds playing are high-lighted.

7. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Sonic Browser - Auditory Audio: Direct representation of sounds Idea -> use “cocktail party effect” to exploit ability to switch between sounds in auditory sceneIdea -> use “cocktail party effect” to exploit ability to switch between sounds in auditory scene Sounds are panned out in a stereo field controlled by the visual location of the sounds under the cursor / aura.Sounds are panned out in a stereo field controlled by the visual location of the sounds under the cursor / aura. The aura is user controlled and can be reduced or increased in sizeThe aura is user controlled and can be reduced or increased in size By reducing the aura allows for “zooming” in on a particular sound/s.By reducing the aura allows for “zooming” in on a particular sound/s.

8. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Sonic Browser – Usage Cursor on object - the object plays, max volume, centered.Cursor on object - the object plays, max volume, centered. Click - show dialogue box with details of the properties of the soundsClick - show dialogue box with details of the properties of the sounds Double-click - access the object, e.g. show the details in the database, edit, etc.Double-click - access the object, e.g. show the details in the database, edit, etc. Single-click & Drag – move the sound in displaySingle-click & Drag – move the sound in display

9. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Study - Overview Explore an interactive method for gathering a audio similarity spaceExplore an interactive method for gathering a audio similarity space Explore if participants noticed any differences between sounds – real versus synthesisedExplore if participants noticed any differences between sounds – real versus synthesised Examine the application’s suitability for this type of scenarioExamine the application’s suitability for this type of scenario Explore if the application provides an engaging interface for participant in this type of scenarioExplore if the application provides an engaging interface for participant in this type of scenario

10. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Users and Data set Postgraduates in the CSIS at ULPostgraduates in the CSIS at UL contained 18 sounds, three typescontained 18 sounds, three types –6 recorded sounds –6 PD-modules modelling impact interactions of two modal resonators simplified to return only one mode –6 PD-modules modelling impact interactions of two modal resonators as well as the dropping event

11. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 PD Module Rath, M. et al, An Introductory Catalog of Computer Synthesized Contact Sounds in Real-Time, Colloquium on Musical Informatics 2003

12. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Tasks FamiliarisationFamiliarisation –practice tasks TestingTesting –similar way with a different sound data set and tasks Post testing debrief & questionnairePost testing debrief & questionnaire

13. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Experiment Screenshot perceived size of the dropped object perceived height of the object drop Aura

14. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 User’s Perceived Scaling

15. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Study Findings ``detailed comparison without reference points is very difficult and would be much easier with only a single scale'‘``detailed comparison without reference points is very difficult and would be much easier with only a single scale'‘ `` aura gives a comparison for similar sounds''`` aura gives a comparison for similar sounds'' ``the longer I spent working with the sounds, the more difficult it was to sort them''``the longer I spent working with the sounds, the more difficult it was to sort them'' Easy to learn and understand how to use the application for the tasksEasy to learn and understand how to use the application for the tasks

16. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Study Conclusions Synthesised sounds, even when found to be unrealistic, can still convey information in two dimensions and even if one mode soundsSynthesised sounds, even when found to be unrealistic, can still convey information in two dimensions and even if one mode sounds Realism of a sound does not affect its ability to convey information about particular perceptual dimensions - sound cartoonification.Realism of a sound does not affect its ability to convey information about particular perceptual dimensions - sound cartoonification.

17. interaction design centre ICAD-03, Boston, USA, July 6-9, 2003 Future Work & Reference Development of zoomDevelopment of zoom Addition of mouse click context menuAddition of mouse click context menu Addition of audio scrubbing for swifter browsing of individual soundsAddition of audio scrubbing for swifter browsing of individual sounds Inclusion of “tagging” or “shading” for files of interest linked to sound editor applicationsInclusion of “tagging” or “shading” for files of interest linked to sound editor applications Use of cue points to play sound areas/regions of interestUse of cue points to play sound areas/regions of interest Website:http://www.idc.ul.ieWebsite:http://www.idc.ul.iehttp://www.idc.ul.ie Sonic Browser available from: http://richie.idc.ul.ie/eoin/ or http://www.soundobject.orgSonic Browser available from: http://richie.idc.ul.ie/eoin/ or http://www.soundobject.org http://richie.idc.ul.ie/eoin/ http://www.soundobject.org http://richie.idc.ul.ie/eoin/ http://www.soundobject.org