1. Visually-induced auditory spatial adaptation in monkeys and humans
Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh
Center for Cognitive Neuroscience, Duke University Hearing Research Center, Boston University Technical University, Košice, Slovakia
Barb suggested making the coordinate-frame animation more obvious
Lid -> Lit
the data are all relative to the no-adaptation baselines. Those things just didn't come across in an obvious way.
Possible reasons for differences:
Monkey don’t try to look at sounds 1

2. Introduction
Visual stimuli can affect the perception of sound location
e.g. the Ventriloquism Effect
Way to go
Red Sox!
Way to go
Red Sox!
ARO abstract:
Visual calibration of auditory spatial perception in humans and monkeys
Norbert Kopco, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh
Duke, BU
When you see and hear something simultaneously, and you will often perceive the sound as coming from the same location as the visual stimulus. This is known as the ventriloquist effect. Indeed, the influence of visual stimuli can often outlast the visual stimulus itself, causing persistent changes in where sounds are perceived to be, even when the sounds are presented by themselves. The neural mechanisms underlying such visual capture of auditory spatial information have been extensively studied in barn owls using a prism adaptation paradigm, but comparatively little is known about the process either perceptually or at the neurophysiological level, in primates. In this study, I will describe our recent experiments on the nature of visual induced-recalibration of sound location in humans and monkeys.
But does effect persist?
Nov 6, 2007
SFN 07 San Diego

3. Introduction
Visual stimuli can affect the perception of sound location
e.g. the Ventriloquism Effect
Way to go
Red Sox!
ARO abstract:
Visual calibration of auditory spatial perception in humans and monkeys
Norbert Kopco, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh
Duke, BU
When you see and hear something simultaneously, and you will often perceive the sound as coming from the same location as the visual stimulus. This is known as the ventriloquist effect. Indeed, the influence of visual stimuli can often outlast the visual stimulus itself, causing persistent changes in where sounds are perceived to be, even when the sounds are presented by themselves. The neural mechanisms underlying such visual capture of auditory spatial information have been extensively studied in barn owls using a prism adaptation paradigm, but comparatively little is known about the process either perceptually or at the neurophysiological level, in primates. In this study, I will describe our recent experiments on the nature of visual induced-recalibration of sound location in humans and monkeys.
But does effect persist?
- barn owls: prism adaptation (Knudsen et al.)
- monkeys: “ventriloquism aftereffect” (Woods and
Recanzone, Curr. Biol. 2004)
Nov 6, 2007
SFN 07 San Diego

4. GOALS
Ventriloquism “aftereffect” in saccade task, in monkeys and humans?
- well-defined sensory-motor paradigm
- bridge to barn owl prism adaptation studies (on different time scale)
Reference frame of plasticity?
- Visual, auditory, or oculomotor reference frame?
Nov 6, 2007
SFN 07 San Diego

5. Methods
Basic idea:
1. Pre-adaptation baseline: Measure auditory saccade accuracy
2. Adaptation phase: Present combined visual-auditory stimuli, with visual location shifted
3. Compare auditory saccade accuracy pre- and post-adaptation
Nov 6, 2007
SFN 07 San Diego

6. Q1: Does it work? Initial experiment Design: Monkey
Pre-adaptation baseline – ~100 Auditory-only trials
Adaptation phase –
80% V-A stimuli, visual stimulus shifted 6 deg. Left or Right
20% Auditory-only
Compare Auditory-only trials from adaptation phase to pre-
adaptation phase
Sounds: Loudspeakers
Visual stimuli: LEDs
Nov 6, 2007
SFN 07 San Diego

7. RESULTS
Nov 6, 2007
SFN 07 San Diego

8. RESULTS
Nov 6, 2007
SFN 07 San Diego

9. RESULTS
Nov 6, 2007
SFN 07 San Diego

10. Q2: reference frame 2a. What is the reference frame? Eye-centered? ? ?
2b. Is the reference frame the same for humans and monkeys?
Eye-centered? ? ?
Oculomotor?
Head (ear) -centered?
Nov 6, 2007
SFN 07 San Diego

11. Method Fix head to face 0° Induce shift: - in only one region of space
- from a single fixation point
Test to see if shift generalizes
to the same sub-region in:
- head-centered space
- eye-centered space
Experiment divided into
1-hour blocks:
(12 for humans, 16 for monkeys)
Within a block, 3 types of
randomly interleaved trials:
- Training AV stim: 50%
- Test A-only stimuli: 50% (25% from trained, 25% shifted FP)
Audiovisual display
Expected behavior
Stimulus Location (°)
Magnitude (°)
Speakers
LEDs FP
Nov 6, 2007
SFN 07 San Diego

12. Results: Humans Head-centered representation,
Audiovisual display
Trained FP A-only
responses:
- Shift induced in
trained sub-region
- Generalization to
untrained regions
(asymmetrical) FP
LEDs
Speakers
Magnitude of Induced Shift (°)
Shifted FP A-only
responses:
- Shift reduced in
center region
After inducing the expected shift in AV trials
Stimulus Location (°)
Expected Responses
Head-centered or Eye-centered
Head-centered representation,
modulated by eye position
Nov 6, 2007
SFN 07 San Diego

13. Magnitude of Induced Shift (°)
Results: Monkeys
Audiovisual display
Trained FP A-only
responses:
- Shift in trained sub-
region weaker
- Generalization to
untrained regions
stronger
(asymmetry oppo-
site to humans) FP
LEDs
Speakers
Magnitude of Induced Shift (°)
Shifted FP A-only
responses:
- Shift decreases on
the right
- Shift increases on
the left
After inducing the expected shift in AV trials
Stimulus Location (°)
Expected Responses
Humans: or
Representation
more mixed
than in humans
Nov 6, 2007
SFN 07 San Diego

14. Summary The main results are consistent across species:
Locally induced ventriloquist effect results in short-term adaptation, causing 30-to-50% shifts in responses to A-only stimuli from trained sub-region.
The induced shift generalizes outside the trained sub-region, with gradually decreasing strength (However, the pattern of generalization differs across the species)
The pattern of induced shift changes as the eyes move. But, overall, it appears to be in a representation frame that is more head-centered than eye-centered.
This shows that plasticty that underlies this adaptation
There were differences
Nov 6, 2007
SFN 07 San Diego

15. (Kandel, Schwartz, Jessel) and (Purves)
Discussion
Posterior
Parietal
Cortex
Neural adaptation could have been
induced at several stages along the
pathway.
Future work
Examine temporal and spatial
factors influencing the eye-
centered modulation.
Look at other trained sub-regions.
Thank You!
Support NIH
Cerebrum
Thalamus
Thalamus
Midbrain
Midbrain
Pons
Pons
Nov 6, 2007
SFN 07 San Diego
(Kandel, Schwartz, Jessel) and (Purves)