1. Report #1 (mid-project) Feride Çetin
Speech Conductor
Report #1 (mid-project)
Feride Çetin

2. Aims :
A gesture interface for driving (“conducting”) a text to speech synthesis system.
Real time modification of text-to-speech synthesis
The “Speech Conductor” will add expression and emotion to the speech flow
Speech signal modification algorithms and gesture interpretation algorithms.
5/31/2019

3. Two Tasks
Expressive speech contest
Baby synthesizer
5/31/2019

4. TASK 1 TASK 1 Expressive speech contest (full TTS)
Try to make the best expressive speech using various gesture controlled devices.
In French, German, Turkish, English
Compare and test various interfaces
Much training would be needed for producing convincing performances
5/31/2019

5. TASK 1 - technicalities MaxMBROLA, MidiMBROLA Device controllers
5/31/2019

6. Technical aspects: gesture control
Devices :
Keyboards, sliders, data glove, joysticks, finger joysticks, buttons, switch pedal board
Protocols:
MIDI (Musical Instrument Digital Interface),
OSC (Open Sound Control)
Demos:
Data glove,
Switch pedal board & finger joystick
5/31/2019

7. TASK 2 TASK 2 Baby synthesizer
Build a parametric synthesizer, made of respiration, glottal source and simple articulation (a bit like a 0-6 month baby)
Producing “pure” expressive voice with only minimal speech content.
Focus on voice quality and prosody
Real time gesture control of respiration, phonation and articulation parameters.
5/31/2019

8. TASK 2 - technicalities Models:
Time domain: waveform from parameters
Spectral domain: filters from parameters
Max/MSP vs. Pure Data: Spectral domain patches, Time domain patches
Articulation and respiration
Mapping between technical parameters and perceptive voice dimensions
Playing and testing, fun
5/31/2019

9. THE DREAM TEAM Christophe d'Alessandro Sylvain Le Beux (zombie)
Nicolas D'Alessandro (zombie)
Juraj Simko (zombie)
Feride Çetin
Hannes Pirker (zombie)
5/31/2019

10. 5/31/2019