1. eNTERFACE 2005 July 18th 2005 Quentin Noirhomme - Jean-Julien Filatriau Communication and Remote Sensing Lab. Université catholique de Louvain Project 3 : Biologically-driven Musical Instrument

2. Biologically-driven musical instrument July 18th 2005 Objectives ElectroEnceph alogram EMG EKG Analysis Sound Synthesis Mapping

3. Biologically-driven musical instrument July 18th 2005 EEG: Origin Brain Facts

4. EEG Recording DTI cap Biologically-driven musical instrument July 18th 2005

5. EEG Biologically-driven musical instrument July 18th 2005

6. EEG Noise Noise Background activities Background activities Activities with partial control Activities with partial control used for Brain Computer Interface used for Brain Computer Interface Training possible Training possible Biologically-driven musical instrument July 18th 2005

7. EEG: Rhythms Alpha Alpha 8-13 Hz 8-13 Hz first discovered first discovered first used for music first used for music Biologically-driven musical instrument July 18th 2005

8. EEG: Rhythms Alpha Alpha Delta Delta 0.1-3.5 Hz 0.1-3.5 Hz Theta Theta 4-7.5 Hz 4-7.5 Hz Biologically-driven musical instrument July 18th 2005

9. EEG: Rhythms Alpha Alpha Delta Delta Theta Theta Mu Mu 8-12 Hz 8-12 Hz Beta Beta 14-30 Hz 14-30 Hz Biologically-driven musical instrument July 18th 2005

10. Mu and Beta rhythms Wolpaw et al., 2002 Biologically-driven musical instrument July 18th 2005

11. Event Related Potentials Biologically-driven musical instrument July 18th 2005

12. Topography Biologically-driven musical instrument July 18th 2005

13. ElectroCardioGram Biologically-driven musical instrument July 18th 2005

14. ElectroMyoGram Biologically-driven musical instrument July 18th 2005

15. Biologically-driven musical instrument July 18th 2005 Objectives ElectroEnceph alogram EMG EKG Analysis Sound Synthesis Mapping

16. EEG Analysis Frequency Analysis Frequency Analysis Fourier, auto-regressive modelling Fourier, auto-regressive modelling Wavelets Wavelets Event detection Event detection Localization Localization Hjort Analysis Hjort Analysis Biologically-driven musical instrument July 18th 2005

17. Biologically-driven musical instrument July 18th 2005 Objectives ElectroEnceph alogram EMG EKG Analysis Sound Synthesis Mapping

18. Biologically-driven musical instrument July 18th 2005 Objectives ElectroEnceph alogram EMG EKG Analysis Sound Synthesis Mapping

19. Sound synthesis - - Sound synthesis = creation of a sound signal from specific algorithms implemented on a computer. = synthesis parameters that influence sound characteristics (pitch, spectral content, timbre...) = ex : additive synthesis, soustractive synthesis, FM synthesis, waveshaping synthesis... - - Ex : Frequency modulation synthesis y(t) = A sin[ ω C t + I sin( ω M t ) ] Synthesis parameters Biologically-driven musical instrument July 18th 2005

20. Digital music instruments - Real-time sound synthesis for « live interpretation » - Sound synthesis parameters controlled by player’s gestures => interaction allows musician to act on the sound like in a traditional musical instrument => Mapping = correspondance functions between gestures and synthesis parameters. Biologically-driven musical instrument July 18th 2005 Synthesis model Sound Gesture transducer Gesture Mapping Synthesis parameters Gesture data

21. Examples of digital music instruments (from LMA, Marseille) The photosonic emulator (training) The Voicer (in live situation) Biologically-driven musical instrument July 18th 2005

22. Biologically-driven musical instrument - Objective of our project : using infos extracted from EEG/EMG analysis as sound synthesis parameters to drive a digital music instrument. - Two main issues : => synthesis algorithm(s) => adequate « mapping » between EEG/EMG data and sound synthesisparameters. - Previous works in interaction biology/music : => Brouse (2001) : Interharmonium (EEG) => Tanaka (2002) : BioMuse (EMG) => Miranda & Brouse (2005) : BCI-Piano (EEG) Biologically-driven musical instrument July 18th 2005

23. References - - Papers : « Brain–computer interfaces for communication and control », Wolpaw & al., 2002 « Toward direct Brain-Computer musical interface », Miranda & Brouse, 2005 « Musical Performance Practice on Sensor-based Instruments », Tanaka, 2002 - Web sites : http://www.tele.ucl.ac.be/~noirhom/eNTERFACE3/index.html http://cmr.soc.plymouth.ac.uk/interaction.htm http://www.music.mcgill.ca/~brouse/conversation/ Biologically-driven musical instrument July 18th 2005

24. Team Benoît MACQ - professor (UCL) Benoît MACQ - professor (UCL) Jean-Julien FILATRIAU - PhD student (UCL) Jean-Julien FILATRIAU - PhD student (UCL) Quentin NOIRHOMME - PhD student (UCL) Quentin NOIRHOMME - PhD student (UCL) Burak ARSLAN - invited professor (TCTS, Mons) Burak ARSLAN - invited professor (TCTS, Mons) Rolando BONAL CACERES - professor (La Havane, Cuba) Rolando BONAL CACERES - professor (La Havane, Cuba) Andrew BROUSE - PhD student (University of Plymouth) Andrew BROUSE - PhD student (University of Plymouth) Julien CASTET - future PhD student (ACROE, Grenoble) Julien CASTET - future PhD student (ACROE, Grenoble) Rémy LEHEMBRE - future PhD student (UCL) Rémy LEHEMBRE - future PhD student (UCL) Cédric SIMON - future PhD student (UCL) Cédric SIMON - future PhD student (UCL) Biologically-driven musical instrument July 18th 2005

25. Thanks for your attention !