2. A framework for the analysis of medical data using data sonification across the GÉANT network and the European Grid infrastructure Roberto BARBERA University of Catania & INFN CT, Italy Francesca FALCETTA The Mario Negri Institute for Pharmacological Research, Milan, Italy Massimo RIZZI The Mario Negri Institute for Pharmacological Research & ARCEM, Milan, Italy Giuseppe LA ROCCA INFN CT & IGI, Italy Mariapaola SORRENTINO ASTRA Project Domenico VICINANZA DANTE, Cambridge, UK

3. 2 Connect | Communicate | Collaborate The Data Sonification process in a nutshell How does the sonification process work ? The Mario Negri Institute The Epilepsy disease – EEGs – Intracranial vs. conventional EEGs recording The mission The role of advanced R&E networking and Computing Infrastructures GEANT Network and the Catania Science Gateway The Sonification process running on a Science Gateway First preliminary results Summary & Conclusions Outline

4. 3 Connect | Communicate | Collaborate  The Data Sonification process in a nutshell How does the sonification process work ? Benefits of the sonification techniques

5. 4 Connect | Communicate | Collaborate Representation of data sets and information through sounds and melodies Data sonification Credits: D. Vicinanza et al. LHC Open SymphonyLHC Open Symphony Same as creating a graph using notes and tones instead of lines and points

6. 5 Connect | Communicate | Collaborate How does sonification work? Associating a musical note to each value Two conditions: Uniqueness: – Same value  same note – Ex: 25  middle C Covariance: – Melody varies as quickly as the data. – Ex: 25  C, 26  D, 27  E …

7. 6 Connect | Communicate | Collaborate From numbers to notes 10 15 10 8 4 1 4 Correspondence between numbers and notes: Large numbers  high pitches Small numbers  low ptiches The same numbers are associated with the same notes

8. 7 Connect | Communicate | Collaborate Example: periodic data

9. 8 Connect | Communicate | Collaborate Periodic melody

10. 9 Connect | Communicate | Collaborate Advantages of sonification Acoustic counterpart of data graphic visualization Sonic representations are particularly useful dealing with complex, high-dimensional data, data monitoring tasks pattern detection Transitions between states become audible Sound is not directional Sound can pack more information Human ear naturally trained to detect patterns and anomalies

11. 10 Connect | Communicate | Collaborate Two main applications: Scientific provide alternative ways to analyse data Artistic write music inheriting from the richness of data – New inspirational ideas

12. 11 Connect | Communicate | Collaborate  The Mario Negri Institute  What is Epilepsy ?  Epilepsy in numbers  EEGs  Intracranial EEGs recording  The mission

13. 12 Connect | Communicate | Collaborate The Mario Negri Institute for Pharmacological Research Not-for-profit biomedical research organisation (est 1961) From the molecular level to the whole human being Developing new drugs and Making existing ones more effective Main research headings: Cancer research Nervous and mental illnesses Cardiovascular and kidney diseases Rare diseases Toxic effects of environmental contaminants

14. 13 Connect | Communicate | Collaborate What is Epilepsy ? Epilepsy is a brain disorder that causes people to have recurring seizures Clusters of neurons, in the brain send out the wrong signals Epilepsy is an illness as old as human kind. It was mentioned in ancient Babylon more than 3,000 years ago Strange sensations and emotions or behave strangely. Violent muscle spasms, auras, convulsions or lose consciousness Many possible causes including illness, brain injury and abnormal brain development. In many cases, the cause is unknown

15. 14 Connect | Communicate | Collaborate Epilepsy in numbers Still an unresolved problem Still one of the most common neurological disorder ~ 1% of the world-wide population affected 50 million people – 90% in developing countries More likely to occur in young children or over 65 ~ 25% of patients not responding to conventional drugs Famous people affected by epileptic seizures: Vincent van Gogh Julius Caesar Alexander the Great Lord Byron

16. 15 Connect | Communicate | Collaborate EEGs An electroencephalogram (EEG) is a test used to record the brain’s electrical activity From EEGs physicians can look for abnormal patterns: seizures encephalopathies Red-colored area highlights the abnormal EEG activity peculiar of a seizure Usually, the onset of a seizure is associated to a rapid build up of 4 to under 8 Hz rhythmic activity of EEG. Diagnose: Brain scans, EEGs…

17. 16 Connect | Communicate | Collaborate Intracranial EEGs recording Usually recorded from scalp, also from electrodes within the brain - intracranial EEGs recording less affected by spurious signals Electrode strip Source: Freiburg’s EEG database

18. 17 Connect | Communicate | Collaborate The mission “inter-ictal” EEGs – EEGs recorded sufficiently far from seizures Analysed for the first time using a sonification technique – to identify a baseline condition (hopefully, a marker) This may help to: highlight and characterize patterns provide a powerful tool for seizure prevention  by listening to how musical patterns change  forecast an impeding seizure in advance. promote the investigation for new therapeutic interventions

19. 18 Connect | Communicate | Collaborate  The role of advanced R&E networking and Computing Infrastructures  The Sonification process running on a Science Gateway The Catania Science Gateway components

20. 19 Connect | Communicate | Collaborate The role of advanced R&E Networking and Computing Infrastructures Sonification and subsequent data analysis requires reliable networks and processing power to produce results –To analyse just a few MB of data using, for example, 100 different configurations would require a few hours on a standard laptop (2.4 GHz Intel Core 2 Duo, 2 GB RAM) and just a few minutes on the grid –EEG sequences needs to be analysed using different parameters and different mapping options –High quality audio files generated and analysed for each EEG data sequence For this analysis we used the MidiToolbox software Other tools are under investigation (i.e. WaveLab)

21. 20 Connect | Communicate | Collaborate Bringing grid computing in hospitals Hospital Grid portal Grid infrastructure + NRENs Grid portal based on Science Gateway allows researchers in the hospital to access grid services through GEANT

22. 21 Connect | Communicate | Collaborate The GÉANT Network (www.geant.net) Connecting EU NRENs GÉANT is co-funded by Europe’s NRENs and the European Commission (EC) under the Seventh Framework Programme (FP7) Project Partners are 32 European NRENs, TERENA and DANTE 150 FTEs’ annual effort (> 350 individuals) working in GÉANT across Europe 25 European POPs 12,000km of dark fibre on 18 routes 50,000km network infrastructure on 44 routes Diversified footprint Serves 40 million users 8,000 institutions Across 40 European countries Courtesy of T. Fryer

23. 22 Connect | Communicate | Collaborate GÉANT: Global Reach http://global.geant.net/ GÉANT connects to 62 countries outside Europe

24. 23 Connect | Communicate | Collaborate The Science Gateway Web-based portal to grid services Nothing to install Based on standards Modular Easy to use Re-usabile Useful to offer grid access anywhere The architecture: JSR 286 standard and Liferay as portlet container Identity Federations based on the SAML 2.0/Shibboleth Computing resources access: (J)SAGA

25. 24 Connect | Communicate | Collaborate Grid transactions Grid transactions: Robot X.509 certificates JAX-RS and PKCS#11 standards Compliant with: EGI VO Portal Policy EGI Grid Security Traceability and Logging Policy

26. 25 Connect | Communicate | Collaborate Science Gateway Science Gateway Standard-based (SAGA) middleware-independent Grid Engine The Catania Science Gateway Components Users from different organizations having different roles and privileges Administrator Power User Basic User

27. 26 Connect | Communicate | Collaborate Access the data Sonification process It provides instructions for users and configuration settings running on http://gw.ct.infn.ithttp://gw.ct.infn.it

28. 27 Connect | Communicate | Collaborate Select the computing resource running on http://gw.ct.infn.it http://gw.ct.infn.it It shows the computing resources where the SW has been successfully deployed It is possible to select & rank the generic computing resource

29. 28 Connect | Communicate | Collaborate running on http://gw.ct.infn.it http://gw.ct.infn.it Upload the data set as ASCII file or using the text-area Configure some advanced settings Job description Enable e-mail notification Select the type of algorithm to be used Enable the analysis of MIDI file with Matlab Select the data set

30. 29 Connect | Communicate | Collaborate Check the status of the simulations Asks for support Automatic jobs output downloading Shows the status of the simulations

31. 30 Connect | Communicate | Collaborate Bringing grid computing in hospitals Hospital Grid portal Grid infrastructure Thanks to the Grid portal data is sent back to the researchers for analysis + NRENs

32. 31 Connect | Communicate | Collaborate  First preliminary results EEG analysis: tonal map EEG analysis: melodic event series EEG analysis: melodic contour EEG analysis: interval distribution  Summary & Conclusions  Links & Contacts

33. 32 Connect | Communicate | Collaborate EEGs analysis: Tonal Map Frame extracted from a tonal map animation showing the evolution of the melody around a tonal centre. The idea: correlate tonal distribution with the disease stages to find a marker

34. 33 Connect | Communicate | Collaborate EEGs analysis: Melodic Event series The series of music events (notes) as a function of time. The ultimate goal is to contribute to the determination of an actual music marker The idea: find clustering and changes in the distribution and correlate them with phases or stages in the disease

35. 34 Connect | Communicate | Collaborate EEGs analysis: Melodic Contour The series of music events (note pitches), connected by line segments, as a function of time. The idea: find clustering and changes in the distribution

36. 35 Connect | Communicate | Collaborate EEGs analysis: Interval Distribution The distribution of music intervals in the analysed EEGs Here as well, the hope is to The ideafind the proper mapping so to correlate that distribution with the disease stages to find a marker

37. 36 Connect | Communicate | Collaborate Summary and Conclusions Sonification techniques are the most appropriate tools for extrapolating temporal-correlated properties in time series as musical patterns Advanced R&E networking and e-Infrastructures have been used to identify “markers” for denoting some specific epileptic EEG states Our research is at early stage.  Only few EEGs have been analysed.  More research is needed to corroborate preliminary results!  Visual approaches don’t take temporal information into account

38. 37 Connect | Communicate | Collaborate Download the Sonification White paper (PDF)White paper (PDF) EEG Sonification For Epilepsy Surgery: A Clinical Work-In Progress [link]link Midi Toolbox [link]link Contacts: Roberto Barbera (roberto.barbera@ct.infn.it)roberto.barbera@ct.infn.it Francesca Falcetta (francesca.falcetta@marionegri.it)francesca.falcetta@marionegri.it Giuseppe La Rocca (giuseppe.larocca@ct.infn.it)giuseppe.larocca@ct.infn.it Massimo Rizzi (massimo.rizzi@arcem.it)massimo.rizzi@arcem.it Mariapaola Sorrentino (mariapaola@astraproject.org)mariapaola@astraproject.org Domenico Vicinanza (domenico.vicinanza@dante.net)domenico.vicinanza@dante.net

39. 38 Connect | Communicate | Collaborate www.geant.net www.twitter.com/GEANTnews | www.facebook.com/GEANTnetwork | www.youtube.com/GEANTtv Connect | Communicate | Collaborate Thank you!