1. 1 Imaginarium Merging science and practice David Kirsh, UCSD Cognitive Science, Assoc. Director ACCHI Erik Viirre M.D. Ph.D., UCSD School of Medicine Sheldon Brown, UCSD Visual Arts, Director of ACCHI Plus collaborators…

2. Objectives 1.Observation: Study the nature of distributed creativity in the natural setting of: 1.Brain storming 2.Design 3.Creative Expression 2.Theory: Build a more comprehensive theory of activity and sensory involvement during brainstorming 3.Design: Explore impact of changes in environment design on imagination, creativity and problem solving

3. Imaginarium Technology Binocular eye tracking among groups Video cameras coordinated with eye tracking Kinects for gesture detection Portable ERP caps Precise sound localization Whiteboard and other surface capture Automated Annotation of design session Remote interaction with other sites

4. Analytic Techniques Ethnographic coding, activity analysis, Datamining for patterns among different capture records Machine learning using VATIC and crowd sourcing to classify events defined via activity analysis Analyses using theories of distributed, situated and embodied cognition

5. Collaborators Relevant Research Basis Kirsh: NSF on Distributed Creative Cognition –In choreography –being expanded to music Makeig, Saygin: ERP (Event Related Potentials) during activity Trivedi: Video and Auditory awareness systems. Viirre: Eye Tracking for wayfinding, medical diagnosis and therapy Brown: User analytics of virtual worlds

6. Observation Set-Up for Distributed Creative Cognition in Choreography - Kirsh  Mandeville Auditorium  Cal-IT2

7. Oculomotor Measuring Equipment

10. Video Engineering Wide Angle Camera to identify head High resolution camera slaved to get face/eye activity Analysis of video image (visible/IR)

11. Body and Gaze Tracking

12. Tool for Collaboration Centered around whiteboard, display table, workbench –Annotation –Command interaction (Kinect)

13. Medical Evaluation Monitoring of Movement control –Fall Risk –Movement control –ADL Evaluation of Disease progression –Vestibular –Parkinsons Diagnosis? –Cerebrovascular vs. Neurodegenerative

14. Research Questions Are binocular indices sensitive indicators of neural disruption What do “natural” eye movements look like? –Combined Eye-Head –Head Free Scan paths

15. Embedded Cognition Theory Interaction with physical objects –Mental planning, practice Learning theory of tasks

16. Principals Erik Viirre, Medicine David Kirsch, Cog Sci Mohan Trivedi, Engineering Sheldon Brown, Visual Arts

17. Funding Sources Industry Partners Medical Research –DOD/VA –NIH NSF Computing, sensors, creativity, social sciences