1. 1 Challenge the future Considering cognitive aspects in designing cyber-physical systems: an emerging need for transdisciplinarity Wilfred van der Vegte and Regine Vroom Delft University of Technology Faculty of Industrial Design Engineering Department of Design Engineering

2. 2 Challenge the future Faculty of Industrial Design Engineering

3. 3 Challenge the future Contents Cyber-physical systems CPSs design – involved disciplines Disciplinary approaches: mono, multi, inter, intra, and trans Flavours of transdisciplinarity Two directions of research: Simulating cognitive loads and processing times Informing systems and mental models Concluding remarks

4. 4 Challenge the future Cyber-Physical systems (CPSs) CPSs are integrations of computation with physical processes, in which embedded computers and networks monitor and control the physical processes, usually with feedback loops where physical processes affect computations and vice versa. Example of a CPS: Swarming Micro Air Vehicle Network (SMAVNET) @ EPFL, CH Rapidly creates communication networks for rescuers in disaster areas Sensor networking technologies Swarm intelligence

5. 5 Challenge the future CPSs design @ IDE – involved disciplines Industrial Design Engineering (IDE), Cognitive Psychology, Psychophysiology, Information and Communication Technology (ICT) Disciplines commonly involved in an interdisciplinary faculty of IDE such as: Materials technology Manufacturing technology Human factors Electronics Mechanical engineering Marketing etc.

6. 6 Challenge the future Addressing cognitive aspects In predecessors of CPSs (mechatronic/smart systems, etc.) ICT and physics were already heavily involved. CPSs will increasingly incorporate (distributed) artificial cognition in interaction with human cognition Handling cognitive psychology issues will be a key challenge in the near future of CPS development Cognition-related issues: Allocation of cognitive tasks between human and CPS Cognitive matching of inputs/outputs between human and CPS Preventing information overload of human users Enabling CPSs as safety-critical systems Objective: cognitive symbiosis between human and CPS

7. 7 Challenge the future Transdisciplinary vs. intra-, inter- and multidisciplinary Flavours of transdisciplinarity engineering design other area of development (e.g., healthcare) end users/consumers (e.g., product users) end users/consumers (e.g., patients) engineering science other area of science (e.g., medical) mono intra inter multi inter trans

8. 8 Challenge the future engineering design other area of development (e.g., healthcare) end users/consumers (e.g., product users) end users/consumers (e.g., patients) engineering science other area of science (e.g., medical) transdisciplinary design = transdisciplinary research Flavours of transdisciplinarity

9. 9 Challenge the future Two directions of research 1.Simulating cognitive loads and processing times 2.Informing systems and mental models

10. 10 Challenge the future 1.0 Simulating cognitive loads and processing times Key application area: deployment of CPSs as safety-critical systems Revision of decision-making responsibilities CPS  human Simulation of human mental processes together with models of products and systems (in particular, CPS) Goal: evaluate CPS during development  identify bottlenecks to be addressed in the CPS’s design including service design, task design/allocation

11. 11 Challenge the future How to simulate human thinking and human reasoning? 1.1 Human-cyber-physical systems – how can we simulate? Interactive simulation vs. fully virtual simulation: Safety-critical systems  identification of incidents happening once in ~1,000 years. Interactive human-in-the-loop simulation must be real-time, but we cannot run a simulation for 1,000 years!  we need faster-than-real-time simulation  fully virtual, even humans Use simulation tools common in embedded systems engineering (procedural logic, state machines) Avoid time-consuming physics simulations based on geometric discretisation (e.g., FEM): use simplified models instead. Take shortcuts: disregard perception, motor skills, etc. human CPS; environment information processing physics

12. 12 Challenge the future 1.2 Simulating human thinking and human reasoning Two aspects: logic of decision making and processing time of decision making Logic of decision making: What action is taken under what condition? e.g. “IF cup is full THEN retrieve cup from machine”: straightforward execution of ‘normal’ use, assuming a particular history of preceding events. But can a simulation predict a user acting according to the production rule “IF cup is full THEN stick finger in it”? → unlikely! Yet we can try to generate typical aberrations from ‘regular use’: so-called error phenotypes (Hollnagel): actions accidentally in wrong order, accidental repetition, etc., by applying systematic variations

13. 13 Challenge the future 1.3 Simulating human thinking and human reasoning: processing time Processing time: How long does it take to accomplish a given action, taking into account aspects such as memory retrieval, memory capacity, learning, multitasking, distraction, etc. These aspects can be simulated using cognitive architectures such as ACT-R A cognitive architecture is a blueprint of the human mind based on findings from brain science filled with psychologically validated task models expressed as production rules

14. 14 Challenge the future 1.4 ACT-R cognitive architecture simulation of CPS & environment ACT-R simulation (human) declarative module(temporal cortex/hippocampus) intentional module (not identified) external world retrieval buffer(ventrolateral prefrontal cortex) visual module (occipital cortex) visual buffer (parietal cortex) motor buffer (motor cortex) goal buffer(dorsolateral prefrontal cortex) motor module(motor cortex/cerebellum) central production system (basal ganglia) ACT-R models are task specific, programmed in LISP by skilled, dedicated cognitive scientists Most tasks require scientists to create new customized models, that have to be validated in laboratory experiments with human subjects Intensive collaboration between cognitive scientists and designers of CPSs seems inevitable

15. 15 Challenge the future 1.5 Example CPS for simulating cognitive loads & processing times: Advanced support of emergency response

16. 16 Challenge the future 2.0 Informing systems & mental models Informing CPSs (e.g. informing public traffic systems) aims to find novel means to inform users and to find new symbiotic relations between human and cyber-physical systems; based on which designers can be supported in the early stages of CPS development; the objective is to avoid situations where users are mentally or perceptually overloaded and to precisely give the information that will help to take a right decision to react.

17. 17 Challenge the future 2.1 Project purpose The purpose of this project is to gain a better understanding in the manner in which MMs influence our interaction with the informing part of CPSs, and to provide guidelines for designers based on these insights Cyber-Physical System Human “system” Processing CPS output adapted to the cognitive capabilities of individual user(s) in a specific situation CPS input Sensors /detectors Human input Senses Brain (cognition, including knowledge, experiences, reasoning) Human output Human output detected by a CPS CPS informs (or offers other functionality) to human Current situation Current detection e.g. through motion detection, smart phone connection, id tag, …

18. 18 Challenge the future 2.2 Informing systems & mental models Goal: Include cognitive insights to influence the adaptability of CPSs. Approach: Study the behavior of mental models Future situation Mental model Cyber-Physical System Human “system” Processing CPS output adapted to the cognitive capabilities of individual user(s) in a specific situation CPS input Sensors /detectors Human input Senses Brain (cognition, including knowledge, experiences, reasoning) Human output Mental model: internal representation that people hold of an external reality that allows them to explain, interact, and predict that reality (from cognitive psychology) Mental model

19. 19 Challenge the future Mental model Cyber-Physical System Human “system” Processing CPS output adapted to the cognitive capabilities of individual user(s) in a specific situation CPS input Sensors /detectors Human input Senses Brain (cognition, including knowledge, experiences, reasoning) Human output 2.3 Future situation Cognitive Science Design Engineering precisely give the information that will help to take a right decision to react Designerly cognitive insights Current situation

20. 20 Challenge the future 2.4 Designerly cognitive insights a.Study behaviour of mental models: Is there inertia when switching from one mental model towards another? E.g. if an unexpected situation occur, will there be a different reaction on the same situation if the person was reading an exciting book than when he was playing football? Difficulty: perception influences can’t be reset (“undo” or “delete”) How to identify inaccuracies and gaps in a mental model (i.e. in a person’s knowledge and experience)? Mental models are inaccurate and incomplete. Insights in how to determine the gaps and the faults incorporate clues to better inform people.

21. 21 Challenge the future 2.4 Designerly cognitive insights cont’d b.Study relationships between mental states and cognition at one side and physical human data (facial expressions, gestures, heart beat etc., i.e. psychophysiology) on the other side. In addition to search for direct determination methods, indirect measurements might be useful: some facial expression may indicate that a person doesn’t understand a message for example. c.How to effectively address the major gaps and faults in a mental model? Effect of senses to address, effect of amplitude of the message (audio volume, pressure level in haptic information, etc.) The bridge towards guidelines for designers of informing CPSs: insight in the mental model states and behaviour will enable designers to design CPSs with adaptive capabilities on the user’s cognition.

22. 22 Challenge the future Concluding remarks – cognitive modelling Two research directions one to allow virtual testing of CPSs by designers, taking into account speed and capacity of human cognition in the interaction with CPSs one to provide knowledge to designers, so that CPSs can adapt themselves to mental models maintained by their users Both entail transdisciplinary collaboration with cognitive scientists – from disjunct research communities with different scientific approaches Cognitive architectures are based on empirical laboratory experiments Mental models are captured based on interviews Common goal: achieve symbiotic relationship between human and CPS

23. 23 Challenge the future Concluding remarks – Transdisciplinary design (or: research) Defining what is transdisciplinary and what is not, is probably not as simple as we have suggested: Knowledge value chain often more complex than research  development/design  application What is one discipline? How to deal with hierarchies of disciplines? (e.g. engineering  electrical, mechanical, civil,...) If we have learned enough from working with expert scientists from another discipline, can we eventually do the trick ourselves? Does it mean that a new discipline has formed and the activity is no longer ‘transdisciplinary’? If so, is that bad?