1. Dimitrios Kotsalis a George Vellis a Demosthenes Akoumianakis a Jean Vanderdonckt b a Department of Informatics Engineering, Technological Education Institution of Crete, Greece b Université catholique de Louvain, Louvain School of Management, Belgium ‘Implementation-agnostic’ instantiation schemes for ubiquitous, synchronous multi-user interfaces

2. Research on advanced, creativity-based User Interface engineering, meaning – non-trivial case scenarios – multi-user interfaces – synchronous collaboration – distributed users – ubiquitous settings Research protocol – Joint supervision of two researchers of the istlab (Department of Informatics Engineering of TEI Crete) in PhD program at the Université catholique de Louvain, Louvain School of Management, Belgium Context of the research

3. Case study problem descriptions and overview of solutions (i.e., current version of the implementation) Brief and non-technical discussion of the engineering challenges and the technicalities of the work – Design-oriented inspiration – Development platforms and tools – On-going work Plan of the presentation

4. Two human collaborators / players – A single shared digital representation The soccer field – Two types of agents Human (collaborators) and non-human (soccer players) Human collaborators virtualized and represented by soccer players – Affordances of soccer players (inspired from semiotic engineering) Each human collaborator operates with the currently selected soccer player Each soccer player operates on a shared space inhabited by other players Each human collaborator operates through different devices (Desktop/Swing and Mobile/Android) The combined experience (i.e., game co-play) entails operations within different representations The system in its current operational settingsystem Scenario in MDSD’14 - The virtual setting

5. How can we design and generate non-trivial multi- user interfaces that support synchronous collaboration between distributed users in ubiquitous settings? Existing approaches – Toolkit-based programming – Model-based UI engineering – Both exhibit limitations which do not allow effective solution to the kind of problem described Research question & related works

6. Theory-based insights – Interaction devices versus affordances – Semiotic engineering for qualifying virtualities by type of agents and kind of operation Engineering approach – Development of an abstraction-based model anchoring interaction in to human intentions and capabilities – Specify capabilities in a model-based fashion – Extend UsiXML as needed to support the new specification Provisions for extensible interaction vocabularies – Build dedicated design tools run-time environment components Proposed approach

7. Current implementation of platform-agnostic UI instantiation schemesplatform-agnostic An illustration of the approach

8. A scheme that relies on implementation agnostic (i.e., abstract) specifications of UIs At run-time and once user and usage context parameters are discovered, the implementation agnostic spec is translated to context-specific interaction vocabularies using dedicated tools Platform-agnostic instantiation

9. Widget gallery

10. Polymorphic classification scheme for the ‘abstract button’ widget Download

11. Introduce new widgets as first-class design objects Widget Specification – Basic concept WSLLibraries Resources Widget Archive

12. Polymorphism & extensible interaction vocabularies Widget Archive

13. Note that polymorphism at the UI-level is a much more demanding notion than polymorphism (i.e., polymorphic method invocation) as implemented in popular Object-Oriented languages. An example of polymorphic specification of the ‘abstract SoccerField’ widget

14. Note that polymorphism at the UI-level is a much more demanding notion than polymorphism (i.e., polymorphic method invocation) as implemented in popular Object-Oriented languages. An example of polymorphic specification of the ‘abstract SoccerField’ widget

15. Run-time scenario (MDSD’14 paper)

16. Some technical challenges resolved Input/output techniques and event models Model View Window selectedtapped activated View Window Breakdowns  Different semantics  Implementation language/toolkit dependent

17. Some technical challenges resolved States and state transitions onMouseDown onMouseRelease onSelectCommand onChangePlayerCommand onTap onUntap moving stillmoving stillmoving still

18. Sharing schemes for Synchronous collaboration: “Common States” sharing  Not abstract enough since there may be no common states or states at all

19. More recent developments (THALES-MusiNet, 2012-2015) Score’ interactive music metaphor: PC, expert users Tablature’ interactive music metaphor: PC, novice users The ‘circular’ interactive music metaphor: Web, novice users

20. Distributed Collaborative music learning User roles – Teacher (Expert) – Student (Novice) User stereotypes – Sighted – Blind Blind Platforms – PC: Java/swing – PC: JNVT2 – Web: HTML5 – Prototype Interactive metaphors – Score – Tablature – Circular Scenarios Multi-user Music Notation Lessons JNVT2

21. HTML5/Prototype Advanced collaborative features – Interim-feedback, Group awareness support ( Multi- user selection, Multi-user highlighting, Radar view, Film- view), Social awareness Novel features YouTube channel (“MusicNet istLab - Tei Crete”)MusicNet istLab - Tei Crete

22. Polymorphic specification

23. Current and on-going work – Support for web2 UIs – Generic support for group awareness – Run-time adaptivity and UI plasticity in distributed and ubiquitous settings – Distributed music learning Acknowledgements – The work is supported by ARCHIMEDES III, THALES – KUL for PhD dissertations (first two authors) Future work & acknowledgements

24. Final remark All screenshots presented were taken from running prototypes. For those interested some of them are already available in our channels on YouTube (“MusicNet istLab - Tei Crete”, iSTLab TeiCrete).MusicNet istLab - Tei CreteiSTLab TeiCrete