1. 1 ICAS’2008 – Gosier, March 16-21, 2008 Model-Driven Engineering of Multi-Target Plastic User Interfaces Benoît Collignon 1, Jean Vanderdonckt 1, Gaëlle Calvary 2 1 Université catholique de Louvain (UCL) Louvain School of Management (LSM) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi 2 Université Joseph Fourier – Grenoble I, Laboratoire LIG 385, rue de la Bibliothèque BP 53 - F-38041 Grenoble Cedex 9 (France)

2. 2 ICAS’2008 – Gosier, March 16-21, 2008 Introduction The problem of designing multi-target user interfaces –Context of use = {user, computing platform, physical environment} –Multiple contexts of use => multiple variations of users, platforms, and environments Independently Simultaneously Notion of plasticity –Plasticity = user interface ability to change according to a change of the context of use while preserving predefined usability properties Context-aware adaptation = mere adaptation depending on context changes Plasticity = more than context-aware adaptation since usability should be guaranteed at a minimum level of satisfaction To address the problem, we combine –Model-driven engineering of user interfaces –Plasticity as a mean to address context-aware adaptation

3. 3 ICAS’2008 – Gosier, March 16-21, 2008 Multiplicity of contexts of use LocationRoleDeviceExperience SportingMultimedia Travel programme WorkingTravel booking site Powerful interface for complex operations TravellingBooking notification Everywhere connectivity for simple data exchange FamilyTV is multi-media family device #1

4. 4 ICAS’2008 – Gosier, March 16-21, 2008 Example of a Plastic User interface Property of plasticity = adaptation to the context of use while preserving predefined usability properties of interest [Grolaux et al.,2002]

5. 5 ICAS’2008 – Gosier, March 16-21, 2008 Example of a multi-target UI Traditional approach: Atomica HTML/Java UI code Target 1: Web terminal Target 2: Personal computer Visual C++ UI code Target 3: Pocket PC Mobile Visual Basic UI

6. 6 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –The mediator solution (model-based approach) [Eisenstein et al.,2001]

7. 7 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –Multiple variations based on a model (here, a platform model) [Märtin et al.,1990]

8. 8 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –Adaptive UI layout: widgets are (un)displayed and laid out according to resolution [Plomp & Keranen,2002]

9. 9 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –Artistic resizing : widgets have different variations according to resolution [Dragicevic et al.,2005] w h wLwL hLhL yLyL xLxL r hBhB wBwB  x L = (w-w L ) / 2  y L = (h-h L ) / 2  w L = 20  h L = 10  w B = 5  h B = 5  r = 20

10. 10 ICAS’2008 – Gosier, March 16-21, 2008 Our solution Combine model-driven engineering and plasticity domains

11. 11 ICAS’2008 – Gosier, March 16-21, 2008 Case study Case study: multi-target plastic UI for date & time (FlexClock) [Grolaux et al.,2002]

12. 12 ICAS’2008 – Gosier, March 16-21, 2008 Case study Dialogue modeled through a Moore machine –16 alternative variations –6 resizing operations Observations –Visualization is hard to achieve

13. 13 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

14. 14 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

15. 15 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

16. 16 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

17. 17 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

18. 18 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

19. 19 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

20. 20 ICAS’2008 – Gosier, March 16-21, 2008 Case study Dialogue modeled through a Mealy machine W2W1 W2 W1 2 / 1 1 11 / 1 2 17 / 13 3 30/15 4 19 / 9 5 42 / 16 6 98 / 16 7 102 / 16 8

21. 21 ICAS’2008 – Gosier, March 16-21, 2008 Case study Main idea: design one UI variation for one context at a time, connect related variations afterwards Horizontal screen resolution Vertical screen resolution Plasticity domain

22. 22 ICAS’2008 – Gosier, March 16-21, 2008 Case study Main idea: design one UI variation for one context at a time, connect related variations afterwards

23. 23 ICAS’2008 – Gosier, March 16-21, 2008 Tool support: PlastiXML Graphical editor for plasticity domains –Direct manipulation of plasticity domains (here, rectangles) –Assignment of a UI variation to each plasticity domain –Automated generation of UsiXML for Layout Navigation between variations <ScreenSizeAspect id="scrd1" shape="rectangle" corners="{(80,10);(80,30);(ScreenSizeXLimit,10); (ScreenSizeXLimit,30)}" allowedOperations="{vertical shrinkage}"/> <ScreenSizeAspect id="scrd2" shape="polygon" corners="{(80,30);(ScreenSizeXLimit,30); (ScreenSizeXLimit,60);(100,60); (100,ScreenSizeYLimit); (80,ScreenSizeYLimit)}" allowedOperations="{vertical shrinkage}"/>

24. 24 ICAS’2008 – Gosier, March 16-21, 2008 Scenario First plasticity domain

25. 25 ICAS’2008 – Gosier, March 16-21, 2008 Scenario Second plasticity domain

26. 26 ICAS’2008 – Gosier, March 16-21, 2008 Scenario Third plasticity domain

27. 27 ICAS’2008 – Gosier, March 16-21, 2008 Scenario Final example

28. 28 ICAS’2008 – Gosier, March 16-21, 2008 Conclusion Advantages –One UI variation is assigned to one context (here, a resolution) at a time –Connect related variations afterwards –Visual design of connected variations insted of separate (unrelated) design –Automated generation of UsiXML specifications Presentation Navigation –Mealy machine –Moore machine Limitations –Apart from copy/paste, no direct reuse of components from one UI variation to another –No factoring out of common components Possible extension –Generate a default UI variation to feed the various resolutions to be supported –Edit afterwards Generalization needed

29. 29 ICAS’2008 – Gosier, March 16-21, 2008 Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs Special thanks to all members of the team!