1. DisplayWall Software Architecture Yuqun Chen, Grant Wallace, Kai Li

2. Motivation We’ve achieved high resolution Next stage: –multiple active surfaces –many types of inputs –new ways of interacting with the surfaces –new ways to visualize information Stanford’s Interactive Space

3. New Application Framework aware of multiple surfaces aware of multiple inputs aware of reliability allows dynamic grouping aware of multiple platforms (PDA, laptop, PC)

4. Future Applications Collaborative Engineering –3D models –2D blue prints/maps –video conferencing –writing workspace (documents, code) –multiple simultaneous user inputs –different types of devices (laptop, PDA, etc.) –many apps work cooperatively

5. Orthogonal Design Spaces Multiple Applications Multiple Inputs Multiple Displays

6. Desired Properties Applications –multiple apps cooperate –take multiple inputs –scalable Input –route many inputs to many apps Display/System –multiple surface management –re-configurable and scalable –fault tolerance (timely recovery) –hardware/OS platform independent

7. Application Space Synchronization and communication –high-level events among multiple apps –large-scale data exchange Select among multiple input events –system support for multiplexing inputs Application-level adaptability –deal with join/leave on the fly new site joins a video conferencing bring the new app’s states up to date

8. Application Scalability Scalable to actual screen size & aspect ratio –e.g., imageviewer Scalable with number of physical nodes –walk-through –load-balancing

9. Inputs Space Multiple physical inputs => one virtual inputs –(gyro-mouse, hand position) ==> virtual mouse –(PDA, keyboard, keyboard) ==> virtual keyboard Multiple virtual inputs –flight simulator: captain and co-pilot

10. Physical to Virtual Input Routing PC keyboard virtual keyboard 1 virtual keyboard 2 virtual mouse 1 virtual mouse 2 virtual speech virtual body wireless keyboard remote keyboard mouse gyro speech recog app

11. Display System Space Surface (=virtual screen) management –which virtual screen to route application output –virtual screen space configuration/grouping Per-surface reconfiguration –assign physical screens to a surface –surface resolution Fault-tolerance –should not worry about connections –state recovery Hardware/OS independent ==> LINUX

12. Common Properties Communication abstraction is the key Sequencing of data/events One-to-many: multicast –high-level app events, graphics primitives Many-to-one: point-to-point –input devices Avoid explicit connection management Safe storage for fault tolerance

13. Communication Choices [1] MPI-style message passing –advantages people are familiar with messages virtual nodes –disadvantages: mutual knowledge between send and receiver –even though physical nodes can change underneath lack of a concept of a “shared space”

14. Communication Choice [2] Shared (Virtual) Memory –advantages single shared memory space at API level, no knowledge about the each others –disadvantages: memory management from many nodes platform dependent (big vs. small endian) coherence may come at a high cost

15. Communication Choices [3] Linda-like tuple space (T Space) –a named storage space independent of any nodes in(name, data), out(name)->data, read(name)->data –messages without direct sender-receiver knowledge –anybody can read from the space

16. Virtual Frame Buffer/Pixelmap Well-defined mappings between virtual areas and the physical memory/frame buffer Coordinated access to the virtual space –everybody reads, barrier, then write Multiple virtual frames to switch between

17. Use of Virtual Pixelmap declare virtual map A and B optional: global frame update: A ==> B barrier() every node: read a region from A every node: write to B barrier() swap A and B barrier()

18. X Server vs. VDD X Window System –virtual screen management –multiple sources –multiple platforms –multiple inputs ?? VDD (partially developed) –Windows Apps –smaller API

19. Software Architecture app App synchronization rendering local X VDD OpenGL inputs mouse hand keyboard XML System communication Virtual FB Virtual inputs