1. The Virtual World Framework: Implementing a Web Based Client Side Simulator Rob Chadwick, Katmai Government Services in support of ADL

2. Outline Introduction Design Principles and Requirements System Architecture Server component Client side application Client system requirements Challenges and Considerations Future Work Examples and Screenshots Questions 2

3. Introduction  The Virtual World Framework is a project funded by the Office of the Secretary of Defense to build a next generation distributed simulation platform  The intent is to provide an environment for application developers to rapidly create multi-user applications that run in a web browser with no client side software requirements  It must serve as a stable, scalable operating system to enable collaborative training experiences to be rapidly developed and easily delivered 3

4. Design Principles and Requirements Open Source Both the client and server side systems are developed with open-source methodologies This makes it possible for all manor of users to copy, modify and contribute code This also fosters more community interaction and communication Web Based The software does not have any dependencies that must be installed on clients beyond a web browser This should mitigate deployment difficulties 4

5. Design Principles and Requirements cont. Client Side All simulation logic is computed client side The server simply connects clients together It may eventually be possible to remove the server completely Allows for a very lightweight server Low bandwidth The VWF uses a principle called ‘replicated computation’ to synchronize clients This paradigm timestamps and transmits only inputs to the simulation – allowing each client to independently compute a synchronized state Messages are queued by the application and executed in order, up to the current time code delivered by the server Clients are not guaranteed to be synchronized in real time, but will remain synchronized relative to the simulation timestamp 5

6. System Architecture Overview A VWF application is designed with an MVC ( Model-View-Controller) architecture The server and client side libraries create a shared model that is automatically synchronized across all clients Instance Application Instance Client Reflector View Static Resource Static Resources Static Resource Static Resources Client View Model 6

7. System Architecture Composed of two major components – a server and a client application Server responsibilities Serves static resources such as images, 3D models, and sounds Organizes clients into groups and bridges the communication between clients within a group Delivers the client side application code, and bootstraps the application Synchronizes late joining clients Client responsibilities Executes simulation logic Interfaces with user Provides user feedback via graphics, sound, etc Broadcasts user input to the server 7

8. Server Design A VWF server can host multiple applications Each application might be loaded by several groups of clients at once No communication between applications or application instances is allowed 8 n Instance n Client Virtual World Framework Server Client Instance Client Application

9. Client Application The client application is a highly modular JavaScript program consisting of a “kernel” and a set of “drivers” These modules work together to execute a simulation in which a hierarchy of nodes modifies its structure and state according to the simulation logic The kernel orchestrates the interaction of all the nodes and drivers, and forwards messages to the server so that other clients so that they may maintain synchronization. Reusable components can instantiate multiple nodes based on a common prototype Application authors may supply custom APIs to the simulation nodes by including additional drivers The VWF supplies a useful set of services to simulation nodes such as scripting, voice and video communications, physics, and audio Each of these capabilities is implemented within a driver module 9

10. Client internals View Drivers (Your Simulation Code) Kernel Model Drivers Component Reflector The VWF system contains ‘drivers’ that provide services to the application ‘Model’ drivers participate in the simulation, while ‘view’ drivers produce output The ‘kernel’ observes the interaction of these modules and forwards message to the server as necessary to maintain synchronization between clients 10

11. Client System Requirements The core VWF kernel has modest system requirements WebSockets - a protocol for creating bidirectional low latency connections between web servers and clients ECMAScript 5 – the current version of the popular web scripting language (sometimes called JavaScript) All modern browsers (Chrome, Firefox, Opera, Internet Explorer, Safari) support these features Out of the box, the VWF comes with several drivers that have additional requirements Rendering requires support for WebGL, a 3D graphics interface for web applications Communication services rely on WebRTC and the HTML5 MediaSource API Audio currently requires HTML5 audio, and may eventually utilize WebAudio However, only the core components are absolutely necessary for a client to join 11

12. System Requirements Cont. Performance No minimum requirement Heavily dependant on the particular logic of a given simulation May or may not scale with the number of users, depending on the application Can be a significant challenge in some situations Some system configurations will use a software renderer to support WebGL – this can cause issues with complex geometries Bandwidth Usually minimal – only user input data must be transmitted, plus a time pulse Can become prohibitive when using the communication features like voice or video chat Immersive scenes must transmit artwork including 3D models and textures It is possible to reduce server bandwidth requirements by hosting assets externally, possibly on a commercial CDN Assets may also be stored locally on the client 12

13. Challenges in Implementation Platform support Initially, this was a significant challenge While the base kernel can run almost anywhere, initially support for the 3D rendering feature was sparse Since the beginning of the project, WebGL and other HTML5 features have become much more widely supported Significantly, Internet 11 will support WebGL, allowing the VWF to provide immersive 3D visualizations on all major browsers Even Android and IOS tablets now support the requirements We expect that further adoption of the web as a platform will mitigate platform support concerns with little direct effort by our staff 13

14. Challenges in Implementation Performance JavaScript execution speed can still be a bottleneck for some applications Specifically, current JavaScript physics simulators still struggle to scale to large immersive environments on commodity hardware There are several interesting projects focused on making JavaScript faster It may be possible to compile the VWF client code into a subset of JavaScript called ASM.js, which can be as fast as half native speed on some JS engines Some of the client side logic may be able to leverage hardware acceleration by utilizing WebCL, a project to bring general purpose GPU acceleration to the web environment. Finally, Google has developed a technology for securely deploying native code on the browser called NativeClient. It may be possible to move some VWF components to this technology 14

15. Challenges in Implementation Maintaining synchronization of user scripts When given properly formatted inputs, the system guarantees that each client will have an identical application state at a given time step. However, it is possible for simulation authors to circumvent the synchronization mechanism By unintentionally storing state information in global variables By writing simulation logic that improperly depends on outside inputs Currently, authors must be careful to follow best practices when creating content We’re actively researching long term solutions, such as sandboxing user provided behavior code We may also periodically detect synchronization failures by computing and comparing the cryptographic hash of the entire simulation state If a mismatch is found, we could re-initialize the clients to a known state It may also be possible to detect incorrect user scripts during authoring by analyzing the scripts within an IDE specific to VWF development 15

16. Future Work Much remains to be done! The basic platform is complete, and work is focused on providing more supporting features to applications User management features Audio features Integration with outside data sources Secure WebSockets and HTTPS content delivery Automatic recovery from synchronization failures Search and retrieval for reusable components and behaviors Persistence services for applications 16

17. Screenshots Two users interacting on a terrain 17

18. Screenshots An environment with several non-player characters 18

19. Screenshots A scale representation of the Solar System 19

20. Questions? … 20

21. Contact Rob Chadwick  Robert.chadwick.ctr@adlnet.gov Robert.chadwick.ctr@adlnet.gov  703-575-2012 21