MIT iCampus iLabs Software Architecture Workshop June , 2006

iLab Batched Experiment Architecture: Client and Lab Server Design iLab Training Course P. Bailey & J. Hardison June 13 – 15, 2006

14/06/06 3 The Big Picture: Batched labs in the iLab Shared Architecture  Service Broker provides generic services, deployment mechanism for the client.  Lab Server and Client contain lab-specific code.  All communications pass through Service Broker. Service Broker Lab Server Client Campus network Internet

14/06/06 4 Lab Developer Tasks  Design Lab Client  Bound by Lab-specific UI requirements, iLab API  Design Lab Server  Bound by lab instrumentation, desired functionality, iLab API  Design Client-Server communication framework

14/06/06 5 Lab Client–Server Communication  Messages passed between Client and Lab Server communicate key lab information.  Lab Hardware Configuration/Status  Experiment Parameters & Results  This information is necessarily lab-specific. Lab Server Client Public Internet Arbitrary Service Broker

14/06/06 6 Lab Client–Server Communication  All Lab Client-Server Messages must be passed through Service Broker.  Generic mechanism.  XML an ideal technology for this application. Service Broker Arbitrary Lab Clients Arbitrary Lab Servers ?

14/06/06 7 Lab Server Design: Basic Requirements  Provide access to lab hardware.  Implement the iLab Lab Server API  Define & utilize format for lab- specific communication with the Client.  Provide any other functionality necessary for lab operation Note: iLab Architecture APIs are platform-neutral. Lab Server technology driven by lab resources, hardware requirements.

14/06/06 8 Lab Client Design: Basic Requirements  Provide and educationally valuable user interface to the lab.  Implement the iLab Client API  Create & Interpret lab-specific communication with Lab Server. Again… iLab Architecture APIs are platform-neutral. Lab developer can select the best technology for their Client.

14/06/06 9 Development Example: The Microelectronics WebLab  Online microelectronic device characterization lab.  First lab deployed using the iLab Architecture.  Used by students, guests & OCW users worldwide.

14/06/06 10 The Microelectronics WebLab: Lab Client-Server Communication  Three distinct message types used for lab- specific communication between Client and Lab Server.  Lab Configuration  Experiment Specifications  Experiment Results  XML is used to encode information.  Passed through the Service Broker as generic text.

14/06/06 11 The Microelectronics WebLab: The Lab Server Lab Server Requirements:  Scalable performance and reliability.  Asynchronous experiment submission and execution  Fault-tolerance  Built-in lab management utilities.  Highly modular, extensible.

14/06/06 12 The Microelectronics WebLab: The Lab Server Built on Windows using.NET Framework and MS SQL Server.

14/06/06 13  Responsible for interaction with outside world  Contains libraries responsible for experiment validation & DB interaction  Shared Library used to parse Experiment Specification messages during validation The Microelectronics WebLab: The Lab Server – Web Server

14/06/06 14 The Microelectronics WebLab: The Lab Server – Data Storage  SQL database used to store lab data:  Connects Web Server & Execution engine.  Provides data persistence.  Submitted Experiment Specifications queued in DB for execution process.  Complex data interactions are internal.

14/06/06 15 The Microelectronics WebLab: The Lab Server – Execution Engine  Governs experiment execution.  Retrieves Experiment Specifications from DB queue.  Communicates with lab hardware via low-level custom drivers.  Stand-alone process  Shared Library used to parse Experiment Specification messages for execution

14/06/06 16 Lab Server Highlights: Experiment Queuing Experiments are processed in an asynchronous manner…  Web Server Layer receives, validates job submissions and adds it to the queue.  Execution Layer de-queues jobs when hardware is available, returns results Removes major performance bottleneck, reduces dependence on any one process

14/06/06 17 Lab Server Highlights: Experiment Validation All experiments are validated on the server before they are queued:  Jobs are checked for:  Basic Correctness  Compliance with Hardware capabilities  Compliance with Server-imposed rules  Reduces resources spent on incorrectly specified jobs.  Server-based validation ensures uniformity, rapid application of changes

14/06/06 18 Lab Server Highlights: Lab Management  Used to view system status/logs, edit system configuration.  Interface geared towards common functions  Allows rapid response to events Most Lab Management functions available online:

14/06/06 19 The Microelectronics WebLab: The Lab Client Client Requirements:  Intuitive interface  Easily deployed on many platforms  Minimal user requirements  Highly modular design  Easily extensible

14/06/06 20 Meeting Client Design Goals: Portability Java used to develop client.  Ubiquitous as client execution environment  Good cross-platform compatibility  Places few special requirements on end-user  Packages/toolkits provide necessary functionality  Graphical UI, Web Services, XML all within reach  Versatility  Few constraints imposed by technology

14/06/06 21 Other Client Technology Options  Stand-alone application (.NET, Java, C/C++, etc.)  Versatile  Typically more platform dependent  User must download/install client  HTML/Web Script based client (.NET, Java/JSP, PHP, etc.)  Typically more portable, easy to deploy  Potentially fewer interface options (limited educational value)  Client development packages (LabView)  Rapid deployment, flexible interfaces  Requires client runtime plugin  More methodology than technology, hard to integrate with Batched-Lab Architecture (outlook much better for Interactive labs)

14/06/06 22 Meeting Client Design Goals: Modularity/Extensibility Client built from three distinct modules:  User Interface Layer  Only presentation code  Main Client Module  Contains core functionality  Server Interface  Translates Core commands to Web Service Calls Many changes can be isolated.

14/06/06 23 Client Extensibility: The Classical Applet A more nimble client needed for deployment in bandwidth starved areas  Graphical Applet’s intuitiveness comes at the expense of size/end-user requirements  A new UI Layer was needed to meet these needs

14/06/06 24 Client Extensibility: The Classical Applet  Both clients use the same Core Functionality and Server Interface modules  Classical applet is smaller (94KB vs. 169 KB) and has fewer requirements

14/06/06 25 Reusability of WebLab Code  Lab Client/Server code is lab-specific  Exception is Client graphing module  However, some parts can be reused with modification  Client/Server – Broker Interfaces, some management tools, Execution queuing, Client/Server infrastructure…  Deployed labs always valuable as working examples

14/06/06 26 Conclusions  iLab Shared Architecture, Service Broker eases lab development.  Turnkey generic functionality  Well-defined client/server APIs  Lab Client & Server interact over generic transport using lab-specific messages.  Remainder of lab development framed by case specifics.  Previous labs can be leveraged in new development  iLab Development Documentation & WebLab code available at