Clara Gaspar, February 2010 DIM A Portable, Light Weight Package for Information Publishing, Data Transfer and Inter-process Communication

Clara Gaspar, February DIM Design Requirements ❚ Efficient Communication Mechanism ❙ Asynchronous Communications ❙ One to Many Communications ❚ Uniformity ❙ The same mechanism for all types of communications ❚ Transparency ❚ Reliability and Robustness

Clara Gaspar, February Communication mechanism ❚ Synchronous ❚ One-to-one ❚ Asynchronous ❚ One-to-many RPC & CORBADIM

Clara Gaspar, February Design Philosophy ❚ Client/Server (Publish/Subscribe) ❚ Services ❙ Set of data (any type or size) ❘ Single items, arrays or structured data ❙ Free name space (but better use a naming convention) ❙ Servers publish Services. ❙ Clients subscribe to Services. ❘ Once, at regular intervals or on change ❙ Clients can also send Commands to Servers ❚ Name Server ❙ Keeps the coordinates of available Services

Clara Gaspar, February DIM Dataflow ❚ Internal mechanism

Clara Gaspar, February DIM transparency ❚ DIM clients do not know where their interlocutors are. ❚ Servers can move from one machine to another, all connections are transparently re-established. ❚ The Name Server can also migrate. ❙ Any pre-established connections will continue working without DNS ❚ All platform differences are handled internally (byte swapping, structure alignment, floating point representation)

Clara Gaspar, February DIM Robustness ❚ DIM detects and recovers automatically from most problems: ❙ Servers, clients or Name Server stops, crashes or hang-ups… ❙ Machines reboots… ❚ Clients are informed when Services not available. ❚ DIM is an old product, has been tested and debugged for many years…

Clara Gaspar, February DIM Monitoring ❚ Large distributed systems are normally difficult to Test, Debug and Monitor ❚ Due to the DIM publishing mechanism: ❙ DIM components (the Name Server and the Servers) publish a set of standard Services used for monitoring ❙ A tool - DID - (using the standard DIM client API) allows access to all information ❙ A browsing API is also available (to be used with care) Can ask for “LHC/magnets/*/current” and then subscribe

Clara Gaspar, February DIM Monitoring - DID

Clara Gaspar, February DIM Performance Fast Ethernet (100Mb/s) Gigabit Ethernet (1000Mb/s) Throughput (kB/s) Message Size (bytes)

Clara Gaspar, February DIM Availability ❚ DIM is available under GPL (GNU Public License) ❙ As Server and Client libraries: ❘ API: C, C++ and Java (also Python…) ❘ at ❙ For the platforms: ❘ VMS (VAX and APHA) ❘ UNIX flavours (HP-UX, Sun-OS, Sun-Solaris, IBM-AIX, DEC-OSF, Linux, MacOSX) ❘ Windows NT/2000/XP ❘ Real-time OSs (OS9, LynxOS, VxWorks)

Clara Gaspar, February DIM example

Clara Gaspar, February DIM Interfaces ❚ LabView (GSI Darmstadt) ❚ PVSS II (fwDIM component) ❙ PVSS II can act as a DIM Client ❘ PVSS datapoints can “subscribe” to DIM services and be kept up to date ❘ PVSS datapoints can be sent as DIM Commands ❙ Or as a DIM Server ❘ PVSS datapoints can be published as DIM services ❘ PVSS datapoints can “receive” DIM Commands ❙ Can also browse and find available DIM Services and Commands

Clara Gaspar, February PVSS – DIM Interface ❚ Datapoint and DIM Service structures have to match

Clara Gaspar, February DIM Usage ❚ DIM is used by ALL LHC experiments ❚ And also in several other projects at CERN and outside CERN ❚ Some selected ones: ❙ DIP – The CERN Data Interchange Protocol is based on DIM ❙ SMI++ - The Finite State Machine toolkit used by the LHC experiments (and others) uses DIM as communication layer ❙ The PVSS Central Log Viewer

Clara Gaspar, February DIM Usage ❚ In LHCb for example, DIM is used for: ❙ The Configuration and Monitoring of all DAQ and Trigger electronics ❙ The Control of all non-standard DCS equipment ❙ The Monitoring and Control of the Trigger farm (2000 PCs), and the Monitoring Farm. ❘ Infrastructure (PC monitoring, reboot, switch on/off, etc.) ❘ Configuration and monitoring of trigger/monitoring processes ❙ Transporting all statistical data for Data Quality checking (all histograms and counters published by the trigger and monitoring tasks -> Histogram Presenter) ❙ The interface to the Conditions and Run Databases ❙ The DAQ/Dataflow Log mechanism ❙ To publish data for the WEB

Clara Gaspar, February LHCb/DIM in images

Clara Gaspar, February Conclusions ❚ Online applications (control, monitoring) have particular needs in terms of communications: ❙ Asynchronous Response ❙ Multiple destination updates ❙ Multiple platforms ❙ Efficiency and Reliability ❚ DIM’s publish/subscribe mechanism is well suited for such applications