Java Root IO Part of the FreeHEP Java Library Tony Johnson Mark Dönszelmann Victor Serbo Max Turri Presented at Computing in High Energy Physics Interlaken, Switzerland, September 2004

Contents  Goals  Java IO library for Root –Implementation –Test Suite –Benchmarks –Networked Root IO (rootd/xrootd) –Demo tools –Root Plugin for AIDA  Examples of Use –AIDATLD Web Interface –JAS3 –WIRED4  Future plans  Links for more information

Java Root IO: Goals  Goals –Pure Java package for reading Root Files  Will be extended to writing later –High Performance  We want to do data analysis so we need very good performance –Should not need to know about objects ahead of time (no need for dll’s,.so files etc.)  But should provide full support for reading user- defined objects –Easy to use –Provide access to Root data, not C++ code

Implementation  Use “TStreamerInfo” objects inside Root files (since Root 3.0) to decode contents of file  Dynamically generate Java proxies for each Root object read  Each proxy has custom “streamer” –Build as Java bytecode using BCEL ( –Converted at runtime to machine code by HotSpot VM  Uses java.nio package (new since Java 1.4) –Standard Java package for high-performance binary IO

Implementation Details TStreamerInfo Java Interface Accessor method for each data member.java file Java Proxy Custom version based on how object is stored in file Implements Interface Generator

Example of Use

Test Suite  Most classes are dynamically generated from info in Root file, so are fairly robust against changes  Need bootstrap process, in particularly need enough custom Java classes to read the streamer info, and interpret it –Relatively few classes, but  This tends to change from one Root release to another  Typically have to make small changes in the bootstrap code for each new Root release  We have test suite that creates a set of test files from each new root release, and verifies that our library can read the files. –Currently test , , , , , ,  In addition we have a set of Root files collected from users, and the test suite verifies that we can (continue) to read these files with each release.

Java Root IO - Benchmark  1.4GHz AMD Athlon, Windows XP, 512MByte DDR, ATA100 disk..  S= Split level = 0,1,2  C= Compression = 0,1  Each test run 5 times in succession, lines show range in results –in later runs file is typically cached in memory == faster if task IO limited  Task: Loop over all events, loop over all tracks, sum px for all tracks event files created with “event” test program distributed with Root ( MByte). CSms/Event

Network Root IO (x)rootd  Recently added ability for Java library to talk directly to rootd or xrootd server –Defined “root:” protocol handler for URL class. –Defined Authenticator for authenticating to rootd –Extended RootFileReader to accept (any) URL argument to constructor  If URL corresponds to file: read the file directly  If URL corresponds to root: protocol use xrootd/rootd (random access)  If URL corresponds to http: and web server supports http 1.1 “ranges”, use extended http  Otherwise buffer URL contents into memory and read from there (suitable for small files only).

Authentification Issues  Authentification can be set programmatically, or via GUI Authenticator  Rootd supports many authentification schemes 1. Anonymous (c.f. anonymous ftp) 2. User/Password (clear text) 3. User/Password (RSA encrypted) 4. SCP 5. SSH like 6. GSI certificate 7. UID/GID Currently FreeHep library supports only 1 and 2Currently FreeHep library supports only 1 and 2 Work underway on 3 (but some compatibility problems between Root and Java RSA implementations)Work underway on 3 (but some compatibility problems between Root and Java RSA implementations) Java libraries exist for all other methods, so can be supported in future.Java libraries exist for all other methods, so can be supported in future.

Network Root IO: Example of Use

Demo Apps: Root Object Browser

Demo Apps: Root Histogram Browser

Root Plug-in for JAIDA  JAIDA = Java implementation of AIDA  AIDA = Abstract Interfaces for Data Analysis = Histograms + Tuples + Fitting + Plotting + IO data analysis package –(See Monday’s talk by Victor Serbo)  Root plugin for JAIDA makes it possible to read root files via the AIDA interfaces and treat the objects contained in the root file as AIDA objects –Useful if:  You like AIDA interfaces better than root interfaces  You want to use language not supported by Root  You want to analyze data in mixed formats  You want to work in J2EE, J2ME, Applet etc.

Root Plug-in for JAIDA: Example

Root Java IO: Examples of Use AIDATLD Web Interface JAS3WIRED4

AIDATLD + Root  AIDATLD = AIDA Tag Library for use in JSP pages –Trivial to put “live” histograms on web –Directly supports Root files

AIDATLD Example: GLAST System Tests Web Interface

JAS3 + Root  JAS3 = Modular, Extensible, Data Format Independent data analysis system.  Among many other formats JAS3 supports Root –Contents of Root files can be accessed directly, or via Root->AIDA interface  Full access to AIDA functionality for fitting, projections, plotting etc. –Can operate on Root data  using GUI  using scripting (Python, Pnuts)  using compiled Java code

JAS3 Access to Root Files

JAS3 + Root: Scripting

Root + WIRED4  Build yourself a super cool event display for your custom Root file in under 1 hour!

Future Plans  Continue to track new Root releases –Root network protocol  More work on authentification –Root file writing  Most of work already done by Petr Vokac/Julius Hrivnac  Easier than reading because only need to support one root protocol version  Support Root XML format –Will be much slower, but perhaps more robust  Use LCGDICT system to allow Java access to C++ methods as well as data? –Probably also slower, but more flexible

More Info  Freehep Java Library –  Root IO component of library –  AIDATLD –  JAIDA –  JAS3 –