Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) HPS Web 2.0 OR Web Apps and Databases (Oh My!) Jeremy McCormick (SLAC)

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) OVERVIEW Database Server Conditions Database Run Database Web Server Run Web Data Catalog Online DQM 2

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) WHY? Databases Central facility for accessing shared information Functional replacement for text files Provides backend for other applications like recon and web apps Change management and authentication (user accounts) Web applications View from “anywhere” without needing any special applications (such as ROOT) Easily share important information such as DQM plots See what’s inside data sources (databases, files, etc.) Because Javascript is magic! 3

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 4 DATABASE SERVER Dedicated HPS MYSQL server hpsdb.jlab.org Hosts conditions, run and datacat databases Full access from inside the JLAB firewall from ‘jlab’ hosts Grants for write operations depend on the account Several users have admin privileges (myself and Maurik) Read only access via an external replica External non-jlab hosts cannot see the primary db directly. Non-production copy at SLAC for Hudson builds Accessible in hps-java through APIs built on JDBC

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 5 CONDITIONS DATABASE Contains detector conditions data from Test Run to present. Each type of condition has its own table with sets of rows assigned to a collection. Collections are assigned valid run number ranges. Sets of collections can be tagged for reproducibility. Data is accessible in reconstruction via an extensive Java API. The ‘hpsuser’ account has a public password and is used for read-only mode. The ‘hpscalibrations’ account can be used to insert, update or delete existing data.

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 6 RUN DATABASE Database of run metadata extracted from crawling the EVIO data Run start and end times Number of events etc. EPICS data Scaler data TI Trigger Time Offset Java API through RunManager Can easily add additional information to DB and API!

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) RUN LOG Run database currently != run log Run database is currently populated from crawling the data after it is copied to MSS and extracting metadata from EVIO. Run log would be updated via hooks in the DAQ so that it happens concurrently with END and PRESTART events. Implementing requires some coordination with Hall B staff. Need to add C code into CODA Determine which values to include (have gotten some good feedback here) Some values also could be filled interactively by pop-ups, GUI interfaces, etc. 7

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 8 WEB SERVER RHEL 7, dual core, 4 GB VM Tomcat 7 Standard Java Servlet container Admin manager only accessible behind JLAB firewall Apache Proxy Specific application URLs mapped to Tomcat Managed by IT Division / CNI Group Terminal access only allowed with crytocard

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 9 RUN WEB Shows information from run database on the web. Sortable run table Run page Summary EPICS data Scaler data EVIO datacat link DQM datacat link

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 10 DATACAT New data catalog server and REST API from SLAC Query and update via HTTP requests Built-in browser interface MySQL backend (runs at JLAB) Java client API Wrapper for URL based requests Add, update, delete, search Crawler for finding data, extracting meta data and adding to the catalog

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) EXAMPLE DATACAT PATHS Display Browser /display/browser/HPS Show a path /r/path/HPS Show a dataset /r/path.json/HPS/test/dataset01;s=SLAC Filter /r/search.json/HPS/dataset01?filter=runMin+>+1000 Recursive Search /r/search.json/HPS/**?filter=runMin+eq

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) 12 DQM ONLINE Example DQM Plots (not production yet!)Example DQM Plots AIDA TLD library of JSP tags (SLAC) Allows easily showing plot files in a web page Can read from AIDA or ROOT files on the file system At JLAB /work can be mounted by server read-only. Export individual plots to graphics files (PNG, etc.)

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) TODO Finish up JLAB web config Enable authentication via scicomp comp certificate Get DQM server working Run database Consolidate multiple sources of run data into database Code for extraction of run (meta)data should be integrated into the crawler module Data catalog Command line interface Integration with reconstruction and Auger jobs 13

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) NOW FOR SOMETHING DIFFERENT! What’s new in SLIC these days? Current versions of Geant4 and other dependencies Support for 3D magnetic field maps Displaced vertices in StdHep files now supported (and maybe even tested) Conversion from StdHep event records into LCIO MCParticle collection uses a standard/shared LCIO class Code for translation of Geant4 tracks into MCParticle output collection rewritten 14