Managed by UT-Battelle for the Department of Energy Best Ever Archive Utility, Yet Creating a BEAST was easy. BEAUtY seems harder. April 2009

2Managed by UT-Battelle for the Department of Energy CSS-based OPI Archive Archive Engine config.xml IOC Binary Data Files Data Server “ASCII” Config. Channel Access XML-RPC Previous Setup History ~2000: Started by Bob Dalesio ~2003: Index Tools, Data Server ~2007: CSS Client

3Managed by UT-Battelle for the Department of Energy Details  Older Java “archive viewer” also available.  Archive Engines running on “sampling” computer, data server on “serving” machine.  Scripts to restart engines, copy data, update indices.  Headaches with data maintenance: –Index time grows with data –Stuck when index files reach 2GB –SNS Users faced with ~80 sub-archives –No clue what needs fixing after network/power problems –No idea who contributes how many samples –No way to remove selected channels or time ranges –Improving on this means implementing an RDB

4Managed by UT-Battelle for the Department of Energy CSS-based OPI ArchiveEngine config.xml IOC RDB (Oracle/MySQL) Channel Access New Setup Samples Config. EngineConfig -Import Other tools for config & samples

5Managed by UT-Battelle for the Department of Energy CSS Data Browser Handles Both  New URL  Just one ‘RDB’ sub-archive  Old and new data can be combined in one plot

6Managed by UT-Battelle for the Department of Energy Configure Sample Engines  Configuration is in RDB –Directly use SQL –EngineConfigImport for legacy config files –View/Edit via web  Hierarchical (as before) –Sampling engine (name, where to run, …)  Groups –Channels  No more duplicate channels!

7Managed by UT-Battelle for the Department of Energy Web Interface to Engine Config in RDB  Tomcat/JSP/Servlets to view and edit

8Managed by UT-Battelle for the Department of Energy Web Config View: Channel Stats

9Managed by UT-Battelle for the Department of Energy Web Config View: Sample Stats OK ??

10Managed by UT-Battelle for the Department of Energy Stats  34 sample engines, channels  Host that runs sampling engine: –CPU load 45%, zero disk I/O wait, very responsive  Oracle performance –8000 samples/sec peak in write tests –Operationally maybe ¼ of that  Better configuration would likely have fewer samples/sec

11Managed by UT-Battelle for the Department of Energy Report Example, SNS Beam Instr. Group  May be SNS-specific, but shows usefulness of data in RDB

12Managed by UT-Battelle for the Department of Energy Summary  Still testing, many Oracle setup issues –Ran since Oct 08, then.. –Oracle cluster setup since ~March –Updated partitioning in April  Plan for about a year now: Make operational in maybe 2 months  Thanks go to –Oracle: Jeff Patton –JSP: Katia Danilova –Beam Instr. Reports: Mariano Padilla, Jeff, Douglas Edwardson –IT: Greg Lawson

13Managed by UT-Battelle for the Department of Energy Stuff

14Managed by UT-Battelle for the Department of Energy Hurdles  Months: Get new Oracle server configured –Interface cards for storage array, fiber switches –Firewall holes for office access, backup, admin  Changes in 10g –SELECT MIN(stamp), MAX(stamp) -> NULL, NULL –No “range” partitioning on Index-Organized-Tables  Configuration issues –ORA-01654: unable to extend … –ORA-00257: archiver error  What used to be impossible is now “easy”, but still expensive –DELETE FROM SAMPLES WHERE …

15Managed by UT-Battelle for the Department of Energy Other Ideas  Jlab’s MyA –Operational, but  Wrapper code around MySQL to create ‘cluster’  Handles less meta info (units, limits, …)  Viewer is one-of C++/TCL/Tk  Gabriele Carcassi mentioned RDDTool –Toolset for logging data with data-aging –Command-line RPN tools, web viewer –May be faster than general-purpose RDB  but lacks advantage of gpp RDB

16Managed by UT-Battelle for the Department of Energy SNS Setup for EngineConfigImport Tool –Imports existing XML config files into RDB –On srv02 in …/imported_config  *.xml files copied from archive1, includes some editing to remove duplicate channels  Review *.log files –'chmk/chmkhebt/PPS_HEBT:PLC_B:Chmk100AlarmContactA’ already in 'pps/pps_hebt’  Maybe edit “your” *.xml file, refer to import.sh to re- import

17Managed by UT-Battelle for the Department of Energy Getting Sources, Building binaries  Get sources from DESY CVS repository –Maybe binaries “real soon” from ChannelArchiver web page  Build binaries –Find somebody who understands “Eclipse RCP”, how to use  org.csstudio.archive.rdb/EngineConfigImport.product  org.csstudio.archive.engine/ArchiveEngine.product  org.csstudio.sns.product/SNS_CSS.product (or DESY CSS.product) –Settings  Compile-time defaults usually in “preferences.ini” and “plugin_customization.ini” files.  Either change, or prepare file for -pluginCustomization command-line option

18Managed by UT-Battelle for the Department of Energy RDB Setup  Currently supported: MySQL or Oracle –Main table difference: Oracle TIMESTAMP has nanosecs, MySQL has separate nanosecs col. –All tools auto-configure based on JDBC URL  Table Schema –ERD, MySQL DBD in org.csstudio.archive.rdb/dbd –Setup for Oracle (table types, partitioning) not final

19Managed by UT-Battelle for the Department of Energy Run Archive Engine  Command-line tool similar to previous engine –More settings (RDB, Logging, JCA/CAJ, …)  Compiled-in defaults from “preferences.ini” and “plugin_customization.ini”  Can specify overrides via -pluginCustomization my_plugin_customization.ini  Mandatory command-line options for instance –engine_name to locate config in RDB –port for consistency, must match settings in RDB  Built-in web server for status, restart, stop:  Empty page (used to be ‘main’)  …./main: Main page  …./reset: Reset statistics  …./restart: Stop, re-read configuration, start again  …./stop: Stop