Presentation on theme: "Channel Archiver Stats & Problems Kay Kasemir, Greg Lawson, Jeff Patton Presented by Xiaosong Geng (ORNL/SNS) March 2008."— Presentation transcript:
Channel Archiver Stats & Problems Kay Kasemir, Greg Lawson, Jeff Patton Presented by Xiaosong Geng (ORNL/SNS) March 2008
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 2 Channel Archiver 1997 design goal: Write 10,000 samples/sec Raw 'write' test on today's computers: >50,000 samples/sec Used by several EPICS sites worldwide
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 3 SNS Stats Need only ~2,000 samples/sec on average. Configuration for ~80,000 channels split into ~70 sub-archives … so that maintenance or problems on one sub-archive won't affect others Means users have to retrieve from correct sub-archive CSS DataBrowser makes that a bit easier (next talk) Disk space: ~170 GB/month Keep running into disk space limitations
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 4 Data Management Limitations Difficult and time consuming Moving data around requires manual index updates (takes hours, could be days) Few Informational Tools Nothing prevents duplication Which channels contribute the most to data growth? Storage only supports "Append new samples" Removal of selected channels impossible Removal of older data limited to complete 'sub archives' No practical way to use Java or Matlab code to replace original samples with reduced sample count, .. or to insert computed data like daily statistics into "archive"
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 5 Question Continue to invest in Channel Archiver's data management tools? End up implementing a relational database? Why not use existing RDB?
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 6 JLab: MySQL Transition (Chris Slominski) Very promising performance tests! Limitations by design Stores every update from IOC. No 'sampling'. 'Double' stored as 'float' to save space. Only small arrays. Metadata: Units. No limits, precision. No status/severity. MySQL Issues Table size is limited Need one table per channel Table count is limited Custom code implements 'clustering' SQL "DELETE" doesn't free disk space, or is very slow (delete ONE week data --> take TWO weeks)
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 7 SNS Oracle Tests Basic JDBC test code: up to 8,000 inserts/second via network Tricks "Batching" ~500 inserts "Partitioning" spreads one big "sample" table over disk partitions Currently one partition each day, automatically added. Details are still being developed. Expensive, but looks like the way to go Avoid MySQL workarounds SNS committed to Oracle anyway, but will need partitioning license
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 8 Great! But what about SLAC? Reported promising performance results for Oracle-based data storage Lee Ann Yasukawa, Robert Hall: "Archiving Into Oracle", ICALEPCS2001 End of 2004: No more. What do we need to learn from that?
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 9 Basic Sample Table Design What data types to support? Time stamp detail, enumerated values, arrays, meta data? One table per channel (JLab)? One table per data type (SLAC)? SNS: One table for all scalar samples Possibly wasting space, but best to use SQL across various channels of different types Array elements in additional table
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 10 Archive Engine Prototype Developed in Java Eclipse/CSS command-line app Reads existing engine config files "Scanned" and "monitored" sampling as before "Disabling"/"enabling" of groups of channels Scalars, arrays, enumerated samples Web-server interface for status which is similar to the original engine Writes into Oracle Write performance OK for scalar tests Reaches the ~8000 samples/sec from original tests
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 11 Engine's Web Server
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 12 Archive Engine Prototype… JNI vs. CAJ Only JNI dependably handles many Channel Access connections (>10,000) JCA/JNI Performance improved on Mar. 11, use “yield” call not “sleep” 10 msec, approximately factor of 30 improvement (from 45 seconds down to 1.5 seconds for 1460 non-blocking connection requests on SNS physics server) Can run multiple engines, but RDB enforces that they handle different channels Arrays are slow One N-element array comparable to N scalars Only "Double" type array elements Need to implement meta data handling (units, limits, …) Plan is to support MySQL, but "you get what you pay for" No custom code to handle table size/count limitations MySQL "Timestamp" only handles seconds, no nanoseconds
O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 13 Summary Testing Oracle setups and prototype sampling engine Oracle space currently only ~30GB Data retrieval via CSS DataBrowser RDB is just another data source Compared to current SNS ChannelArchiver setup, performance expected to be A little less … but sustainable in the long run.