1. BKh - 16/01/2007LHC PM Workshop1 SDDS to LabVIEW The path from client data to viewing and analysis Greg Kruk ➔ 08/2006 SDDS converter package Boris Khomenko 2005 ➔ PM module viewer/analyser - PMX for SDDS Hubert Reymond2005 Graph sheets Dmitriy Kudryavtsev 2006 ➔ Generic SDDS converter

2. BKh - 16/01/2007LHC PM Workshop2 SDDS Converter PMD ➔ SDDS pmdata PM Server subscription PMD name dev-time.pmd sys-class-pmxNN-dev-time.pmd pmdata.bin PMX SDDS_bin SDDS_ascii NB: Duplicated PMD files are recognized and saved differently. Reprocessing is possible scan_pmbinscan_sdds scan_pmclog log

3. BKh - 16/01/2007LHC PM Workshop3 SDDS Converter is an Event Builder SYS Class evdays devlst year device evd_day dvm_month ev_evid dv_evid *.sdds *_ascii.sdds … pmdata.bin pmx_*-vNN.txt s-link

4. BKh - 16/01/2007LHC PM Workshop4 SDDS: Self Describing Data Set HeaderHeader SDDS1 &parameter name=PMM_System, type=string, &end &parameter name=PMM_Class, type=string, &end &parameter name=PMM_Source, type=string, &end &parameter name=PMM_Timestamp, type=llong, &end &array name=AQNTIME, type=llong, &end &array name=MB.LOC:ST_PWR_PERM, type=byte, modifier=ST:, &end &array name=MB.LOC:U_2, type=float, format_string=%.4f, &end &array name=MB.LOC:U_1, type=float, format_string=%.4f, &end &array name=MB.LOC:U_QS0, type=float, format_string=%.4f, &end &data mode=ascii, &end DataData QPS DQAMCMB C16L2 1165859504818000000 2548 1165859503784000000 1165859503789000000 1165859503794000000 …… 2548 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 … …….

5. BKh - 16/01/2007LHC PM Workshop5 Added features for PM ➔ SDDS+  New data type “llong’ (int-64, timestamp standard for PM)  Additional description field “Modifier”, used for: oName definition for digital data ( states and bit flags ) “Modifier=ST:” - boolean (0=False, 1=True) “Modifier=ST:/Name_0/Name_1/…” - names for states 0,1, … “Modifier=FL:/Flag_0/Flag_1/…” names for bit flags 0,1,… oArithmetic operations on analog values while file reading “Modifier=*1e-3” - f.ex. conversion mV -> V oIndividual customization of string data representation (examples) “Modifier=u1” - string length format is byte (for binary SDDS) “Modifier=sep:t” - set string separator to tab (for ascii SDDS)

6. BKh - 16/01/2007LHC PM Workshop6 PMX: PM data index and control Simple text file which makes SDDS converter generic A text file fully defining the conversion process of raw data into SDDS file(s). Is very simple for simple cases, is clear to treat special requirements, is open for new (reasonable) complications. Must contain o Raw data block description (formats and names) May contain o Additional info for SDDS object attributes o Instructions for data generation/modification o Instructions for SDDS file formation

7. BKh - 16/01/2007LHC PM Workshop7 PMX/Converter extended options As it was asked by users, the conversion takes additional efforts to make SDDS files more comfortable to read by their applications possibly without additional operation on data. So such operations may be performed by converter: (Yet from Greg’s version) Table data are represented as column arrays, not rows as in standard SDDS practice. NZ segmentation: automatic detection and cut of leading and trailing zeros in timestamp array, then all table columns will be cut in the same way Timestamp column generation as N samples at rate F or Δt, then adjusting f.ex. last element to acquired value Generation and/or modification of columns via arithmetic and some others operations on existing ones Creation several SDDS files per data module

8. BKh - 16/01/2007LHC PM Workshop8 PMX examples $SET Endian LITTLE i4 TYPE i4 CIRCUIT i8 QUENCHTIME $TABLE Rows 2550 SegmNZ AQNTIME i8 AQNTIME u1 MB.LOC:ST_PWR_PERM {ST:} u1 DQQDL.LOC:ST_COHER {ST:} u1 MB.LOC:ST_MAGNET_OK {ST:} u1 MB.LOC:ST_NQD0 {ST:} +80 f4 MB.LOC:U_2 (%.4f) f4 MB.LOC:U_1 (%.4f) f4 MB.LOC:U_QS0 (%.4f) f4 MB.LOC:U_HDS_1 (%.4f) f4 MB.LOC:U_HDS_2 (%.4f) +72 $TEND $FILE $SET Endian BIG # # Buffer 'status' ……. $TABLE Rows 21000 SegmNZ TIMESTAMP_SEC i4 TIMESTAMP_SEC i4 TIMESTAMP_USEC {*1e-6}...... u2 ST_MEAS_A {FL:I_MEAS_OK/V_MEAS_OK/...} u2 ST_MEAS_B {=}...... i2 I_DIFF_MA {*1e-3} (%.3f) …… f4 V_REF (%.4f) f4 V_MEAS (%.4f) $TEND $GENCOL f4 I_DIFF = I_DIFF_MA/1000; $MAXCOL 30 $MINCOL -30 $FILE Suffx status

9. BKh - 16/01/2007LHC PM Workshop9 Conversion flowchart PM server unzip PMDbin PMX SDDSSDDS a s c ii - b i n Data objects pmdata

10. BKh - 16/01/2007LHC PM Workshop10 Conversion at client site (proposal) optional use of block build methods PM serverPMX SDDSSDDS bin imagebin image Data objects Data buffer initDataModule(…) addDataIntParam(…) addDataStrParam(…) addDataLlongArr(…) … sendDataModule(…)

11. BKh - 16/01/2007LHC PM Workshop11 SDDS Converter Status Is in production for QPS and PC Full processing time for module: – QPS/MB - 44ms – FGC/51 - 440ms (9 SDDS files) All user requests are implemented Demo for “conversion at client site” is almost ready, then contacts with interested users, if any, might be useful

12. BKh - 16/01/2007LHC PM Workshop12 Are PMD and SDDS data in security ? PM Server and SDDS Converter run under “OPERA” account It is a garbage account, 20 years old PM project needs to be normally legitimated in this sense, f.ex. by creation PMS, PMA and PMU accounts under group “PM” with permission scheme like: PMDSDDSPM Results PMSWW- PMA-RW PMU-RR

13. BKh - 16/01/2007LHC PM Workshop13 PMM_VA PM Module data View and Analysis PMM data locator SDDS Read SYS spec data modif SYS Class Option AnalysisAnalysis Result Views Analog + State Data View Result Table

14. BKh - 16/01/2007LHC PM Workshop14 PMM_VA in automatic mode (reduced application) SDDS Read SYS spec data modif SYS Class Option AnalysisAnalysis Result Views A+S Data View Result Table PMM data locator File(s) Event Result Out callparamscallparams

15. BKh - 16/01/2007LHC PM Workshop15 PMM Data Locator-1

16. BKh - 16/01/2007LHC PM Workshop16 PMM Data Locator-2

17. BKh - 16/01/2007LHC PM Workshop17 SDDS reading SDDS Data Parameters Analog data channels State data channels Timestamp channel Internal object types

18. BKh - 16/01/2007LHC PM Workshop18 QPS MB Graphs

19. BKh - 16/01/2007LHC PM Workshop19 QPS analysis result NB. QPS team runs its own analysis locally. Nothing requested from PMM_VA. The shown result is from older frosen version, just for impression

20. BKh - 16/01/2007LHC PM Workshop20 S-Map for FGC

21. BKh - 16/01/2007LHC PM Workshop21 PMM_VA status New data locator OK to read SDDS binary or ascii for QPS and PC OK to display limited number of analog and state (A+S) channels Is in progress to show many channels by scrolling, selecting, sorting and regrouping channel lists QPS analysis no more requested by the team PC automatic analysis is in development Version 2.00 to the end of January