1. Archiving System Deployed at Soleil
Tango Meeting (Soleil, March 2007)

2. Archiving System Deployed at Soleil
Archiving System General Description
HDB
System Architecture
Assessment
Improvements
TDB
Mecanism Evolution
HDB / TDB Implementation Remarks
Tango Meeting (Soleil, March 2007)

3. Archiving System General Description
The tools :
Historical DataBase (HDB)
Data are kept permanently
The minimal archiving period is 10 seconds
Temporary DataBase (TDB)
Data are kept 3 days
The minimal archiving period is 100 milliseconds
The two are managed via the graphical tool named Mambo
Snap DataBase (SNAP)
Allows to have a snapshot of a set of attributes values
Allows to apply values on a set of attributes
It is managed via the graphical tool named Bensikin
Its Goal : the same system must be useful for the machine and the beam lines. The main distinction is
Machine : Oracle DataBase
BeamLine : MySql DataBase
Tango Meeting (Soleil, March 2007)

4. Archiving System General Description
On the Machine Control System
HDB and TDB is largely used by machine operators
To log the values of their equipments
To analyze somes particulars phenomena
To help them diagnose (beam interlocks …)
Snap not used enough
On the Beam Lines Control System
Snap will be deployed as part of the Control System
HDB and TDB will be installed on demand
Tango Meeting (Soleil, March 2007)

5. HDB – System Architecture
Jacquard
SNAP
HDB
Devices
(Watcher, Extractor, etc)
Knuth
HdbArchiver
Currently : 2600 archived attributes
2574 scalars + 26 spectrums
380 at 10 s period
1350 at 30 s
700 at 60 s
150 at [ 100 – 1000] s
The rest is upper 1000 s
25 DServers * 5 devices, so 125 devices
Now each device supports about 20 attributes
Number of devices has been estimated to support 6000 attributes so about 50 attributes / device
Tango Meeting (Soleil, March 2007)

6. HDB - Assessment Now : Secured HDB archiving system. How to notice it
Less call from the operators
Via the Supervision Device : the Watcher
Diagnosis tool : detecting abnormal archiving interruption
Recovery : reactivate archiving on the failed attributes
Tango Meeting (Soleil, March 2007)

7. HDB - Assessment How do we reached this goal ?
The code is more reliable
Some attributes were archived by more than one archiver
Lots of abnormals behaviors has been resolved
Recovery
Oracle Database optimization
The storage mode (data are splitted in n partitions)
Oracle server memory management
Tango Meeting (Soleil, March 2007)

8. HDB - Improvements Spectrum storage in the Oracle Database
The Oracle System foundation must be reexamine
The space storage is not enough to guarantee a correct DataBase backup copy system
This is a very important point for the control : backup / restore quickly in case of crash
In the initial machine specification the maximum archived attributes must be 6000 now users request near attributes
The Oracle configuration must be clearly reviewed
Tango Meeting (Soleil, March 2007)

9. TDB – System Architecture
AHO
Devices
TdbArchiver
Blaise
TDB
Devices
(Watcher, Extractor)
Currently : 2950 archived attributes
2940 scalars + 10 spectrums
15 at 200 ms period
1030 at 500 ms
1840 at 1 s
50 at [ 10 – 60] s
The rest is upper 60 s
40 DServers * 5 devices, so 200 devices
Now each device supports about 15 attributes
Number of devices has been estimated to support 6000 attributes so about 30 attributes / device
Tango Meeting (Soleil, March 2007)

10. TDB – Mechanism Evolution
1 - At the beginning
Considering the minimum required period (100 ms) it was not possible to directly insert the attributes values in the DataBase
Solution : work with intermediates files
1 - File Creation
2 - Filling
Thread
File
4 – Read  Insertion line / line
TDB
3 – Read order
Observations :
Too many simultaneous insertions in DB
DataBase was saturated  the Archiver threads were blocked
In this configuration the TDB Archiving System was hang with only 600 attributes (6000 must be stored !!! )
Remark : This solution is always true for MySql DataBase
Tango Meeting (Soleil, March 2007)

11. TDB – Mechanism Evolution
2 – Read the files thanks to Oracle SqlLoader
Massives insertions file by file called by an Oracle Job
Thread
File
1 - File Creation
2 - Filling
TDB
3 – One New file is available
Every 4 minutes Block Insertion
Job
Observations :
Too many insertion in DB : the Job is blocked and so the DB itself
On the other hand, the Archivers are no more blocked but we cannot support more than 1500 attributs
Tango Meeting (Soleil, March 2007)

12. TDB – Mechanism Evolution
3 – Creation of 10 Oracle Jobs
Call every 10 minutes
Each of them managed the files of different attributes based on the last digit of the attribute identifier.
Observations :
Now near 3000 attributes are archived
The system can support the 6000 attributes
Tango Meeting (Soleil, March 2007)

13. TDB - Improvements Spectrum storage in the Oracle Database
Some peace of code must be improved (still too many exceptions)
New Specification : permanently store, on demand, a subset of logged attributes values
Require to lay out a particular storage space in the DB (Oracle and MySql) to comply to this new requierement.
Tango Meeting (Soleil, March 2007)

14. HDB / TDB – Implementation Remarks
There are some divergences between the code for MySql and Oracle
See the Tdb mechanism
DataBaseApi design is being reviewed to well distinguish the MySql and Oracle code  refactoring ?!
We don’t manage the milliseconds in the MySql Database : not supported by the TIME type
Tango Meeting (Soleil, March 2007)