1. Multiple Federations Issues Where to use multiple FDs How to produce FD “copies”

2. Multi-Federation Issues n Currently we use multiple consistent FDs –Decoupling n avoid unwanted lock contention n avoid unwanted changes to schema or catalogue –Partial copy for distribution or backup n transfer data to machines with slow and/or unreliable network connections to the original FD n save a backup of central meta data n Mainly workarounds to deficiencies in Objectivity/DB –lock contention on global resources (catalogue) –lack of private schema/catalogue –lack of security –lack of support of partial backups

3. Decoupling Processes with different Priority n Problem: exclude locks from low priority processes –e.g. DAQ needs to perform independent of locks obtained by offline analysis n Solution: two federations with independent lock servers –offline starts as copy of the online FD n copied FD uses a different FDid and typically a different lockserver host –periodically copy a consistent state of all updated database files to the offline FD n alternatively one could attach read-only files to both federations n Limitations: –simple if data flow is uni-directional –special care needed if data is propagated back –complete file copies needed for updated file (e.g. registries)

4. Decoupling Schema & Catalogue n Problem: share data but some additional private schema & catalogue –Production data (read-only) is shared between many end users. –Each of them needs private databases and schema n Solution: Multiple Cloned Federations –User federations start as clone of the production federation n complete catalog and schema is copied n files are shared read-only between master and clones –Clones could use the same FD-id and the same lockserver –users n add new schema to their clone n add new databases to their clone, n add instances of private classes to their databases

5. Decoupling Schema & Catalogue n Limitations: –Special care for propagating updates n between different users or production federation and users –databases pre-allocated to users –named schema pre-allocated to user –Many copies of the (large) federation file n Inefficient since user changes are very small –Single lockserver may become bottleneck n Could use multiple lockservers and different FD id if shared databases are not updated

6. Objects using new Schema User1FD User1 Boot PrivateSchema User2FD U2DB1 U2DB2 User2 Boot U1DB1 U1DB2 Decoupling Schema & Catalogue Prod ProdFD DB1DB2DB3DBn Clone FD Prod Boot

7. Federation Copies n Objy tools to be used are oocopyfd or ooinstallfd –not oonewfd + ooschemaupgrade + ooattachdb –ooattach reads each single object in a database n currently a scalability problem for large databases n not needed if databases are attached to their original database id –bug in ooattachdb currently breaks e.g. oobackup & oorestore n Use ooattachdb only if really needed –e.g. if db ids have changed –otherwise consider oochangedb -catalogonly –oochangedb can not create a new catalogue entry,

8. Federation Copies n oocopyfd –insures that original FD is in a consistent state –copies all files to another location (may need running ooams on destination host) n ooinstallfd –requires the user to transfer all files –requires the user to make sure that files are in a consistent state n Partial Copies –Just copy only part of the database files n make sure they don’t contain references that will be dangling in the destination FD –ooinstallfd -nocheck will complain, but setup the catalogue for all files found –one may later “detach” other files from the catalogue using oodeletdb - catalogonly

9. Federation Backup n Our backups are typically a partial copies! –Often mostly meta data backups since data is kept often read- only on tape n experiment registry n schema & catalogue –oobackup/oorestore don’t support partial backups n they would bring in all data from tape n no way to declare data to be read-only n Making sure that the backup is consistent –partial federation copy to temporary area (e.g. flip-flop) –release locks on original –install backup federation in temporary area –check new backup for “consistency” n e.g. using ootidy or oodump >/dev/null

10. Making sure that a FD is in a consistent state n Goal: copy all database files –in the same transactional state –as one atomic operation n No updates during the copy process –require no update locks on the copied databases (read locks are ok) –copy process should acquire a read lock for each copied database to make sure that no update will be allowed during the copy n alternatively: use list of locks from oolockmon –if a update lock is found: try again n need a reliable way to wait for db/fd locks n Perl example script available –HepODBMS/examples/admin

11. Summary n Multiple Federation are used in production as workaround for Objectivity shortcomings –significant management needed by the db-admin n Implementing a consistent FD copy is often application dependent and not trivial –should we still try to merge the different scripts that have been developed?  Be careful! –Keep a consistent copy of all files before starting to play with ooattach & Co –Keep a backup for all data you would not like to lose!