1 CNAP 22nd March 2004 Summary of Atlas Petabyte Data Store User Group Meeting March 4 th 2004

2 Summary of recent developments LHC, PP community and hardware upgrade, and media migration (Tim Folkes) SRB interface (Bonny Strong) SE interface for GRIDPP (Jens Jensen) Belt and braces: Improved environmental monitoring disaster recovery: New off site back-up service. OAIS, the RLG and trusted digital repositories (David Giaretta)

3 9940B connections Switch_1Switch_2 RS6000 fsc0fsc1 fsc0 9940B fsc1fsc0fsc1fsc rmt1 rmt4rmt3rmt2 rmt5-8 AAAAAAAA STK 9310 “Powder Horn” Gbit network 1.2TB

4 SRB Example: CMS Largest project using CCLRC SRB services at present is the CERN CMS experiment. SRB chosen for Pre-Challenge Production in 2003, producing data for Data Challenge ADS driver for SRB was developed to meet CMS immediate needs. SRB server installed for CMS which interfaces to ADS.

5 Future Plans for SRB to ADS The SRB driver developed for CMS will be expanded for use by other projects. ADS will run an SRB server for integration into any SRB domain. Will translate the SRB user name and/or domain name into an ADS owner name. Will use the pathtape server to map SRB collection names to ADS 6-character tape names.

6 APS Recent New Users & Potential New Users Recent New Users National Crystallography Service, Southampton University (~2TB/yr?) WASP (30TB/yr?) VIRGO Consortium (3TB/yr?) Potential New users Integrative Biology (15TB/yr?) Diamond? (1-3PB/yr?) BBSRC (BITS)? 10-20TB/yr?) Arts and Humanities Data Service? (2TB/yr)

7

8 Questionnaire responses 62% from CCLRC; 38% external 75% currently using ADS;25% not currently using or not users. Average years of use7.4 Max years of use20.0 Min years of use0.8 SD years of use6.6 Some role descriptions of those responding: “Sys admin”, “Data Analysis and data provision”, “Experiment coordinator”, “Archiver”, ”User”, “Project Data Storage Manager”, “Responsible for project back-ups”, “Project Manager”.

9 Questionnaire – Motivation and assessment “Convenient”, “Easy”, “Reliable”, “Support available”, “Secure”, “Long term back up”, “Large volume” “No need to get involved with tape storage”; “No perceived alternative” Mean Score (out of 10)8.2 Min5.0 Max10.0 SD1.8

10 Questionnaire – Web page usage Web page usage% Never21 Rarely14 Occasional57 Often 7

11 Questionnaire – Communication & Awareness Preferences for improved methods of communication % For% Against% Maybe Need for list server Need for user group meeting User awareness of recent developments Awareness ofAware (%)Not aware (%) Hardware upgrade7921 SE interface2971 SRB interface5050

12 Improvements or changes required to the service (1) Backup service available on wide platform i.e Windows PC etc Require SRM interface Need to store data sets with long names (I.e. > 6 chars) - and better than pathtape look-up is required Native support for full path names (ie. not having to use the pathtape service). Tiny tape names Use more for known downtimes etc Ability to store large files (> 2Gb)

13 Improvements or changes required to the service (2) More online storage / caching (depending on future requirements) Web / Grid interface User-queryable database of usage statistics, e.g. to find out my top-100 datasets, or to see how many times this year / month / etc a particular item has been accessed. Having this as a database that I can query using JDBC from my own management applications would be even better than static reports. Metadata lookups: it would be useful to check the file size directly from flfsys

14 Improvements or changes required to the service (3) Transparent file access (HSM) so that we could forget about (virtual) tapes Fix the problem between Solaris and the ADS software regarding multiple files on ADS datasets; Provide a backup and archive interface for NT servers. Really good tape changer driver mapped into Windows server (More support required) Quicker access to off line tapes to improve speed of restores. More documentation. More user-friendly commands for such things as rules Price control.

15 Ranked User issues questionUser specified IssueMean response (A-K) 3Need to store data sets with long names (I.e. > 6 chars) - and better than pathtape look-up is required 7.9 4Native support for full path names (ie. not having to use the pathtape service).Tiny tape names 7.7 6Ability to store large files (> 2Gb)7.3 18Price control.6.5 8Web / Grid interface6.4 5Use more for known downtimes etc6.2 16More documentation.6.1 7More online storage / caching (depending on future requirements)5.6 17More user-friendly commands for such things as rules5.6 1Backup service available on wide platform i.e Windows PC etc5.4 15Quicker access to off line tapes to improve speed of restores.5.3 9User-queryable database of usage statistics,5.0

16 Conclusions (1) Responses have been received mainly from technical, hands-on users with a good balance from both within CCLRC and from external users. The majority of responses have been received from people who are currently using the Data store. Most have many years of experience of using the Data Store. The responses received represent approximately 20% of the active users. (Total number of active[1] users = 84)[1] Given 1,2 and 3 above, the responses received are from a knowledgeable section of experienced users both internal and external to CCLRC, who comprise a representative proportion of all current active users. On this basis the responses can be believed and should be used reliably.

17 Conclusions (2) Most users understand the advantages of the ADS. I.e. they know what they want. Overall, most users get what they want from the service (8.2/10). We now have a measure from which to improve. Some of the improvements identified by the users have already or are now being addressed. Of those that are not, further clarification is required in order to understand how important the issue is to other users, and to clarify the problem adequately to consider appropriate solutions. What mechanisms could be used to achieve this? Most users were aware of the recent hardware upgrade, although a surprisingly high proportion of users (21%) were not. Most users were unaware of the SE interface, and only half were aware of the SRB interface. This matters because there are improved services coming on line from the development team, which some users may wish to take advantage of.

18 Conclusions (3) Most users (64%) use the web page at least occasionally, whereas 35% use it rarely or never. Communication between users and development team needs to be improved. Given that most users make at least occasional use of the web pages, the most simple and effective means of doing so is to keep the web site up-to-date with current developments. However, this will not be successful for around one third of users. Almost 80% of users are in favour of a list serv. Service. The combination of this with an improved web site should be adequate. Almost 60% of users are in favour of User group meetings. These should be continued, probably yearly.

19 Backups

20 Digital Curation Centre (DCC) Joint collaboration between CCLRC, UKOLN, and Edinburgh and Glasgow Universities. Provide advice, support, research and development into aspects of Digital Curation for the UK HE community Funded jointly by JISC and EPSRC - £1m/year for three years initially. Feb Establish collaboration with industrial partners…

21

/9940 Drive connections (old) STK 9310 ~6000 slots 3590 RS G216G108G 100Mbit Network 9940

23 Real drive performance Upgrade