Storage Workshop Summary Wahid Bhimji University Of Edinburgh On behalf all of the participants…

What was it? Monday / Tuesday this week. Focused on Tier 2 Grid Storage in the UK. Good attendance from UK storage community: – All our SRM technologies dCache/DPM/ Storm – Expertise in the demanding customers - CMS/ ATLAS – but not forgetting the others

Stated Aims Data at LHC – Tier 2 storage facing significant challenges. Workshop - take a step back and: 1.Identify significant issues from site experiences. 2. Provide training and spread good practice for current SRM/Storage technologies. 3.Consider other/emerging technologies: – Do these actually address the challenges – If so – what do we need to do to test/develop them.

Overview Sessions on: Site Experiences – bottlenecks SRMs – experiences/tutorials/plans/ comparisons Hardware Filesystems and File Access The Future Some highlights here – see upcoming write-up for full info…

Site Experiences Sites (big and small) running well Dealing with all kinds of operational issues Issues: Upgrade schedule – can gridpp manage process, to getting features and stability Thin(ner) SRM layers -­ do all use cases need the full SRM spec -­ longer term discussion

SRM – Storage Resource Manager Variety of SRM implementations (3 in UK Tier2s) All increasingly mature – no losers dropping out Do have different strengths

SRMs dCache: Getting easier to use Well liked by current sites Certain key valuable features such as replication DPM: Most used in UK ( and world) Support from CERN improved Development of some features desirable (replication, draining, checking) we can push /help them StoRM Thin SRM - exploits filesystem Now mature and more widely used Questions over support and feature implementation Needs increasing UK expertise

SRMs Converging (in some ways)– eg NFS4, file:// But is there a point in making DPM more like dCache (in features) or Storm (in Lustre support) …

Hardware SSDs: Ongoing Studies at Glasgow Performance for sure but is it needed if LHC data structures and access improve Networking: 10 Gig for those who can afford it… channel bonding for others

Procurement Discussions Advantages of teaming together with others (inc university or T2 or T1) for hardware purchases to get bulk rates. Even sharing purchase information (where allowed) is useful.

Filesystems – File Access File access: Changes in LHC file access have big impacts on site storage efficiencies, tuning etc. Need to continue studies and keep in close contact with developments

Filesystems Other advanced filesystems - GPFS, HDFS can offer features that can be exploited for HEP storage As can models of analysis such as Hadoop Map-Reduce

The Future Brainstorming session covered both short term needs and longer term strategy Compile to proper output -> workplan but, to give some examples…

Discussions Local file access User space expectations Upgrade schedule getting the features and stability Super lightweight SRMs Current solutions made more easily deployable and configurable Support: Recipes gridpp certified? Coordinated testing of implementations Regular Workshops! Emerging Issues / Questions Dream storage solutions Actions for GRIPP STORAGE

Conclusions Valuable workshop – In and outside sessions Indications of challenges ahead – And work to be done

Conclusions People say storage is hard – So we are lucky to have the GridPP Storage Community – We need to maintain fora for it.

Some Issues Local file access – random file patterns cause load Priorities for Evaluation – Hardware / SRM-Filesystems Which SRM is best for a Tier 2- not just a big=dCache small= DPM issue. Do I buy SSDs? 10 Gig E? More reliable disks? User space expectations- I have 2 TB in my pocket why do you give me 1 on the grid Should GridPP package/certify its own releases – ensure they are stable + get early access to features Do we need (to use) a lighter way to access some things on a grid – at Tier 3s etc. Can we get more value through shared procurements