1. KUKDM’2011, Zakopane Semantic Based Storage QoS Management Methodology Renata Słota, Darin Nikolow, Jacek Kitowski Institute of Computer Science AGH-UST, Krakow, Poland ACK Cyfronet AGH, Krakow, Poland Research supported by MNiSW grant nr N N516 405535

2. KUKDM’2011, Zakopane Outline Introduction Common mass storage system model and ontology Storage system performance monitoring and estimation Use cases Summary

3. KUKDM’2011, Zakopane Introduction There are applications (called data intensive applications), which: – use data storage systems intensively, – Have constantly growing demands concerning capacity and storage system efficiency. Example of applications dealing with huge amounts of data – Scientific applications (simulations, out-of-core computations, HEP experiments), – Backup & restore, archiving, disaster recovery. The performance of the data intensive applications depend on the performance of the underlying storage system Applications running in distributed environments need some QoS (Quality of Service) concerning data access

4. KUKDM’2011, Zakopane Introduction (cont.) Storage QoS metrics include – data access latency – data access bandwidth – storage space – data availability Diversity of storage systems Hierarchical Storage Management (HSM) systems Disk arrays Tiered storage Problem of efficient storage performance utilization while respecting the storage QoS constrains Storage performance monitoring and data transfer scheduling are necessary

5. KUKDM’2011, Zakopane Goal of research The subject of this research is the development of semantic-based storage management methodology allowing to achieve QoS for storage performance metrics. As part of this research the following has been done: Development of common mass storage system model ( C2SM ) and relevant ontology Development of storage performance monitoring sensors and estimators Implementation of two use cases demonstarting our approach The research has been done within the OntoStor project http://www.icsr.agh.edu.pl/ontostor

6. KUKDM’2011, Zakopane C2SM Is a common mass storage system model, which can be used to describe the state of a storage system Consists of set of well defined storage performance related parameters Algorithms specifying the storage system behaviour Is based on the Common Information Model – CIM Is used in our methodology for unifying the description of storage performance parameters of heterogenious storage systems

7. KUKDM’2011, Zakopane C2SM class diagram

8. KUKDM’2011, Zakopane OntoStor ont o logy Developed based on C2SM Has been created semi - automatically using the ‘cim2owl’ tool, which has been developed at the DCS AGH The ontology is used in our methodology to find storage resources using semantic queries

9. KUKDM’2011, Zakopane OntoStor ontology diagram

10. KUKDM’2011, Zakopane Storage System Performance Monitoring Two components have been defined in our methodology: –sensors – obtaining storage performance parameters, Storage system dependent –estimators – estimating the future storage performance based on the data from the sensors. Simulational, rule-based, statistical Three types of systems supported – local disk, disk array, HSM system

11. Testbed

12. KUKDM’2011, Zakopane OntoStor Portal

13. KUKDM’2011, Zakopane Disk array monitoring

14. KUKDM’2011, Zakopane Monitoring of loaded disk array

15. KUKDM’2011, Zakopane Estimation test

16. KUKDM’2011, Zakopane Use cases Two use cases have been implemented using the proposed methodology Data access optimization with replication Finding the best location for newly created replica for write access Selecting the best existing replica for read access Components used: Sensors, estimators, monitoring system, C2SM SLA (Service Level Agreement) monitoring Components used: Sensors, monitoring system, OntoStor ontlogy, QoS metrics ontology, C2SM

17. KUKDM’2011, Zakopane SLA monitoring 1.QoS metric limits 2.How to obtain values of QoS metrics 3.How to monitor performance parameters 4. Description of parameters

18. KUKDM’2011, Zakopane Summary Semantic-based storage management methodology based on monitoring and estimation of storage performance has been presented Common mass storage model and its relevant ontology has been proposed A set of sensors and estimators has been implemented Two use cases have been implemented using the proposed methodology The proposed methodology has been used in FiVO/QStorMan.