January 2013 CDMI: An Introduction

Big Data Complexity Volume Speed “Big Data” refers to datasets whose size is beyond the ability of typical tools to capture, store, manage and analyze 2

Trends Shaping the Big Data Market  Scale –10’s PB of data, millions of users –“Life time retention” of data  Object Access –Object = Data + Metadata –APIs/Protocols to provide object access  Automatic Data Management –Object granular automated data mobility and management  Distributed Access –Multi-location repositories, data mobility, local view/access 3

 Commonly used term to describe a data store deployment  Apt to describe a Big Data store, increasingly characterized by: –Object based access and APIs –Multi-protocol support (NAS: NFS/CIFS/Object, SAN: FC/iSCSI) –Automatic policy-driven management based on object metadata –Provides services like billing, audit, security –Globally federated clouds  Formal term: Data Storage as a Service (DaaS) Cloud Storage

55 NetApp Confidential - Internal Use Only What is CDMI?  Cloud Data Management Interface  CDMI is a open standard for cloud storage –Defines client-to-cloud and cloud-to-cloud interactions –Defines object semantics for content stored in the cloud

6 NetApp Confidential - Internal Use Only What does CDMI let you do?  CDMI allows clients to store and retrieve from the cloud –Named Objects (Accessed by name, similar to files) –Nameless Objects (Accessed by ID) –Queues (first-in-first-out structured storage)  CDMI allows clients to organize stored content –Containers (Groups of named objects, similar to directories)  CDMI allows clients to manage stored content –User Metadata (Structured metadata attached to objects) –Data System Metadata (Allows clients to request storage characteristics) –Versions of objects  CDMI allows clients to discover content –Query and Notifications

7 NetApp Confidential - Internal Use Only Why a standard?  End customers want it: –Customers want cross-cloud interoperability –Customers want to be able to avoid vendor lock-in, switch cloud vendors and second-source cloud solutions  Application developers want it: –Developers don’t want to develop custom code to talk with each vendor’s cloud –Developers want to leverage common libraries, documentation, test tools, expertise, etc.  Cloud storage vendors want it: –Tired of inventing yet another new API that needs to be documented, tested and maintained –Don’t want their API to be controlled and lead by vendors  Cloud service providers don’t want it (yet): –They like API lock-in, and feel they can adopt CDMI later.

88 NetApp Confidential - Internal Use Only Who created CDMI? And many more…  A Technical Working Group in SNIA comprising several major vendors in the Storage and Networking industry  Each of the major contributors have provided at least 20 pages of spec text

9 NetApp Confidential - Internal Use Only What does CDMI look like?  CDMI is a wire protocol, like CIFS and NFS, that provides access to objects  CDMI is based on RESTful HTTP, and is designed to be a very simple protocol  CDMI uses JSON, the Web 2.0 data format, and can be directly accessed by any web browser  Basic CDMI functionality includes Create, Read, Update and Delete of objects  Advanced CDMI functionality includes notifications, query, snapshots, serialization, and compliance

10 NetApp Confidential - Internal Use Only What does CDMI look like?  CDMI GET /mycontainer/photos/eqd.jpg HTTP/1.1 Host: cdmi.example.com Date: Mon, 26 Mar :37: Authorization: Basic cm9vdDpyb291

11 NetApp Confidential - Internal Use Only Opportunities for Partners!  Develop CDMI clients to make legacy applications CDMI aware  Develop CDMI protocol conformance and performance testing tools  Develop tools to migrate data from non-CDMI compliant clouds to CDMI compliant clouds  Develop utilities/services for cloud provides using CDMI data system metadata  Design applications using object metadata

12 NetApp Confidential - Internal Use Only How to learn more?  CDMI specification: –  Participate in CDMI PlugFests: –  Use the opportunities to develop