1. Section 6 Object Storage Gateway (RADOS-GW)

2. Objectives Understand the Ceph Object Storage Gateway (RADOS- GW)
Installing the Object Storage Gateway with ceph-deploy

3. Understand Ceph Object Gateway
Objective Notes:

4. Common Object Storage APIs
OpenStack Swift
Native object storage API for OpenStack
“The OpenStack Object Store project, known as Swift, offers cloud storage software so that you can store and retrieve lots of data with a simple API. It's built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift is ideal for storing unstructured data that can grow without bound.” - wiki.openstack.org
Amazon S3
Commercial Storage solution with well defined API
“Amazon Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable cloud storage. Amazon S3 is easy to use object storage, with a simple web service interface to store and retrieve any amount of data from anywhere on the web” - aws.amazon.com

5. Object Storage - Review
All data in an object storage system is stored as objects
Objects have a unique identifier, data and metadata
Access to the object store is via an API, not as a filesystem
Ceph provides a native object store API called ‘librados’
Applications written specifically for Ceph must use librados to access the object store
Other OpenSource projects and commercial products also offer object storage with different APIs
OpenStack Swift
Amazon Web Services ‘Simple Storage Service’ (S3)

6. Connecting to 3rd Party Object APIs
In addition to the native librados Ceph API, Ceph supports both Swift and S3
The RADOSGW gateway provides access to the underlying RADOS object store by providing a proxy service for S3 and Swift clients
S3 and Swift requests are translated to native librados API requests by the RADOSGW gateway
Any application written for either S3 or Swift can then function with a Ceph based storage system such as SUSE Enterprise Server
Ceph’s Object Storage uses the term object to describe the data it stores. S3 and Swift objects are not the same as the objects that Ceph writes to the Ceph Storage Cluster. Ceph Object Storage objects are mapped to Ceph Storage Cluster objects. The S3 and Swift objects do not necessarily correspond in a 1:1 manner with an object stored in the storage cluster. It is possible for an S3 or Swift object to map to multiple Ceph objects.

7. RADOSGW
The Ceph Object Storage daemon, radosgw, is a FastCGI service that provides a RESTful HTTP API to store objects and metadata.
It layers on top of the Ceph Storage Cluster with its own data formats, and maintains its own user database, authentication, and access control.
Gateway specific information is stored in pools created for the gateway service
The RADOS Gateway uses a unified namespace, which means you can use either the OpenStack Swift-compatible API or the Amazon S3-compatible API.
For example, you can write data using the S3-compatible API with one application and then read data using the Swift- compatible API with another application.

8. Swift, S3 and RADOSGW Swift Client S3 Client REST RADOS GW
(single namespace)
librados
Socket
RADOS Cluster

10. Gateway Details Uses CivetWeb as the web server Lightweight and fast
Shared namespace between S3 and Swift
Write data to an S3 Bucket or a Swift Container
Read back via either S3 or Swift

11. Gateway Created Pools .rgw -Domain root
.rgw.root -Region/Zone information for the RGW
.rgw.gc -Garbage collection pool. Stores information on objects marked to be deleted
.users -Access keys for S3 users
.users.uid -User names
.rgw.buckets -Buckets
.rgw.buckets.index -An index on buckets and objects in it
.users.swift -Swift subuser info
.log Bucket creation info
.intent-log -Stores a copy of the object before an update in case a undo is required.
.usage -Per user usage log

12. Swift Protocol Support

13. S3 Protocol Support

14. Testing the gateway RGW listens on the port 7480
If successful, the following XML should be printed
# curl
<ListAllMyBucketsResult xmlns="
<Owner>
<ID>anonymous</ID>
<DisplayName/>
</Owner>
<Buckets/>
</ListAllMyBucketsResult>

15. Suggested Clients for the Gateway
There are various available clients which can communicate with S3 or Swift APIs.
Useful for learning about the behaviour of Ceph Object Gateway
Provide a way to access data on S3/Swift for testing, migration etc
S3 Cmd
Python S3 tool
Available for SLES 12
OpenSource
Download from s3tools.org
Cyberduck
Commercial version available – not for Linux at present
Fully featured client

16. Installing Ceph Object Gateway

17. Command: ceph-deploy rgw
Syntax: ceph-deploy rgw OPTIONS node:gatewayname
Option Description
create -Create a new Ceph Object Gateway
delete -Delete an existing gateway
list Lists existing Ceph Object Gateways
The --overwrite-conf statement is often required when creating or deleting a Ceph Object Gateway
Notes:

18. Command: radosgw-admin
Syntax: radosgw-admin mode options
Mode Description
user create or remove users
subuser -create or remove Swift Subusers
key manage access keys
quota -manage storage quotas on the gateway
Note: radosgw-admin is a native RADOS command, it does not operate via the RESTful interface
Notes:

19. Creating Users

20. Notes on OpenStack use
Ceph Object Gateway supports Swift V1 authentication
Can create swift users directly within Ceph
More normal to authenticate Ceph against Keystone
Ceph becomes a drop in replacement for Swift
Visible in Horizon etc

21. Section 6 Exercises
Objective Notes: