1. Open Object Storage as the Foundation for Open Sync and Share in Science
Simon Traill –
© SwiftStack All Rights Reserved

2. Characteristics: File Sync and Share
Deals with unstructured data
Highly distributed over multiple geographic regions
Highly concurrent access
Requires infrastructure resilience
Requires high durability
Ever Growing!
© SwiftStack All Rights Reserved

3. Characteristics: Object Storage
Designed for unstructured data
Consistency Model enables multi datacenter deployments
Scales Linearly
Routes around hardware failure
Data is highly durable
Start small and expand later
© SwiftStack All Rights Reserved

4. Anatomy of a SwiftStack Cluster
Applications
SwiftStack Controller (and Agents)
Deployment and Provisioning over ZeroMQ based fabric
Monitoring & Utilization
Metrics and Reports
Services exposed via REST API
Site 1
Site 2
Site 3
Standard servers, drives, & networking
Proxy Services
File Services
NFS SMB
Swift API
SwiftStack Controller
Account, Container and Object Services
Consistency Processes
SwiftStack Agents
S3 API
© SwiftStack All Rights Reserved

5. Anatomy of a SwiftStack Cluster
Applications
Access via Swift and S3 REST APIs
PUT to upload
GET to download
POST to modify
DELETE to remove
Site 1
Site 2
Site 3
Standard servers, drives, & networking
Proxy Services
File Services
NFS SMB
Swift API
SwiftStack Controller
Account, Container and Object Services
Consistency Processes
SwiftStack Agents
S3 API
© SwiftStack All Rights Reserved

6. Anatomy of a SwiftStack Cluster
Applications
Proxy Services
Serve client requests via Swift / S3
Each object stored as many replicas:
Site 1
Site 2
Site 3
Standard servers, drives, & networking
Proxy Services
File Services
NFS SMB
Swift API
SwiftStack Controller
Account, Container and Object Services
Consistency Processes
SwiftStack Agents
S3 API
OBJECT NODES
HTTP
PROXY
© SwiftStack All Rights Reserved

7. Anatomy of a SwiftStack Cluster
Applications
Site 1
Site 2
Site 3
Standard servers, drives, & networking
Proxy Services
File Services
NFS SMB
Swift API
SwiftStack Controller
Account, Container and Object Services
Consistency Processes
SwiftStack Agents
S3 API
ACO Services
Store data on disk.
All data in Swift is namespaced into an account, container and object.
© SwiftStack All Rights Reserved

8. Anatomy of a SwiftStack Cluster
Applications
Site 1
Site 2
Site 3
Standard servers, drives, & networking
Proxy Services
File Services
NFS SMB
Swift API
SwiftStack Controller
Account, Container and Object Services
Consistency Processes
SwiftStack Agents
S3 API
Consistency Processes
Constantly audit and replicate data to where it is needed
© SwiftStack All Rights Reserved

9. Anatomy of a SwiftStack Cluster
Applications
Commodity Hardware
Standard x86 servers run Swift processes
Data is initially placed as uniquely as possible across disks, servers, racks and sites
Consistency processes replicate in the background where needed
SwiftStack Controller
File Services
NFS SMB
Swift API
S3 API
Proxy Services
Account, Container and Object Services
Consistency Processes
SwiftStack Agents
Standard servers, drives, & networking
Site 1
Site 2
Site 3
© SwiftStack All Rights Reserved

10. File Sync and Share
© SwiftStack All Rights Reserved

11. FS&S Integration Points
Scale capacity as adoption rises (and preferably not before…)
Provision space for users easily
Ideally:
Have dual access to data – via both FS&S and Swift / S3 aware applications
© SwiftStack All Rights Reserved

12. Scaling
© SwiftStack All Rights Reserved

13. Scaling Hardware with Swift
Scale by the box, or scale by the rack.
Use SSDs for account and container storage.
Think carefully about your load balancing strategy
File Services
NFS SMB
Proxy Services
Account, Container and Object Services
SwiftStack Reference Architectures:
© SwiftStack All Rights Reserved

14. Scale a box at a time: PACO
© SwiftStack All Rights Reserved

15. Real World Example: Scale By the Box
Load Balancer
HA (10G)
SwiftStack
Controller
LEGEND
Outward-facing (10G)
Cluster-facing (10G)
Core Switch
Core Switch
Aggregation Switch
Aggregation Switch
Rack | Zone 1
Rack 2 | Zone 2
Rack Zone 3
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
© SwiftStack All Rights Reserved

16. Real World Example: Scale By the Box
Load Balancer
HA (10G)
SwiftStack
Controller
LEGEND
Outward-facing (10G)
Cluster-facing (10G)
Core Switch
Core Switch
Aggregation Switch
Aggregation Switch
Rack | Zone 1
Rack 2 | Zone 2
Rack Zone 3
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
© SwiftStack All Rights Reserved

17. Real World Example: Scale By the Box
Load Balancer
HA (10G)
SwiftStack
Controller
LEGEND
Outward-facing (10G)
Cluster-facing (10G)
Core Switch
Core Switch
Aggregation Switch
Aggregation Switch
Rack | Zone 1
Rack 2 | Zone 2
Rack Zone 3
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
© SwiftStack All Rights Reserved

18. Real World Example: Scale By the Box
Load Balancer
HA (10G)
SwiftStack
Controller
LEGEND
Outward-facing (10G)
Cluster-facing (10G)
Core Switch
Core Switch
Aggregation Switch
Aggregation Switch
Rack | Zone 1
Rack 2 | Zone 2
Rack Zone 3
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
Proxy | Account | Container | Object
© SwiftStack All Rights Reserved

19. Scale by a rack at a time: P/ACO*
* Or PACO.
© SwiftStack All Rights Reserved

20. Real World Example: Scale By the Rack
Load Balancer
HA (10G)
Firewall
SwiftStack
Controller
LEGEND
Outward-facing (10G)
Cluster-facing (10G)
Controller network (1G)
Out-of-band (1G)
Core Switch
Core Switch
Aggregation Switch
Aggregation Switch
Rack | Zone 1
Rack 2 | Zone 2
Rack Zone 3
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
Proxy
Proxy
Proxy
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Mgmt Switch (1G)
Mgmt Switch (1G)
Mgmt Switch (1G)
© SwiftStack All Rights Reserved

21. Real World Example: Scale By the Rack
Load Balancer
HA (10G)
Firewall
SwiftStack
Controller
LEGEND
Outward-facing (10G)
Cluster-facing (10G)
Controller network (1G)
Out-of-band (1G)
Core Switch
Core Switch
Aggregation Switch
Aggregation Switch
Rack | Zone 1
Rack 2 | Zone 2
Rack Zone 3
Rack Zone 4
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
ToR Switch (10G)
Proxy
Proxy
Proxy
Proxy
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Account | Container | Object
Mgmt Switch (1G)
Mgmt Switch (1G)
Mgmt Switch (1G)
Mgmt Switch (1G)
© SwiftStack All Rights Reserved

22. Provisioning
© SwiftStack All Rights Reserved

23. Provisioning Strategy
SwiftStack, and every FS&S solution spoken about earlier, can authenticate users using LDAP, including Active Directory
Combine Swift account auto provisioning with FS&S quotas for the path of least resistance
Not all FS&S products optimise their storage strategy for Swift.
Swift performs best when objects are reasonably well dispersed between accounts and containers:
Dual access – where FS&S file share the same names as objects in Swift - is easier to accommodate, without added need for ACLS
Examine your chosen FS&S provider and choose provisioning strategies that enable the above where possible.
© SwiftStack All Rights Reserved

24. Example: Provisioning With OwnCloud
OwnCloud utilises two types of logical storage.
One Primary storage area:
Uses single container in swift
Renames Objects
>> Will not scale; no dual access model
Many External Storage areas:
Can be tied to many accounts / containers
Do not rename objects
Quotas operate on user level.
Best practices then:
Provision user folders using External Storage
Apply quotas on OwnCloud user level
SwiftStack and OwnCloud integration guide:
© SwiftStack All Rights Reserved

25. Example: Provisioning With OwnCloud
3. OwnCloud: provision user’s external storage with OCC CLI:
# cat << ‘EOF’ > /tmp/straill.json [
    {
        "mount_point": “\//home/straill",
        "storage": "\\OC\\Files\\Storage\\Swift",
        "authentication_type": "openstack::openstack",
        "configuration": {
            "service_name": "Swift Object Storage (
            "region": "default",
            "bucket": "straill_home",
            "timeout": "",
            "user": "owncloud",
            "password": "owncloud",
            "tenant": "AUTH_straill",
            "url": "
        },
        "options": {
            "encrypt": true,
            "previews": true,
            "enable_sharing": false,
            "filesystem_check_changes": 1
        "applicable_users": [ "straill" ],
        "applicable_groups": []
    } ]
EOF
# occ files_external:import /tmp/straill.json
# curl –X PUT –H ‘x-auth-token: XXXX’
1. Add user to “owncloud_users” LDAP group in your directory.
2. Swift: Create a container for straill_home:
3. OwnCloud: Set user quota via user provisioning REST API:
# curl -X PUT -d key="quota" -d value="100MB"
© SwiftStack All Rights Reserved

26. Example: Provisioning With Seafile
Seafile professional supports Swift and S3 backends.
Uses three containers in one account at most to store objects, named with UUIDS.
Won’t scale out of the box, and no dual access, but…
...is suitable for use with SwiftStack autosharder middleware to remediate scaling concerns.
Quotas operate on user level, as with OwnCloud.
Best practices (currently):
Enable Swift autosharding for Seafile account
Allow users to handle their own libraries or provision via Seafile API
© SwiftStack All Rights Reserved

27. Example: Provisioning With Seafile
1. Add user to “seafile_users” LDAP group in your directory.
2. Enable autosharding for user account via SwiftStack API.
3. Provision user home directory via Seafile API, as user in question:
# curl -v -d "name=home_straill&desc=home folder" \
-H 'Authorization: Token xxxxxxxx'
-H 'Accept: application/json; indent=4' \
4. Set user quota, as admin, via Seafile API:
# curl -v -X PUT -d "storage= ”\
-H "Authorization: Token yyyyyyyy” \
-H 'Accept: application/json; indent=4' \
© SwiftStack All Rights Reserved

28. Summary OpenStack Swift and FS&S are largely complimentary
Open source or not, look for deployment and provisioning strategies that maximize Swift’s performance
SwiftStack and OwnCloud Integration:
Seafile with Openstack Swift backend: Deploying SwiftStack Object Storage for Storage Made Easy:
© SwiftStack All Rights Reserved

29. Thank you. Simon Traill – straill@swiftstack.com
© SwiftStack All Rights Reserved