1. HAIL (High-Availability and Integrity Layer) for Cloud Storage
Alina Oprea
Joint with Kevin Bowers and Ari Juels
RSA Laboratories

2. Cloud storage Cloud Storage Provider Client Mostly static data:
Storage server
Web server
Mostly static data:
Back-up
Archival
Is my data available ?
Client

3. Proofs of Retrievability (PORs)
Cloud Storage Provider
Corrects small corruption F
Encoding
Client k

4. Proofs of Retrievability (PORs)
Cloud Storage Provider F F
Challenge
Response
Requires integrity checks on server or client
Detects large corruption
Client k

5. When PORs fail F F Unrecoverable Cloud Storage Provider decoder
Challenge
Response
Unrecoverable
Client k

6. HAIL Goals
Resilience against cloud provider failure or temporary unavailability
Amazon S3 went down several times, once for 8 hours
Linkup lost 45% of its customer data
Use multiple cloud providers to construct a reliable cloud storage service out of unreliable components
RAID (Reliable Array of Inexpensive Disks) for cloud storage
Provide clients verification capabilities
Efficient proofs of file availability by interacting with cloud providers

7. Replicate across multiple providers
Amazon S3
Google
EMC Atmos F F F
Naïve approach F
Sample and check consistency across providers
Client

8. Roadmap Adversarial model for HAIL
Small-corruption attack on replication scheme
Encoding layer for each replica individually
Reduce storage overhead by dispersal
Increasing file lifetime with secret keys

9. Adversarial model
Static: corrupts a fixed number b of the n total providers over time
Create enough redundancy in the file to handle this (b+1 replicas)
Is this realistic?
Mobile (proactive): corrupts b out of n providers in each epoch
Separate each server into code base and storage base
At the beginning of an epoch code base of all servers is cleaned (through reboot, for instance)
All servers might have residual data corruption
Reactive design: check integrity and redistribute

10. Attack on replication scheme
Amazon S3
Google
EMC Atmos F F F F F F
File can not be recovered after [n/b] epochs
The probability that client samples the corrupted block is low
Client

11. Replication with POR F F F F Amazon S3 Google EMC Atmos Client POR POR
ECC
Cons: requires integrity checks for each replica
Client

12. Replication with POR F F F F Amazon S3 Google EMC Atmos Client
Sample and check consistency across providers
Client

13. Replication with POR F F F F Amazon S3 Google EMC Atmos Clientєd єd єd
Large storage overhead due to replication
File lifetime still limited by [n/b] (єc/ єd)
єc correction threshold of POR encoding
єd detection threshold of POR
Sample and check consistency across providers
Client

14. Reduce storage overheadF
decode
m fragments
n fragments
dispersal
(n,m) F
Client

15. Dispersal code parity blocks
(n,m) F F
Dispersal code parity blocks
Client

16. Dispersal code parity blocks
Stripe
POR encoding F
Dispersal code parity blocks
How to increase file lifetime?
Check that stripe is a codeword in dispersal code
POR encoding to correct small corruption
Client

17. Increasing file lifetime with MACsP1 P2 P3 P4 P5
MAC
MAC
MAC
MAC
MAC
Can we reduce storage overhead?
Client

18. Integrity-protected dispersal codem
hk1(m)
UHF
hk2(m) +
PRF
Reed-Solomon dispersal code
Client

19. Integrity-protected dispersal codem +
PRF
MACs embedded into parity symbols
Client

20. Current work and open problems
Proofs of Retrievability
Lower bounds akin to Naor and Rothblum’s lower bounds for memory checking
What is the cost of file updates?
HAIL
K. Bowers, A. Juels and A. Oprea – “HAIL (High-Availability and Integrity Layer) for Cloud Storage”, CCS 2009
Different adversarial models
Investigate alternative constructions
Supporting file updates

21. Proofs of Retrievability (PORs)
Cloud Storage Provider F F A
Challenge
Response
Requires integrity checks on server or client
Detects large corruption
Client k

22. POR requirements F F Cloud Service Provider Client
Efficient file encoding
Low storage overhead
Low bandwidth for challenge and response
Efficient proof construction and verification
Efficient file recoverability
[Juels, Kaliski 07]
[Shacham, Waters ]
[Dodis et al. 09]
The requirements in designing a POR protocol are:
That the file encoding is efficient
There is low storage overhead on both client and server
The ch/res protocol is efficient in both computation and bandwidth
It is also efficient to recover the file from a fraction of correct server responses.
There are a number of POR constructions: JK 07, SW08, Dodis et al. 09 that offer different tradeoffs in these metrics. I will not get into details here, I just wanted to point out that the POR problem has been studied a lot and there are good solutions for it.
But do PORs solve our problem?
Client k

23. Reed-Solomon parity blocks
HAIL P1 P2 P3 P4 P5 F
Reed-Solomon parity blocks
POR encoding protects against small corruption
Protects static files availability against mobile adversary
Client

24. HAIL P1 P2 P3 P4 P5
Aggregates stripes for efficient integrity checking
Periodic checking and reconstruction upon failure
MACs embedded into parity symbols
Client