PROOFS OF RETRIEVABILITY VIA HARDNESS AMPLIFICATION Yevgeniy Dodis, Salil Vadhan and Daniel Wichs

Remote Data Storage Average Computer User: Bob Remote Storage Server: Lots of data (music, photos, s, forms…) Lots of devices (desktop, laptop, music player, phone, camera…) Accessibility: Wants ability to access all data at all time from all devices. Reliability: Should never loose data. Provides greater accessibility and reliability. (for a cheap price)

Does all of my data still exist? Is my data private? Is it authentic? Bob : Remote Storage Server: Remote Data Storage Encrypt and MAC data before storing it remotely

Proofs of Retrievability Introduced by [Juels, Kaliski 07]. An audit protocol between Bob and the server in which Bob checks that his data still retrievable. Formalized using the extraction paradigm (as in proofs of knowledge). If the audit fails then Bob may not get his data back but… Its better to know about a problem than to be ignorant. Bobs insistence on running audits may help enforce honest behavior. Naïve Protocol: To run an audit, Bob downloads all his data and verifies signature. Too costly! Bob does not actually need the data at the time of an audit. Goal: An audit protocol that has: Low communication complexity. Locality (server only accesses few locations of the data). Low storage overhead for Bob and server.

Proofs of Retrievability (PoR) Introduced by [Juels, Kaliski 07]. An audit protocol between Bob and the server in which Bob checks that his data still retrievable. Formalized using the extraction paradigm (as in proofs of knowledge). Naïve Protocol: To run an audit, Bob downloads all his data and verifies signature. Too costly! Bob does not actually need the data at the time of an audit. Goal: An audit protocol that has: Low communication complexity. Locality (server only accesses few locations of the data).

Direct-Product Scheme (One Audit) Bob: Bobs file F Server file S Error Correcting Code Remote Storage Server: Store t random blocks S [ r 1 ],…, S [ r t ]. r1r1 r2r2 rtrt Enrollment

Direct-Product Scheme (One Audit) Server file S Remote Storage Server: r1r1 r2r2 r3r3 Bob: e = r 1,…, r t S [ r 1 ],…, S [ r t ] Verify that received blocks are correct. If one block is lost, unlikely to get caught! Note: Locality requires redundancy. Store t random blocks S [ r 1 ],…, S [ r t ]. Audit

Direct-Product Scheme (One Audit) Server file S Remote Storage Server: r1r1 r2r2 r3r3 Bob: e = r 1,…, r t S [ r 1 ],…, S [ r t ] Verify that received blocks are correct. Store t random blocks S [ r 1 ],…, S [ r t ]. Audit

Direct-Product Scheme (One Audit) Intuition for security: If the server knows enough blocks of the server file S, then can decode F. If the sever knows too few blocks of S, then it cannot pass an audit. Unfortunately, intuition does not translate into a proof since the server does not gives us blocks of S. Question 1: Is this scheme secure in general? Question 2: Is the tradeoff between server storage overhead, communication, and locality optimal? Know Server file S Know Dont know Know Dont know Know Dont know

Direct-Product Scheme (One Audit) Arbitrary Adversarial Server: Intuition for security: If the server knows enough blocks of the server file S, then can decode F. If the sever knows too few blocks of S, then it cannot pass an audit. Unfortunately, intuition does not translate into a proof since the server does not gives us blocks of S. Question 1: Is this scheme secure in general? How do we extract the file? Question 2: Is the tradeoff between server storage, communication, and locality optimal? e = ( r 1,…, r t ) C*(e)C*(e) Answers ² fraction of challenges correctly with C *( e )= ( S [ r 1 ],…, S [ r t ])

Prior Work The direct-product scheme was introduced by [Naor, Rothblum 05] in the context of sublinear authenticators. PoR schemes were studied by [Juels, Kaliski 07], [Ateniese et al. 07], [Shacham, Waters 08]. Question 1: Is the direct-product scheme secure? Yes if… [JK07]: Make simplifying assumptions on behavior of the adversary. [JK07,SW08]: Add MACs to authenticate the responses. Good: gives us many-time scheme + proof of security. Bad: increased server storage overhead (and computation/communication). Question 2: Is the tradeoff between server storage overhead, communication, and locality optimal? An optimization to direct-product scheme appears as part of an optimized MAC/Sig based scheme of [SW08]. Nearly optimal parameters required Random Oracles.

Direct-Product Protocol (One Audit) Server file S Remote Storage Server: Bob: e = r 1,…, r t C ( e ) = S [ r 1 ],…, S [ r t ] Verify that received blocks are correct. Store t random blocks S [ r 1 ],…, S [ r t ].Store key k for a MAC. Tags S[r]S[r] ¾ [ r ] = mac k ( S [ r ]) ¾ [ r 1 ],…, ¾ [ r t ]

Prior Work The direct-product scheme was introduced by [Naor, Rothblum 05] in the context of sublinear authenticators. PoR schemes were studied by [Juels, Kaliski 07], [Ateniese et al. 07], [Shacham, Waters 08]. Question 1: Is the direct-product scheme secure? Yes if… [JK07]: Make simplifying assumptions on behavior of the adversary. [JK07,SW08]: Add MACs to authenticate the responses. Good: gives us many-time scheme + proof of security. Bad: increased server storage overhead (and computation/communication). Question 2: Is the tradeoff between server storage overhead, communication, and locality optimal? No, e.g. Optimizations to communication complexity appear in [SW08] but utilized Random Oracles to get nearly optimal parameters. Remove R.O. ? Further improvements?

Our Results Introduce new primitive called PoR codes. Abstract key component of PoR into a clean coding-theoretic problem. Three ways to turn PoR codes into PoR schemes with various tradeoffs. 1. Security of PoR, efficient (list) decoding algorithms for such codes. 2. Efficiency of PoR, optimizing various parameters of PoR codes. Construct nearly optimal PoR codes (and therefore PoR schemes). Along the way, answer questions 1, 2. Answer 1: The direct-product scheme is secure. First storage efficient PoR scheme (optimization of [JK07]) with full proof of security. First information-theoretically secure PoR. Answer 2: Further optimize all previous schemes. In particular, remove Random Oracles from [SW08]. Key Step: Connect (list) decoding of PoR codes to seemingly unrelated area of hardness amplification.

Our abstraction: PoR Codes Bobs file F Server file S 2 ¦ n PoR Codeword C 2 § N … Coordinate C [ e ] corresponds to servers response on challenge e. In particular C can be exponential as it is never stored explicitly. Locality: C [ e ] can be computed from only a few positions in S. Ignores how Bob decides whether responses are correct/incorrect. e Storage Server: Bob: Direct Product PoR ECC All t -tuples e C[e]C[e] SF

Decoding PoR Codes – Attempt 1 Efficiently decode F given oracle access to C * that is ² -close to C (where ² ¸ nelgigible) But we cannot uniquely decode from an error- rate ² ¸ ½. Remote Storage Server: … C*(e)C*(e) e Decoder

Decoding PoR Codes (Attempt) Remote Storage Server: Given oracle access to C * that is ² -close to C, decode F. But we cannot uniquely decode when ² · ½. … Incorrect codeword C * C*(e)C*(e) e Decoder

Decoding PoR Codes: Two variants Remote Storage Server: Error List Decoding: Given oracle access to C * that is ² -close to C, produce a (short) list containing F Corresponds to basic scheme. Erasure Decoding: given oracle access to C * that is ² -close to C and C *[ e ] 2 { C [ e ], ? }, recover F Corresponds to MAC based scheme. Efficiency: Run-time poly(| F |, 1/ ² ). … Incorrect codeword C * C*(e)C*(e) e Decoder

PoR Schemes from PoR codes Sheme 1: Bob stores (challenge, response) pairs locally. Good: Information Theoretic security. Optimal server storage. Bad: Bounded Use. Large client storage. Scheme 2: Offload storage to the server (encrypt/MAC). Good: Optimal client storage. Small additive overhead to server storage. Bad: Bounded use. Scheme 3: Authenticate each block of server file. Good: Unbounded use. Optimal client storage. Bad: Server storage roughly doubles. Basic ideas of Schemes 1,2,3 come from [NR05], [JK07],[SW08]. Efficiency of all schemes improved with optimized PoR codes. Security of schemes 1& 2 requires error list-decoding which has not been known before (optimized or not).

PoR Schemes from PoR codes # of auditsSecurityServer Storage Client Storage CommunicationDecoding Scheme 1Bounded s-time Information theoretic OptimalBad OptimalError Scheme 2Bounded s-time Comp. Good for small s Optimal Error Scheme 3UnboundedComp.Bad Optimal Erasure Basic ideas of Schemes 1,2,3 comes from [NR05], [JK07],[SW08]. Security of schemes 1& 2 requires error decoding which has not been known before. Efficiency of all schemes improved with optimized PoR codes.

List decoding direct-product codes Bobs file F Server file S … ECC All t -tuples Given oracle access to C * which is ² -close to C, output a small list containing F. Hardness Amplification (direct-product theorems) If S ( r ) is ± -hard then the direct-product function C ( e ) = ( S ( r 1 ),…, S ( r t )) e = ( r 1,…, r t ) is ² -hard, where ² ¿ ±. PoR Codeword C

List decoding direct-product codes Hardness Amplification (direct-product theorems) 9 adversary computing C ( e ) = ( S ( r 1 ),…, S ( r t )) on an ² -fraction of tuples ) 9 adversary that computes S ( r ) on a ± -fraction of inputs. Bobs file F Server file S … ECC Given oracle access to C * which is ² -close to C, output a small list containing F. PoR Codeword C All t -tuples

List decoding direct-product codes … ECC Hardness Amplification (uniform direct product theorems) [Trev05], [IJK06], [IJKW08] Bobs file F Server file S Given oracle access to C * which is ² -close to C, output a small list containing F. Given oracle access to an adversary that computes C ( e ) = ( S ( r 1 ),…, S ( r t )) on an ² -fraction of tuples, construct a short list of adversaries one of which computes S ( r ) on a ± -fraction of inputs. PoR Codeword C All t -tuples

List decoding direct-product codes … ECC Bobs file F Server file S Step 1: C * ) short list containing S * which is ± -close to S. Step 2: Short list containing S * ) short list containing F. Hardness Amplification (uniform direct product theorems) [Trev05], [IJK06], [IJKW08] Given oracle access to an adversary that computes C ( e ) = ( S ( r 1 ),…, S ( r t )) on an ² -fraction of tuples, construct a short list of adversaries one of which computes S ( r ) on a ± -fraction of inputs. PoR Codeword C All t -tuples

Parameters of Direct-Product Codes Tradeoff between locality and server storage is optimal. Easy to show that challenge/response size must be O( ¸ ). Does the challenge/response size need to depend on t ? Parameters Security param ¸. Server Storage = ° | F |. Any ° ¸ 1. Locality t = O( ¸ /( ° -1)) Chall. Size = t log( n ) Resp. Size = t log(| ¦ |) … ECC Bobs file F Server file S 2 ¦ n PoR codeword C 2 ( ¦ t ) N e = ( r 1,…, r t ) All t -tuples

U = S [ r 1 ],…, S [ r t ] Two optimizations Shorter Responses: Instead of sending response U = ( S [ r 1 ],…, S [ r t ]), ask server to send a random position in an error-correcting encoding of U. [SW08]: Implicitly use Hadamard which increases challenge. Can be replaced by Reed-Solomon. Making this optimization work with MAC based scheme was major contribution of [SW08]. Shorter Challenges: Use a randomness efficient hitter to sample indices ( r 1,…, r t ) with a shorter challenge. Works for erasure decoding. Removes Random Oracles from [SW08]. Open for efficient error decoding. (works for inefficient decoding) Storage Server: Bob: S e U = S [ r 1 ],…, S [ r t ] ECC( U )[ p ] =( r 1,…, r t ),p,p e

Conclusions Introduce PoR codes. Give nearly optimal constructions. Proves security of storage-efficient PoR schemes. First information-theoretic scheme. Remove the use of Random Oracles from [SW08]. Open questions: Can we show efficient list-decoding for optimized PoR codes with a hitter? Do unbounded use schemes require poor server storage overhead?

Two optimizations Shorter Responses: Instead of sending response U = ( S [ r 1 ],…, S [ r t ]), ask server to send a random position in an error-correcting encoding of U. In [SW08], challenge is indices ( r 1,…, r t ) and coefficients ( a 1,…, a t ). Response is a i S [r i ]. This is a random position in the Hadamard encoding of U. Increases challenge size. In [SW08], R.O. was used to reduce challenge. Notice: Can use Reed-Solomon where challenge is a single coefficient a and response is a i S [r i ]. Erasure/Error decoding is not much more complicated. Shorter Challenges: Use a randomness efficient hitter to sample indices ( r 1,…, r t ) with a shorter challenge. Unfortunately, cannot show that efficient error decoding works. Would require special derandomized version of [IJKW08]. Inefficient decoding may be enough. Alternatively, use R.O. to sample challenge.

PoR Schemes from PoR Codes. ( I ) Bob stores s random (challenge, response) pairs ( e 1, C [ e 1 ]),…,( e s, C [ e s ]) locally and a short almost-universal hash of his file: ( h, ½ = h ( F )). Server only stores the server file S. To run audit i, Bob sends challenge e i and verifies that response is C [ e i ]. Server can pass next audit: ) answers an ² ¸ 2 - ( ¸ ) fraction of challenges correctly ) Error-decoding recovers a short list containing F. ) Unique value in list that hashes to ½ must be F. Information theoretic security. Low server storage. Large client storage.

PoR Schemes from PoR Codes. ( II ) Essentially same as before but Bob chooses challenges e i pseudo-randomly and encrypts/authenticates responses C [ e i ]. Server stores the server-file S and the s authenticated-ciphertexts. Bob only stores short keys to later re-compute challenges e i and decrypt/verify ciphertexts. To run audit i, Bob sends challenge e i and server responds with C [ e i ] and the i th authenticated ciphertext. Bob verifies that these match. Argument for security is essentially same as before. Requires error- decoding of PoR codes. Low client storage, Low server storage for small s.

PoR Schemes from PoR Codes. ( III ) Bob remembers a keys k for a MAC.

PoR Schemes from PoR Codes. ( II ) Bob remembers two keys k 1, k 2, for a PRF f and the hash ( h, ½ = h ( F )). Bob computes s challenges e i = f ( k 1, i ) and responses C [ e i ]. Also, Bob computes the tags ¾ i = f ( k 2, C [ e i ]). Server stores the server file S and tags ¾ 1,…, ¾ s To run audit i, Bob sends challenge e i = f ( k 1, i ). Server sends response C [ e i ] and tag ¾ i. Bob verifies tag. Server that passes next audit must answer an ² ¸ 2 - ( ¸ ) fraction of challenges correctly. Security based on error-decoding. Low client storage. Increased server storage.

Computational PoR Schemes w/ Low Server Storage (Bounded Use)

Unbounded Use Computational Scheme

List decoding direct-product codes … ECC Direct-products Connection between Hardness Amplification and approximate list decoding of the direct-product code was originally noted by Trevisan [Tre05]. An efficient uniform direct product theorem appeared recently in [IJK06], [IJKW08]. Bobs file F Server file S Direct-Product Codeword C Hardness Amplification (uniform direct product theorems)

Optimizing PoR codes. … ECC 1 Direct-products Bobs file F Server file S 2 ¦ n Direct-Product C 2 ( ¦ t ) N e1e1 In [SW08], challenge is ( r 1,…, r t ) and coefficients ( a 1,…, a t ). Response is a i S [r i ]. Response is O( ¸ ) but challenge is O( ¸ 2 ). In [SW08], R.O. was used to reduce challenge. Notice: this is a random position in the Hadamard encoding of S [r i ]. Nothing special about Hadamard. Can use any code, e.g. Reed-Solomon. Oracle that is ² -correct on Q yields one which is ² -correct on C where ² = poly( ² ). … Challenge-Response Q 2 ( § ) NM Challenge: e = ( e 1, e 2 ) Response: ECC 2 ( C [ e 1 ])[ e 2 ] ECC 2 For Reed-Solomon, response is: a i S [r i ].

Optimizing PoR codes. … ECC Direct-products Bobs file F Server file S 2 ¦ n Direct-Product C 2 ( ¦ t ) N e1e1 To reduce the challenge size, use a randomness efficient hitter to select positions in S. Hit( e 1 ) = r 1,…, r t. Hits every large enough subset of S with large probability. The size of the challenge goes down to O( ¸ ). Unfortunately, we do not know how to error decode the direct-product code efficiently when a hitter is used. Can still use it for erasures. … Challenge-Response Q 2 ( § ) NM Positions: e = ( e 1, e 2 ) Value: ECC( C [ e 1 ])[ e 2 ]