1. An Introduction to Randomness Extractors Ronen Shaltiel University of Haifa Daddy, how do computers get random bits?

2. Randomized algorithms and protocols Randomized algorithms/protocols: Receive stream of independent unbiased coin tosses. Necessary for Crypto. deterministic algorithm outputinput Provably help in distributed settings. Randomized algorithms are often simpler and more efficient than known deterministic ones. (even though we conjecture that BPP=P). Randomized

3. “weak source of randomness” Randomized algorithm Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix) These distributions are “ somewhat random ” but not “ truly random ”. Paradigm: randomness extractors Input: one sample from arbitrary “weak source of randomness”. Output: independent coin tosses. How do computers obtain random coin tosses? Randomness Extractor Extensively studied area, dates back to von-Neumann in 1951. outputinput Coins may be biased and correlated

4. How do computers obtain random coin tosses? “weak source of randomness” Randomized algorithm Randomness Extractor outputinput Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix) These distributions are “ somewhat random ” but not “ truly random ”. Paradigm: randomness extractors Input: one sample from arbitrary “weak source of randomness”. Output: independent coin tosses. Extensively studied area, dates back to von-Neumann in 1951.

5. Extractors have applications in: Randomized complexity theory. Cryptography. Network design. Ramsey theory. Coding theory. Combinatorics. Algorithm design. Data structures. Extractors have many applications Often not directly related to randomness! Gives additional motivation to extractors (in addition to the initial motivation of extracting randomness for randomized algs). “weak source of randomness” Randomized algorithm Randomness Extractor inputoutput

6. Several notions of extractors Deterministic extractors Restrict to specific families of “ allowed sources ”. Multiple sources extractors Extractor gets samples from several independent sources. Seeded extractors Allow extractor to get a seed of few truly random bits.

7. Deterministic extractors: Formal definition “weak source of randomness” Randomness Extractor Distribution X from C Dfn: Let C be a set of distributions over {0,1} n (family of “allowed sources”). E:{0,1} n ! {0,1} m is an extractor for C if 8 X 2 C, random variable E(X) is uniform over {0,1} m. Two distributions Y,Z over the same domain are ² -close if 8 event A, |Pr[Y 2 A]-Pr[Z 2 A]| ≤ ². Goal: Design efficiently computable extractors for interesting and general families of sources. Maximize number of extracted bits. Minimize error ². ² -close to

8. Example: von-Neumann’s sources and extractor (1951!) Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X 1,..,X n ) s.t. X 1,..,X n i.i.d. p ≤ Pr[X i =1] ≤ 1-p. vN extractor E(x) (extracts one bit): on input x 2 {0,1} n Scan input bits from left to right. If you see pair “01” stop and output “0”. If you see pair “10” stop and output “1”. Observation: Pr[“01”] = Pr[“10”] (implies correctness). Subsequent work on extracting many bits [Elias72,Peres92]. X has entropy ≥ pn.

9. Impossibility of extraction from Santha-Vazirani sources Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X 1,..,X n ) s.t. X 1,..,X n i.i.d. p ≤ Pr[X i =1] ≤ 1-p. An SV-source is a distribution X=(X 1,..,X n ) s.t. Source bits can be correlated. Every next bit is somewhat unpredictable. More formally, 8 i, 8 x 1,..,x i-1 2 {0,1}, p ≤ Pr[X i =1|X 1 =x 1,..,X i-1 =x i-1 ] ≤ 1-p. Thm: [SanthaVazirani86] No extractors for such sources. Historically => research on other notions of extractors. X has entropy ≥ pn.

10. Bit-fixing sources [ChorGoldreichFriedmanHastadRudichSmolensky85] Let k be a parameter. A k-bit-fixing source is a distribution X=(X 1,..,X n ) s.t. k bits are uniformly distributed. remaining n-k bits are fixed to arbitrary values. Easy to extract one bit: E(X 1,..,X n )=parity(X 1,..,X n ) Thm: [CGFHRS] Impossible to extract 2 bits with zero error for k<n/3. Probably not a good example for “extraction story”. Naturally arise in cryptographic scenarios. x1x1 x2x2 x3x3 xnxn k random bits

11. (Non-interactive) Privacy amplification Alice and Bob share a uniformly chosen key Z 2 {0,1} n. Can use random key to encrypt communication on public channel. Eve somehow learns n-k bits of key. Alice and Bob don’t know which bits. Eve’s view: Z is a k-bit fixing-source. Eve’s view: E(Z) is (close to) uniform. E(Z) is a new and secure key. Motivates extractors: Extract many bits (hopefully k bits). Explicit (poly-time computable). Alice Bob Eavesdropper public channel Z 2 R {0,1} n Use bit-fixing source extractor. E(Z) k random bits From my point of view Z is distributed like: Extract m=(1-o(1))k bits [CGFHRS85] [CohenWigderson89] [KampZuckerman07] [GabizonRazShaltiel06] [Rao09]

12. Affine sources Let F be a finite field (typically F 2 ={0,1}). An affine source is a distribution that is uniform over some affine subspace with dimension k of F n. Affine sources generalize bit-fixing sources. Extractor is E:F n ! {0,1} is in particular “anti-linear”: non-constant on any affine subspace of dimension k. (In extractor jargon, this is called a “disperser”). Exist for k=O(log n) by probabilistic method. Explicit constructions: (poly-time computable) Extractor : k=o(n) [Bourgain07]. Disperser: k=n o(1) (“anti-linear function”) [Shaltiel11].

13. Feasibly samplable sources [Blum86,TrevisanVadhan00]. Sources defined by considering an allowed “sampling process”. Source distribution = Sampler(uniform bits). Restrictions on complexity of sampler induces family of sources. Small space, Small circuits, Constant depth circuits… [TV00,KampRaoVadhanZuckerman06, KonigMaurer05,Shaltiel06,Viola11,DeWatson11]. Orthogonal notion of “Feasibly recognizable sources” suggested in [Shaltiel09]. Source uniform on {x:P -1 (x)=1} for some procedure P. Restrictions on complexity of procedures induce family.

14. Several notions of extractors Deterministic extractors Restrict to specific families of “ allowed sources ”. Multiple sources extractors Extractor gets samples from several independent sources. Seeded extractors Allow extractor to get a seed of few truly random bits.

15. Multiple sources extractors No deterministic extractors for SV-sources. Possible if you get samples from two independent sources! Can allow a more general family than SV-sources. C={distributions X with “high entropy”}. Best we can hope for. X nn Y 2-source extractor

16. Dfn: (min-entropy) X has min-entropy ≥ k if ∀ x: Pr[X=x] ≤ 2 -k “Can hope to extract k random bits from X”. Seen examples of sources with min-entropy ≥ k. vN-sources. SV-sources. Bit-fixing sources. Affine sources. Another example: flat distributions: X uniformly distributed on a subset of size 2 k of {0,1} n. subset  flat distribution Measuring the entropy of the source distribution “weak source of randomness” Distribution X over n bits size 2 k {0,1} n A more stringent variant of Shannon entropy X

17. Formal definition of Multiple sources extractors Definition: (emerged from [SanthaVazirani86]) A (k,ε)-2-source-extractor is a function E(x,y) s.t. for every two independent dist. X,Y over n bit strings each having min-entropy ≥ k, E(X,Y) is ε-close to uniform. Realistic model for generating random bits. Unfortunately, we don’t have good explicit constructions. X nn Y 2-source extractor Can be generalized to t>2 sources.

18. Explicit 2-source extractors imply explicit Ramsey graphs 2-source extractor E(x,y) that outputs one bit is a matrix (w.l.o.g. symmetric) Property: Every X x Y rectangle of size 2 k is balanced.  Every X x X rectangle of size 2 k is not monochromatic.  Adjac. matrix of a 2 k -Ramsey graph: Graph with no 2 k -clique or 2 k -independent set. Explicitly constructing r-Ramsey graphs for small r is a longstanding open problem. 001000010 010011101 101110010 011000010 010100110 101000100 010101101 101011010 010100100 2n2n X Y x y X

19. Explicit constructions of 2-source extractors and Ramsey graphs 2 k -Ramsey graphs on 2 n nodes Erdős 47: Probabilstic method achieves k≈log n Frankl and Wilson 81: Explicit construction k≈(n log n) ½ [BKSSW05,BRSW06]: Explicit construction k=n o(1) (Extractor techniques). Construct bipartite Ramsey graphs (stronger than Ramsey graphs but weaker than 2-source extractors). (k,ε)-2-source extractors Probabilstic method achieves k≈log n Chor and Goldreich 88: E(x,y)= mod 2 works for k ≥ n/2. Bourgain 05: Explicit construction k=0.4999n. Progress on t-source extractors [BIW04,BKSSW05,Rao06]. Rao06: extract from log n/log k sources with min-ent k.

20. Several notions of extractors Deterministic extractors Restrict to specific families of “ allowed sources ”. Multiple sources extractors Extractor gets samples from several independent sources. Seeded extractors Allow extractor to get a seed of few truly random bits.

21. We allow an extractor to also receive an additional seed of (few) independent random bits. Makes sense as long as: # bits extracted > seed length. Handle all high min-entropy sources! Seeded extractors [NisanZuckerman92] source dist. X on n bits Extractor seed Y random output Randomness Definition: A (k,ε)-extractor is a function E(x,y) s.t. for every dist. X with min-entropy ≥ k, E(X,Y) is ε-close to uniform. Lower bounds [RadhakrishnanTaShma98] : seed length ≥ log(n-k) + 2log(1/ε) Probabilistic method: Exists optimal extractor which matches lower bound and extracts all the k random bits in the source distribution. Explicit constructions: E(x,y) can be computed in poly-time. uniformly distributed Current milestones in explicit constructions: [LuReingoldVadhanWigderson03, GuruswamiUmansVadhan07,DvirWigderson08,DvirKoppartySarafSudan09]. “Optimal up to constants”: seed = O(log(n) + log(1/ε)) output  (k) bits. For constant error: seed = O(log(n)) output (1-o(1))∙k bits.

22. Simulating randomized algorithms using weak random sources Goal: Run rand algorithm with a weak random source of randomness. Where can we get a seed? Idea: Go over all seeds. Given sample X from source. ∀ y compute z y = E(X,y) Compute Alg(input,z y ) Answer majority vote. seed=O(log n)=>poly-time. Explicit constructions. Unsuitable for crypto protocols. Randomized algorithm input output random coins Randomness Extractor seed source dist. X on n bits

23. Something about the tools used in explicit constructions 2-wise independent hash functions [ImpagliazzoLevinLuby89,NisanZuckerman92]. E(x,h)=h(x),h where h is chosen from small family of 2-wise independent hash functions. Disadvantage: huge seed. List decodable error correcting codes [Trevisan99]. E(x,y)=Enc(x) y,y where Enc is a binary list decodable error correcting code. (also works vice-versa). Rate ≥ 1/poly(n) => logarithmic seed. Disadvantage: extract only one additional bit. Can try and exploit properties of specific codes [TaShmaZuckeramanSafra01,ShaltielUmans01,GuruswamiUmansVadhan07]. Various composition methods […]

24. long seed extractor Composing short seed extractor with long output extractor x1x1 x2x2 x3x3 xnxn k bits of min-entropy short seed extractor Short random output Seeded Extractors are only guaranteed to work when the source and seed are independent. correlated! !long random output Nevertheless, many constructions make this “go through” by modifying initial extractors to have additional properties.

25. Seeded extractors as graphs with “volume expansion”. Extractor is a bipartite graph. Given extractor E(x,y) N=2 n (# of inputs) M=2 m (# of outputs) K=2 k (# of source elements) D=2 d (# of seeds) Connect x to E(x,1),..,E(x,D). Small seed length d ~ log n => small deg D ~ log N. D=2 d edges x N≈{0,1} n M≈{0,1} m E(x,1) E(x,D)..

26. Extractor graphs: volume expansion property Extractor property: ∀ dist X of min-entropy≥k, E(X,Y) ε-close to uniform. => “ expansion ” property: ∀ set X of size K=2 k, |Γ)X)| ≥ (1-ε)M. Such graph/function is called “Disperser”. X N≈{0,1} n M≈{0,1} m K=2 k Γ(X) (1-ε)M

27. Extractors and Expander graphs X N≈{0,1} n M≈{0,1} m Γ(X) (1-ε)M Extractor N≈{0,1} n X Γ(X) D=2 d edges (1+δ)-Expander (1+δ)K K N≈{0,1} n K=2 k

28. Volume expansion: K -> (1-ε)M K/N -> (1-ε) Extractors and Expander graphs X N≈{0,1} n M≈{0,1} m Γ(X) (1-ε)M Extractor N≈{0,1} n X Γ(X) (1+δ)-Expander (1+δ)K N≈{0,1} n Size expansion: K -> (1+δ)K K K=2 k Extractors produce better results in some applications of expanders

29. Expanders with expansion that beat the eigenvalue bound [WigdersonZuckerman93] Goal: Construct low deg expanders with huge expansion. Line up two low degree extractors. ∀ set X of size K, (where K<<N) |Γ)X)| ≥ (1-ε)M > M/2. ∀ sets X,X’ of size K X and X’ have common neighbour. Contract middle layer. Bipartite graph in which every set of size K sees N-K vertices. Trivially degree ≥ (N-K)/K ≈ N/K. Obtain low degree ND 2 /K. Eigenvalue methods cannot yield graphs with such parameters. N≈{0,1} n X X’X’

30. v1v1 v2v2 v3v3 vDvD Randomness efficient (oblivious) sampling using expanders [AjtaiKormlosSzemeredi87] Random walk variables v 1..v D behave like i.i.d: ∀A of size ½M Hitting property: Pr[∀i : v i ∊A] ≤ δ = 2 -Ω(D). Chernoff style property: Pr[#i : v i ∊A far from exp.] ≤ δ = 2 -Ω(D). # of random bits used for walk: m+O(D)=m+O(log(1/δ)) # of random bits for i.i.d. m∙D=m ∙ O(log(1/δ)) M≈{0,1} m Random walk on constant degree expander

31. Randomness efficient (oblivious) sampling using extractors [Sipser86,Zuckerman96] Given parameters m, δ: Use E with k=m, n=m+log(1/δ) ε<½ and small seed d. Choose random x: m+log(1/δ) random bits. Set v i =E(x,i) Expansion property ⇒ Hitting prop. ∀A of size ½M Call x bad if ∀i: E(x,i) inside A. # of bad x’s < K=2 k Pr[x is bad] < 2 k /2 n = δ D edges x N≈{0,1} n M≈{0,1} m E(x,1) E(x,D).. bad x ’ s (1-ε)M A

32. Every (oblivious) sampling scheme yields an extractor An (oblivious) sampling scheme uses a random n bit string x to generated D random variables. Thm: [Zuckerman06] if the scheme has sampling property then the derived graph is an extractor. Extractors  oblvs Sampling D=2 d edges x N≈{0,1} n M≈{0,1} m E(x) 1 E(x) D..

33. Extractors come in several flavors and have many applications in diverse fields. Goal: Explicitly construct extractors with parameters that match existential bounds. Many open problems. See article in proceedings for more details. Conclusion “weak source of randomness” Randomized algorithm Randomness Extractor inputoutput

34. Thank You… Daddy, can you tell me that story again?