Xin Li, Weikang Qian, Marc Riedel, Kia Bazargan & David Lilja A Reconfigurable Stochastic Architecture for Highly Reliable Computing Electrical & Computer Engineering University of Minnesota GLSVLSI, Boston – May 12, 2009

Opportunities & Challenges Topological constraints. Inherent structural randomness. High defect rates. Novel materials, devices, technologies: Challenges for logic synthesis: High density of bits/logic/interconnects.

Opportunities & Challenges Strategy: Cast synthesis in terms of arithmetic operations on real values. Synthesize circuits that compute logical values with probability corresponding to the real-valued inputs and outputs.

Probabilistic Signals Claude E. Shannon 1916 –2001 “A Mathematical Theory of Communication” Bell System Technical Journal, deterministic random deterministic

Probabilistic Analysis Circuit Reliability –Probabilistic fault models. –Random test pattern generation. Statistical Timing Power (circuit level). Statistical Performance Measures (architectural level).

Probabilistic Analysis “There are known knowns; and there are unknown unknowns; but today I’ll speak of the known unknowns.” – Donald Rumsfeld, 2004 Independent Known Unknown Probabilistic Inputs Probabilistic Outputs Digital Circuit

Probabilistic Analysis “There are known knowns; and there are unknown unknowns; but today I’ll speak of the known unknowns.” – Donald Rumsfeld, 2004 Probabilistic Inputs Probabilistic Outputs Digital Circuit Synthesis of Probabilistic Circuits Unknown (for us to design) SpecifiedIndependent Known Unknown

Synthesis of Probabilistic Logic Shannon and von Neumann: –“Probabilistic Logic,” –“Reliable Circuits Using Less Reliable Relays”. K. Nepal, R. Bahar, J. Mundy, W. Patterson, and A. Zaslavsky, “Designing Logic Circuits for Probabilistic Computation in the Presence of Noise.” L. Chakrapani, P. Korkmaz, B. Akgul, and K. Palem, “Probabilistic System-on-a-chip Architecture.”

Stochastic Logic Probability values are the input and output signals. combinational circuit

combinational circuit t Stochastic Logic Probability values are the input and output signals tt   tt Functions of a probability value t.

X Y X Y Z Z (independently) tZX  )1Pr()1 3.0)1  Y t t t t tt   tt Stochastic Logic

Stochastic Bit Streams A real value x in [ 0, 1 ] is encoded as a stream of bits X. For each bit, the probability that it is one is: P( X=1 ) = x. x = 2/5 0,1,0,1,0 X

Probabilistic Bundles X A real value x in [ 0, 1 ] is encoded as a stream of bits X. For each bit, the probability that it is one is: P( X=1 ) = x. x = 2/5

Stochastic Logic 5/8 3/8 4/8 3/8 4/8 8/8 Probability values are the input and output signals. combinational circuit

Stochastic Logic Probability values are the input and output signals. 1,1,0,1,0,1,1,0… 1,0,0,0,1,1,0,0,… 0,1,1,0,1,0,1,0,… 0,1,1,0,1,0,0,0,… 1,0,1,0,1,0,1,0,… 1,1,1,1,1,1,1,1,… serial bit streams combinational circuit

Stochastic Logic Probability values are the input and output signals. parallel bit streams 4/8 3/8 4/8 8/8 5/8 3/8

combinational circuit Randomness Analog interface with fractional weighting of 1’s. parallel bit streams A/D

combinational circuit Randomness Analog interface with fractional weighting of 1’s. parallel bit streams LFSR Accumulator LFSR

Nanowire Crossbar (idealized) Randomized connections, yet nearly one-to-one.

Fault Tolerance Conventional approach: binary radix encoding (7/8) (2/8) (1/8)

Fault Tolerance Bit flips can result in large error. Conventional approach: binary radix encoding (7/8) (6/8) (5/8)

Fault Tolerance … (7/8) … (2/8) … (1/8) Stochastic Logic Highly redundant. Complex operations can be performed with simple logic.

Fault Tolerance … (7/8) … (3/8) … (2/8) Stochastic Logic Highly redundant. Complex operations can be performed with simple logic. Bit flips never result in large errors.

Arithmetic Operations Multiplication(Scaled) Addition ba BPAP CPc    )()( )( ) )1( ()](1[)()( )( bsas BPSPAPSP CPc    0 1

Synthesizing Stochastic Logic combinational circuit )(tg t Only polynomials… Questions: What kinds of functions can be implemented in the probabilistic domain? How can we synthesize the logic to implement these?

Synthesizing Polynomials combinational circuit )(tg t Only polynomials… Implement polynomials using AND (multiplication) and MUX (scaled addition). Must consider polynomials with coefficients less than 0 or larger than 1…

A little math… Bernstein basis polynomial of degree n

A little math… Bernstein basis polynomial of degree n Bernstein polynomial of degree n is a Bernstein coefficient

A little math… Obtain Bernstein coefficients from power-form coefficients: Given, we have

Example: Converting a Polynomial Power-Form Polynomial Bernstein Polynomial coefficients in unit interval

Synthesizing Polynomials combinational circuit )(tg t Synthesis steps: 1. Convert the polynomial into a Bernstein form. 2. Elevate it until all coefficients are in the unit interval. 3. Implement this with “generalized multiplexing”.

Probabilistic Multiplexing ) )1( ()](1[)()( )( btat BPTPAPTP CPc    Bernstein polynomial

X 1, …, X n are independent Boolean random variables with Pr(X i =1) = t, for 1 ≤ i ≤ n Z 0, …, Z n are independent Boolean random variables with Pr(Z i =1)=, for 0 ≤ i ≤ n Probabilistic Multiplexing

A Reconfigurable Architecture Implement different functions by setting the coefficients:

Example Implement

Example Convert to

Example

with, such that is minimized. Non-Polynomial Functions Find a Bernstein polynomial to approximate the function:

Non-Polynomial Functions Example: Gamma correction function. Degree 6 Bernstein coefficients are: b 0 = , b 1 = , b 2 = , b 3 = , b 4 = , b 5 = , b 6 = f (t) = t 0.45

Deterministic v.s. Stochastic Implementation of Gamma correction function with 10% noise injection. Conventional Implementation Stochastic Implementation 1%2%10% Stochastic Implementation: no pixels with errors > 20%! Deterministic implementation: 37% pixels with errors > 20%

Comparison with Conventional Hardware Implementation of Image Processing Functions * The entire ReSC architecture, including Randomizers and De-Randomizers. ** The ReSC Unit by itself. Number of LUTs in FPGA mapping

* Software using math function from ‘Math.h’ Speedup (1024 cycles needed) ** Software using direct function table lookup Comparison with Conventional Software Implementation of Image Processing Functions

Percentage of Output Pixels with Errors Greater than 25% Noise is injected in the form of a percentage of bit flips. Comparison of Fault Tolerance for Image Processing Functions The stochastic implementation never produces such errors!

Sixth-order Maclaurin polynomial approx., 10 bits: sin(x), cos(x), tan(x), arcsin(x), arctan(x), sinh(x), cosh(x), tanh(x), arcsinh(x), exp(x), ln(x+1) Comparison of Fault Tolerance for Mathematical Functions

Conclusions The hardware cost is comparable. Stochastic computation is much more error tolerant. Advantage for applications where large errors are critical but small fluctuations can be tolerated is dramatic. (Also some pretty interesting math…) Future Directions Apply the method at the processor level. Apply the method at the circuit level (e.g., with PCMOS).

Quantities of Different Types Probability Distribution on outcomes Biological Process [computational] Synthetic Biology

Z        YX X Prwith Y X fixed Biological Process