New Ways of Generating Large Realistic Benchmarks for Testing Synthesis Tools Petr Fišer, Jan Schmidt Faculty of Information Technology Czech Technical University in Prague

IWSBP 2010, Freiberg2 Outline Motivation New benchmark generation methods Experimental results Conclusions

IWSBP 2010, Freiberg3 Motivation … why another artificial benchmark generator? To test logic synthesis tools Capabilities of synthesis processes Immunity to “bad” structures Ability to discover “good” structures Iterative power Scalability …

IWSBP 2010, Freiberg4 Motivation J. Cong, K. Minkovich: Optimality study of logic synthesis for LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 small They created artificially large circuits, functionally equivalent to their small origins (70 LUTs) Synthesis produced 10k – 30k LUTs

IWSBP 2010, Freiberg5 Motivation P. Fišer, J. Schmidt, J: Small but Nasty Logic Synthesis Examples, IWSBP'08 XOR tree is appended to the circuit outputs and the circuit is collapsed Synthesis produced >400 LUTs instead of 11

IWSBP 2010, Freiberg6 Motivation Will my synthesis tool produce the same result for different descriptions (versions) of one particular circuit? (a.k.a. iterative power) Most probably not! (if things go bad) What went wrong? What descriptions are bad for me? What structures caused my failure? What should I do to perform better?

IWSBP 2010, Freiberg7 Proposed Benchmarks Starting with seed circuit (could be small) Functionally equivalent “big” circuit is created The size of the benchmark circuit is adjustable Result Transformation 1 Transformation 2 Transformation 3 Bench circuit 1 Bench circuit 2 Bench circuit 3 Seed circuit Ideal case: Synthesis

IWSBP 2010, Freiberg8 Proposed Benchmarks Starting with seed circuit (could be small) Functionally equivalent “big” circuit is created The size of the benchmark circuit is adjustable Transformation 1 Transformation 2 Transformation 3 Bench circuit 1 Bench circuit 2 Bench circuit 3 Seed circuit Real case: Synthesis Result 1 Result 2 Result 3

IWSBP 2010, Freiberg9 Cong’s LEKU Benchmarks J. Cong, K. Minkovich: Optimality study of logic synthesis for LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 LEKU = Logic Examples with Known Upper Bound Based on elimination of the original circuit structure … and bad decomposition

IWSBP 2010, Freiberg10 1. Realistic LEKU Benchmarks Any circuit may be used as a seed (instead of g25) Possible chance of success Global BDDs may be used instead of collapsing Upper bound = size of the original circuit

IWSBP 2010, Freiberg11 1. Realistic LEKU Benchmarks Size increase by collapsing 250 ISCAS and IWLS benchmarks Size increase in 61% of circuits

IWSBP 2010, Freiberg12 1. Realistic LEKU Benchmarks Experimental results Benchmark circuit Synthesis (# of 4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 c432367original c432367global BDD2,0171,0311,0231,333 c432367ABC collapse2,6581,2461,5481,648 c432367SIS collapse7,0753,3613,8724,738 c original c global BDD407,09893,190174,983N/A c ABC collapse13,7277,4378,1099,460 c SIS collapse30,01519,78720,48728,017

IWSBP 2010, Freiberg13 2. Parity Benchmark Circuits XOR tree is appended to the circuit outputs, then the structure is destroyed (collapsing, BDD) No guarantee of circuit size increase Upper bound = size of the core circuit + XOR tree

IWSBP 2010, Freiberg14 2. Parity Benchmark Circuits Size increase by appending parity & collapsing 100 ISCAS and IWLS benchmarks Size increase in 25% of circuits

IWSBP 2010, Freiberg15 2. Parity Benchmark Circuits Experimental results Benchmark circuit Synthesis (# of 4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 s original s global BDD6,2823,8494,0553,839 s ABC collapse31,83919,74121,87525,793 s SIS collapse39,63626,31328,254N/A b4331original b4331BDD16,9636,3476,0994,285 b4331ABC collapse1, b4331SIS collapse4,0872,0362,4221,627

IWSBP 2010, Freiberg16 3. Tautology Benchmarks Large random SOP is generated When the number of terms exceeds some threshold, the SOP is a tautology Then, the big SOP is mapped into 2-input gates (SIS tech_decomp)   Big network Upper bound = 0 The benchmark size may be adjusted by Number of input variables Dimension of SOP terms

IWSBP 2010, Freiberg17 4. Partial Collapsing Only parts of the network are collapsed Choose one pivot gate Extract its transitive fan-in and fan-out to a given radius Collapse the extracted network part Decompose into 2-input gates Put it back Iterate several times Upper bound = size of the original circuit The benchmark size may be adjusted by Size of the extracted circuit Number of iterations

IWSBP 2010, Freiberg18 4. Partial Collapsing Example – c432

IWSBP 2010, Freiberg19 4. Partial Collapsing Example – big tautology

IWSBP 2010, Freiberg20 4. Partial Collapsing Example – big tautology

IWSBP 2010, Freiberg21 4. Partial Collapsing Experimental results Benchmark circuit Synthesis (4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 c432367original c432367Part. coll., size 981, c432367Part. coll., size 1093,0771,4451,6992,422 c432367Part. coll., size 1385,0262,5982,7613,727 c432367Part. coll., size 14011,5316,6476,8449,255 c original c Part. coll., size 1291, c Part. coll., size 1715, ,3943,769 c Part. coll., size 20110, ,0107,887

IWSBP 2010, Freiberg22 5. Replicating Shared Logic Duplicate a part of the logic that is shared Find a branching signal Duplicate its transitive fan-in, to a given depth Upper bound = size of the original circuit The benchmark size may be adjusted by Number of duplicated branches Depth of duplication

IWSBP 2010, Freiberg23 5. Replicating Shared Logic Experimental results Benchmark circuitSynthesis (4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 c432367original c k dup., depth 11, c k dup., depth 24, c k dup., depth 38, c k dup., depth 411, c k dup., depth 516,

IWSBP 2010, Freiberg24 6. Adding Inverters (special bonus – not included in the proceedings) Add pairs of inverters to random locations The network size may be arbitrarily expanded And all the synthesis tools… Are completely immune to this!

IWSBP 2010, Freiberg25 Summary Experiments

IWSBP 2010, Freiberg26 Summary Experiments

IWSBP 2010, Freiberg27 Conclusions Several new benchmark generation methods proposed Artificially “big” circuits are generated from seed circuits Benchmarks are functionally equivalent to the seed circuits  the complexity upper bound is known Tested on ABC and 2 commercial tools Unfortunate result – the bigger the circuit going to synthesis, the bigger the result