Data Reduction Processes Using FPGA for MicroBooNE Liquid Argon Time Projection Chamber Jinyuan Wu (For MicroBooNE Collaboration) Fermilab May 2010

The MicroBooNE Detector MicroBooNE detector:  150 tons total Liquid Argon  89 tons active volume  TPC: ~2.5 x 2.3 x 10.4m long  Ionization electrons drift to beam right  30 PMTs peek through the wire chambers on beam right  Will use BNB and NuMI beams at FNAL for physics program RT2010, May 20102Wu Jinyuan, Fermilab,

MicroBooNE Liquid Argon Time Projection Chamber Drift Time Data from BO detector of FNAL Induction #1 Induction #2 Collection Wire Number  Passing charged particles ionize Argon.  Electric fields drift electrons to wire chamber planes.  Waveforms of all wires are digitized. RT2010, May 20103Wu Jinyuan, Fermilab,

Data Reduction on Liquid Argon TPC Data Hit waveforms in TPC carry useful information (e.g. track angle etc.). Digitizing the waveforms creates large volume of data. Data reduction without losing useful information is necessary. Drift Time Wire Number Data from BO detector of FNAL Induction #1 Induction #2 Collection RT2010, May 20104Wu Jinyuan, Fermilab,

Character of Waveform More than 99% points differ from previous points by -1, 0 or +1 in this example. Good predictability exists in waveform data. DFF Q A B A-B U(n+1) D U(n+1)-U(n) RT2010, May 20105Wu Jinyuan, Fermilab,

The Huffman Coding The U(n+1)-U(n) value with highest probability is assigned to shortest code, i.e., single bit 1. Values with lower probabilities are assigned with longer codes, e.g., 01, 001, 0001 etc. Huffman coded words and regular words are distinguished by bit-15. U(n+1)- U(n) Code -4 and others Full 16 bits word ADC value (13-bit) Regular ADC data for first point or when U(n+1)-U(n) is outside +-3 Huffman Coded Padding or Continue to Next Word In this example, 6 differences of the data samples are packed in the 16-bit data word RT2010, May 20106Wu Jinyuan, Fermilab,

Huffman Coding (A Lossless Compressing) Shorter codes (1-7 bits) are assigned to differences with higher probability ( in our case -3 to +3). Any differences outside +-3 use 16 bits. In this example, coding rate is 1.53 bits/sample. In other events, coding rate is also ~1.5 bits/sample. U(n+1)-U(n)CountProbability (P)CodeNo. of bits (N)P*N -4 and others Full 16 bits word E total RT2010, May 20107Wu Jinyuan, Fermilab,

 On typical TPC events a compression ratio of about 10 can be achieved. N N/(10.7) The Compress Ratio of Huffman Coding RT2010, May 20108Wu Jinyuan, Fermilab, Huffman Coding Huffman Coding

The Huffman Coding Block Difference of Data Points Huffman Code Lookup Table Huffman Code Composer Huffman Code or Raw Data Selector 245 Logic Cells (245/39600)*$129 = $0.80 RT2010, May 20109Wu Jinyuan, Fermilab,

The Data Paths Serial to Parallel Conversion 16MHz to 2MHz Decimation Data Merging RAM Dynamic Decimation External Memory Output Interface Huffman Coding Huffman Coding Serial to Parallel Conversion 16MHz to 2MHz Decimation Serial to Parallel Conversion 16MHz to 2MHz Decimation Serial to Parallel Conversion 16MHz to 2MHz Decimation ADC Accelerator Neutrino Events Sync to Beam 7% duty cycle Lossless Compression Supernova Data 100% duty cycle, Deeper compression is needed. Loss unavoidable RT2010, May Wu Jinyuan, Fermilab, Accelerator neutrino events are in sync with beam which is 7% duty cycle. So lossless compression with small compression ratio is fine. The supernova data path needs 100% duty cycle digitization. Compression with large ratio is needed and loss is unavoidable.

Dynamic Decimation (DD)  Only small time intervals, i.e., region of interest (ROI) must be sampled at high rate.  Most time intervals can be sampled with lower rate, without losing useful information. RT2010, May Wu Jinyuan, Fermilab, Data are not thrown away in pedestal region

Dynamic Decimation Block  The two blocks are able to operate at up to 250MHz clock.  The Dynamic Decimation in our case reduces data by a factor of 10.  The supernova data will go through two compression stages. All data Supernova Data N N/(10) RT2010, May Wu Jinyuan, Fermilab, Dynamic Decimation Huffman Coding

The Dynamic Decimation Block 217 Logic Cells (217/39600)*$129 = $0.71 Region of Interest Finder Decimation Filter Raw Data Pipe Decimation or Raw Data Selector RT2010, May Wu Jinyuan, Fermilab,

Any Differences ? Raw With Dynamic Decimation RT2010, May Wu Jinyuan, Fermilab,

Noise Sensitivity of Huffman Coding  Sampling Theorem!  采样定律！  Teorema De Amostragem!  Abtast Theorem!  Theorie d’Echautillonage!  Follow the sampling theorem strictly! RT2010, May Wu Jinyuan, Fermilab,

A “Mystery” of Huffman Coding Ratios on Down Sampled Data The 5MHz data is down sampled to 1MHz. The Huffman Coding compress ratio drops from 10.7 to 7.5 when the data is down sampled. N N/(10.7) (N/5) (N/5)/(7.5) RT2010, May Wu Jinyuan, Fermilab, Huffman Coding Huffman Coding Huffman Coding Huffman Coding

Decimation != “Throwing Data Away” Decimation: Anti-aliasing low-pass filter Down sampling The Huffman Coding compression ratio is sensitive to the aliasing noise, good filter must be applied to the data first. RT2010, May Wu Jinyuan, Fermilab, Down Sampling Down Sampling Down Sampling Down Sampling Anti-alias Low-pass Filter Anti-alias Low-pass Filter

Noise Sensitivity of Huffman Coding Ratios for 5MHz to 1MHz Decimation The Huffman Coding compress ratio improves as the filter in Dynamic Decimation improves. RT2010, May Wu Jinyuan, Fermilab, Original No Filter No Filter Poor Filter Poor Filter Good Filter Good Filter Better Filter Better Filter

Huffman Coding Ratios for Dynamic Decimation The Huffman Coding compress ratio improves as the filter in Dynamic Decimation improves. RT2010, May Wu Jinyuan, Fermilab, Original No Filter No Filter Poor Filter Poor Filter Good Filter Good Filter Better Filter Better Filter

Summary  Simple Huffman Coding scheme with fix coding table is suitable for data compression on digitized waveforms.  A compression ratio about (1/10) is achievable while further improvements can still be anticipated.  Huffman Coding is sensitive to the aliasing noise and Anti-aliasing filter should be carefully designed.  Dynamic Decimation provides another (1/10) compression with data loss.  Dynamic Decimation cascaded with Huffman Coding provides sufficient compression on continuously digitized data for supernova study. RT2010, May 2010Wu Jinyuan, Fermilab,

The End Thanks

Data Words with Huffman Coding and Dynamic Decimation 1 00DD 1 ADC value (13-bit) 0X Regular ADC data when U(n+1)-U(n) is outside +-3 Reserved Huffman Coded DD=0: 5M samples/s RT2010, May Wu Jinyuan, Fermilab, 10X Reserved 1 Huffman Coded DDADC value (13-bit) DD=1: (5/16) M samples/s