Outline Experiments Setup and DAQ configuration Fastbus for SBS: performance and status GEn/GMn - trigger and DAQ GEp - trigger and DAQ GEM readout Outline Experiments Setup and DAQ configuration Fastbus for SBS: performance and status GEn/GMn - trigger and DAQ GEp - trigger and DAQ GEM readout SBS DAQ E. Cisbani / INFN Sanità SBS DOE Review 4-5/Nov/ JLab 4/Nov/2014 DOE SBS Review / SBS DAQ 1 Main guidelines -Reuse available equipment (Fastbus) to reduce cost -Exploit JLab CODA3 VME hardware (FADC...) -GEM readout based on APV25 + MPD Contributions from: Sergey Abrahamyan Alexandre Camsonne Mark Jones Bob Michaels Paolo Musico Igor Rachek …

Experimental Setup 4/Nov/2014 DOE SBS Review / SBS DAQ 2 Neutron form factor (GEn/GMn) Reaction : Quasifree electron scattering on 3He or 2H Trigger: Single arm electron Electron singles rate:<5 kHz Electron arm: BigBite Magnet 4 GEM chambers (FT) Gas Cerenkov (GRINCH) 1 Large GEM chamber (BT) Scintillator paddle array Preshower/Shower Calorimeter Hadron Arm: Super BigBite Magnet Coordinate Detector Hadron Calorimeter Proton form factor (GEp) Reaction : Elastic electron-proton Trigger: Elastic ep coincidence Electron singles rate:200 kHz Hadron singles rates:2 Mhz Coincidence trigger rate:5 kHz Electron arm: Coordinate Detector Electron Calorimeter Hadron arm: Super Bigbite Magnet Front GEM tracker (FT) Analyzer 5 Rear GEM tracker (BT) Analyzer 5 Rear GEM tracker (BT) Hadron Calorimeter

DAQ configuration for SBS experiments 4/Nov/2014 DOE SBS Review / SBS DAQ 3 Reuse the NIM and Fastbus equipment already available at Jlab Exploit the new JLab CODA VME hardware for the «rest»

FASTBUS for SBS experiments 4/Nov/2014 DOE SBS Review / SBS DAQ 4 Struck Fastbus Interface (SFI) is the Fastbus Master (18 available at JLab) Allows control the Fastbus modules through any VME CPU. Had slot for standard JLab Trigger Interface Module 64 channel Lecroy ADC 1881M (113 available at JLab) 9  s encoding time in 12 bit resolution and 12  s in 13 bit resolution. GEp experiment will use the fast clear feature in which the module is ready to accept another event after 1ms. 96 channel Lecroy TDC 1877s (236 available at JLab) Built-in Data Zero Suppression and Data Compaction (sparsification) Capable of multihit with an event buffer of 8 events. Encoding time 1.7  s plus 50 ns per hit per channel giving a maximum encoding time of 78  s. Fast clear settling time < 250ns Fastbus Crates holds up to 25 modules (30 available at JLab) Plenty of FASTBUS modules but: Fastbus standard transfer rate: 40 MB/s (sustainable 15 MB/s)  25% dead time at 5 kHz  Need to reduce Fastbus dead time!

Common Stop background signal window 0-32 µs sparsify Gate Triggers Readout Overhead Triggers Readout Overhead II. Event Blocking (8 events in TDC and ADC) Blocklevel=4 should work with pipelining VME Buffers the deadtime and reduces overhead I. Sparsification (built-in feature in TDC and ADC) Throws out background hits III. Event Switching 3 parallel crates, triplicate equipment, but reduces rate by 3 Status: tried, works Status: test about to start (expected to be straightforward) Sergey Abrahamyan Igor Rachek Making Fastbus Faster 4/Nov/2014 DOE SBS Review / SBS DAQ 5 4x(20+50) us 20+4x50 us

Fastbus expected performance and status 6 We can merge Fastbus with the rest of the DAQ if: All components use blocklevel = 4 All crates conform to the CODA standard. Needs to be tested ~10% deadtime at 20kHz For a simple level-1 trigger 4/Nov/2014 DOE SBS Review / SBS DAQ Two large Fastbus systems are being assembled for test in the test lab! TDCADCCrateSFI Have Need

Neutron Form Factors (GEn / GMn) 4/Nov/2014 DOE SBS Review / SBS DAQ 7 Neutron form factor (GEn/GMn) Reaction : Quasifree electron scattering on 3He or 2H Trigger: Single arm electron Electron singles rate:<5 kHz Electron arm: BigBite Magnet 4 GEM chambers (FT) Gas Cerenkov (GRINCH) 1 Large GEM chamber (BT) Scintillator paddle array Preshower/Shower Calorimeter Hadron Arm: Super BigBite Magnet Coordinate Detector Hadron Calorimeter

BigBite Electron Single arm trigger (GMn,GMn) 4/Nov/2014 DOE SBS Review / SBS DAQ 8 Use existing NIM logic for preshower/Shower coincidence

BigBite Shower Trigger 4/Nov/2014 DOE SBS Review / SBS DAQ 9  2 x   +   Shower 7 x27 Preshower 2 x27  To discriminator Bigbite trigger is OR of discriminated superblocks of Shower + Preshower 27 rows

GEn and GMn: Hadron Arm DAQ 4/Nov/2014 DOE SBS Review / SBS DAQ 10 2 VME switched Serial (VXS) Crates JLAB FADC250, 16-channel 12-bit FADC sampling at 250 MHz Capable of 300 ps time resolution in Hall D tests TS ( Trigger Supervisor) Accepts electron arm trigger Check status of all ROCs Outputs L1 accept as stop to TDCs gate for ADCs Readout signal for GEM MPDs Readout signal to HCAL TI GEM/MPD’s SSPSSP Electron Arm Trigger Optical Link L1A See more, next in the GEp DAQ

Proton Form Factor (GEp) 4/Nov/2014 DOE SBS Review / SBS DAQ 11 Proton form factor (GEp) Reaction : Elastic electron-proton Trigger: Elastic ep coincidence Electron singles rate:200 kHz Hadron singles rates:2 Mhz Coincidence trigger rate:5 kHz Electron arm: Coordinate Detector Electron Calorimeter Hadron arm: Super Bigbite Magnet Front GEM tracker (FT) Analyzer 5 Rear GEM tracker (BT) Analyzer 5 Rear GEM tracker (BT) Hadron Calorimeter

GEp: Electron Trigger 4/Nov/2014 DOE SBS Review / SBS DAQ 12 Sum analog signals to form “superblock” of 4x8 blocks Total of 204 “superblocks” go to discriminators with threshold of 80-90% of elastic maximum energy L1 trigger is OR of the 204 superblocks logic signals 204 superblock signals sent to L2 trigger processor → next slide Elastic electrons at Q 2 = 12 at the calorimeter E thr /E max (%) L1 Rate (kHz) Data Rate (Mb/s) ECAL blocks

GEp: Hadron Arm / HCAL DAQ and trigger 4/Nov/2014 DOE SBS Review / SBS DAQ 13 HCal Signals to FADC inputs Same acquisition scheme of GEn, GMn  Integrated signal and timing from FADC channels collected and sent to a General Trigger Processor (GTP) every 32 ns (over optical link)  GTP  compute all 4x4 sums of adjacent channels (HCAL clusters)  get electron Arm cluster information (204 superblocks signals)  check angular e-p correlation  If correlation send level 2 trigger to Trigger Supervisor → next slide But now HCAL is in the trigger logic:

Gep: DAQ Configuration / both arms 4/Nov/2014 DOE SBS Review / SBS DAQ 14 Trigger Supervisor Globally controls readout of all crates Receives T1 from ECal sends L1 to FASTBUS crates. The V1495 selects which group of FB crates to read out. If T2 from GTP arrives then readout of VME crates and FASTBUS ( event buffer size of 4) If no T2 then FASTBUS crates get Fast Clear  e-arm T1 triggers processing for generation of T2  T2 triggers acquisition GTP (GEM)

GEM – Readout Electronics DOE SBS Review / SBS DAQ 15 4/Nov/2014 Up to 16 APV25 cards (2048 chs) on a single MPD (parallel readout) Altera Arriga GX FPGA / RAM: DDR2 (128 MB) Optical Link interface (either ETH 1Gb/s or Aurora Gb/s protocol) 110 MHz system clock and Front panel coax clock Used HDMI-A connectors only for analog and digital signals SD-card / All spare signals go to PMC compliant connectors VME/32, VME64, VME64-VXS compliant 4 high speed line on the VXS available for data transfer ChannelsAPV25MPDs Front Tracker Rear Tracker analog ch / APV25 ASIC 3.4  s trigger latency (analog pipeline) Capable of sampling signal at 40 MHz Multiplexed analog output (100 kHz readout rate) MPD

MPD v4.0 VME interface performance All VME cycles tested including 2eSST (with STRUCK SIS-3104) –2eSST supported by new firmware release –Readout speed measured by software: 100 transfer 4MB each. Data integrity checked for each block. –Bus speed is measured directly on VME bus CYCLE DATA period [ck] / [ns] Bus / Simulated / Measured Speed [MB/s] BLT (32 bit)16 / / - / 24.3 D64-MBLT17 / / - / eVME20 / / - / eSST1606 / / 148 / 117 2eSST2674 / / 222 / 124 2eSST3203 / / 296 / 124 Speed limited by SIS3104 2Gb/s fiber connection Optical link with ETH 1Gb tested Optical link with Aurora protocol connected to SSP to be tested (no surprise expected): speed up to 390 MB/s (sustainable  200 MB/s)  sustainable 200 MB/s 4/Nov/2014 DOE SBS Review / SBS DAQ 16

Strip Occupancy: 60% Single MPD Transfer Rate: 45.2 MB/s (after sparsification) 200 kHz Rear Tracker – same occupancy and transfer rate CDR rate estimation Expected Hits Rate (Front Tracker):  500 kHz Samples/Events: 3 GEM signal width:  25 0 ns Cluster width: 2.5 strips Trigger rate: 5 kHz conservative APV25 signal output Real time data reduction needed ! Cluster Width 4/Nov/2014 DOE SBS Review / SBS DAQ 17 GEp: GEM readout performance Pure VME64 (original design)MPD + Optical Link + SSP + VME64 MPD/VME64 = 4 need 15 VME64 crates MPD/SSP=4, SSP/VME64=1 need 15 SSP, 15 VME64 crates

GEM Making Data smaller Deconvolution logic in the MPD (approx. 40% FPGA resource available) –Deconvolution algorithm to time-correlation at the level of 1/3 * 250 ns  80 ns (can even use larger number of samples)  reduce data by a factor of  3 SSP additional processing of the data –Geometrical correlation using BigBite & ep scattering & HCAL information reduce the «signal area» to  40x40 cm2 (even smaller)  keep only information of 8+8 cards/chamber;  reduce data by a factor of  3 or more –Further processing likely possible in SSP (e.g. clustering with x/y charge correlation...) 4/Nov/2014 DOE SBS Review / SBS DAQ 18 Single MPD transfer rate = 45.3 / 3 = 15 MB/s Single SSP transfer rate with 32 MPD = 15 * 32 / 3 = 161 MB/s  Two SSPs on two separate VME64 crates

DAQ: Man power 4/Nov/2014 DOE SBS Review / SBS DAQ 19 CoordinationVME-DAQFastbusHCAL-TriggerGEM-MPD A. CamsonneXXXX M. JonesX D. AdikaramX R. MichaelsX B. MoffitX B. RaydoX V. BelliniX E. CisbaniX P. MusicoX J. Campbell (SMU student) X + Support from JLab CODA and Electronics group

DAQ: Short Term plan Fastbus –Crates switching test in progress: 3 months MPD –Integration in CODA (partially done) need additional 6 weeks –Deconvolution algorithm: 4 months –Optical link protocol with SSP (proto-firmware of Aurora available but not implemented): 4 months SSP –SSP available, processing firmware development in 2015 HCAL –trigger development and testing: 6 months Small scale full setup Fastbus + HCAL trigger in /Nov/2014 DOE SBS Review / SBS DAQ 20

Summary 4/Nov/2014 DOE SBS Review / SBS DAQ 21 Neutron ExperimentsProton Experiment Single arm BigBite electron trigger at low rate: reuse existing NIM / Fastbus setup High rate T1 triggers formation of T2 on hadron arm Low rate L2 generated from: ECAL & HCAL & ep angular correlation → trigger readout Fast Clear on Fastbus (if no L2) Fastbus: Plenty of ADC, TDC and Crates Use sparsification, event buffering and crate switching to reduce dead time VME: FADC amplitude and time information for HCAL MPD + Optical Link + SSP for GEM (smart processing in MPD and SSP to reduce data by 1/10) JLab CODA3 hardware/software framework

4/Nov/2014 DOE SBS Review / SBS DAQ 22 Backup.....

GEp Electron Trigger (part 2) 4/Nov/2014 DOE SBS Review / SBS DAQ 23 Groups of 32 or 4x8  Combine four of the “groups of 8” analog sums in linear FI/FO to get summed signal for “group of 32” or 4x8 blocks  Each “group of 32” overlaps by a “group of 8” in the horizontal and vertical direction.  Total of 204 “groups of 32” summing analog signals which go to discriminators. Send to 16 channels discriminator in groups with match momentum  Set discriminator to 80-90% of elastic maximum energy  Level One trigger (T1) is OR of the 204 “group of 32” logic signals and his gate sent to TI and then to FASTBUS ADCs and TDCs  204 “group of 32” logic signals sent to FPGA to determine angular correlation with the HCAL and form the Level 2 trigger.  If no coincidence trigger then Fast Clear sent to FASTBUS crates.

Hadron Arm - HCAL DAQ: proton trigger 4/Nov/2014 DOE SBS Review / SBS DAQ 24 2 VME switched Serial (VXS) Crates JLAB FADC250, a 16-channel 12-bit FADC sampling at 250 MHz 16 FADC in VXS Crate 1 2 FADC in VXS Crate 2 If signal pass threshold Integrates signal and subtracts pedestal Sends time frame info CTP (Crate Trigger Processor) Located in VXS crate 1 Collects integrated signal and timing from FADC channels Sends data to SSP in crate 2 for processing over optical link HCal Signals to FADC inputs

HCAL DAQ 4/Nov/2014 DOE SBS Review / SBS DAQ 25 CTP (Crate Trigger Processor) Located in VXS crate 1 Collects integrated signal and timing from FADC channels Sends to SSP in crate 2 for processing over optical link SSP (Sub System Processor) Collects crate 1 CTP data Other 2 SSP for processing GEM data VETROC Input register for ECAL logic signals GTP (Global Trigger Processor) Located in VXS crate 2 Reads HCal info from SSP and 2 FADCs Reads Ecal info from VETROC Forms HCAL clusters and checks angular correlation with ECAL In good HCAL-ECAL coincidence then sends trigger signal T2 to Trigger Supervisor Trigger Supervisor Globally controls readout of all crates Receives T1 from ECal sends L1 to FASTBUS crates. The V1495 selects which group of FB crates to read out. If T2 from GTP arrives then readout of VME crates and FASTBUS ( event buffer size of 4) If no T2 then FASTBUS crates get Fast Clear GTP 204 Groups of 32 from ECAL Extensive use of JLab CODA hardware

Fastbus: GEp is the most demanding 26 For Fastbus, the plan is to use second-level (L2) triggers to issue a fast clear. Singles L1 trigger rate 100 – 300 kHz depending on thresholds. L2 rejection 80% or greater. L1 4/Nov/2014 DOE SBS Review / SBS DAQ

MPD (ADC + Controller) Block Diagram EP1AGX60DF780C6N 4/Nov/2014 DOE SBS Review / SBS DAQ 27

MPD Event Builder Implemented multi event block structure as suggested by DAQ people. Native data width: 24 bit, packed to 32 bit on 64 bit boundary for efficiency. Implemented 128MB FIFO data buffer using DDR2 SDRAM. Quite complicate machinery used to arbiter read and write to/from DDR2. Output can be read in 32/64 bit format in any of the supported VME cycles, including 2eSST. Performed functional simulation (FPGA + DDR2 + ADC) of quite simple events to follow all the signals. Some effort has to be put in DAQ driver to recover packed data. 4/Nov/2014 DOE SBS Review / SBS DAQ 28