1. The optical readout chain for the ALICE Transition Radiation Detector Presentation at IRTG Seminar Felix Rettig 29. Juni 2007

2. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 2 About me 1996Abitur in Heppenheim 1997Undergraduate Studies in Physics at University of Surrey, Guildford, UK 1999 -Physics Studies at Uni Heidelberg 2002-Employment at Curagita AG 2005 2007Physics Diploma in Heidelberg 2007 -PhD Student in Heidelberg

3. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 3 Outline The ALICE Experiment The Transition Radiation Detector (TRD) Optical Readout Interface Board (ORI) –features, high-speed laser modulation –error rates, irradiation test results Readout datapath components –Global Tracking Unit (GTU) and its sub-components –low-latency transmission for triggering –high-bandwidth buffering of raw data –multi-event data buffering Current Status of GTU development and assembly Outlook

4. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 4 ALICE Experiment Pb-Pb collisions at up to 1150 TeV 8000 collisions per second Goal: Quark-Gluon- Plasma Various detectors: TPC, TRD,... A Large Ion Collission Experiment (ALICE)

5. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 5 Transition Radiation Detector 1.2 million channels 65664 MCMs

6. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 6 Transition Radiation Detector 1.2 million channels 65664 MCMs Radiator Drift Chamber Drift Ampl. z y Cathode Pads Cathode Wires Anode Wires

7. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 7 TRD Front-End Electronics (FEE) Multi-Chip Module (MCM) –10-bit ADC for each of the 18 channels –up to 32 time bins, 10 MHz sampling rate ADC raw data reduction to stiff "tracklets" characterized by y-, z-position and deflection (compound 32-bit value) Filter Preprocessor Processor 10... PASA... 18x... Analog signals from 18 cathode pads Readout Tree ADC Network Interface Data Buffers Tracklet Processor (TRAP) Multi-Chip Module (MCM) deflection time bins y Local Tracking

8. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 8 TRD FEE Readout Tree Readout Boards (ROB) –16 MCMs (288 channels) in 4-level readout tree Module: 2 Half-Chambers –3 / 4 Readout Boards, Half-Chamber Merger (HC-MCM) –Optical Readout Interface Board (ORI) for laser trans- mission of tracklets and raw data to GTU MCM Board Merger Readout Board z y Board Merger CCCC CCCC CCCC CC C C HC Merger ORI-Board z y 2x per Module Half Chamber Merger 4 Pad Rows 18 x 4 = 72 Channels12 or 16 Pad Rows 2 x 72 Channels Laser to GTU located outside of L3 magnet

9. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 9 TRD FEE Readout Tree Readout Board (ROB): –16 MCMs in readout tree –4 column mergers –1 board merger Module: –2x 4 (3) Readout Boards –2 Half-Chamber Mergers –2 Optical Readout Inter- face Boards (ORI) half chamber data concen- trated by HC Merger optical transmission from ORI to Global Tracking Unit outside of L3 magnet Board Merger CCC C CCCC CCCC CCCC HC Merger ORI-Board z y 2x per Module 12 or 16 Pad Rows 2 x 72 Channels Optical laser link to GTU located outside of L3 magnet The full TRD comprises 65664 + MCMs on 4104 ROBs 540 modules in 90 stacks 1080 optical readout links

10. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 10 Optical Readout Interface Board HC-MCM LVDS CPLD 8B/10B Serial. Laser Driver PROM Key features: conversion 120 MHz 8-bit DDR to 125 MHz 16-bit SDR 8B/10B encoding and serialization to 2.5 Gbit/s high-speed modulation of laser driving signals 850nm Laser Diode (VCSEL) with Monitoring Diode ROB 120 MHz125 MHz2.5 GHz I²C bus Laser Diode

11. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 11 Optical Readout Interface Board CPLD LVDS Transceivers Laser Driver Laser Diode Connectors to Readout Board 8B/10B Encoder Serializer

12. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 12 Laser Diode Vertical Cavity Surface Emitting Laser Diode (VCSEL) built-in monitoring diode for power regulation feedback C I forward

13. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 13 Laser Modulation - Shunt Switching diff. data signal DAC + - S Temp Comp Temp Sensor + - + - + - ADC DAC S LOG AMP CURRENT ATTENUATOR C Temp Comp Laser Driver APC Mode ADC Servo Controller Modulation CurrentI M tc1/ 2 Servo Controller Source Current I M Gain T Nom I M Nom Peaking I M TC 1,2 I MD TC 1,2 APC Gain I M Rng I MD Nom I M Rng I MD RTRT ISIS IMIM I LD Laser Diode Monitor Diode

14. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 14 Laser Modulation - Eye Diagrams eye closed big parameter space: 15 setup parameters in combination with various properties of board components determine low- and high-speed characteristics of laser driving signals hard to find a common setup to be used for all 1080 boards very fast signals to be optimized, intricate to measure 8 mA Laser Forward Current 250 ps clear eye openingeye closed

15. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 15 Laser Modulation - Eye Diagrams eye closed huge parameter space: 15 setup parameters in combination with various properties of board components determine low- and high-speed characteristics of laser driving signals hard to find a common setup to be used for all 1080 boards signals to optimize are very fast, intricate to measure 8 mA Laser Forward Current 250 ps clear eye openingeye closed 6 mA final setup with "peaking" 125 ps

16. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 16 ORIs in first Supermodule at CERN Number of Optical Links Optical Power [µW] First Supermodule installed at CERN high reliability of data transmission achieved: broad safety margin ensures sufficient opti- cal power in case of laser degeneration laser diodes operating in lower region of forward current and optical power output minimum power for BER <10 -10 Bit Error Rates < 10 -15

17. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 17 ORI Irradiation Tests Irradiation tests carried out at OCL 27.7 MeV proton beam All components meet safety margin of factor 4 life-time Assumptions about TRD radiation environment: D IP ~ 1.6 Gy, D tot ~ 1.8 Gy in 10 years of operation

18. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 18 Full TRD Readout Chain Global Tracking Unit (GTU) Track Matching Unit (TMU) Supermodule Unit (SMU), Trigger Generation Unit (TGU) ORI Module 6x Stack Trigger Design Event Buffering Design Tracklets only Tracklets & Raw Data RX TMU 5x SMU Trigger Handling & Control GTU Supermodule Segment Front-End Electronics within L3 magnet Half Chamber DCS Board TTCrx DDL SIU DATE Software DIU D-RORC Mass Storage DAQ System Central Trigger Processor TGU 1 8 x 5x Track Concentr. GTU Racks 12 fibres Racks below Muon System Storage

19. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 19 ALICE Trigger Timing Time after Collision CollisionLevel-0 Trigger Level-2 Trigger Window 1.273.8 µs 493.8 µs Tracklets 6.2 µs Raw Data Data Forward to DAQ Accept/Reject Level-1 Trigger

20. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 20 GTU Requirements 1080 optical links deliver both –tracklets:low-latency required, small volume, high rate –raw data:big volume, latency subordinate, lower rate huge bandwidth –each opical link: 1.94 Gbit/s –links into one TMU: 23.3 Gbit/s –full detector: 2.1 Tbit/s = 244 GByte/s fast track reconstruction and decision taking for TRD's level-1 trigger contribution uni-directional optical links, no flow control –buffering of incoming raw data with full bandwidth forward buffered event data to DAQ system or discard support for trigger interlacing, buffering of multiple events

21. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 21 TMU Requirements low-latency deserialization and 8B/10B decoding of optical data received from front-end electronics 12 independent 16-bit data streams at 125 MHz clocks single 4-Mbit SRAM with 128-bit interface at 200 MHz multi-event buffering of up to five events, appropriate data forwarding or discarding on L2 messages error detection / handling for each separate link aborting of incomplete events, discarding associated data delicate balance between speed and ressource utilization to accomodate trigger and buffer designs in one FPGA → complex FPGA design, many clock domains, highly pipelined data path design needed to allow for operation at 200 MHz

22. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 22 Receiving of Optical Data optical-electrical converter modules Virtex-4 FX100 FPGA 16 Multi-Gigabit Transceivers –highly configurable, ~100 parameters –clock reconstruction –deserializing –comma detection, comma aligning –8B/10B decoding –rate matching incoming 16-bit data streams synchronized to one 125 MHz clock

23. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 23 Event Buffering Design 16/128 Event Shaper SRAM Controller writeread 16/128 Read Address Logic & Control Data Formatting TMU/SMU Interface Readout Unit 4-Mbit SRAM SMU Control (L0-/L1-Trigger) Scheduling Memory Management Event (n, 0) Event (n+1, 0) Event (n, 11) Event (n+1, 11) Event Info FIFO Buffer Supermodule Unit Links from one Stack... 12x... SMU Control (L2-Message) Data Block, Link 0Data Block, Link 11... 12x... empty address data status Data stream Merging

24. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 24 Data Stream Merging collation of 16-bit values to 128-bit lines for each link padding at the end of datastreams 12 data path clock domain crossings 12 independent 16-bit data streams at 125 MHz with gaps single SRAM with 128-bit wide interface at 200 MHz 128-bit Line

25. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 25 Data Collation = FIFO ring counter BRAM0 1 2 3 Link Data Data Valid End Marker Line Data Last Line Line Advance Line Valid 3216 128 read address write address write enable Block-RAM primitives –16-bit / 32-bit collation –safe datapath clock domain crossing –buffering of filled 128-bit lines until scheduled forward to SRAM write commas discarded, only valid data words are collated write enable logic performs padding without need for extra endmarker write cycles end of data flag stored explicitly, no 32-bit compares necessary in later stages 125 MHz clock domain 200 MHz clock domain Aligning Buffer

26. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 26 Buffer Organization SRAM divided into separate parts, one for each link blocks independently organized as ring buffers data of multiple events dynamically double write pointers allow discarding of incomplete events, directly freeing occupied memory locations overrun protection, anticipatory busy signaling 4-Mbit SRAM Event (n, 0) Event (n+1, 0) Event (n, 11) Event (n+1, 11) Data Block, Link 11... 12x... empty memory Data Block, Link 0 rp end (n) end (n+1) rp end (n) end (n+1)

27. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 27 Buffer Address Management Event n -2 Event n-1 Event n (incomplete) + + Write Pointer A 1 Block RAM Bank B Bank A write address read address 1 0 R 18 SRAM write address offset Data Block, Link 0 Read Pointer AB Write pointers Write Pointer A 0 Write Pointer B 1 Write Pointer B 0... 12x... base address event end pointer 0 2 write pointers –first word offset –current offset normal operation: –first constant –increment current event complete: –pointers swap meaning discard incomplete event: –use first word pointer, copy to current pointer only one RAM block needed for all pointers, no 12:1 multiplexers

28. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 28 Event Buffering Design - Detailed

29. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 29 Event Buffering Design - Detailed

30. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 30 Event Buffering Design - Detailed

31. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 31 Event Buffering Design - Detailed

32. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 32 TMU FPGA Design Current TMU design: –ressources produce new pictures!

33. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 33 CERN Tests

34. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 34 Tests and Integration Progress GTU segment running in Münster for super-module assembly and testing all segments being assembled and tested currently transfer to CERN and installation scheduled for end of july some CTP signal handling and DDL/DAQ issues open trigger design not yet integrated

35. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 35 Prospective Work Finish development of GTU –a lot of open detail issues –multi-event buffering under full CTP control Installation at CERN Beam and performance tests, refinements Monitoring system, GTU event display Physics applications of GTU –various trigger schemes based on comprehensive track and p t information, e.g. Jet Triggers

36. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 36 Thank You for Your Attention Questions?

37. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 37 TMU Board - Final Layout

38. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 38 SMU Board - Final Layout

39. Felix Rettig – KIP – ALICE TRD Global Tracking Unit – IRTG Seminar 2007-06-29 p. 39 TMU-SMU-Hybrid