1. Data transmission needs and challenges for Frontier Particle Physics: Part 1 Why HEP needs special? Radiation-hard high speed optical links Lasers and p-i-n diodes  Suen’s talk Radiation tolerance of fibres Single Event Upsets Solution for LHC phase 1 upgrades – Versatile Link: optoelectronics – GBT: rad-hard chipset Tony WeidbergInfieri 20141

2. HEP Requirements Need high speed links – ~ 100 M detector channels – Typical read out rates for LHC detectors ~ 100 kHz Usual requirements – High speed links (5 to 10 Gbps) – Low costs Unique requirements – Radiation tolerant up to ~ 500 kGy (Si) – Single Event Upsets – Very low mass (material degrades resolution) and small – Very low power (cooling  more material) – Avoid magnetic materials (e.g. ferrite cores) – Access very difficult  very high reliability Tony WeidbergInfieri 20142

3. Example: CMS silicon tracker (LHC) Tony WeidbergInfieri 20143 Silicon detectors Readout and optoelectronics (space limitations) Fibre ribbons Access to optoelectronics clearly very difficult  very high reliability

4. Tony WeidbergInfieri 20144 Step Index Multi Mode (SIMM) Fibres Simplest fibre: Step Index Multi-mode fibre. Light trapped by total internal reflection. Maximum angle Problem: – Larger angles   longer optical path length  longer flight time. – Many modes with different  propagate (multi-mode) – large modal dispersion  very low bandwidth. Typical diameters (  m) Core50 Cladding125 Buffer (protection)250 OBSOLETE

5. Tony WeidbergInfieri 20145 Graded Index (GRIN) fibres Adjust refractive index profile n(r) vs r to minimise modal dispersion. RIA  Im(n) must change  Re (n) must also change  small change in  could increase dispersion (degrade bandwidth).  Dispersion vs  Very deep minimum 

6. Singlemode fibres Use small diameter core ~ 5 to 10  m. – Must use wave optics (not geometrical) – Fibre is dielectric waveguide: solve Maxwell’s equations subject to boundary conditions. Continuity at core/cladding boundary: D perp, E //, B perp,H //. Propagating wave in core Evanescent wave in cladding (exponentially decreasing with radius) Allowed solutions  propagating modes in fibre. – If diameter is small enough  only one mode allowed Tony WeidbergInfieri 20146

7. Pros & Cons of Singlemode fibres Pros: no modal dispersion  higher bandwidth Cons: only one mode and smaller core diameter  more precise alignment, i.e. harder to couple light into singlemode fibre Multi-mode fibres benefit from cheap low power VCSELs (850 nm) Use Singlemode fibres for long-haul and Multimode fibres for short distances (up to ~ 1 km). Tony WeidbergInfieri 20147

8. Dispersion in Singlemode Fibres Two sources of dispersion in SM fibres: – Chromatic dispersion, variation of dn/d with – Intra-modal dispersion: effect of waveguide v g (k) is a function of k, “different path length for different k” (very naïve picture). 1310 nm better for chromatic dispersion but 1550 better for absorption (also for Er doped fibre amplifiers). – Telecoms fibres use dispersion shifted fibres Waveguide dispersion designed to cancel material dispersion at 1550 nm. Tony WeidbergInfieri 20148

9. Tony WeidbergInfieri 20149 Fibre Dispersion and Attenuation Dispersion is a minimum ~ 1.3  m Attenuation is minimum ~1.5  m Long distance uses ~ 1.3 or 1.5 um Short distance use 850 nm because of availability of cheap VCSELs (  m)

10. Radiation Tolerance Fibres Radiation creates “colour centres”, states which allow increased absorption  Irradiated fibre goes yellow! Radiation Induced Attenuation (RIA) – Important for other applications, e.g. nuclear power stations, fusion reactors etc. Fibre bandwidth Fibre mechanical reliability after radiation Tony WeidbergInfieri 201410

11. Radiation Induced Attenuation Most fibres use P dopants  slows down annealing of radiation damage and results in very poor radiation tolerance. Normal dopant to create refractive index profile is GeO 2 but F increases annealing rate  better radiation tolerance with F doping (n.b., F decreases refractive index but can still design correct doping profile). Other tricks commercially sensitive … Tony WeidbergInfieri 201411

12. Radiation Source 60 Co  sources at RITA (SCK SEN) Belgium &BNL. Annealing of damage is very important – Temperature and dose rate important At HL-LHC Si detectors will operate ~ -25 o C – Warm and cold measurements of fibres Dose rates : 0.0265 to 22.5 kGy(Si)/hour Tony WeidbergInfieri 201412

13. Radiation Induced Attenuation Measure returned optical power for – Reference fibre (not in radiation zone) – Fibre in radiation zone – Difference gives Radiation Induced Attenuation (RIA). Tony WeidbergInfieri 201413

14. Dose Rate Effects Damage for same dose lower for lower dose rates because of annealing. Spike at very high dose rate not seen at lower dose rates. Tony WeidbergInfieri 201414 No cooling

15. Tony WeidbergInfieri 201415

16. Tony WeidbergInfieri 201416 Note “spurious annealing” when fibre removed from source

17. Cold Fibre Irradiation (1) Some fibre types survived high dose rate (HL-LHC in 24 hours) at T=-25C Tony WeidbergInfieri 201417 Singlemode Fibre X CO 2 cooling

18. Cold Fibre Irradiation (2) Other fibre types did not survive high dose rate (HL-LHC in 24 hours) at T=-25 o C (but was good in warm irradiation). Tony WeidbergInfieri 201418 Corning Infinicor SX+ Damage so large it saturated our measurement system! Inconclusive!

19. Next Tests We have fibres that are – radiation tolerant at high dose rate and warm temperature. – Not radiation tolerant at high dose rate and low temperature (T~ -25 o C). Need long exposure at low dose rate at T=-25 o C. – Not trivial … used evaporative CO 2. – Blow-off system  changed CO 2 bottles every day for 10 days. Tony WeidbergInfieri 201419

20. Radiation Induced Attenuation Measure RIA vs dose. T=-25 o C Low dose rate – 0.7 kGy(Si)/hour Combine this with expected dose at HL-LHC 5 fibres qualified for use at HL-LHC. Tony WeidbergInfieri 201420 Cold irradiation

21. Fibre Bandwidth Need high bandwidth – speed * distance (units MHz. km) – Distance for detectors ~ 0.1 km, readout rate ~ 10Gbps  bandwidth > ~ 1000 MHz. km Commercial fibres can meet these requirements – OM3: 1500 - 2000 MHz. km – OM4: 3500 - 4700 MHz · km Will bandwidth be affected by radiation? Tony WeidbergInfieri 201421

22. Fibre Bandwidth & Radiation RIA  imaginary part of n(  ) changes with irradiation – can affect profile of n(  ) which is precisely optimised for GRIN fibre – Change in is frequency dependent, Kramers- Kronig relation  expect to change and hence change chromatic dispersion. Tony WeidbergInfieri 201422

23. Differential Mode Delay (DMD)Principle Tony WeidbergInfieri 201423 Scan injected pulse over fibre core and measure arrival time High power laser coupled to SM fibre Sensitive to modal dispersion.

24. DMD Results Mode delay varying across fibre as expected Negligible change with irradiation. Tony WeidbergInfieri 201424

25. Chromatic Dispersion Measured by Time of Flight (TOF) for different wavelengths. Units are ns/(km mm). Negligible change after 500 kGy(Si). Fibre qualified as radiation tolerant from RIA perspective. Tony WeidbergInfieri 201425 Draka Elite SRH-MMF

26. Fibre Mechanical Reliability Extensive studies by fibre manufacturer’s predicted excellent long-term reliability. Confirmed by reliability of installed fibres. Does radiation damage change reliability? – Some evidence that it can happen, e.g. F in cladding can produce aggressive radicals that damage the fibre. – Perform tests on qualified radiation tolerant fibres. Tony WeidbergInfieri 201426

27. Fibre Mechanical Reliability Long term reliability methodology based on statistical analysis of fibre breaks in destructive tests. Measure median breaking strain at a given stress rate. Repeat at different stress rates Stress corrosion parameter n  long term reliability. Tony WeidbergInfieri 201427 2 point bend tester Jaws move together at constant speed until fibre breaks. Detected by microphone. Record breaking strain.

28. Micro-bending Mechanical strength dominated by glass. Buffer coating protects fibre and minimises loss from micro-bending Assess quality by fibre winding machine with sandpaper to introduce micro-bending. Compare fibre before and after irradiation. Small improvement with radiation! Tony WeidbergInfieri 201428

29. Summary Fibre Radiation Tolerance Have identified suitable multi-mode and singlemode fibres – Small increase in attenuation with radiation, can absorb loss into optical power budget. – No significant change in fibre bandwidth with radiation. – Small improvement in mechanical reliability with radiation! Tony WeidbergInfieri 201429

30. Single Event Upsets in ATLAS SCT Single Event Upsets (SEU) studied for ATLAS & CMS in test beams – Study of SEU in ASICs in LHC operation. Expectations for (SEU) from test beam data. SEU in SCT operation and comparisons with test beam – p-i-n diodes in TTC link. Mitigation for ATLAS operation. Mitigation strategy for SEUs at HL-LHC. Tony WeidbergInfieri 201430

31. SEUs in SCT, how and where? Particles deposit sufficient charge in small region of silicon  bit error (SEU) – Typically needs nuclear interaction to deposit sufficient energy, i.e. MIPs are harmless. In p-i-n diode that receives optical TTC signal – Single bit error  loss of synchronisation of a FE module. Also In static registers in ABCD – Don’t care about dynamic memory (pipeline) but static registers will stay wrong after an SEU until reset. – Look at effects in DAC threshold register. Tony WeidbergInfieri 201431

32. SEU Studies Measure SEU rates for prototype in test beams: – Low energy  /p beams (mainly 200 – 500 MeV/c) – Extrapolate to LHC spectrum? – No synchronisation with beam bunches. – Angle of incidence. Measure actual SEU rates in ATLAS operation and compare with test beam based predictions. – Results shown for barrel SCT only. Tony WeidbergInfieri 201432

33. SEU In SCT Optical Links On-detector p-i-n diode is Sensitive to SEU – Small electrical signal before amplifier stage. Measure BER with loopback – With beam – Without beam – Difference  SEU Tony WeidbergInfieri 201433 TTC

34. SEU in p-i-n diode – Test Beam Measured SEU vs current in p-i- n diode I PIN (simple loopback test). – No errors with beam off. – No errors for MIPs. – Measured Bit Error Rate vs I PIN with beam on. – ac coupled  charge required to cause bit flip is proportional to I PIN.  higher for 300 MeV/c  because of  resonance  large variation of  with energy  difficult to predict rates for LHC operation. Tony WeidbergInfieri 201434  (SEU)=# bit errors/fluence J.D. Dowell et al., Single event upset studies with the optical links of the ATLAS semiconductor tracker, Nucl. Instr. Meth. A 481 (2002) 575.

35. SEU in ATLAS Operation p-i-n diode receives optical TTC signal. Indirect measurement BER Signature for SEU in p-i-n diode is loss of synchronisation for L1A trigger: – TTC sends full L1A number to ROD: L1A(full) L1A signal to detector FE via optical links. – On-detector 4 bit counter counts L1A and returns 4 LSBs in data stream: L1A(4) – SEU causes 0  1 can cause loss of L1A on-detector. – Compare L1A(full) with L1A(4). Persistent discrepancy is SEU. No errors seen in “physics mode” running with no beam  suspect that these errors during beam are due to SEU. Tony WeidbergInfieri 201435

36. Are errors really SEU ? SEU rate should scale with module occupancy (proxy for particle flux). Occupancy changes from luminosity variations and decreases as radius of barrels increase Shows expected linear behaviour Total number SEU – Predicted: 2504 – Observed: 1949 Tony WeidbergInfieri 201436

37. Mitigation Strategies for ATLAS Operation SEU in TTC links – Use large values of I PIN (> 100  A) to reduce  (SEU) – Reset pipeline in FE chips and all counters if this de-synchronisation detected by DAQ (20 to 50s). Mitigation strategies reduce effects of SEU to negligible level. Tony WeidbergInfieri 201437

38. SEUs @ HL-LHC Expect SEUs to be more important @ HL-LHC because of higher Luminosity. What can we do to mitigate SEU? – Triple event redundancy in gates – Error correction on TTC link. – Propose to correct for sequence of error bursts up to 16 bits long  slide. Tony WeidbergInfieri 201438

39. Versatile Link TTC SEU Measured BER vs optical power, Optical Modulation Amplitude (OMA). SEU killed by error correction (FEC)  Error correction required for TTC links Tests to determine if it is also required for data Tony WeidbergInfieri 201439 A. Jimenez Pacheco et al., Single-Event Upsets in Photoreceivers for Multi-Gb/s Data Transmission, IEEE Trans. Nucl. Sci., Vol. 56, Iss. 4, Pt. 2 (2009), pp. 1978 – 1986.

40. Versatile Link Optical links for data read out from detector and Timing, Trigger and Control data needed by the detector. Generic system for LHC detector upgrades Tony WeidbergInfieri 201440

41. Versatile Link Need radiation-tolerant optoelectronics for on-detector components. – Select VCSELs and p-i-n diodes from radiation studies (see Suen’s talk). – Want very reliable and low mas/non-magnetic package. – Commercial optical transceivers can’t be used – Need radiation-tolerant ASICs. – Too much material, magnetic and use ferrite cores as inductors in laser drivers. Keep optical sub-assembly and commercial connectors for reliability but use plastic package. Tony WeidbergInfieri 201441

42. Performance Measure Bit Error Rate (BER) over 400m fibre (pessimistic) – Designed for 4.8 Gbits/s but works up to ~ 10 Gbps. – Small penalty in minimum power required. Tony WeidbergInfieri 201442 dBm is optical power in dB relative to 1 mW. dB=10*log10(p2/p1)

43. The GBTx System Tony WeidbergInfieri 201443

44. Other Radiation Tolerant ASICS GBLD – Laser driver GBTIA – Receives optical signal from p-i-n diode, transimpedance amplifier/discriminator Produced in 130 nm IBM process. Radiation tolerance qualified. Tony WeidbergInfieri 201444 4.8 Gb/s, pre-emphasis on Total jitter: ≈ 25 ps

45. Optical Power Budget Is there enough power for receiver to work allowing for all losses? In particular consider effects of radiation damage. Margin represents additional safety margin with worst case assumptions for all other components, so 1.8 dB is fine! Tony WeidbergInfieri 201445

46. Summary Optoelectronics used for high speed data transfer in HEP. Some special requirements, particularly radiation tolerance. – Fibre radiation tolerance demonstrated – Laser and p-i-n diodes in Suen’s talk Versatile Link for HL-LHC detectors. Tony WeidbergInfieri 201446

47. Backup Tony WeidbergInfieri 201447

48. Absolute Rates (2) Naïve prediction: – N(SEU) =  (SEU) * Fluence – Ignore variation in  (SEU) with LHC spectrum. – Corrected for variation of  (SEU) with I PIN. – Fluence: use – Reject long SEU bursts (>60s) 13% uncertainty – Reject modules with multiple errors in one run: 5 to 6% bias. Number SEU in data set – Luminosity 7.81 fb -1 – Measured: 2504 – Predicted : 1949 – Good agreement within large uncertainties. Tony WeidbergInfieri 201448

49. Tony WeidbergInfieri 201449

50. SEU in ATLAS Operation (2) L1A signal is 110 Short code vulnerable to single bit error (minimize latency). Assume 0  1 transitions more probable than 1  0 because of high value of I PIN. Most probable error “110”  “111” In ATLAS energy deposition synchronised to bunch crossing, unlike test beam Creates large uncertainties in extrapolating test beam cross section to ATLAS operation. Tony WeidbergInfieri 201450

51. Some References The radiation induced attenuation of optical fibres below −20°C exposed to lifetime HL-LHC doses at a dose rate of 700 Gy(Si)/hr, 20012 JINST 7 C01047 doi:10.1088/1748-0221/7/01/C01047 doi:10.1088/1748-0221/7/01/C01047 The Versatile Link common project: feasibility report, 2012 JINST 7 C01075 doi:10.1088/1748-0221/7/01/C01075. The Versatile Link common project: feasibility report, 2012 doi:10.1088/1748-0221/7/01/C01075 A Study of the effect of a 500 kGy(Si) radiation does on the bandwidth of a radiation hard multi-mode fibre, 2012_JINST_7_P10021, http://dx.doi.org/10.1088/1748-0221/7/10/P10021 http://dx.doi.org/10.1088/1748-0221/7/10/P10021 A study of the effect of radiation on the mechanical strength of optical fibres, 2013_JINST_8_P05011, http://dx.doi.org/10.1088/1748-0221/8/05/P05011http://dx.doi.org/10.1088/1748-0221/8/05/P05011 Further studies of the effect of radiation on the mechanical strength of optical fibres, 2013 JINST_8_P12002, http://dx.doi.org/10.1088/1748- 0221/8/12/P12002http://dx.doi.org/10.1088/1748- 0221/8/12/P12002 The Optical Links of the ATLAS SemiConductor Tracker, 2007_JINST_2_P09003, http://www.iop.org/EJ/abstract/1748-0221/2/09/P09003 http://www.iop.org/EJ/abstract/1748-0221/2/09/P09003 Single Event Upset Studies Using the ATLAS SCT, 2014_JINST_9_C01050.2014_JINST_9_C01050 Tony WeidbergInfieri 201451