ATCA carrier layout 10/100 GTS /SEGMENT Config PLD CORE /SEGMENT Ethernet Switch Data Mgm FPGA GTS /SEGMENT Zone 3 RTM TCLK PORT Config PLD To JTAGs CORE /SEGMENT ZONE 2 FAST ETHERNET MAIN FPGA SEGMENT ZONE 2 ETHERNET 1000 PCI Express Data Mgm FPGA SEGMENT 32x128M SDRAM 8x8M FLASH 32x512k Static DPRAM POWER & IPM IMP µP DC-DC +5  +3.3 DC-DC From -48v to +12/+5 4x DC-DC +5  +3.3 DC-DC +5  +2.5 DC-DC +5  +1.2

200Mhz Clock distribution tpd1 , skew1, jitter1 GTS GTS_CLOCK FANOUT 1:2 TCLK PORT 2 1 tpd2 , skew2, jitter2 Jitter 10ps DELAY LINE Jitter 14ps CORE SYNC SYNC_RTN tpd4 , skew4, jitter4 tpd3 , skew3, jitter3 FANOUT 1:6 SWITCH SEGMENT 4 tpd5 , skew5, jitter5 Jitter 10ps MAIN FPGA SMB INSP SEGMENT SEGMENT FANOUT 1:2 TCLK PORT DELAY LINE SEGMENT SYNC SYNC_RTN 5 FANOUT 1:6 SWITCH tpd Master = tpd1+tpd2+tpd3+tpd4 tpd Slave = tpd1+tpd5+tpd3+tpd4 skew Master = skew1+skew2+skew3+skew4 skew Slave = skew1+skew5+skew3+skew4 jitter Master2 = jitter_clock2+jitter12+jitter22+jitter32+jitter42 jitter Master2 = jitter_clock2+jitter52+jitter22+jitter32+jitter42 SEGMENT 6 3 MAIN FPGA SMB INSP SEGMENT

Bcast & Msg Handler 1/2 TRG/BCT FPGA MAIN FPGA GTS MEZZANINE FROM REMOTE (TCLK) 8 B_cast_data (7 downto 0) B_cast_str0 FANOUT TO MAIN FPGA, LOCALS MEZZANINES, AND REMOTE MEZZANINES (TCLK) SERIALIZERS 81 8 B_cast_str1 GTS Status (7 downto 0) TRG/BCT FPGA 8 GTS MEZZANINE MAIN FPGA LLP Status (7 downto 0) Msg_data (7 downto 0) Msg_str0 Msg_str1 FROM REMOTE (TCLK) Concentrator SEG/CORE MEZZANINE 8 LLP Status (7 downto 0) SERIALIZERS 81 8 TO REMOTE (TCLK) Msg_data (7 downto 0) Msg_str0 8 Msg_str1

Bcast & Msg Handler 1/2 GTS CORE SEGMENT SEGMENT SEGMENT SEGMENT TCLK PORT Serializers FPGA CORE SEGMENT MAIN FPGA SEGMENT SEGMENT TCLK PORT Serializers FPGA SEGMENT SEGMENT MAIN FPGA SEGMENT

TRIGGER Handler 1/2 MAIN FPGA GTS MEZZANINE CORE MEZZANINE Trig_val (1 downto 0) Trig_Rej (1 downto 0) 8 Lt_data (7 downto 0) FANOUT TO OTHERS DEST SERs / DESERs 81 8 Lt_Strobe GTS MEZZANINE 8 Tv_data (7 downto 0) Tv_Strobe Trig_req (1 downto 0) MAIN FPGA Trig_req (1 downto 0) CORE MEZZANINE

TRIGGER Handler 2/2 GTS CORE SEGMENT SEGMENT SEGMENT SEGMENT SEGMENT TCLK PORT Serializers FPGA CORE SEGMENT MAIN FPGA SEGMENT SEGMENT TCLK PORT Serializers FPGA SEGMENT SEGMENT MAIN FPGA SEGMENT

Data readout engine Serializer 1024x18 Serializer DPRAM Serializer Using 1 16 bit port : 128 words/event 256 bytes/event 6 channels  1536bytes 16bit bus @ 100MHz Need 7.68µs (20µs avaible @ 50KHz) Data readout engine 1536*4 = 6144 byte/Event 32bit bus @ 100MHz Need 15.36µs (20µs avaible @ 50KHz) 8 pairs ; 16 I/O Data_A (15 downto 0) Serializer Deserializer Serializer Empty_A Serializer Serializer 32bit Data_Ready_A 32bit 1024x18 DPRAM Data_Request_A Data_A (15 downto 0) 9bit 9bit Empty_A Data_Ready_A Data_Request_A 1024x18 DPRAM X4 Mezzanines

MGT Clocking Layout RocketIO 101 MUX MGTclk M34/N34 RocketIO 102 USER SMA MUX MGTclk M34/N34 RocketIO 102 ATCA FABRIC CHANNEL6 INSPECTION PADS 200MHz GTS Clock MUX ATCA FABRIC CHANNEL5 MGTclk AP28/AP29 RocketIO 103 ATCA FABRIC CHANNEL4 MUX ATCA FABRIC CHANNEL3 RocketIO 105 ATCA FABRIC CHANNEL2 MUX ATCA FABRIC CHANNEL1 200MHz to 250MHz PLL RocketIO 106 (**) The ATCA FABRIC channels are routed from CHANNEL1 to CHANNEL12 by switches MUX EPSON 250MHz RocketIO 109 MUX MGTclk AP3/AP4 RocketIO 110 MUX MGTclk J1/K1 RocketIO 112 RTM PCI EXPRESS LANE3 MUX RTM PCI EXPRESS LANE2 FROM SFP TRANSCEIVER PCI Express JITTER ATTENUATOR RocketIO 113 RTM PCI EXPRESS LANE1 FROM RTM MUX RTM PCI EXPRESS LANE0 INSPECTION PADS RocketIO 114 MUX USER SFP TRANSCEIVER (***) User SFP could be used as 1GEnet or PCIExpress DAQ without FABRIC

MGT Layout MGT112A MGT112B MGT113A MGT113B MGT114A MGT102A MGT102B RocketIO ZONE3 (RTM) PCI Express Lane 0 MGT112B RocketIO ZONE3 (RTM) PCI Express Lane 1 MGT113A RocketIO ZONE3 (RTM) PCI Express Lane 2 MGT113B RocketIO ZONE3 (RTM) PCI Express Lane 3 MGT114A RocketIO iSFP Cage (1GEthernet,PCI Espress,User …) (*) To clock management Clock in Cage (*) MGT102A RocketIO FABRIC CHANNEL12 TEST FABRIC CHANNEL11 RocketIO FABRIC CHANNEL10 MGT102B TEST FABRIC CHANNELL9 MGT103A RocketIO FABRIC CHANNELL8 TEST FABRIC CHANNELL7 The ATCA FABRIC is a DUAL STAR. This means that CHANNELS 1 of slot from 3 to 14 are are routed on CHANNEL1 to CHANNEL12 on HUB1 and CHANNELS 2 of slot from 3 to 14 are are routed on CHANNEL1 to CHANNEL12 on HUB2 FABRIC CHANNELL6 MGT103B RocketIO TEST FABRIC CHANNELL5 MGT105A RocketIO FABRIC CHANNELL4 TEST FABRIC CHANNELL3 FABRIC CHANNELL2 MGT105B RocketIO (*) (*) These cnannel must be routed on LLP cards, the others Channels are needed only on central switch. TEST FABRIC CHANNELL1 (*)

MGT topology template AC VIAS N P N P P VIAS AC VIAS N N P P AC VIAS N

PCI Express topology template AC AC N P VIAS N P VIAS N P VIAS AC AC VIAS N N P N P N P N P P AC AC N P VIAS N P VIAS N P VIAS AC AC VIAS N N P N P N P N P

ATCA Configuration and status TCK0 TMS0 TDI0 TDO0 Xc9500 pld TCK_RTM TMS_RTM TDI_RTM TDO_RTM MEZZANINE 0 TCK1 TMS1 TDI1 TDO1 TCK_PPC TMS_PPC TDI_PPC TDO_PPC MEZZANINE 1 RTM/PPC TCK2 TMS2 TDI2 TDO2 MEZZANINE 2 CONF[1..0] MEZZANINE 0 CONF[1..0] MEZZANINE 1 CONF[1..0] MEZZANINE 2 TCK3 TMS3 TDI3 TDO3 CONF[1..0] MEZZANINE 3 MEZZANINE 3 FPGA_INIT/PROGRAM MEZZANINE 0 FPGA_INIT/PROGRAM MEZZANINE 1 TCK4 TMS4 TDI4 TDO4 FPGA_INIT/PROGRAM MEZZANINE 2 MAIN FPGA FPGA_INIT/PROGRAM MEZZANINE 3 FPGA_INIT/PROGRAM MAIN FPGA FPGA_INIT/PROGRAM TRG FPGA FPGA TRIGGER TCK5 TMS5 TDI5 TDO5 SEL PROGRAM MEZZANINE 0 SEL PROGRAM MEZZANINE 1 SEL PROGRAM MEZZANINE 2 AUX_1 TCK6 TMS6 TDI6 TDO6 SEL PROGRAM MEZZANINE 3 SEL PROGRAM MAIN FPGA SEL PROGRAM TRG FPGA AUX_2 TCK7 TMS7 TDI7 TDO7 I2C_SCK I2C_SCL

ATCA I2C Layout and addressing Mezzanine 1 Address $00 Mezzanine 1 Address $00 Mezzanine 1 Address $00 Mezzanine 1 Address $00 Monitoring ADC Address $00 Serial EEPROM Address $00 Temp Sensor 1 Address $00 Temp Sensor 2 Address $00 Address $00 IPMB1 Temp Sensor 3 Address $00 Address $00 IPMB0 Jtag Supervisor Address $00 Address $00 Main Fpga SWITCHS Supervisor Address $00

ATCA Power Supply 4x POWER ONE YS12S10 55W DC to DC Converter P3V3-5A 16.5W MEZZANINE 1 P3V3/P2V5 Linear Reg P2V5-1.5A VCCAUX Fpga 1 Fusing Filtering Protection Hot Swap DC to DC Converter P3V3-5A 16.5W MEZZANINE 2 P5V0/P2V5 Linear Reg P2V5-1.5A VCCAUX Fpga 2 -48V DC DC to DC Converter P3V3-5A 16.5W MEZZANINE 3 P5V0/P2V5 Linear Reg P2V5-1.5A VCCAUX MGT DC to DC Converter P3V3-5A 16.5W MEZZANINE 4 P5V0/P1V8 Linear Reg P1V8-0.5A PROMS ENABLE P5V0/P1V2 Linear Reg P1V2-0.5A VTTTXs ATC210 (210W) P12V-14.7A 176.7(160.6)W M48/P12 DC DC P12/P5V0 DC DC P5V0-6A 30W P5V0/P1V2 Linear Reg P1V2-0.5A VTTRXs M48V-4.0A 194.4(176.7)W P3V3_BOOT P12/P3V3 DC DC P3V3-7A 23.1W MAIN BOARD P12/P2V5 DC DC P2V5-7A 17.5W MAIN BOARD P12/P1V2 DC DC P1V2-7A 8.4W FPGAs CORE DC-DC Efficency is estimated at least 90% P12/P1V2 DC DC P1V2-4A 4.8W FPGA MGT P12/P1V8 DC DC P1V8-6A 10.8W MGT BUFFERS 6x POWER ONE YS12S10 55W

Segment / Core mezzanine I/O model 8 Data_A (15 downto 0) 8 MII port Empty_A Data_Request_A 8 Data_Ready_A Jtag and config 8 Data_B(15 downto 0) Serial_Tx Empty_B 4 Serial_Rx Data_Request_B Data_Ready_B 2 I2C_Sck Trig_req (1 downto 0) I2C_Sda Trig_val (1 downto 0) Cfg(1downto 0) Trig_Rej (1 downto 0) 2 16 pairs 32 I/O SER / DESER 81 Lt_data (7 downto 0) 8 Lt_Strobe 8 SEGMENT/ CORE MEZZANINE Tv_data (7 downto 0) 6 Tv_Strobe Msg_data (7 downto 0) 8 Msg_str0 Segment / Core mezzanine I/O model Msg_str1 2 B_cast_data (7 downto 0) 8 B_cast_str0 B_cast_str1 2 SMB INSP LLP Status (7 downto 0) 8 GTS Status (7 downto 0) 8 8 SPARE OUT 6 8 SPARE IN 2 CLOCK

GTS mezzanine I/O model MII port Jtag and config Serial_Tx Serial_Rx I2C_Sck Trig_req (1 downto 0) I2C_Sda Trig_val (1 downto 0) Cfg(1downto 0) Trig_Rej (1 downto 0) 2 9 pairs 18 I/O SER / DESER 81 Lt_data (7 downto 0) 8 Lt_Strobe 8 GTS MEZZANINE Tv_data (7 downto 0) 6 Tv_Strobe Msg_data (7 downto 0) 8 Msg_str0 Msg_str1 2 B_cast_data (7 downto 0) 8 GTS mezzanine I/O model B_cast_str0 B_cast_str1 2 SMB INSP LLP Status (7 downto 0) 8 GTS Status (7 downto 0) 8 8 SPARE OUT 6 8 SPARE IN 2 CLOCK

GTS mezzanine I/O model PREPROC 1 PREPROC 2 PREPROC 3 PREPROC 4 TCLK_RTM MAIN_FPGA Trig_req (1 downto 0) 8 Trig_val (1 downto 0) Trig_Rej (1 downto 0) 8 Lt_data (7 downto 0) 8 Lt_Strobe 8 FANOUT TO OTHERS DEST Tv_data (7 downto 0) Tv_Strobe 8 GTS MEZZANINE B_cast_data (7 downto 0) SER / DESER 81 8 B_cast_str0 B_cast_str1 GTS Status (7 downto 0) 8 8 LLP Status (7 downto 0) 8 Msg_data (7 downto 0) Msg_str0 Msg_str1 SMB INSP FROM TCLK (SLAVE MODE)

Rise time 35ps  0.35/35ps = 10Ghz !!!!

1 2 3 4 5 6 GTS CLOCK tpd1 OUT FIRST FANOUT 1:2 tpd5 OUT AFTER TCLK PORT 3 tpd4 OUT AFTER FANOUT 1:8 4 Tpd2 OUT AFTER DELAY LINE 5 tpd4 OUT AFTER FANOUT 1:8 6