August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology KM3NeT CLBv2 1

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Visual Status 2 Rx_mac2buf I2C Fifo 31 TDCs TDC 0 Management & Control Data Control Wishbone bus RxPacket Buffer 64KB IP/UDP Packet Buffer Stream Selector (IPMUX) Rx_buf2data RxPort 1 RxPort 2 RxPort_m Management & Config. Tx_pkt2mac Tx_data2buf TxPort 1 TxPort 2 TxPort_m Flags Rx Stream Select TxPacket Buffer 32KB Flags Tx Stream Select 31 PMTs Pause Frame ADC Management & Control Hydrophone Fifo TDC 30 Fifo Nano Beacon GPIO Debug LEDs I2C Debug RS232 Temp Compass Tilt Point to Point interconnection Xilinx Kintex-7 Start Time Slice UTC & Offset counter since Time Slice Start MEM S 2 nd CPU LM32 M M WB Crossbar (1x7) WB Crossbar (3x2) S M S M M S S M M M SS S UART S M M S S M M State Machine SPI S M Flash UTC time & Clock (PPS+RefClk) aux_master ext_wb wrf_src wrf_snk

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Done: ◦ LM32 + WB-Crossbar + DPRAM + UART ◦ Soft-PLL FMC layout ◦ WR without PCI-express ◦ Deterministic PHY ◦ 1-wire Currently: ◦ Soft PLL (hardware + software). Still under study… ◦ Endpoint (= MAC) <= under investigation, focus on user interface ◦ Fabric redirector <= focus on user interface To do (in order of priority): ◦ Mini-nic <= Complex, but seems to work (PTP flows) ◦ PPS generator <= relatively straightforward ◦ SysCon <= easy? Status Listing 3

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Ethernet connection to Linux-PC 4 1Gb optical network card (Intel PRO/1000 PF Server AdapterIntel PRO/1000 PF Server Adapter Gigabit Ethernet Base-SXGigabit Ethernet Base-SX) WR core debug window via UART Wireshark receiving PTPv2 packets Linux PC SFP on SoftPLL FMC 850 nm multimode (double) fiber (Tx Rx) No WR support w.r.t. timing but suitable for Ethernet communication

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Install optical network card 5 `OpticalEth on eth2` Maximum Transmisstion Unit => Allow jumbo frames Assign IP address (for example )

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology WR on KC ) Get WR MAC address 2) Put it in the arp table (network card has IP assign WR to ) 3) Start wireshark 4) Put WR in Master mode (so that it is the initiator of a PTP transfer) 5) Start the PTP deamon

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology wireshark output 7 PTPv2 Sync Message

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology KC705 WR Ethernet port Block diagram 8 Ethernet traffic: PTP frames Ethernet MAC aux_master (Etehrbone) wrf_src wrf_snk

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Connect IPMUX => WRPC simulation 9 IPMUX Frame Generator During simulation packets are lost in the endpoint since the packet classifier is implemented using a VLIW processor that needs a “program”. Need the initialization of registers and memory Start simulation of hardware & software (wrc.elf)

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Co-Simulation hardware/software (wrc.elf) 10 LM32 crt0.S Stack pointer is set Start “main” mprintf(“up\n"); UART output S S x75u x70p x0DCR x0ALF

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology 11 During simulation packets are lost in the endpoint since the packet classifier is implemented using a VLIW processor that needs a “program”. Need the initialization of registers and memory Start simulation of hardware & software (wrc.elf) Co-Simulation hardware/software (wrc.elf) Lots of Endpoint registers to be set! Now properly initialized by software For example: MAC address Dropped!? Unicast IP\UDP Broadcast ARP Accepted Packet Filter fails! => under study

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Focus on ipmux connection to WR ◦ Understand Endpoint and wr fabric sink and source => simulation ◦ Packet Filter failure Timing Servo (still) does not lock => PPS not enabled (but focus now on ipmux connection). ◦ Servo State: “Uninitialized” => “SYNC_SEC” => “SYNC_NSEC“ ◦ …but not yet “SYNC_PHASE” and “TRACK_PHASE” Again PHY RXCDRLOCK_OUT lost in simulation… ◦ We thought this was understood but it apparently isn’t! ◦ Prepared testcase for Xilinx support showing:  Received data is of influence  Reference clock phase is of influence ◦ Work around (for the moment) digitally filter the lock signal Summary WR status 12

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology CLBv2/trunk/fw/CLBv2_Design now contains: ◦ lm32_2 nd Needs some modifications to connect:  ipmux  lm32_1 st ◦ ipmuxNeeds “Wishbonization” ◦ clb_wrpcThe KM3NeT implementation of White Rabbit  Setting up simulation environment.  Struggle with packet filter classifier… SVN 13

August 22, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology There is no way to reset the CLBv2 Watchdog timer? Talked to Eric Heine; it is foreseen to be able to switch the power for a complete string (not for individual DOMs). If a CLB hangs then the complete string could be power cycled… Is this the preferred solution? Implement a Watchdog timer? (Multiboot) Other concerns 14