1. Kostas Manolopoulos Tasos Belias
University of Athens
Physics Dpt.
Electronics Lab.
NCSR Demokritos
CLBv2 Shore Station
Kostas Manolopoulos
Tasos Belias

2. Readout Electronics on Shore
Shore station handles
Data readout
Control
Calibration
Main functions:
Point to point data reception
Broadcasting data distribution
Clock/Commands distribution
Time measurement of the optical network for each point  (calibration)
GPS reference time stamping interface

3. Shore Station Overview
WR Switch
Server1
x.1
GPS
PPS
10MHz
Shore
Sub Sea
Switch
Buffers
DOM 1 2 16
12798
12799
12800
Server2
x.2
Server n
y.z
Server
Each DOM synchronizes to the absolute time (using White Rabbit)
Each DOM receives a look up table with IP addresses while configuring the detector.
All DOMs start at an absolute point in time which was communicated via a command over the White Rabbit network.
All DOMs start their first time-slice at exactly the same time.
All data is IP/UDP formatted and passed to the IP number corresponding to the time slice
After ‘n’ time slices, first PC is again selected to process the data

4. Shore Station Functional Diagram
DOM
Buffer
SFP
Broadcast
Optical
Network
Start Tx
t4 Stop1
Reference
Clock
PTP
Time Stamp t1
Time Stamp t4
t4 Stop2
t4 Stop3
t4 Stop4
Shore Station interface Rx
j: DDMTD

5. Issues identified so far
Adjust Switch Routing Table
Software changes?
PTP timestamps t1 same for all DOMs (broadcasted from the outgoing port)
Firmware/Software changes
MAC Control-Level multicast MAC addresses need to be handled correctly
Broadcast of pause frames, control commands etc. will end up to every DOM
Need to be able to address individual DOMs
Implement address filtering?

6. More things to consider
Inter-switch synchronization
How to synchronize multiple WR switches…
Required bandwidth to send data from the WR switches to their dedicated servers
DAQ Networking: buffering and routing data of whole detector to dedicated processing units
 10GbE, RDMA capability (?), traffic modeling to ensure scalability and robustness against burst rates
Manpower
2 persons working in Athens currently (looking for more)
Collaborate closely with 7Solutions
Possibility to get IT-experts of departments involved ….

7. Next steps (outcome of this workshop)
Separate tasks, prioritize and allocate them
7Solution collaboration can help speedup things
Identify bottlenecks and critical paths
U-Athens and NCSR Demokritos ready to undertake tasks
Make a realistic time-schedule
We want to expand our test-bench in Athens used to develop broadcast down-link & p2p up-link
Kintex-7 available
2 WR switches needed
1GbE / 10GbE switch available

8. Lots to be done…