1. Status of the OPERA DAQ D.Autiero, J.Marteau
S.Gardien, C.Girerd, C.Guérin (electronics)
T.Descombes (informatics)
OPERA week STRASBOURG, january 22, 2003

2. OPERA week STRASBOURG, january 22, 2003
GLOBAL DAQ SCHEME
Minutes of the cabling & installation meeting (CERN, november 28th)
(Presents : D.Autiero, S.Buontempo, K.Borer, J.Marteau,
I.Montero,U.Moser, A.Paoloni, H.Pessard, M.Spinetti, L.Stanco)
Reported in the Project Leaders board (Gran Sasso, december 6th).
Internal note circulated.
1) All the cables will be collected on the top of the detector and routed
towards the counting room placed at the top of the detector close to the entrance wall of the HALL C. The CR will contain the switches and the PCs needed for the event building and slow control. A larger terminal room for the shifters will be setup in HALL B.
2) In order to preserve the access to the PM of the target tracker and at the same time to the racks containing the electronics 2 passerelles will be added at the top of the detector along the right and left sides (total width 2 meters).
At the back of the crates a cable tray (about 50 cm width) will collect all the cables from the racks and route them to the CR.
3) Given the compatibility with the crane operations the height of the racks should not exceed 1.7 m (max. 4 crates). It was also pointed out that the voltages and the fan units of the crates should be monitored. A standard for the slow control of the crates is not defined yet (it could be the CAN bus).
OPERA week STRASBOURG, january 22, 2003

3. OPERA week STRASBOURG, january 22, 2003
GLOBAL DAQ SCHEME
OPERA week STRASBOURG, january 22, 2003

4. OPERA week STRASBOURG, january 22, 2003
GLOBAL DAQ SCHEME TT
spectrometer
Magnet power supply
RPC
racks
PT racks
TT power
supply
hubs
Control room
Cable
tray
MaPMTs 2m
0.5m
OPERA week STRASBOURG, january 22, 2003

5. OPERA week STRASBOURG, january 22, 2003
GLOBAL DAQ SCHEME
OPERA week STRASBOURG, january 22, 2003
Summary for the TT:
The total power consumption for the TT amounts to ~32 kW for the 2 super-modules.
The total number of horizontal cables routed to the control room is (31+5+1) x 2 x 2 ~150.
The different options to be fixed concern the choice of power supply units, the choice of the hubs
(or switches) at the top of the detector and/or the replacement of these repeaters by local repeaters
at the controller board level.

6. OPERA week STRASBOURG, january 22, 2003
GLOBAL DAQ SCHEME
OPERA week STRASBOURG, january 22, 2003
Summary for the PT:
The total power consumption for the TT amounts to ~30 kW for the 2 super-modules.
The total number of horizontal cables routed to the control room is (11) x 2 = 22.
The slow control interface to the central DAQ remains to be defined.

7. OPERA week STRASBOURG, january 22, 2003
GLOBAL DAQ SCHEME
OPERA week STRASBOURG, january 22, 2003
Summary for the RPC Tracker:
The total power consumption for the TT amounts to ~20 kW for the 2 super-modules.
The total number of horizontal cables routed to the control room is (36 x 2) = 72.
The slow control interface to the central DAQ remains to be defined.

8. OPERA week STRASBOURG, january 22, 2003
RPC DAQ SCHEME
Meeting in Naples (december, 19th –20th) on RPC electronics & DAQ
( L.Stanco’s talk in he afternoon session).
Long discussion under way to proceed for controller and trigger.
The final decision can be summarized as:
- There will be a single controller card for each RPC plane
- On each controller card there will be the timestamp
- Naples will develop a new controller card with LVDS system
- The controller card will allocate also the "mezzanine" for DAQ
- A trigger card can be developed in a next time to operate
in a "parallel" system with the DAQ system of OPERA.
This "parallel" DAQ will allow debugging and data taking
before the foreseen start up of the experiment in 2006.
- Even if Naples is really willing to develop the trigger
card, its available future human resources are not clear yet.
At present, group candidates to develop the trigger card
are: Naples, Gran Sasso and Bologna.
OPERA week STRASBOURG, january 22, 2003

9. GENERIC CONTROLLER BOARD
Clock unit
External clock
EPLD
(clock decoder)
Commands
Synchro
orders
(optional for delay measurement &
ajustement)
clock
trigger
Clock shift
Reset
Readout
signal
FPGA
Processor
core with Ethernet
Interface
FIFO
OPERA week STRASBOURG, january 22, 2003
F/E readout
Ethernet
JTAG
Validation input
Ethernet controller
F/E controller
Power
unit
Digital Power
5V 3.3V
Analog Power
0-5V
Power supply
0-8V

10. GENERIC CONTROLLER BOARD : BFOOT FAMILY
BFOOT version 2 validated !
( see demonstration)
OPERA week STRASBOURG, january 22, 2003
Vers.2
Vers.1

11. GENERIC CONTROLLER BOARD : BFOOT FAMILY
OPERA week STRASBOURG, january 22, 2003

12. OPERA week STRASBOURG, january 22, 2003

13. ETRAX version 1 PROTOTYPE
VHDL simulation model of ETRAX bus + FPGA + FIFO version 0.1
VHDL RTL (Register Transfer Level) design version 0.1
ETRAX bus interface
Front-End Control and configuration registers
Readout sequencer
Time Stamp interface
Serial DAC interface
Schematic version 0.1
Connector 120 pins 1.27 pitch
FPGA ALTERA APEX 20KE BGA 324 pins (low cost FPGA Cyclone EP1C6 will be used)
FIFO IDT 128K x 32 bits
MCM ETRAX 100 LX with 8 MB SRAM and 2 MB flash EEPROM
Mezzanine board under layout
We hope to have a prototype in February
We hope to have a first prototype completely tested in June or July
OPERA week STRASBOURG, january 22, 2003

14. ETRAX version 1 PROTOTYPE
Ethernet
FPGA
ALTERA
APEX 20KE
BGA 324
FIFO
IDT
32 x 128 K
ETRAX 100 LX
Fash
EEPROM
2M Bytes
SRAM
8 MBytes
Tranceiver
Fifo control
Fifo data In
Data Bus
Address Bus
Flash Load control
JTAG FPGA configuration
Front End
control
Reference Clock 16 32
MCM control
+3.3V
0 V
PPS / Cycle Incr
Raz Cycle
Time Control Serial Link
irq
RS232 Debug
External Reset
Extern Calibration Pulse
ADC_A
OPERA week STRASBOURG, january 22, 2003
ADC_B

15. ETRAX version 1 PROTOTYPE
Address
Decoder
Front-end
Control
Gain /
disable triggers
Cycle Counter
PPS/Cycle Time Stamp
Dac Control
Threshold
High voltage
Led pulser
Slow control ADC
High Voltage mon.
Readout
Sequencer.
Data bus In
Address Bus
Fifo Rd Enb
Fifo Out Enb
Register
Select
Trigger Time Stamp
Extern Trigger
Time Stamp
Fifo
Data In
ADC
Front end
channel
selection
Chip select Wr Rd
IRQ control
irq
Fifo flags
Event cycle
counter
Fifo Wr Enb
Fifo configuration
Clock
Distribution
Serial interface
Node
coordinate
PLL
DataRqst
DataIn
Clk Ref GPS
100 MHz
50 MHz
Incr
cycle
Full Cycle
Flag
Event load
Cycle number
New cycle
Data Dac
High voltage ADC
Load front-end
config rqst
Load Dac rqst
Sequencer config reg
Channel
Set Chan rqst
type
hold
Front-end Control (clk,Din)
Dac serial link
ADC serial Link
Raz
Cycle
Extern
trig
Led Pulse
Test
Pulse
trigger
reg
Control and
configuration
registers
OPERA week STRASBOURG, january 22, 2003

16. ETRAX version 1 PROTOTYPE
Hold (ADC)
ADC Rqst
Trigger
Detect
Main sequencer / data load and format
ADC Done
Load PPS
Trigger Src
Intern Trig
Wait event
PPS Rqst
PPS / Cycle
Time Stamp
Incr Cycle/PPS
Load Ext
Time
Stamp
Load Front
End Trig
Time
PPS done
Ext Rqst
Extern Trig
Extern Trigger
Time Stamp
Get
ADC
Ext Done
FE Rqst
Front-end
Trigger
Time stamp
Front-End Trig
OPERA week STRASBOURG, january 22, 2003
Zero
Supp
FE Done
Current channel
ADC (after zero supp)
Current cycle
Mux data
Data Type
Incr
Channel
Channel = 63
Load ADC
type
Channel/cycle
Data 16 bits
Fifo Data In
Fifo Wr Enb
Fifo Full

17. Physical view ETRAX version 1 PROTOTYPE
6 cm 1 2
119
120
………………
FPGA
EP20KEFC324
EPC2
120 pins
( 4 x 30 )
1.27 pitch
OPERA week STRASBOURG, january 22, 2003
…………………….
… ….
6 cm
MCM
IDT72V39110L10 59 60 61 62

18. CLOCK DISTRIBUTION SYSTEM
Node card i
SM1
SM2
Master card 0
MLVDS
MLVDS
O/E converter
OPERA week STRASBOURG, january 22, 2003
1:2
splitter
General features :
bi-directionnality (propagation delays measurements & correction)
dual level chain (master cards & node cards) using MLVDS electrical link
commands are encoded into the clock
the distribution line has to be 100% efficient
Optical fiber
Master clock
PCI card +
ESAT adapter

19. CLOCK DISTRIBUTION SYSTEM
Master card
OPERA week STRASBOURG, january 22, 2003
Node card

20. PCI card architecture Hot Link TX 923 APEX 20KE PLX 9080 Hot RX1 Link
Inputs from GPS receiver
(pps, 10Mhz, analog Irig B, digital Irig B)
10Mhz
5.10E-11
O/E
Converter
Must be defined
Optical fiber
from the master
clock
Master clock date
EPC2
Hot
Link
923 TX
Optical fiber
to the O/E
converter
Local
bus
APEX
20KE
PECL
PLX
9080
OPERA week STRASBOURG, january 22, 2003
Hot
Link
933
RX1
Optical fiber
From the SM1
RX2
Optical fiber
From the SM2
EEPROM
To the station

21. PCI #2 card architecture
pps
10Mhz
Irig B
gets the GPS signals (standard : PPS, 10MHz, irig-b or non standard)
could be plugged on the PCI bus (supplies)
10Mhz
5.10E-11
Master clock date
EPC2
Hot
Link
923
Local
bus
APEX
20KE
OPERA week STRASBOURG, january 22, 2003
PLX
9080
Hot
Link
933
EEPROM

22. OPERA week STRASBOURG, january 22, 2003
PRESENT STATUS
ETRAX controller board :
Design is completed (soft & hard), routing under achievement
Prototype available for tests in February 2003
Full R/O chain by June 2003
CLOCK distribution system :
Prototypes of master cards + 8 receivers cards under construction
Tests (MLVDS, EPLD…)  march 2003
// developments of the PCI GPS receiver boards (ESAT coll.?)
CORBA event building (see T.Descombes talk) :
Event display produced (JAVA applet)
GUI on the form of a WEB page (for acquisition, monitoring, slow control, event display…)
¿ trigger strategy ?
New tests on network load (4 more ETRAX E.B. ordered)
OPERA week STRASBOURG, january 22, 2003

23. OPERA week STRASBOURG, january 22, 2003
PRESENT STATUS
« Cabling & installation procedure : baseline proposal », D.Autiero, J.Marteau
« General presentation of the OPERA event builder »,
T.Descombes
« Ethernet DAQ core mezzanine. User manual : draft », C.Girerd soon on the web « perhaps one day you will join us… »
OPERA week STRASBOURG, january 22, 2003