1. Technion – Israel Institute of Technology Department of Electrical Engineering High Speed Digital Systems Lab Part A Presentation System Design Performed by: Roni Ruach Yoni Tzur Supervised by: Evgeny Fiksman Triple Processor Redundancy System

2. Presentation Contents Background High Level Design Processors Synchronizer Voter Error Correction Interrupts Image Storage Testbench

3. Background Computer system reliability is never perfect: Electromagnetic noise effects the signals Extreme physical conditions Murphy ’ s Law One solution approach is to take an odd number of components (three), give them the same job and use the result most components agree on.

4. High Level Design Proc 1 Proc 2 Proc 3 SYNCHRONIZER FIFO 2 FIFO 3 VOTER ECU ErrorOn1 ErrorOn2 ErrorOn3 Valid1 Valid2 Valid3 WatchdogTester BusLock Monitor Interrupt signals Reset signals FIFO 1

5. Processors INPUT P_Clk - same clock for all processors RESET signal from ECU Interrupts via the Int. Controller OUTPUT D_Out to the Synchronizer thru OPB, can be blocked by FIFO INPUT-OUTPUT Code & data memory thru LMB Image Storage thru OPB FUNCTION Execute program from RAM Write output data to Synchronizer Interrupt Service Routines for error correction using the Image Storage (see Interrupts) Synchronizer Microblaze Image Storage OPB LMB Memory Interrupt Controller Report Realign Reset P_Clk

6. Synchronizer – FIFO & WD INPUT: D_In from proc’s data bus OUTPUT D_Out pass the data to the Voter Valid signal invalid words due to Watchdog Timeout V_Enable signals the Voter to read the next set of words to compare BusLock when FIFO full FUNCTION FIFOs hold data during burst writes by the procs, serving the Voter with ordered triplets Watchdog handles excess delay Self-lock input bus before overflow Input controlled by P_Clk, output controlled by V_Clk Watchdog FIFO 1 E FIFO 2 E FIFO 3 E Valid 1 Valid 2 Valid 3 Full D_In1 D_In2 D_In3 V_Enable D_Out1 D_Out2 D_Out3

7. Synchronizer Description Each processor writes to it’s FIFO via the OPB bus. The data from the 3 FIFOs is read by the Voter unit, 3 parallel words at a time so they can be compared. Watchdog: if 1 or 2 of the FIFOs are empty while others fill up, signal invalid word(s) to the Voter. Algorithm: count time starting when 1st FIFO goes from Empty to non-Empty state. After WD_MaxDelay time every FIFO that is Empty is signaled not Valid. Bus Lock: if a FIFO fills up, it uses the AlmostFull signal to lock the OPB bus and prevent overflow. FIFOs must be Bus Masters with higher priority than the processors. Algorithm: AF raises OPB_Mrequest, when OPB_Mgrant is received, OPB_busLock is raised and locks the bus.

8. Voter Unit INPUT: D_In from Synchronizer Valid signal invalid words OUTPUT D_Out pass valid data to the external output OPB ErrorOn signal which FIFO delivered different/invalid data word FUNCTION Do Majority Vote between the 3 words, passes correct word to external OPB and signals errors Compare and Select D_In1 D_In2 D_In3 ErrorOn1 ErrorOn2 ErrorOn3 External Output OPB Valid1 Valid2 Valid3

9. Voter Description The compare between the three words is done by combinatorial logic gates. We can upgrade the system to 16 or 32 bit words by using shift- registers. The voter is an a-synchronic unit between two synchronic units The v_clock is used to get new word from the synchronizer FIFO and also to make sure that the ECU is in the right state.

10. Error Correction – State Machine C1=C2=C3=0 Out=>bus C1=1 Out=>bus C2=1 Out=>bus Ci=3 Process 2 FIFOs C3=1 Out=>bus ErrorOn1 3:0 ErrorOn1 Ci=2 Realign (Interrupt) 3:0 ErrorOn3 ErrorOn2 3:0 ErrorOn2 ErrorOn3 ErrorOn2 3:0 ErrorOn1 ErrorOn2 ErrorOn3 2:0 RESET 1:1:1 1:1 1:0:0, 1:1:1 3:0

11. Error Correction Bad Processor Realign The ECU reads Voter results on rising edge of V_Clk In a 2:1 situation, the 2 “good” procs agree on the output data, and the “bad” proc outputs a different value (or invalid) The good procs are sent the Report interrupt (see Interrupts) They stop and save their register image in the Image Storage The Image Storage compares the two images. In case of a mismatch, all processors are reset The bad proc is sent the Realign interrupt. It stops, clears it’s FIFO, and loads the register and memory image from the Image Storage. When the bad proc is ready it releases the good procs so they all continue to run the same code at the same time

12. Software Interrupts In case that the system is not working properly, for example :the voter gives a 2:1 result, we need to solve it. The first and light solution is to execute an interrupt that will realign the bad processor and flush his FIFO. If it ’ s not enough we will have to RESET the system. The system is using two kinds of interrupts: Hold & Report, Realign.

13. Report Interrupt Report: The two “ good ” processors execute this ISR(interrupt service routine) the usual program holds and the two processors “ reports ” -they put their register image & stack register into another FIFO. We compare between the two FIFO ’ s-if it ’ s not equal we RESET the system. Pseudo-Code: Mov R1,Im_FIFO Mov R2,Im_FIFO ……. Hold

14. Realign Interrupt Realign: The bad processor reads the data from the FIFO after we assure its o.k. He flushes his normal FIFO and bring the three processors back to work in c1=c2=c3=0 state just after the three FIFO’s are empty Pseudo-Code Mov Im_FIFO,R1 Mov Im_FIFO,R2 ……. When(good FIFO’s ==Empty) Release all procs

15. Image Storage In order to bring the system back to usual work with three processors after a 2:1 situation we need to copy the “ good ” Image from the “ good ” processors to the one that fails. In a case of fault we keep in a special two FIFO ’ s the image of the “ good ” processors, compare between them and copy it to the “ bad ” processor.

16. Test Bench The Testbench includes the Tester unit and selectors, and the Monitor logic. The Tester unit will be implemented with a 4 th PowerPC processor. It enables the user to create error events in a pre- determined file or by manual control. These error events cause 1 or more selectors to intercept the original processor’s data, replace it with different data or delay it to create a Watchdog Timeout. The Monitor takes data from the external OPB as well as other signals in the system and displays them on the Virtex board (on LCD, LEDs or thru UART).