1. Project – Video manipulator (based on Zed Board) Final presentation
Yakir Peretz
Idan Homri
Supervisor - Rolf  Hilgendorf
Semester - winter 2014
Duration - one semester

2. agenda Project goals Component description Data flow
Required tests and check points
Clock definitions
Software description
Critical issues and solutions
Complete Program (without Uart)

3. Project goals
Creating a system that enables reading images from an external device, saving it in the memory and displaying it by VGA.
Creating a programmable logic design that will handle the transportation of the data from the main memory to the VGA output via video direct mapped accessed (VDMA) component.
agenda

4. Component description -ZYNQ
In the design we use the following components:
ZYNQ processor
the ZYNQ is actually the PS (processing system) part of the design, which means all the software programmable part.
This part is very powerful and includes many features, but we use the following:
UART connection
memory controller
the memory itself - DDR3
One ARM processor –CORTEX A9
All the needed interface connections to the other components in the PL side are built in.
All the clocks of the design are generated by the zynq, and given to the relevant components.
PS side overview

5. Component description -VDMA
The VDMA is the core of the PL side of the design. It is responsible for the transportation of the data from the memory to the stream part. It is connected to three other components via three buses:
To the processor via AXI4-lite – to get data regarding the address and size of the data to get from the memory.
To the memory controller via full AXI4 – to get the data from.
To the “stream_to_video_out” via AXI4-stream – to send the data to.
the data transportation to the stream part is done with respect to the VTC timing signals.

6. Component description - VTC
Video timing controller
this component is responsible for timing the data transfer from the VDMA to the stream to video out component.
It generated signals regarding the vertical data transfer (line count) and the horizontal data transfer (pixels per line) as well as the active video signal.
It works with a clock that is set in order to fit the data size and rate of pictures per second - Clock definitions

7. Component description - stream
Stream to video out – AXI4-Stream to Video Out core converts AXI4-Stream Video protocol from Xilinx video processing cores such as VDMA, that use this protocol, to video output with explicit sync and timing such as the unit we built to communicate with the VGA port.
In our project, the unit is used to convert the output of the VDMA in AXI_stream protocol to an actual video protocol that consists of:
Active data signal
Vertical sync and horizontal sync
Blank periods
“Stream to video out” interface

8. Component description – RGB_out
this unit was built by us, to convert the data from 8 bit per color (for red green and blue) to 4 bit per color.
The output of this unit is the input of the VGA ports
RGB – 4 bits per color
Vsync
Hsync.
agenda
RGB_out

9. Data flow agenda PS side overview Step 1 :
Sending the data from an external device to the uart
(bitmap to pixels only)
PS side overview
Step 2 :
Extracting the data from the Uart and saving it to the memory
the ZYNQ processor triggers the VDMA by sending the start address and the size of the data in the memory, on an AXI_LITE.
Block Diagram
Step 3:
Step 4:
The data is being transferred to the VDMA via memory controller and saved in a frame buffer.
VDMA
Step 5:
The VDMA sends the data to the “stream to video out” unit, with respect to the VTC timing.
“Stream to video out” interface
Video timing controller
Step 6:
the data is transferred from the stream to a RGB_output component to be transferred in the right form
RGB_out
agenda

10. Required tests and check points
We have some strategic check points for validating our design:
Uart to memory – we first check that the data we delivered from an external device thru the Uart is saved in the memory where we wanted it to be saved.
Memory to VDMA – we check that the data is transferred correctly from the memory to the frame bufers inside the VDMA.
VDMA to “stream_to_video_out” – we check that the data is transferred correctly from the VDMA to the stream to video out by reading the data runs on the AXI_stream bus.
Control signals – we need to check that the “video_timing_controller” is sending the timing signals as we assumed it will.
“stream_to_video_out” to VGA – we check if the data from the “stream_to_video_out” is sent as we wanted in a 24 bit (8 bit per color and 3 colors R,G,B) format.
VGA output – we need to check that the data in the output of the VGA component is the picture we delivered. This should be displayed on the screen.
Block Diagram
agenda

11. Clock definitions
There are 2 main clocks in the design (beside the ARM clock & DDR clock)
The faster clock is used for the AXI4_lite bus that connects the ARM and the VDMA. On that bus the ARM transfers the data regarding the address and the size of the picture in the memory. the clock is set to 200MHZ
The slower clock is used for the full AXI4 bus and the AXI_stream bus. On that bus we move the data from the memory to the VDMA and then from the VDMA to the “stream to video out” unit.
That clock is defined to be mega pixels per second. That is calculated to fit the amount of data being transferred in one second, calculated as:
(number of lines including blank)*(number of pixels per line including blank)*(number of pictures per second) – for us *1125*60 = 148.5[MHz]
In order to fit to the screen in the lab we needed a 1080*1920, and there are 60 pictures per second. (the sizes represent pixels).
agenda
Component description - VTC

12. Block Diagram Data flow Required tests and check points
VDMA sub system
ZYNQ sub system
Data flow Required tests and check points

13. PS side overview Data flow Component description -ZYNQ To VDMA via
Axi lite
Data flow
To VDMA via AXI-4
Component description -ZYNQ

14. Pin to Pin

15. “Stream to video out” interface
From VDMA
To VGA out
From video timing controller
Data flow
Component description - stream

16. Video timing controller
For write channel – not in use
Optionally – can be controlled by the processor. Not in use
Output timing signals for the stream_to_video_out unit
Data flow
Component description - VTC

17. RGB_out Data – 4 bits per color Input Sync signals Output Sync signals
Data – 8 bit per color
Data flow
Component description – RGB_out

18. VDMA
Connected to the memory on a full AXI4 bus. Required for data transfer
This is the connection to the processor to get the address and the size of the picture from
Connected to the memory on an AXI4_stream bus. Required for data transfer
Those are the 3 clocks of the desine
Data flow
Component description -VDMA

19. Software description

20. Software Flowchart Board -C code SDK Host - MATLAB agenda
Rescaling of BMP Image,
open Uart for writing
and sending the Image
Matlab
Host - MATLAB
Board -C code
SDK
Vdma Configuration and Setup
Vdma Start Transfer
Load the incoming Image from Uart into DDR
HAND SHAKE PROTOCOL
agenda

21. Critical issues and solutions
Problem
Solution 1
Uart Buffer is limited and Uninitialized
uart issues 2
Stream To Video Out – Always Output 0.
Hardware – Connect reset to active low, and clocken to const 1.
Software – Parking on frames with appropriated Xilinx function. 3
SAMSUNG Screen requires format picture of -1080* [Hz]
Hardware – Configure Peripheral clock to 2200*1125*60 = 148.5[MHz] 4
Image with “Noise” - Inconsistent writing problem
Software – disable caches 5
Video Timing Controller driver isn’t loaded into SDK.
SDK loads drivers only for components that are connected directly to ZYNQ with AXI-lite.
The driver (C code) can modify the registers during running via the AXI-lite interface (like the VDMA), but driver is not always necessary. In some cases you need to define settings only during invocation of the hardware in Vivado (like the VTC).
Since there is no need for connection with the CPU, because the VTC behavior is fixed it was not connected to the AXI_LITE, and no drive is needed. 6
The output of the "stream to video out" is 8 bit per color, while VGA is 4 bit per color.
The RGB_out unit is taking the top 4 bits from each color.
agenda

22. Uart Problem Uart Solution1
Colors of the picture are completely mixed
Matlab – the same data was sent in a loop until it was read by the board, instead of sending it once and waiting for ACK. 2
The shape was almost correct but the colors were different than the original picture
The UART buffer works as a FIFO, therefore the data is saved in the memory in reverse way. 3
The picture on the screen starts with an offset.
Cleaning the FIFO of the UART at the beginning of the read operation. The function is set Options(UART, reset) 4
The transportation of the data was stack every time after a different amount of transfers.
A simple handshake protocol was created in which the matlab sends the data to the UART in a 64 byte blocks (UART’s buffer size) and waits for the acknowledge from the board. 5
Every some amount of transfers - one transfer is unsuccessful. When it happens, the picture on the screen has a shift and the colors are changed.
The problem is that if the communication problem happens due to acknowledge lost, which means that the data was receive but the matlab think it didn’t because no ack was detected, data is then resent even though it was successfully received by the board
ACKNOWLEDGE LOST (SHIFT RIGHT)
It seems like the timeout of the SDK to resend acknowledge was equal to the timeout of the matlab before sending the image again. That way each time acknowledge is lost, the data is being sent twice.
The timeout of the matlab was changed to be 10 times bigger than SDK and problem was solved. 6
The same thing happens when the communication is unsuccessful due to transmitted data lost.
DATA LOST (SHIFT LEFT)
Create a protocol that knows to differentiate the reason for the error.
In that case it should resend the data.

23. FINAL RESULTS
Image initialization
final presentation

24. SUMMERY AND CONCLUTION
The project goals were to load data from an external device and save it to the internal memory, and create a data path in which the data is transferred from the memory, and being eventually streamed aout to the VGA port in the right format.
The first goal, even though seems to be less complicated required more "mind work" in the sense of going into small details when creating a communication protocol. A lot of thought should be taken when setting the rate of the timing controller, the timeouts of the two parts of the communication and more.

25. …
The second goal of this project required a great amount of learning. Starting with the board being used, the vivado tool and the SDK tool. After that all the IPs that were mentioned here had to be completely understood in order to be configured correctly and connected correctly to the other components.
Connecting a system that contains more than a few parts is a very gentle task, and should be done with a great deal of attention to small details.
Creating such a system can be a good preparation before doing any project even outside the academy.

26. … There are a few conclusions for this project:
When getting a fixed component, one should fully understand its use before getting to work with it.
If one is "stuck" in some part of the project, it is best to overcome the problem in some way possible and move on with the work while thinking how to solve it. That way the work is in progress most of the time.
One should always use all the help he can get. Most of the time when not being able to proceed because of technical issues, the solution is found with some technical person or the supervisor.

27. APPENDIX SOFTWARE

28. Matlab
The main goal of the program is to create Matlab GUI interface between the PC and Zedboard in order to load the desired image.
Step One:
Determine the desired uart port configuration.
(8 data bit, 1 stop bit, baud rate)
Step two: Load the bitmap image into Matlab and make dimensions’ rescale: 1080 * 1920
Step Three: Open the port and send information (Hand shake protocol)
software flowchart

29. C code – SDK Load Picture into DDR
Program should read the incoming data and load it into the DDR.
At this point of the Project we encounter a technical problem – Zedboard buffer size at polling mode is 65 bytes only, so there is no option at this time to load the whole picture.
In order to continue with the development, the pictures were written manually into the DDR.
software flowchart

30. C code – SDK Vdma Configuration and Setup
Initialize DMA engine – A VDMA instance is set to VDMA Physical address
Setup the Read channel- The VDMA module use only Read Channel (mm2s). Setup of vertical and horizontal lengths, frames store start address, and other unused Registers.
software flowchart

31. C code – SDK Vdma Start Transfer
Start the DMA engine to transfer – the VDMA read channel is activated.
parking on a frame –The vdma reads the same image, in order to display image on screen Continuously.
software flowchart

32. HAND SHAKE PROTOCOL
שקופית 20

33. Inconsistent writing problem

34. Solution – disable cahches
Critical issues and solutions

35. ACKNOWLEDGE LOST (SHIFT RIGHT)
שקופית 22

36. DATA LOST (SHIFT LEFT)
שקופית 22

37. VDMA sub system
Block Diagram

38. ZYNQ sub system
Block Diagram