1. Task 10.3: Remote Visualization / Luigi Calori

2. Remote Visualization concepts
Remote Visualization tecniques address the problem of providing system and applications enabling remote users to interactively visualize large data sets residing on centralized data infrastructure .
Data could be collected and shared centrally or produced on HPC systems by numerical simulations codes.
remote users could possibly connecting from home (low speed WAN networks).
The traditional approach to visualizatin has been to transfer the remote data to a local workstation and then run locally a visualization application. This approach has several drawbacks:
The local system has to be powerful enough to handle data
The latency is concentrated into the initial data transfer step
The need of explicit complete data transfer prevent tight coupling between simulation and visualization applications ( steering) 2 2

3. Remote Visualization approaches
Remote Visualization solutions mainly differentiate by which kind of information travel thru the wire and weather the solution is application transparent. 3 3

4. Survey activity
From the former table, only the colored options have been surveyed.
Transparent filesystem mount through sshfs on Wan line is quite unstable and anyway is not specific to Remote Visualization but to general data access.
Regarding applications able to be deployed as client/server, such as ParaView, Visit, StarCCM, they have been scouted but not tested thorougly, as the deployment is somewhat complicated as the two sides, one on the HPC platform and the other on the user workstation must be usually kept in sync and deployed separately.
Focus on application neutral, VNC-like systems 4 4

5. using high performance machine
Perform scientific visualization on large amounts of data produced on HPC systems
without moving data
using high performance machine
Data
GPU
Rendering
2D images
Data
GPU?
Rendering

6. Survey activity: VNC
A VNC system consists of a client, a server, and a communication protocol
The VNC server is the program on the machine that shares its screen. The server passively allows the client to take control of it.
The VNC client (or viewer) is the program that watches, controls, and interacts with the server. The client controls the server.
The VNC protocol (RFB) is very simple, based on one graphic primitive from server to client ("Put a rectangle of pixel data at the specified X,Y position") and event messages from client to server.
The server receive from client input events and dispatch them to running applications, it also control which parts of the screen have been updated by the running apps, grab them from the Framebuffer, compress them and provide them to the client(s).
More than one client can connect to the same server sharing the framebuffer and controlling applications (desktop sharing) by each one providing input control events. 6 6

7. Survey activity: VNC Different VNC implementations Thin clients
Commercial
RealVNC:
TightVNC:
UltraVNC:
TeamViewer:
Open Source
VirtualGL:
TurboVNC:
TigerVNC:
Html5
noVNC:
Guacamole: 7 7

8. Survey activity: Other proprietary solutions
Nvidia (projet Monterey):
Promise very low latency
Support GPU virtualization
Support OpenGl, DirectX,Cuda
Integrates with virtualiaztion vendors Citrix,Vmware
First support for Windows OS
Integrated appliance:
targets also remote cloud gaming market:
PCoIP (Teradici):
Completely transparent to applications and Operating System
Low est latency
High end system (require additional hw)
1:1 connection: only one client per workstation
Nice DCV: 8 8

9. Survey of partner visualization services
Results of scouting activities seems to indicate that:
VirtualGL / TurboVNC sw stack is currently a common building block for remote visualization services supporting also wide area networks Has been adopted by
CINECA (integrated into Remote Connection Manger pilot)
SARA (used for high resolution remote connections)
RZG
PCoIP (TeraDici) has been adopted and deployed at LUND as an High end solution for campus wide fully transparent, OS-independent Remote visualization stack
Other system like NICE DCV are still used for specific users /applications (Cineca ) 9 9

10. Evaluation of partner visualization services
In order to evaluate remote visualization services:
Remote test accounts to other partners has been granted to allow cross-testing and qualitative evaluation of the whole service
Some repeatable test sequence is under definition to estimate quantitative indicators correlated to prceived performance for VNC stacks ( real frame-rate)
Other procedures for latency measurement, suited also for HW based systems as PCoIP is under evaluation. 10 10

11. Dissemination:
2012
In june , Bologna key parners of Task took part in the Cineca Scientific Visualization School ( organized within Prace framework)
RZG introduced Visit visualization framework and presented theri visualitation experiences based on it
SARA presenetd VirtualGL remote visualization stack
CINECA presented the VirtualGL / TurboVNC deployment services recently deployed that has been used during the school lab session
2013
SARA
Internal SURFsara workshop on visualization, Amsterdam, May 20th, 2013
Several Collaboratorium demonstrations for external parties, in which the remote visualization cluster was promoted
CINECA
10-14 june, Milan Cineca Scientific Visualization summer school, presentation of RCM:
Cientific 11 11

12. Pilot projects: Remote Connection Manager
CINECA was willing to deploy its VirtualGL - TurboVNC based visualization service using a GPU-enabled computing cluster.
this kind of deployment require that the visualization nodes are configured and administered as computing nodes :
No inbound connection allowed from internet
No interactive (ssh) access allowed
Computing resources handled by job scheduler (PBS) 12 12

13. Remote Connection Manager cont.
CINECA deployment consist of some compute nodes which have the X11-GLX server enabled and that are assigned to specificically defined visualization PBS queues: in order to facilitate access:
A python based server side script takes care of simplifying vncserver job creation,inspection and removal
A cross platform, client side, python based GUI handle all client side operations related to connection management.
This client side python application has been packaged as a single executable including TurboVNC client 13 13

14. NODES:
2 PLX compute nodes (96, 97) (no inbound connection):
Processors: 2 six-cores Intel(R) Xeon(R) CPU E GHz per node
GPU: 2 NVIDIA Tesla M2070Q per node
RAM: 48 GB/node
2 RVN (01,05) (inbound connection and login allowed):
Processors: 2 Quad-core Intel(R) Xeon(R) CPU X GHz per node
GPU: 2 NVIDIA QuadroPlex 2200 S4
RAM: 128 GB/node
The infrastructure is scalable: more nodes can be added
Common home and scratch filesystem mounted by login node, compute nodes and RVN nodes

15. Remote Connection Manager cont.
The setup of VNC connection from user desktop / laptop to the compute node running the VNC server job results to be quite cumbersome (Job submission + ssh tunneling on cluster login node) 15 15

16. RCM: login

17. RCM: display info

18. RCM: new display

19. RCM: new display

20. RCM: connect & kill

21. RCM: upgrades
RCM has been upgraded to use the new versions of the VirtualGL / TurboVNC stack
RCM has also been ported to Fermi Tier 0 Blue Gene Q front end nodes to allow for limited (no OpenGL) interactive post-processing (LoadLeveler support)
RCM has also be extended to allow direct connection (ssh, no scheduler) to support industry customers who want complete control on the viz node 21 21

22. Viz Software on PLX Paraview 3.14 - http://www.paraview.org/
Blender -
Vaa3D -
Tecplot (under license) -
StarCCM+ (under license)-
Ansys (under license) -

23. References
RCM user documentation and download page:
RCM code base:
VirtualGL:
TurboVNC:
the link to the user documentation of the RCM, which contains also the link to download page.
The to ask for service activation of your user and to report problems.
Some references; in particular:

24. Challenges: Data movement
How to visualize large data produced in exascale Tier 0 machines?
File system sharing between visualization cluster and Tier 0 machines?
In situ visualization: how to integrate visualization / postprocessing into simulation code? 24 24