A Communication Relay Mechanism toward Construction of Virtual Cluster on Orverlay Network PRAGMA14 Working Group March 2008 Yasuyuki Kusumoto Osaka University, Cyber media Center

Motivation of this study It is difficult to unify resources on Grid and provide a single o ptimum computational system with a user Heterogeneous computational environment Disability on network connectivity, such as NAT, FW Requirement for virtual cluster Homogeneous computational machines virtualization technology (Xen) End to end communication among nodes of virtual clusters orverlay network(P2P framework: PIAX) siteA siteC siteB os:solaris library:2.0.1 os:redhat library:1.0.0 os:debian library:1.1.2 Grid resources Firewall NAT Heterogeneous Disconnectivity

Virtual cluster on overlay network(P2P) Our approach for virtual cluster Gathering computational machines on overlay network Deployment of virtual machines on gathered machines Providing a private network among virtual machines goal providing a customizable computational environment with a user virtual cluster Grid resources Overlay network virtualization virtual cluster Virtual network

A communication relay mechanism Private network communication among virtual machines End-to-end communication with IP regardless of NAT,FW Virtual private network over a P2P environment Relaying VPN communication on orverlay network This mechanism : delivers VPN packets on overlay network regardless of NAT, FW allows virtual machines to communicate with IP each other Overlay network relay Communication relay mechanism Virtual machine A Virtual machine B Virtual machine C Virtual machine D Virtual machine E

The communication relay mechanism architecture OrverlayNetwork VPN workerAppmasterApp VirtualNIC IP: a VirtualNIC IP: b vpnclientvpnserver PIAX ID communication IP communication Tunnel Agent Tunnel Agent hostA(worker)hostB(master) forward vpn packets by using logical ID communication End-to-end communication with IP

Performance evaluation of the proposed mechanism Communication throughput master-worker communication (bandwidth:100Mbps) Execution of distributed computation applications Dock cpu utilization: big network utilization: small Himeno Benchmark cpu utilization: big network utilization: big communication throughput[Mbps] App:Dock exaction time[sec] App:HimenoBench score[Mflops] Physical cluster Virtual cluster8.8(-90.3%)779(-0.6%)205(-72%)

Summary The proposed mechanism provides virtual private network across multiple organizations realizes a virtual cluster on a P2P environment Future works Improvement of communication throughput VPN hole-punching on overlay network On-demand construction of virtual clusters Automatic deployment of virtual machines by utilizing data transfer functions of P2P Collecting resources broadly by using the resource powerful searching ability of PIAX

fin

Summary The proposed mechanism provides virtual private network across multiple organizations realizes a virtual cluster on a P2P environment Future works Improvement of communication throughput On-demand construction of virtual clusters Automatic deployment of virtual machines by utilizing data transfer functions of P2P

Virtual cluster

Virtual cluster on overlay network(P2P) Advantages of P2P Scalability of computational resources Resource Discovery Network connectivity(solves problem of NAT, Firewall) Goal To provide a customizable computational environment (virtual cluster) with a user virtual cluster Resources of grid Overlay network virtualization

Requirements for communication among virtual cluster nodes virtualization Virtual Cluster Virtual Private Network Overlay network Private network communication among virtual machines End-to-end communication with IP regardless of NAT,FW Virtual private network over a P2P environment

A communication relay mechanism relays VPN communication on orverlay network This mechanism: delivers VPN packets on overlay network regardless of NAT, FW allows virtual machines to communicate with IP each other Overlay network relay Communication relay mechanism Virtual machine A Virtual machine B Virtual machine C Virtual machine D Virtual machine E

Motivation for study Virtual cluster on orverlay network(P2P) Scalability of computational resources Discovery for resources Virtual machine (Xen)

Virtual cluster on overlay network siteA siteB siteC Virtualization virtual cluster

Necessity for new communication mechanism Virtual Private Network Virtual Cluster Users job

Performance of proposed system Communication throughput Execution of application Dock :CPU-intensive application()

siteC siteA siteB Virtualizat ion Virtual Private Network Virtual Cluster Users job overlay network PIAX virtual machine Xen

Summary and Future work Summary Proposed system enable computers on orverlay network to communicate using IP address Realization for Construction of virtual cluster utilizing our proposed communication system Future work Improvement of communication throughput On-demand construction of virtual cluster

n overlay network1/2 siteAsiteCsiteB Virtualization Virtual Private Network Virtual Cluster Users job overlay network PIAX virtual machine Xen

Virtual cluster on overlay network siteA siteB siteC Virtualization virtual machine Xen overlay network PIAX virtual cluster

Necessity for new communication mechanism Virtual Private Network Virtual Cluster Users job

Necessity for new communication mechanism Users job virtualization Virtual Private Network Virtual Cluster Overlay network

Necessity for new communication mechanism Virtual Private Network Virtual Cluster virtualization Overlay network Users job

Virtual cluster on overlay network(P2P) Integration of virtual machines on wide area distributed environment(P2P) Scalability of computational resources Discovery for resources Goal virtual cluster Resources of grid Overlay network virtualization

Virtual network Grid resources overlay network virtual resource virtual cluster virtual resource virtual cluster