Virtual Private Grid (VPG) : A Command Shell for Utilizing Remote Machines Efficiently Kenji Kaneda, Kenjiro Taura, Akinori Yonezawa Department of Computer Science, University of Tokyo Improvement of network technology Connection of clusters and super computers with high-speed network Many people has accounts on a large number of machines Across multiple subnets and geographically distributed places Background These machines are managed by different administrators Various restrictions are imposed Firewall (IP filtering) DHCP Private IP, etc. Working around these restrictions requires human intervention It is difficult to utilize all the remote machines efficiently Problem Utilize remote machines through a shell at the local host Design and implement Virtual Private Grid Working around the restrictions automatically Utilizing a large number of machines in multiple subnets securely and efficiently Virtual Private Grid : Goals Restrictions Bypass of firewall Submission of jobs to nameless hosts A Possible Solution Keeping connections permanently between all the hosts Problem A large number of connections User involvement to minimize connections i. Nicknaming (= each host has a unique nickname) ii. Job submission to any nicknamed host iii. Redirection from/to a file at any nicknamed host iv. Network pipe between any nicknamed host Functions of VPG do not change administrative policy can combining existent programs with standard input/output through the network Job submission Redirection Pipe Syntax of @ Selection of hosts which have no DNS name with nickname Automatic detection of the route to hosts File | Local shellRemote Example : Difficulty of Job Submission Across Firewall Private IP It is cumbersome to implement remote job submission across multiple subnets

i. Daemons boot up at hosts ii. Each daemon creates and keeps necessary connections with SSH port forwarding DHCP clients create a connection to the outside Multiple subnets are connected iii. Finally, daemons finish creating connections Network becomes connected Using self stabilizing spanning tree algorithm iv. The daemon at home host keeps track of the topology of the whole network It detects the route to hosts for job submission Overview of Implementation Developing Virtual Private Grid Working around the restrictions automatically Utilizing remote machines securely and efficiently Easier and more efficient utilization of remote machines Automatic and parallel resource selection  Simple scheduler  Control of dependency relation of files  Automatic generation of executables  File sharing/sync, etc. Summary and Future Work Globus[I.Foster et al. Providing basic services for global computing RMF[Y.Tanaka et al. IWCC ’ 99] Utilizing resources inside the firewall Ufo[Albert D. Alexandrov et al. USENIX ’ 87] Providing global file system Secure Shell Providing secure access to remote machines Virtual Private Network Constructing a private network on the Internet Related Work Methods to utilize a large number of machines with SSH Giving a shell window to each host  It requires a large number of windows Creating connections whenever submitting a job  It entails high overhead Using SSH port forwarding  It requires an user involvement Advantages of VPG Minimum connections for all the hosts to communicate with each other Easy and efficient network pipe and redirection Job submission to nameless hosts Difference between VPG and SSH Host C (DHCP client) Host D (Private IP) Host B Subnet X Subnet Y Subnet Z Host A (Home host) Host C (DHCP client) Host D (Private IP) Host B Subnet X Subnet Y Subnet Z Host A (Home host) Host C (DHCP client) Host D (Private IP) Host B Subnet Y Subnet Z Subnet X Host A (Home host) Live Connection Dead Connection > File | Live Host Dead Host the host which the user initially logins VPG constructs a private network at the user level VPG places major emphasis on remote job submission Difference between VPG and VPN