1. JPDPS 2003 Grid computing SGCB1 Secure Generic Connection Brokering – SGCB enhancing secure submission of grid jobs across firewalls David Front, Lorne Levinson, Morton Taragin Weizmann Institute of Science, Rehovot Miron Livny, Se-Chang Son, University of Wisconsin, Madison Itzhak Ben-Akiva, Tel Aviv University, Tel Aviv

2. JPDPS 2003 Grid computing SGCB2 Agenda The problem Requirements Architecture Performance, evaluation References

3. JPDPS 2003 Grid computing SGCB3 The problem In order to use (Grid) servers, incoming connections should be created Organization security policies restrict connections to prevent malicious acts Incoming connections are more threatening than outgoing connections Hence, organization security managers object to allow incoming connections to grid domain firewalls

4. JPDPS 2003 Grid computing SGCB4 The problem: 2 use cases server Server firewall client (Grid) network applications fail to create incoming connections, because of organization security policy, enforced by firewall/s In addition, client ’ s firewall prevents Connections to client. No direct connection is possible Client firewall 2 Server ’ s firewall prevents Connections to server. Server may connect client. 1

5. JPDPS 2003 Grid computing SGCB5 Requirements A solution to securely submit jobs across firewalls should: 1.Satisfy security managers: allow incoming connections, yet not violate security policies 2.Not require dynamic firewall changes 3.Support communication with standard sockets 4.Not require changes at communicating applications 5.Not require kernel changes 6.Support various security schemes 7.Not require root privilege to install/run

6. JPDPS 2003 Grid computing SGCB6 SGCB architecture Generic Connection Brokering (GCB) by Sechang Son and Miron Livny +Bypass by Douglas Thain and Miron Livny + Security layer for management messages: –Trivial security –GSI security Generic Connection Brokering (GCB)

7. JPDPS 2003 Grid computing SGCB7 GCB use case 1: reversed TCP connection broker server Server firewall client 1 Register me 2 I want to connect server 3 Connect client connect() A GCB management message data Time

8. JPDPS 2003 Grid computing SGCB8 GCB use case 2: relayed TCP connection server Server firewall client Client firewall 1 Register me 2 I want to connect server A GCB management message Connect() 3 Connect me 4 Connect me Connect() Data is relayed broker Time

9. JPDPS 2003 Grid computing SGCB9 firewalls holes without GCB Server incoming Server firewall Client outgoing Client firewall Holes for Server-client connections

10. JPDPS 2003 Grid computing SGCB10 firewalls holes with GCB Broker incoming Server outgoing Server firewall Client outgoing Client firewall Holes for management messages Holes for Server-client connections Broker firewall

11. JPDPS 2003 Grid computing SGCB11 GCB socket SW layer GCB socket calls GCB_bind(), GCB_connect(), GCB_accept()... GCB calls do whatever is needed to connect, such as: communicate with other entities reverse connection direction initiate standard socket calls: GCB calls replace standard socket calls at server and client applications bind(), connect(), accept() … Standard socket calls call

12. JPDPS 2003 Grid computing SGCB12 GCB_bind GCB_listen GCB_accept Time GCB_connect GCB layer: reversed TCP connection GCB_socket client machine broker machine server machine

13. JPDPS 2003 Grid computing SGCB13 GCB layer: relayed TCP connection Time GCB_bind GCB_listen GCB_accept GCB_connect GCB_socket

14. JPDPS 2003 Grid computing SGCB14 SGCB architecture Applications must call GCB socket functions in order to use GCB Using Bypass avoids this need

15. JPDPS 2003 Grid computing SGCB15 Bypass accept() 2 Application issues a system call GCB_accept() 3 Agent intercepts, and runs agent code 4 For example: call connect() connect() 1Agent squeezes in between application and system calls call Bypass is a code generator software, for making C++ interposition agents.

16. JPDPS 2003 Grid computing SGCB16 Bypassing GCB Client Server Application connect accept socket bind listen broker GCB_socket GCB_bind GCB_listen GCB_accept GCB_connect Agent GCB Bypass agent implements GCB seamlessly to applications Time

17. JPDPS 2003 Grid computing SGCB17 SGCB architecture GCB management messages are not secure: SGCB Security layer adds security

18. JPDPS 2003 Grid computing SGCB18 GCB: No management message security Server/Client Broker Accept Data Connect Data Time

19. JPDPS 2003 Grid computing SGCB19 SGCB security scheme 1: trivial security Server/Client Broker Data Accept Data Connect Applicable for management messages AUTH_assert userid@hostname AUTH_accept userid@hostname Time

20. JPDPS 2003 Grid computing SGCB20 SGCB security scheme 2: GSI security Server/Client Broker Data Accept Data Connect Applicable for GCB management messages AUTH_assert certificate AUTH_accept certificate encrypt decrypt encryptdecrypt Time

21. JPDPS 2003 Grid computing SGCB21 Broker location Broker incoming Server outgoing Client outgoing Broker DMZ A brokers has relaxed security policy, allowing incoming connections It is recommended to locate a broker at a DMZ with no other computers

22. JPDPS 2003 Grid computing SGCB22 GCB TCP performance test passing across a private network Relayed connection Reversed connection Time [msec] of 1030 Connection avg. time 11020 Data (echo) avg. time GCB does not coerce a big time penalty

23. JPDPS 2003 Grid computing SGCB23 Evaluation SGCB does satisfy its requirements, however: Scalability: The broker is a potential traffic bottleneck. Brokering of up to thousands machines, yet to be tested Robustness: The broker is a single point of failure Complexity: Adding SGCB and Bypass SW layers to a grid application adds complexity and causes a debugging challenge Experimental: Globus problems with GCB and bypass Applicability: SGCB is relevant for GT2 more than for GT3

24. JPDPS 2003 Grid computing SGCB24 References Globus Toolkit Firewall Requirements: Von Welch http://www-fp.globus.org/security/firewalls/Globus%20Firewall%20Requirements-5.pdf http://www-fp.globus.org/security/firewalls/Globus%20Firewall%20Requirements-5.pdf GCB: Recovering Internet Symmetry in Distributed Computing, Sechang Son and Miron Livny, Computer Science Department, University of Wisconsin http://www.cs.wisc.edu/condor/doc/CCGRID2003.pdfhttp://www.cs.wisc.edu/condor/doc/CCGRID2003.pdf Bypass: Douglas Thain and Miron Livny http://www.cs.wisc.edu/condor/bypass/http://www.cs.wisc.edu/condor/bypass/ SGCB user guide: David Front www.weizmann.ac.il/~dfront/sgcb.htmwww.weizmann.ac.il/~dfront/sgcb.htm SSH tunnels and Globus (alternative attitude to connect across firewalls): Globus Grid and Firewalls: Issues and Solutions in a Utility Data Center Environment1, Sven Graupner, Carsten Reimann, HP Laboratories Palo Alto, HPL-2002-278, October 2nd, 2002 http://www.hpl.hp.com/techreports/2002/HPL-2002-278.pdf http://www.hpl.hp.com/techreports/2002/HPL-2002-278.pdf

25. JPDPS 2003 Grid computing SGCB25 Spare slides

26. JPDPS 2003 Grid computing SGCB26 Status SGCB works with test applications: creates Bypass’ed connections across firewalls with trivial or GSI security. SGCB support for Globus is under development

27. JPDPS 2003 Grid computing SGCB27 Web services, GT2/3 and SGCB Web servicesNon Web services All services use port 8080Different services use different (possibly multiple) ports. ports Only if firewall understands XML According to port numberFirewall control over services GT3GT2 Mixture: Some services are web services and some are not NoWeb services? (8080,) 2811, (22,) 7512 + ephemeral ports range, controllable by GLOBUS_TCP_PORT_RANGE 2219, 2135, 2811, (22,) 7512 + ephemeral ports range, controllable by GLOBUS_TCP_PORT_RANGE Server ports used The non web servicesAll servicesSGCB is relevant for

28. JPDPS 2003 Grid computing SGCB28 GCB Sechang Son and Miron Livny’s GCB architecture GCB allows Condor to seamlessly work across private networks and over firewalls SGCB is based on and enhances GCB, adding a security layer * Mainly for presentation simplification reasons, GCB entity names is slightly changed in following slides

29. JPDPS 2003 Grid computing SGCB29 SGCB Firewall settings Broker machine Allow creation of incoming connections, from legal machines: GCB connections: Any port to be brokered by GCB, from clients and servers. Management connections to GCB management port: Internal server machines: BROKER_LOCAL_PORT, 65430. External client machines: BROKER_ PUBLIC _PORT, 65432. Server machine Allow creation of outgoing connections to legal machines: GCB connections: Any server ports, towards the broker and possibly to clients. Management connections: BROKER_LOCAL_PORT, 65430 towards the broker. Client machine Allow creation of outgoing connections towards the broker and possibly from servers: GCB connections: Any client ports, towards the broker, and possibly from servers. Management connections: BROKER_ PUBLIC _PORT, 65432 towards the broker.

30. JPDPS 2003 Grid computing SGCB30 Standard connection-creation ClientServer 1)bind() port to socket listen(), wait for connect() 2) Connect() to ip:port 3) accept() connection ClientServer 0) No connection Connection established: both parties may communicate Time

31. JPDPS 2003 Grid computing SGCB31 Bypass Bypass, a research project of Douglas Thain and Miron Livny,Douglas ThainMiron Livny is a compiler like, code generator software, for making C++ interposition agents. An interposition agent is software that squeezes itself into an existing program, between the program and the operating system, and transforms the program: When the program attempts certain system calls, the agent grabs control and runs the agent code, as supplied by the programmer. Agents can be used to emulate operations that otherwise might not be available.

32. JPDPS 2003 Grid computing SGCB32 Using Bypass Write an agent specification. Partial example: calls GCB_accept() when accept() is called int accept(int fd, struct sockaddr * peer, socklen_t *addrlen) agent_action {{ return GCB_accept(fd, peer, addrlen); }}; Compile and dynamically link the agent Bypassed application should be dynamically linked. Activate agent at user session: Export LD_PRELOAD=

33. JPDPS 2003 Grid computing SGCB33 Bypassing GCB GCB socket calls GCB_bind(), GCB_connect(), GCB_accept()... GCB calls do whatever is needed to create connection, such as: communicate with other entities reverse connection creation direction initiate standard socket calls: Application does not have to be changed in order to use GCB

34. JPDPS 2003 Grid computing SGCB34 SGCB Security schemes GCB created connections security Applications that use GBC to create connections are responsible for the security of those connections. GCB management messages security The security of the underlying GCB management messages between applications and the broker is: –independent to the security of the GCB connections –under the responsibility of GCB –determined at GCB configuration time –may be one of the following: 1. GSI, 2. Trivial, 3. none.

35. JPDPS 2003 Grid computing SGCB35 AUTH security layer SGCB uses AUTH simple (generic) security layer for authentication, based on similar code from Bypass. Auth consists of the following functions: Auth_register: An application that implements AUTH registers at startup to the security mechanism/s that it supports. (Currently,) the security mechanism may be ‘GSI’ or ‘trivial’. Auth_assert: Called by clients that wishes to authenticate itself towards a server. After AUTH negotiates with the other party for the best protocol supported by both, the selected specific assert function is called to perform authentication. Auth_accept: Counter function of Auth_assert. It grants or does not grant authentication, via negotiated protocol accept function.

36. JPDPS 2003 Grid computing SGCB36 SGCB management messages security schemes 1.GSI security GSI is the de-facto security basis of (Globus based) grid systems. If you use GCB for an application that has GSI infrastructure, this is the recommended scheme to use. 2.trivial security Trivial security uses a simple user repository of the form @, residing at the broker, to authenticate SGCB users. 3.no security Applications that use GCB to create connections must use the same security scheme at both ends of a connection SGCB applications (as opposed to GCB) negotiate the security scheme per connection request

37. JPDPS 2003 Grid computing SGCB37 Related work

38. JPDPS 2003 Grid computing SGCB38 ssh tunneling A secure means to pass 2 firewalls ssh is an open source, accepted security standard, available for unix machines. ssh supports certificates, and GSI-enabled ssh programmatically connects ssh and GSI. Once an ssh connection is made across a firewall, channels may be opened by it, to various ports, from both directions, without the firewall being aware of this because the traffic within the tunnel is encrypted

39. JPDPS 2003 Grid computing SGCB39 Globus Grid and Firewalls: Issues and Solutions.. Sven Graupner, Carsten Reimann, HP Laboratories Palo Alto HP researches tested ssh tunnels as a means to submit Globus jobs Through one or two firewalls.

40. JPDPS 2003 Grid computing SGCB40 SSH - drawbacks Connection between application and sshdserver is not secure Unless you have a ‘bridge host’, there should be a hole in the server or client firewall Either all traffic is encrypted or the whole thing is not secure

41. JPDPS 2003 Grid computing SGCB41 GGF Grid specific firewall planned by OGSA

42. JPDPS 2003 Grid computing SGCB42 More approaches For a discussion of more approaches, see Sechang Son’s http://www.cs.wisc.edu/condor/doc/CCGRID2003.pdf. http://www.cs.wisc.edu/condor/doc/CCGRID2003.pdf It refers to ‘Application-specific connection brokering’, such as Napster server and Gnutella And generic IETF approaches, such as SOCKS and Realm Specific IP (RSIP), explaining there relevancy, characteristics and reasons for not using them.

43. JPDPS 2003 Grid computing SGCB43 Globus: GSI used once, to connect gatekeeper GRAM and GSI

44. JPDPS 2003 Grid computing SGCB44 SGCB firewall sGCB host The Internet The Internet sGCB listener clients sGCB listener clients sGCB connectors sGCB connectors sGCB firewall sGCB server - No out going connections - Allow incoming connections to management port, from any ip:port (or from friends ip:ports) Dynamic configuration Dis/allow incoming connections, as ordered by firewall manager relayers commands broker firewall manager Static configuration

45. JPDPS 2003 Grid computing SGCB45 GCB Performance Private to private (2 firewalls) Public to private (1 firewall, outgoing) Private to public (1 firewall, incoming) GCBRegularGCBRegularGCBRegular 7 (0.2) 1 (0.0) 34 (2.9) 20 (0.6) 34 (9.3) 2 (0.3) Conn. 108 (1.7) 104 (0.7) 19 (1.6) 22 (0.9) 28 (7.4) 23 (3.8) Data GCB TCP performance tests of connection and data (echo) time average and standard deviation (in parentheses), in msec. Private to public is similar to 1 firewall, incoming connection to server Public to private is similar to 1 firewall, outgoing connection from server Private to private is similar to 2 firewalls, at server and at client applications. Conclusions: - GCB causes connection time to be longer than regular, but still connection time is smaller than echo time - Relaying data only causes a small time penalty

46. JPDPS 2003 Grid computing SGCB46 GT2 network traffic characteristics Server ports: 2219, 2135, 2811, (22,) 7512 + ephemeral ports range, controllable by GLOBUS_TCP_PORT_RANGE

47. JPDPS 2003 Grid computing SGCB47 GT3 network traffic characteristics Server ports: (8080,) 2811, (22,) 7512 + ephemeral ports range, controllable by GLOBUS_TCP_PORT_RANGE

48. JPDPS 2003 Grid computing SGCB48 GCB broker implementation GCB broker is implemented as a daemon that can be run with the least privilege. GCB broker does not assume that it can directly talk to its clients and It can be placed anywhere both clients and public nodes can talk to it. When GCB broker receives a register request from a client, it creates a proxy socket and a record for the client. Connection requests to the client are brokered by referring the record. When a connection is accepted to a proxy socket, the broker creates a relay record and uses it to relay packets. Relaying is delegated by the broker to (one of) a (set of) local relayServer process(es). GCB broker, needs not maintain information persistent over broker restart or machine reboot. Instead it just does fresh start when it restarts. To keep the correct set of client records, GCB broker asks the client from which a heartbeat is received but whose record is not in the broker to register again.