1. 1 IPSec: Security at the IP Layer Rocky K. C. Chang 15 March 2007

2. Rocky, K. C. Chang2 Outline  An overview The security threats to IP The security services afforded by IPSec IP security associations and protocols  Authentication Header  Encapsulation Security Payload  A few more words

3. 3 An overview

4. Rocky, K. C. Chang4 Security problems with IP  Forge the IP addresses. Source authentication problem  Modify IP datagram’s content. Data integrity problem  Replay old IP datagrams. Sequence integrity problem Used for replay attack, a type of denial-of-service attack  Inspect IP datagram’s content. Data confidentiality problem

5. Rocky, K. C. Chang5 Security services  IPSec provides security services between a pair of hosts or a pair of security gateways or between a host and a security gateway on the IP level. A security gateway is an intermediate system that implements IPSec, e.g. router or firewall. For both IPv4 and IPv6.  Security services: Data origin authentication: verify the identity of the claimed source of data.  This service is usually bundled with connectionless integrity service. Integrity: ensure that modifications of data are detectable.  Connectionless integrity: detect modification of an individual IP datagram, without regard of the packet ordering.

6. Rocky, K. C. Chang6 Security services  Partial sequence integrity (or anti-replay integrity): detect arrival of duplicate IP datagrams within a constrained window. Confidentiality (encryption): protects data from unauthorized disclosure.  Data confidentiality: conceal application data.  Limited traffic flow confidentiality: conceal source and destination addresses, message length, or frequency of communication. Access control to  a host’s data and/or computing cycles  network or network resources behind a security gateway.

7. Rocky, K. C. Chang7 Two main security protocols  The IP Authentication Header (AH) in RFC 2402 provides data origin authentication, connectionless message integrity, and an optional anti-replay protection.  The Encapsulating Security Payload (ESP) protocol in RFC 2406 provides all the services provided by AH, plus data confidentiality, and limited traffic flow confidentiality.  These two protocols can be used individually or together to provide different security services.  Why two?

8. Rocky, K. C. Chang8 Two main security protocols  ESP (transport mode)  AH (transport mode) IP header ESP header ESP trailer upper layer data encrypted authenticated IP header AH header upper layer data authenticated

9. Rocky, K. C. Chang9 Two modes: transport and tunnel  Each protocol, AH or ESP, supports two modes of use: transport and tunnel. In transport mode, the protocols provide protection primarily for upper layer protocols. In tunnel mode, the protocols are applied for tunneled IP packets.  Transport mode applies to host-to-host IPSec, while tunnel mode applies to host- to-gateway, gateway-to-host, and gateway-to-gateway.

10. Rocky, K. C. Chang10 Two modes: transport and tunnel  For tunnel mode: ESP AH Outer IP header ESP header ESP trailer upper layer data encrypted authenticated Outer IP header AH header upper layer data authenticated Inner IP header

11. Rocky, K. C. Chang11 Security associations in IPSec  Before two nodes can use IPSec to protect their traffic, a security association must be established between them.  A security association (SA) is a simplex “connection” that affords security services to the traffic carried by it. To secure bi-directional communications between two nodes, two SAs are required. SAs are security protocol specific, i.e., each SA is associated with either AH or ESP, but not both.  An SA is uniquely identified by a triple: A security parameter index (SPI), Destination’s IP address (unicast, multicast, broadcast), and A security protocol identifier (AH or ESP).

12. Rocky, K. C. Chang12 Security associations in IPSec  Transport mode and tunnel mode SAs: A transport mode SA: An SA between two hosts A tunnel mode SA: An SA applied to an IP tunnel  When either end is a security gateway, the SA must be tunnel mode.  Two hosts may also establish a tunnel mode SA between themselves.  SA management: Creation of SAs: manually or automatically through a key exchange protocol, such as IKE. Management of SAs Deletion of SAs  Nested SAs

13. Rocky, K. C. Chang13 Security policies and selectors  A Security Policy Database (SPD) specifies what services to be offered to IP datagrams and in what fashion. The SPD contains an ordered list of policy entries. Each entry is keyed by one or more selectors that define the set of IP traffic encompassed by this policy entry. Each entry includes also an indication of whether traffic matching this policy will be bypassed, discarded, or subject to IPSec processing. In the last case, the entry includes an SA (or SA bundle) specification, listing the IPSec protocols, modes, and algorithms to be employed.

14. Rocky, K. C. Chang14 Security policies and selectors  Example one: Two security gateways in 140.10.0.0/16 and 142.89.0.0/16 may form an opaque tunnel between them, forming a VPN. The security gateway in 140.10.0.0/16 may have this policy entry:  Source address: 140.10.0.0/16  Destination address: 142.89.0.0/16  Protocol: any  Ports: any  Algorithms: tunnel ESP with 3DES with tunnel destination equal to the other security gateway’s external IP address.

15. Rocky, K. C. Chang15 Security policies and selectors  Example two: Two hosts with IP addresses 140.10.1.1 and 142.89.1.1 require authentication services between them. The host with address 140.10.1.1 may have this policy entry:  Source address: 140.10.1.1  Destination address: 142.89.1.1  Protocol: any  Ports: any  Algorithms: transport AH with HMAC MD5

16. Rocky, K. C. Chang16 Security policies and selectors  Valid selectors: Source and destination IP addresses Names, such as DNS name, X.500 Distinguished Name. Transport layer protocol Source and destination port numbers  The SPD must be consulted during the processing of inbound and outbound traffic, including nonIPSec traffic.

17. Rocky, K. C. Chang17 Outbound packet processing  Match the outbound packet’s selector fields against the outbound policies in the SPD. If no policy defined for the packet, it is discarded. IPSec processing is not required by the policy (normal IP). IPSec processing is required by the policy.  In the last case, locate the first entry for the packet. If there is no SA or SA bundle specified in the entry, Use Internet Key Exchange (IKE) to establish the SA. The newly established SA is then stored in the node’s SA Database (SAD).

18. Rocky, K. C. Chang18 Outbound packet processing  Each SA in the SAD is specified by: IPSec protocol mode: tunnel or transport; AH or ESP. Algorithms and parameters:  AH authentication algorithms, keys, etc  ESP encryption algorithm, keys, IV mode, IV, etc  ESP authentication algorithm, keys, etc Lifetime of this SA Anti-replay attack related parameters:  sequence number counter  sequence counter overflow flag  anti-replay window  If there is an SA or SA bundle specified in the entry, go to the corresponding SA entry in the SAD, and perform the required IPSec processing specified there.

19. Rocky, K. C. Chang19 Inbound packet processing  If the inbound packet does not contain any IPSec headers, the IPSec layer checks with its SPD to determine how to process the packet.  If the inbound packet contains an IPSec header, Use the packet’s destination IP address, IPSec protocol, and SPI to look up the SA in the SAD.  Drop the packet if not found. Perform appropriate IPSec processing according to the SA found for this packet. Find an incoming policy in the SPD that matches the packet (using backpointers from the SAs to the SPD or by matching the packet's selectors). Deliver the packet to the transport layer or forward the packet.

20. Rocky, K. C. Chang20 IPSec document roadmap

21. 21 Authentication header

22. Rocky, K. C. Chang22 Security services provided by AH  Security services: data origin authentication, connectionless message integrity (jointly referred to as authentication), and an optional anti-replay protection.  In addition to upper level protocol data, AH provides authentication for as much of the IP header as possible. Some IP header fields that may change in transit (mutable fields), such as hop limit, will not be protected. The primary difference between the authentication provided by ESP and AH:  ESP does not protect any IP header fields unless those fields are encapsulated by ESP (tunnel mode).

23. Rocky, K. C. Chang23 Authentication header format  The protocol header (IPv4, IPv6) immediately preceding the AH will contain the value 51 in its Protocol field (IPv4) or Next Header field (IPv6).  The Next Header identifies the type of the next payload after the AH.

24. Rocky, K. C. Chang24 Authentication header format  SPI, together with the destination IP address and security protocol, uniquely defines the SA for this datagram.  The sequence number is to fend off replay attacks. An attacker may obtain a copy of a valid authentication packet and replay it. Sequence numbers start from 0 and must not re-cycle.  The authentication data is a variable-length field that contains the Integrity Check Value (ICV) for this packet. This field must be an integral multiple of 32 bits in length. Padding may be included to ensure that the length of the AH is an integral multiple of 32 bits for IPv4 and 64 bits for IPv6.

25. Rocky, K. C. Chang25 AH in transport mode  The AH is placed after the IP header (and any options that it contains), but before the “upper layer protocol,” e.g. TCP, UDP, ICMP, etc.

26. Rocky, K. C. Chang26 AH in tunnel mode  In tunnel mode, the inner IP header carries the ultimate source and destination addresses.  The outer IP header may contain distinct addresses, e.g. addresses of security gateways.  In tunnel mode, AH protects the entire inner IP packet, including the entire inner IP header.

27. Rocky, K. C. Chang27 Authentication algorithms  The authentication algorithm employed for the ICV computation is specified by the SA.  Mandatory-to-implement authentication algorithms for conformance requirement test are HMAC with MD5 (RFC 2403) HMAC with SHA-1 (RFC 2404)  ICV computation: The mutable fields in the IP header are set to 0. The Authentication Data field in the AH is set to 0, and include padding, if necessary.

28. Rocky, K. C. Chang28 AH processing  Inbound packets: Upon receiving a packet containing an AH, The node determines the appropriate SA, based on the destination IP address, AH, and the SPI. It may or may not check the sequence number. It computes the ICV over the appropriate fields of the packet, and compares with the authentication data.  Outbound packets: AH is applied to an outbound packet only after determining that the packet is associated with an SA that calls for AH processing. Generate a sequence number. Compute the ICV using an algorithm specified by the SA. Insert the AH into the packet and send it out.

29. Rocky, K. C. Chang29 Sequence number verification  A receiver uses sequence numbers and a sliding window to defend against replay attacks.  A receiver receives a window of size at least 32.  When a packet is received, the first AH check applied to the packet after identifying the SA is sequence number verification. If the received packet is within the window and is new, or is to the right of the window, then the receiver proceeds to ICV verification.  If the ICV verification is successful, slide the window to include the corresponding sequence number. If the received packet is within the window but is old, or is the left of the window, the packet will be dropped.

30. Rocky, K. C. Chang30 Sequence number verification  The receiver is required to keep track of two parameters and a list  L: The highest sequence number (SN) for the set of contiguous packets that have been received correctly,  R: The highest SN received so far,  List(L, R): A list that contains SNs between L and R for which the packets have been received correctly.  Sliding window algorithm: If SN  L, discard the packet, else  If SN > R, receive the packet and update R with the new SN, else  If SN  List(L, R), discard the packet, else  Receive the packet and update List(L, R), and possibly update L if the list update results in a new set of contiguous SNs.

31. 31 Encapsulation Security Payload

32. Rocky, K. C. Chang32 Security services provided by ESP  Security services: all the services provided by AH (the difference in the authentication service is mentioned in the AH slides), plus data confidentiality, and limited traffic flow confidentiality.  Both authentication and confidentiality are options, e.g., confidentiality + authentication confidentiality only, but without authentication authentication only, but without confidentiality illegal to have neither Traffic flow confidentiality requires selection of tunnel mode, and is most effective if implemented in a security gateway.

33. Rocky, K. C. Chang33 ESP in transport mode  The ESP header is placed after the IP header (and any options that it contains), but before the “upper layer protocol,” e.g. TCP, UDP, ICMP, or before any other IPSec headers that have already been inserted.

34. Rocky, K. C. Chang34 ESP in tunnel mode  In tunnel mode, the inner IP header carries the ultimate source and destination addresses.  The outer IP header may contain distinct addresses, e.g. addresses of security gateways.  In tunnel mode, ESP protects the entire inner IP packet, including the entire inner IP header.

35. Rocky, K. C. Chang35 ESP packet format

36. Rocky, K. C. Chang36 ESP packet format  The protocol header (IPv4, IPv6) immediately preceding the ESP packet will contain the value 50 in its Protocol field (IPv4) or Next Header field (IPv6).  SPI, sequence number: same usage as for the AH.  Payload Data is a variable-length field containing data described by the Next Header field, consisting of the encrypted data, and possibly the cryptographic synchronization data, e.g. an Initialization Vector (IV) (the IV’s length is defined in the SA) for cipher block chaining, required by the encryption algorithm.

37. Rocky, K. C. Chang37 ESP packet format  Padding Data alignment for encryption algorithms, e.g., block size of a block cipher Data alignment for the ESP packet Conceal the actual length of the payload, in support of partial traffic flow confidentiality.  The Authentication Data is a variable-length field containing an ICV computed over the ESP packet minus the Authentication Data. The authentication does not cover headers in front of the ESP packet.

38. Rocky, K. C. Chang38 Authentication and encryption algorithms  The encryption algorithm is specified by the SA. The ESP is designed for use with symmetric encryption algorithms.  The authentication algorithm is specified by the SA.  Mandatory-to-implement encryption and authentication algorithms for conformance requirement test are DES in CBC mode (RFC 2405) HMAC with MD5 (RFC 2403) HMAC with SHA-1 (RFC 2404) NULL authentication algorithm NULL encryption algorithm

39. Rocky, K. C. Chang39 ESP outbound processing  ESP is applied to an outbound packet only after determining that the packet is associated with an SA that calls for ESP processing. Encapsulate (into the ESP Payload field)  just the original upper layer protocol information (for transport mode) or  the entire original IP datagram (for tunnel mode). Add any necessary padding. Encrypt the result using the key, encryption algorithm, algorithm mode indicated by the SA, and cryptographic synchronization data, if any.

40. Rocky, K. C. Chang40 ESP outbound processing  If authentication is selected, encryption is performed first, before the authentication. Why not the other way around? With this order, the encryption does not encompass the AH field. By simply checking the ICV, it allows a rapid detection and rejection of replayed or bogus packets by the receiver, prior to decrypting the packet. Besides, it allows parallel processing of packets at the receiver, i.e., decryption takes place in parallel with authentication.  Because the Authentication Data is not protected by encryption, a keyed authentication algorithm must be employed to compute the ICV.

41. Rocky, K. C. Chang41 ICV computation in ESP  Unlike AH, the ICV is computed over the SPI, Sequence Number, Payload Data (may be encrypted), Padding (if present, may be encrypted), Pad Length (may be encrypted), and Next Header (may be encrypted)

42. Rocky, K. C. Chang42 ESP inbound processing  Upon receipt of a packet containing an ESP header, the receiver determines the appropriate SA, based on the destination IP address, ESP, and the SPI. It may or may not check the sequence number. If authentication has been selected, compute the ICV over the ESP packet minus the Authentication Data, and compare with the ICV included in the Authentication Data field.  Packet decryption: Decrypt the four fields. Process the padding. Reconstruct the original IP datagram from  the original IP header plus the original upper layer protocol information in the ESP Payload field (for transport mode), or  tunnel IP header + the entire IP datagram in the ESP Payload field.

43. 43 A few more words

44. Rocky, K. C. Chang44 Combining security associations  IP datagrams transmitted over an individual SA are afforded protection by exactly one security protocol, either AH or ESP, but not both.  SAs may be combined into bundles in two ways: Transport adjacency refers to applying more than one security protocol to the same IP datagram, without invoking tunneling. Iterated tunneling refers to the application of multiple layers of security protocols effected through IP tunneling.

45. Rocky, K. C. Chang45 Combining security associations  Transport adjacency applies to only transport mode SAs. AH is used in conjunction with ESP in the same IP packet. AH should appear as the first header after IP, prior to the appearance of ESP.

46. Rocky, K. C. Chang46 Combining security associations  Iterated tunneling: Both endpoints for the SAs are the same -- The inner and outer tunnels could each be either AH or ESP.

47. Rocky, K. C. Chang47 Combining security associations  One endpoint of the SAs is the same -- The inner and outer tunnels could each be either AH or ESP.

48. Rocky, K. C. Chang48 Combining security associations  Neither endpoint is the same -- The inner and outer tunnels could each be either AH or ESP.  These two approaches also can be combined, e.g., an SA bundle could be constructed from one tunnel mode SA and one or two transport mode SAs.

49. Rocky, K. C. Chang49 Counterpane’s critique: complexity  IPSec is too complex to be secure: security’s worst enemy is complexity?  IPSec’s complexity in terms of providing many different ways to accomplish more and less the same thing. Two modes (transport and tunnel) and two protocols (AH and ESP) Two hosts authenticate each other can use transport+AH, tunnel+AH, transport+ESP with NULL encryption, or tunnel+ESP with NULL encryption.  The functionality of tunnel mode seems to be superior than that of the transport mode.  Conclusion: Eliminate the transport mode?

50. Rocky, K. C. Chang50 Counterpane’s critique: complexity  AH authenticates the IP header in addition to the payload. Why do we need to authenticate the header? Authentication of the payload already proves that it came from someone who knows the proper authentication key. The IP header is not used for interpreting the data. Handling of mutable fields are also problematic.  The tunnel+ESP provides the same service but does not need to handle mutable fields.  Conclusion: Eliminating transport mode allows the elimination of the AH protocol as well, without loss of functionality?

51. Rocky, K. C. Chang51  The ESP protocol allows the payload to be encrypted without being authenticated. In virtually all cases, encryption without authentication is not useful.  Conclusion: Without the transport mode or AH protocol to worry about, ESP should always provides its own authentication. Counterpane’s critique: complexity

52. Rocky, K. C. Chang52 Summary  The new IPSec headers were added after thought.  The new IPSec headers have successfully addressed several major security issues at the IP layer.  The main criticism of IPSec is its complexity. ESP + tunnel mode is sufficient?

53. Rocky, K. C. Chang53 Acknowledgments  The notes are prepared mostly based on S. Kent and R. Atkinson, “Security Architecture for the Internet Protocol,” RFC2401, Nov. 1998. N. Doraswamy and D. Harkins, IPSec, Prentice Hall, 1999. S. Kent and R. Atkinson, “IP Authentication Header,” RFC2402, Nov. 1998. S. Kent and R. Atkinson, “IP Encapsulation Security Payload (ESP),” RFC2406, Nov. 1998. N. Ferguson, and B. Schneier, “A Cryptographic Evaluation of IPSec,” http://www.counterpane.com/ipsec.html