IPsec HOWTO
Ralf Spenneberg
ralf (at) spenneberg.net
2003-08-18
Revision History
Revision 0.9.95 2005-09-03 Revised by: RS
Added iptables rule setting the MSS and one minor correction
Revision 0.9.94 2005-07-19 Revised by: RS
Added some remarks about routing
Revision 0.9.93 2005-03-3 Revised by: RS
fwd-policy corrected, p12 added
Revision 0.9.92 2005-02-1 Revised by: RS
fwd-policy added
Revision 0.9.91 2005-01-31 Revised by: RS
/etc/ipsec.conf replaced by /etc/setkey.conf
Revision 0.9.9 2004-12-22 Revised by: RS
Nat-Traversal added Changed Document structure
Revision 0.9.6 2004-01-28 Revised by: RS
Correction modp768
Revision 0.9.5 2004-01-08 Revised by: RS
Added Compilation of certpatch and keyconv
Revision 0.9.4 2003-08-28 Revised by: RS
Corrections
Revision 0.9.3 2003-08-22 Revised by: RS
Fixed a typo
Revision 0.9.2 2003-08-19 Revised by: RS
Fixed a typo
Revision 0.9.1 2003-08-18 Revised by: RS
Minor corrections
Revision 0.9.0 2003-08-15 Revised by: RS
Added: Using the OpenBSD isakmpd
Revision 0.8.3 2003-05-13 Revised by: RS
Further typos corrected. Some sentences rephrased.
Revision 0.8.2 2003-05-03 Revised by: RS
Bugfixes
Revision 0.8.1 2003-04-30 Revised by: RS
added chapter covering certificates
Revision 0.8 2003-04-18 Revised by: RS
first draft
This HowTo will cover the basic and advanced steps setting up a VPN using IPsec based
on the Linux Kernels 2.6. Since there is a vast amount of documentation available for the
Linux Kernel 2.4, this HowTo will concentrate on the new IPsec Features in the 2.6 kernel.
Table of Contents
Introduction ...........................................................................................................................3
Theory ..................................................................................................................................... 4
Openswan running on Linux 2.6 .......................................................................................9
Linux Kernel 2.6 using KAME-tools .................................................................................9
Linux Kernel 2.6 using OpenBSD’s isakmpd................................................................20
Generating X.509 Certificates............................................................................................24
Advanced Configuration ...................................................................................................28
Links...................................................................................................................................... 28
Introduction
The latest version of this document can always be found at The Linux Documentation
Project1 and at the official homepage http://www.ipsec-howto.org.
Reasons to write this HowTo
I have used numeruos HowTos in the past. Most were very valuable to me. When the
new IPsec features in the Linux Kernel were implemented I started to play around
using them. Soon I found out that only very little documentation exists. That started
me writing this HowTo.
Format of this document
This document is broken down into 7 chapters.
Section 1: Introduction
This section
Section 2: Theory
IPsec theory. Essentially the IPsec protocols.
Section 3: Openswan
This section will describe how to setup Openswan on the Kernel 2.6.
Section 4: Racoon running on Linux Kernel 2.6
This section describes how to setup an IPsec VPN using the KAME tools setkey
and racoon. This now includes NAT-Traversal.
Section 5: Isakmpd running on Linux Kernel 2.6
This section describes how to setup an IPsec VPN using OpenBSD isakmpd IKE
daemon.
Section 6: Generating X.509 Certificates
This section describes how to generate X.509 Certificates using the
openssl-Command.
Section 7: Advanced Configuration
This section gives some hints on XAUTH and on useful iptables-rules.
Contributors to this document
• Matija Nalis
• Fridtjof Busse
• Uwe Beck
• Juanjo Ciarlante
• Ervin Hegedus
• Barabara Kane
• Alois Schmid
3
IPsec HOWTO
Legal Information
Copyright
Copyright (c) 2003 Ralf Spenneberg
Please freely copy and distribute (sell or give away) this document in any format.
It’s requested that corrections and/or comments be fowarded to the document main-
tainer. You may create a derivative work and distribute it provided that you:
• Send your derivative work (in the most suitable format such as sgml) to the LDP
(Linux Documentation Project) or the like for posting on the Internet. If not the
LDP, then let the LDP know where it is available.
• License the derivative work with this same license or use GPL. Include a copyright
notice and at least a pointer to the license used.
• Give due credit to previous authors and major contributors.
If you’re considering making a derived work other than a translation, it’s requested
that you discuss your plans with the current maintainer.
Disclaimer
The author assumes no responsibility for anything done with this document, nor
does he make any warranty, implied or explicit. If your dog dies, the author may not
be made responsible!
Related Documents
• Networking Overview HOWTO3
• Networking HOWTO4
• VPN-Masquerade HOWTO5
• VPN HOWTO6
• Advanced Routing & Traffic Control HOWTO7
Theory
What is IPsec?
IPsec is an extension to the IP protocol which provides security to the IP and the
upper-layer protocols. It was first developed for the new IPv6 standard and then
“backported” to IPv4. The IPsec architecture is described in the RFC2401. The fol-
lowing few paragraphs will give you a short introduction into IPsec.
IPsec uses two different protocols - AH and ESP - to ensure the authentication, in-
tegrity and confidentiality of the communication. It can protect either the entire IP
datagram or only the upper-layer protocols. The appropiate modes are called tunnel
mode and transport mode. In tunnel mode the IP datagram is fully encapsulated by
a new IP datagram using the IPsec protocol. In transport mode only the payload of
4
IPsec HOWTO
the IP datagram is handled by the IPsec protocol inserting the IPsec header between
the IP header and the upper-layer protocol header (see Figure 1).
original packet
transport mode
new IP header
mode
tunnel
TCP
TCPAHIP
IP Data
Data
TCPIPAHIP Data
Figure 1. IPsec tunnel and transport mode
To protect the integrity of the IP datagrams the IPsec protocols use hash message
authentication codes (HMAC). To derive this HMAC the IPsec protocols use hash
algorithms like MD5 and SHA to calculate a hash based on a secret key and the con-
tents of the IP datagram. This HMAC is then included in the IPsec protocol header
and the receiver of the packet can check the HMAC if it has access to the secret key.
To protect the confidentiality of the IP datagrams the IPsec protocols use standard
symmetric encryption algorithms. The IPsec standard requires the implementation
of NULL and DES. Today usually stronger algorithms are used like 3DES, AES and
Blowfish.
To protect against denial of service attacks the IPsec protocols use a sliding window.
Each packet gets assigned a sequence number and is only accepted if the packet’s
number is within the window or newer. Older packets are immediately discarded.
This protects against replay attacks where the attacker records the original packets
and replays them later.
For the peers to be able to encapsulate and decapsulate the IPsec packets they need a
way to store the secret keys, algorithms and IP addresses involved in the communi-
cation. All these parameters needed for the protection of the IP datagrams are stored
in a security association (SA). The security associations are in turn stored in a security
association database (SAD).
Each security association defines the following parameters:
• Source and destination IP address of the resulting IPsec header. These are the IP
addresses of the IPsec peers protecting the packets.
• IPsec protocol (AH or ESP), sometimes compression (IPCOMP) is supported, too.
• The algorithm and secret key used by the IPsec protocol.
• Security Parameter Index (SPI). This is a 32 bit number which identifies the security
association.
Some implementations of the security association database allow further parameters
to be stored:
• IPsec mode (tunnel or transport)
• Size of the sliding window to protect against replay attacks.
• Lifetime of the security association.
5
IPsec HOWTO
Since the security association defines the source and destination IP addresses, it can
only protect one direction of the traffic in a full duplex IPsec communication. To
protect both directions IPsec requires two unidirectional security associations.
The security assocations only specify how IPsec is supposed to protect the traffic.
Additional information is needed to define which traffic to protect when. This in-
formation is stored in the security policy (SP) which in turn is stored in the security
policy database (SPD).
A security policy usually specifies the following parameters:
• Source and destination address of the packets to be protected. In transport mode
these are the same addresses as in the SA. In tunnel mode theymay differ!
• The protocol (and port) to protect. Some IPsec implementations do not allow the
definition of specific protocols to protect. In this case all traffic between the men-
tioned IP addresses is protected.
• The security association to use for the protection of the packets.
The manual setup of the security association is quite error prone and not very se-
cure. The secret keys and encryption algorithms must be shared between all peers in
the virtual private network. Especially the exchange of the keys poses critical prob-
lems for the system administrator: How to exchange secret symmetric keys when no
encryption is yet in place?
To solve this problem the internet key exchange protocol (IKE) was developed. This
protocol authenticates the peers in the first phase. In the second phase the security
associations are negotiated and the secret symmetric keys are chosen using a Diffie
Hellmann key exchange. The IKE protocol then even takes care of periodically rekey-
ing the secret keys to ensure their confidentiality.
IPsec Protocols
The IPsec protocol family consists of two protocols: Authentication Header (AH) and
Encapsulated Security Payload (ESP). Both are independent IP protocols. AH is the
IP protocol 51 and ESP is the IP protocol 50 (see /etc/protocols). The following two
sections will briefly cover their properties.
AH - Authentication Header
The AH protocol protects the integrity of the IP datagram. To achieve this, the AH
protocol calculates a HMAC to protect the integrity. When calculating the HMAC the
AH protocol bases it on the secret key, the payload of the packet and the immutable
parts of the IP header like the IP addresses. It then adds the AH header to the packet.
The AH header is shown in Figure 2.
Next
Header
Payload
Length
Reserved
Sequence Number (Replay Defense)
Security Parameter Index (SPI)
Hash Message Authentication Code
Figure 2. The AH Header protect the integrity of the packet
6
IPsec HOWTO
The AH header is 24 bytes long. The first byte is the Next Header field. This field spec-
ifies the protocol of the following header. In tunnel mode a complete IP datagram is
encapsulated; therefore the value of this field is 4. When encapsulating a TCP data-
gram in transport mode the corresponding value is 6. The next byte specifies the
length of the payload. This field is followed by two reserved bytes. The next dou-
ble word specifies the 32 bit long Security Parameter Index (SPI). The SPI specifies the
security association to use for the decapsulation of the packet. The 32 bit Sequence
Number protects against replay attacks. Finally the 96 bit holds the hash message au-
thentication code (HMAC). This HMAC protects the integrity of the packets since only
the peers knowing the secret key can create and check the HMAC.
Since the AH protocol protects the IP datagram including immutable parts of the IP
header like the IP addresses the AH protocol does not allow NAT. Network address
translation (NAT) replaces an IP address in the IP header (usually the source IP) by a
different IP address. After the exchange the HMAC is not valid anymore. The NAT-
Traversal extension of the IPsec protocol implements ways around this restriction.
ESP - Encapsulated Security Payload
The ESP protocol can both ensure the integrity of the packet using a HMAC and
the confidentiality using encryption. After encrypting the packet and calculating the
HMAC the ESP header is generated and added to the packet. The ESP header consists
of two parts and is shown in Figure 3.
Sequence Number (Replay Defense)
Security Parameter Index (SPI)
Initialization Vector (IV)
Next
Header
Padding
Length
Data
Padding
Hash Message Authenication Code
Figure 3. The ESP header
The first doubleword in the ESP header specifies the Security Parameter Index (SPI).
This SPI specifies the SA to use for the decapsulation of the ESP packet. The second
doubleword holds the Sequence Number. This sequence number is used to protect
against replay attacks. The third doubleword specifies the Initialization Vector (IV)
which is used in the encryption process. Symmetric encryption algorithms are sus-
ceptible to a frequency attack if no IV is used. The IV ensures that two identical pay-
loads lead to different encrypted payloads.
7
IPsec HOWTO
IPsec uses block ciphers for the encryption process. Therefore the payload may need
to be padded if the length of the payload is not a multiple of the block length. The
length of the pad is then added. Following the pad length the 2 byte long Next Header
field specifies the next header. Lastly the 96 bit long HMAC is added to the ESP
header ensuring the integrity of the packet. This HMAC only takes the payload of
the packet into account. The IP header is not include in the calculation process.
The usage of NAT therefore does not break the ESP protocol. Still in most cases NAT
is not possible in combination with IPsec. The NAT-Traversal offers a solution in this
case by encapsulating the ESP packets within UDP packets.
IKE Protocol
The IKE protocol solves the most prominent problem in the setup of secure commu-
nication: the authentication of the peers and the exchange of the symmetric keys. It
then creates the security associations and populates the SAD. The IKE protocol usu-
ally requires a user space daemon and is not implemented in the operating system.
The IKE protocol uses 500/udp for it’s communication.
The IKE protocol functions in two phases. The first phase establishes a Internet Se-
curity Association Key Management Security Association (ISAKMP SA). In the second
phase the ISAKMP SA is used to negotiate and setup the IPsec SAs.
The authentication of the peers in the first phase can usually be based on pre-shared
keys (PSK), RSA keys and X.509 certificates (racoon even supports Kerberos).
The first phase usually supports two different modes: main mode and aggressive
mode. Both modes authenticate the peer and setup an ISAKMP SA, but the aggres-
sive mode uses only half the number of messages to achieve this goal. This does have
its drawbacks though, because the aggressive mode does not support identity protec-
tion and is therefore susceptible to a man-in-the-middle attack if used in conjunction
with pre-shared keys. On the other hand this is the only purpose of the aggressive
mode. Because of the internal workings of the main mode it does not support the us-
age of different preshared keys with unknown peers. The aggressive mode does not
support identity protection and transfers the identity of the client in the clear. The
peers therefore know each other before the authentication takes place and different
pre-shared keys can be used for different peers.
In the second phase the IKE protocol exchanges security association proposals and
negotiates the security associations based on the ISAKMP SA. The ISAKMP SA pro-
vides the authentication to protect against a man-in-the-middle attack. This second
phase uses the quick mode.
Usually two peers negotiate only one ISAKMP SA, which is then used to negotiate
several (at least two) unidirectional IPsec SAs.
NAT-Traversal
What is NAT-Traversal and why is it needed?
Often one peer in the VPN is behind a NAT-device. I just assume Source-NAT devices
here. Whenever I talk about NAT I mean Source-NAT or Masquerading. What does
this mean concerning the VPN? Well, first of all the original IP address of the peer is
hidden by the NAT-device. The NAT-device conceals the original source IP address
and replaces it by its own IP address.
This make the IPsec AH protocol immediately unusable. But ESP can still be used if
both sides are configured correctly.
So why do you need NAT-Traversal? Because as soon as two machines behind the
same NAT device try to build a tunnel to the outside, both will fail.
8
IPsec HOWTO
Why is this happening? The NAT device needs to keep track of the "natted" connec-
tions to be able to "de-nat" the reply packets back to the original client. Therefore
the NAT device maintains an internal table where all "natted" connections are stored.
Lets assume one client connects to a webserver on the Internet. The NAT device con-
ceils the original address by replacing it with its own address. It then makes a note in
its internal table that all packets coming back on the chosen client port have to be send
to the original client1. As soon as the second client starts a connection, it handles that
connection identical. If the second client chose the same client port by coincidence the
NAT device will also modify the client port for unambuigity. This works very well
using TCP and UDP because those protocols provide ports. ESP does not use ports.
Therefore the NAT device can only use the protocol distinguish the packets. When
the first client connects it stores the information in the table that all ESP packets have
to be "denatted" to the first client. When the second client connects it will overwrite
this entry with the appropiate entry for the second one thus breaking at least the first
connection.
What does NAT traversal do to help? NAT-traversal again encapsulates the ESP pack-
ets in UDP packets. These can easily be handled by a NAT device since they provide
ports. By default port 4500/udp is used. NAT traversal is specified in several drafts.
There are no RFCs at the moment. A nice feature of NAT traversal is the fact that once
activated the peers automatically use it when needed.
Openswan running on Linux 2.6
ToDo
Linux Kernel 2.6 using KAME-tools
This chapter explains the usage of the native IPsec stack of the Linux Kernel ≥2.5.47
and 2.6.*. The installation and the configuration of this IPsec stack differs greatly from
FreeS/WAN and is similar to the *BSD variants like FreeBSD, NetBSD and OpenBSD.
I will first cover the configuration and installation of the Linux kernel and the user
space tools. Then the setup of a manually keyed connection in transport and tunnel
mode will be explained. Finally we will cover the setup of automatically keyed con-
nections using preshared keys and X.509 certificates. The support of roadwarriors
will be explained last.
Installation
The installation requires at least a Linux kernel of version 2.5.47 or 2.6.*. The ker-
nel source may be downloaded at http://www.kernel.org. After downloading the
source the kernel source package must be extracted, configured and compiled.
cd /usr/local/src
tar xvjf /path-to-source/linux-.tar.bz2
cd linux-
make xconfig
make bzImage
make modules
make modules_install
make install
These are the most often used commands to configure and compile the Linux kernel.
If you need a special setup please refer to the Kernel-Howto.
When configuring the kernel, it is important, to turn on the following features:
9
IPsec HOWTO
Networking support (NET) [Y/n/?] y
*
* Networking options
*
PF_KEY sockets (NET_KEY) [
本文档为【ipsec-howto】,请使用软件OFFICE或WPS软件打开。作品中的文字与图均可以修改和编辑,
图片更改请在作品中右键图片并更换,文字修改请直接点击文字进行修改,也可以新增和删除文档中的内容。
该文档来自用户分享,如有侵权行为请发邮件ishare@vip.sina.com联系网站客服,我们会及时删除。
[版权声明] 本站所有资料为用户分享产生,若发现您的权利被侵害,请联系客服邮件isharekefu@iask.cn,我们尽快处理。
本作品所展示的图片、画像、字体、音乐的版权可能需版权方额外授权,请谨慎使用。
网站提供的党政主题相关内容(国旗、国徽、党徽..)目的在于配合国家政策宣传,仅限个人学习分享使用,禁止用于任何广告和商用目的。