Understanding the Operation of SSL Remote Access VPNs

Before getting into the design and implementation of SSL remote access VPNs, it is a very good idea to take a look at the underlying mechanisms that allow their operation. This section examines the protocols and mechanisms are used to enable SSL remote accessVPNs.

The Secure Sockets Layer (SSL) is a security protocol that is used to secure e-commerce, web transactions, and more recently, to provide remote access VPN connectivity.

SSL was invented by Netscape Communications, and there are a number of versions:

• SSL version 1 (SSLv1) This version was not released by Netscape.
• SSL version 2 (SSLv2) This version was released and is still supported by many web browsers, but it has a number of well-known weaknesses and deficiencies, including its vulnerability to downgrade attacks (an attacker can force the negotiation of a weak cipher suite [which is then subject to attack]); its vulnerability to truncation attacks (an attacker can cause either the server or the client to believe that data transfer has finished by simply forging a TCP FIN); the fact that SSLv2 relies on MD5 MACs and SHA-1 is not available for use; the fact that the same key is used for encryption and authentication (so breaking the key of one leads automatically to the breaking of the other); and the ability of an attacker to intercept and hijack a client's connection. So, SSLv2 is definitely not recommended.
• SSL version 3 (SSLv3) This version addresses the weaknesses of SSLv2.
• Transport Layer Security (TLS version 1.0, RFC 2246) This is an IETF standard that is based on but not compatible with SSLv3.

In this chapter, the term SSL is used as a generic term to describe SSLv3 and TLS.

SSL Overview: TCP, the Record Layer, and the Handshake Protocol

SSL sits on top of a reliable protocol, such as TCP. Application data can then be carried on top of SSL.

Figure 10-2 shows the overall SSL packet format.

Figure 10-2. Overall SSL Packet Format

The SSL protocol itself consists of the record protocol plus the handshake protocol, the alert protocol, the change cipher spec protocol, and the application data protocol.

Figure 10-3 shows the relationship between the record protocol and the handshake, alert, change cipher spec, and application data protocols.

Figure 10-3. Relationship Between the Record Protocol and the Handshake, Alert, Change Cipher Spec, and Application Data Protocols

The Record layer (protocol) has a number of functions, including the following:

• Fragmentation The record protocol fragments/reassembles data on transmission/reception, if required.
• Compression If negotiated between a client and server, the record protocol compresses/decompresses data.
• Applies MAC The record protocol applies/verifies a MAC.
• Encryption The record protocol encrypts/decrypts data.

As shown in Figure 10-3, the handshake, alert, change cipher spec, and application data protocols sit on top of the Record layer. These protocols have the following functions:

• Handshake protocol (type 22) This protocol consists of a number of messages exchanged between peers (the client and server) that allow those peers to negotiate SSL version, cryptographic algorithms, and parameters; optionally authenticate each other; and generate shared secret cryptographic keys using public key techniques.

  Table 10-1 summarizes SSLv3 and TLS handshake protocol messages.

  Table 10-1. Handshake Protocol Messages

  Handshake Protocol Message	Type	Sent By	Description
  HelloRequest	0	Server	Notification that the client should begin negotiation again.
  ClientHello	1	Client	Used to (re-)initiate an SSL connection. Specifies list of cryptographic algorithms/parameters, plus optional compression method and random number.
  ServerHello	2	Server	Specifies chosen cryptographic algorithms/parameters and optional compression method. Includes random number.
  Certificate	11	Client/server	Contains certificate or certificate chain.
  ServerKeyExchange	12	Server	Used in ephemeral RSA or Diffie-Hellman handshake when server certificate message does not allow client to exchange premaster secret.
  CertificateRequest	13	Server	Used by the server to request the client's certificate. Used when client authentication is required.
  ServerHelloDone	14	Server	Used to indicate that the server will not send any more messages at this stage of the handshake.
  CertificateVerify	15	Client	Used by the client to prove that it is the legitimate owner of its certificate.
  ClientKeyExchange	16	Client	Used to send the premaster secret to the server.
  Finished	20	Client/server	Ensures integrity of handshake exchange.

• Alert protocol (type 21) This is used to signal error conditions. These error conditions include those that occur during an SSL handshake, when decrypting or integrity checking, or when closing a connection.
• Change cipher spec protocol (type 20) This protocol (which consists of a single message type) is used by the client or server to signal that subsequent SSL messages will be protected using negotiated cryptographic algorithms, parameters, and keys.
• Application data protocol (type 23) The application data protocol carries data from whichever application or applications are running over SSL. This data can include protocols such as HTTP, FTP, POP3, IMAP4, SMTP, as well as other higher-level protocol data.

Establishing an SSL Connection Between a Remote Access VPN User and an SSL VPN Gateway Using an RSA Handshake

As previously described, the function of SSL is to negotiate cryptographic algorithms, authenticate the server (VPN gateway) and optionally the remote access VPN client, and establish cryptographic keys.

The most common method of establishing an SSL connection between a remote access VPN client and a VPN gateway is using the Rivest, Shamir, and Addlemen (RSA) handshake with VPN gateway authentication. In this case, the remote access VPN client is not authenticated during the handshake, but may be authenticated using a separate mechanism after the SSL connection has been established.

Figure 10-4 illustrates the RSA handshake with VPN gateway only authentication.

Figure 10-4. SSL RSA Handshake

As shown in Figure 10-4, the RSA handshake with VPN gateway only authentication consists of the exchange of nine messages:

1. The first step in the RSA handshake is when the remote access client sends a ClientHello message to the VPN gateway (server). This message is used to propose a number of cryptographic parameters and algorithms (cipher suites), compression methods (if any), and well as being used to transmit a random number that is later used to generate cryptographic keys.
2. The VPN gateway now selects one of the cryptographic proposals (cipher suites), together with any compression method sent in the ClientHello and includes these chosen proposals in the ServerHello message that it sends to the remote access client.

   The ServerHello message also includes a random number generated by the VPN gateway, which is again used to later generate cryptographic keys.

3. Next, the VPN gateway's sends a Certificate message. This message contains the VPN gateway's certificate (including its public key) along with any other certificates in a certificate chain.
4. Immediately after the Certificate message is ServerHelloDone. The VPN gateway sends this message to signal that it will not send any more messages at this stage of the handshake sequence.
5. The remote access client now sends the ClientKeyExchange message. This message includes a key, called the pre_master_secret, which is encrypted using the VPN gateway's public key.

   The pre_master_secret, together with the random numbers sent in the ClientHello and ServerHello messages is used to generate the keys later used to encrypt and authenticate SSL traffic.

6. The ClientKeyExchange message is followed by the ChangeCipherSpec and Finished messages.

   The ChangeCipherSpec message signals that SSL messages from the client (starting with the Finished message) will from now on be protected using the negotiated cipher suite.

   The Finished message contains a hash of all the messages previously sent in the handshakethis ensures that an attacker has not modified any of the previous handshake messages.

7. The VPN gateway now completes the RSA handshake by sending its own ChangeCipherSpec and Finished messages.

Now that you know what happens in theory, it is time to take a look at an SSL connection in practice.

SSL Connection Establishment: ClientHello Message

Figure 10-5 shows a packet capture of an SSL ClientHello message.

Figure 10-5. SSL ClientHello Message

If you look at the highlighted line in the main portion of the screen shown in Figure 10-5, you can see the message is, in fact, a ClientHello message. Below the highlighted line there are a number of message fields that can be described as follows:

• Record layer header:

  - Content TypeThis field specifies the record type, which in this case is Handshake (type 22).

  - VersionThis defines the SSL version (major and minor versions). SSLv3 is 3.0 (0x0300), and TLS is 3.1 (0x0301).

  - LengthThe length of the record. According to the specification, a record can be up to a maximum of 214 bytes, although at least one browser type did erroneously send records of up to 216-1 bytes.

• Handshake protocol header:

  - Handshake TypeThe handshake protocol message type. Here, the message type is ClientHello (1).

  - LengthThe length of the message.

  - VersionThe highest version of the SSL protocol that the client supports.

  - Random number (Random.gmt_unix_time, plus Random.bytes)This is the previously described random number that is later used to generate cryptographic keys.

  Notice that the random number is, in fact, made up of the number of seconds since midnight, January 1, 1970 (4 bytes in UNIX format), as well as a 28-byte randomly generated number. The UNIX time is included in the random number to ensure that the same random number is not chosen twice (this would be possible if all 32 bytes of the random number were randomly chosen!).

  - Session ID LengthThe length of the Session ID field.

  The Session ID length is zero here, and this indicates that the Session ID field itself is not present. The Session ID field is only present in the ClientHello if the client wants to resume a previous SSL session with the VPN gateway.

  - Cipher Suites LengthThe length of the Cipher Suites list.

• Cipher SuitesThere are a number of SSL cipher suites, including a number that require the VPN gateway to provide an RSA certificate, a number that are used for server authenticated (and optionally client authenticated) Diffie-Hellman, and a number that are used for anonymous Diffie-Hellman communications where neither the VPN gateway nor the client is authenticated.

  The cipher suites themselves are fairly self-explanatory. For example, TLS_RSA_WITH_RC4_128_MD5 specifies RSA authentication with the RC4 stream cipher for encryption and Message Digest 5 (MD5) for integrity checking. One thing you might notice if you closely examine the cipher suites in Example 10-5 is the lack of any support for AES. This is due to the lack of any AES cipher suites in the base TLS v1.0 specification (RFC 2246 [SSLv3 includes no support for AES, by the way]). RFC 3268 describes support for AES cipher suites with TLS, and this support is integrated into the TLS v1.2 specification, which is under discussion in the IETF TLS working group at the time of this writing. It may be interesting to note that support for AES with HTTPS is added in Microsoft Vista (support is also included in Firefox).

  - Compression Methods LengthThe length of the Compression Methods field.

  - Compression MethodsThis specifies methods of compression that the client is willing to support. RFC2246 does not specify any compression methods because of intellectual property concerns, and so this client specifies NULL compression, meaning no compression.

  Note that RFC3749 does specify the DEFLATE compression algorithm for use with SSL (DEFLATE itself is defined in RFC1951).

SSL Connection Establishment: ServerHello, Certificate, and ServerHelloDone Messages

The next messages in the SSL handshake sequence are ServerHello, Certificate, and ServerHelloDone. The VPN gateway sends these messages (see Figure 10-6).

Figure 10-6. SSL server_hello, Certificate, and ServerHelloDone Messages

If you take a look in the main portion of Figure 10-6, you can see the common SSL record header (Content Type, Version, Length), followed by the ServerHello, Certificate, and ServerHelloDone messages.

The ServerHello message is similar to the ClientHello message, but there are one or two differences:

• The Handshake Type field now specifies that this message is a ServerHello (2).
• The Session ID Length field specifies a 32-byte length.
• The Session ID field specifies a Session ID that can be used to resume the session at a later stage if desired.
• The Cipher Suite field specifies the cipher suite chosen by the VPN gateway from those sent by the client in the ClientHello message.

Immediately after the ServerHello is the Certificate message. The Certificate message is relatively simple and consists of the following fields:

• Handshake Type In this case, this field is used to specify that this is a certificate message (11).
• Length The length of this message.
• Certificates Length The length of the certificate(s) included in this message.
• Certificates The certificate or certificates sent by the VPN gateway, including the following:

  - Certificate LengthThe length of this certificate.

  - CertificateThe certificate itself.

  Note that a number of certificates can be included.

Next is the ServerHelloDone message. As you can see, it consists of only two fields:

• Handshake Type The type of handshake message, which is ServerHelloDone (14).
• Length The length of the message, which in this case is zero (remember that ServerHelloDone simply indicates that the VPN gateway does not have any more messages to send at this stage).

SSL Connection Establishment: ClientKeyExchange, ChangeCipherSpec, and Finished Messages

After the VPN gateway sends the ServerHello, Certificate, and ServerHelloDone messages, the client responds with the ClientKeyExchange, ChangeCipherSpec, and Finished messages.

Figure 10-7 shows the transmission of the ClientKeyExchange, ChangeCipherSpec, and Finished messages. The Finished message is the first in the handshake to be encrypted.

Figure 10-7. Transmission of the ClientKeyExchange, ChangeCipherSpec, and Finished Messages

Figure 10-7 again shows the common Record layer header, followed by the ClientKeyExhange, ChangeCipherSpec, and Finished messages:

• The common Record layer header, including the following

  - Content Type Specifies that this is a Handshake message (22).

   

  - Version This is a TLS (SSL version 3.1) message.

   

  - Length The length of this message is 134 bytes long.

   

• A ClientKeyExchange message, including the following

  - Handshake Type This is a ClientKeyExchange message (16).

   

  - Length This message is 130 bytes long.

   

• A ChangeCipherSpec message, including the following

  - Content Type This is a ChangeCipherSpec message (20).

   

  - Version This is a TLS message.

   

  - Length The length of this message is 1 byte.

   

• An encrypted Finished message ("Encrypted Handshake Message").

The remote access client now completes the handshake by sending ChangeCipherSpec and Finished messages.

Figure 10-8 shows the ChangeCipherSpec message sent by the client.

Figure 10-8. ChangeCipherSpec Message Sent by the Client

Figure 10-9 shows the Finished message sent by the client.

Figure 10-9. Finished Message Sent by the Client

The format of the Record layer header has already been described, and so will not be described again here. Notice, however, the final line in the output in the main portion of the screen shown in Figure 10-9this line shows that the Handshake message is encrypted. This should be no surprise, as you will remember that the Finished message is, in fact, the first message that is encrypted in the handshake.

And that is itthe handshake is complete, and the client and VPN gateway are ready to send application data to each other over the SSL connection. Figure 10-10 shows application data sent over the SSL connection.

Figure 10-10. Application Data Sent over the SSL Connection

The Record layer header Content Type field shows that this is an Application Data message. The length of the message is 264 bytes, but the payload is encrypted and so is not shown.

SSL Handshake: SSLv2, SSLv3, or TLS?

Before finishing this section, it is worth pointing one apparent anomaly that is sometimes seen in the SSL handshake (see Figure 10-11).

Figure 10-11. SSL Handshake Anomaly

First, take a look at the upper portion of the screen capture. Look at the SSL messages; you will notice the familiar beginning of the RSA handshake (compare with Figure 10-4). If you are the observant type, you will have noticed one apparent anomalythe TLS handshake begins with an SSLv2 ClientHello message!

Now take a look at the lower portion of the screen and you see the detail of the SSLv2 ClientHello. If you look closely at the fields of the Record header, you will notice that although this is an SSLv2 ClientHello, the Version field in the record header specifies TLS!

What is going on? Well, this is a common method that web browsers/clients use to check whether a server/VPN gateway can support SSLv3 or TLS. This method of checking support for SSLv3 or TLS works because the Version fields in SSLv2 and SSLv3/TLS are interpreted differently (SSLv2 is represented as 0x0002, whereas SSLv3 and TLS are represented as 0x0300 and 0x0301 respectivelynote the difference in the way the high-order and low-order octets are used).

So, if a client sends an SSLv2 ClientHello with the Version field set to TLS (0x0301), and the server/VPN gateway supports TLS, it will parse the SSLv3/TLS version field and identify support for TLS, then respond using TLS handshake messages, as appropriate! A server/VPN gateway that supports only SSLv2, on the other hand, will not identify this as TLS.

Note

SSLv3 and TLS (RFC2246) also include a variation on the regular RSA handshake called an ephemeral RSA1, as well as another type of handshake that takes advantage of the Digital Signature Standard (DSS) and Diffie-Hellman algorithms.

The ephemeral RSA handshake permits SSL connections to be established between export clients (which use weak cipher suites) and U.S. domestic servers (which use strong cipher suites) using a strong key.

The DSS/DH handshake, on the other hand, was designed to overcome any issues with patents on the RSA algorithm.

Since U.S. export controls for strong cryptographic software have been relaxed, the ephemeral RSA handshake has become much less relevant. The patent for the RSA algorithm has now expired, and so the DSS/DH handshake has also become less relevant. For these reasons, neither the ephemeral RSA handshake nor the DSS/DH handshake is discussed in this chapter.

Understanding the SSL RSA Handshake with Client Authentication

The RSA handshake described in the previous section ensures that the server/VPN gateway is authenticated by the client (using the certificate sent by the server in the Certificate message). But, how about if you want the server to authenticate client during the handshake, too? In this case, the SSL handshake with client authentication, illustrated in Figure 10-12, is used.

Figure 10-12. SSL Handshake with Client Authentication

If you compare Figure 10-12 with Figure 10-4, you can see that the handshakes are very similar. There are one or two differences, however:

• In response to the ClientHello, the VPN gateway sends a CertificateRequest message in addition to the ServerHello, Certificate, and ServerHelloDone messages.

  As the name suggests, the CertificateRequest message is used to request that the client send its certificate (or certificate chain) to the VPN gateway as well as to specify which key types are acceptable (for example, RSA) and the CAs it will accept as issuers of certificates (the client's certificate must be issued by one of these).

• The client replies with Certificate and CertificateVerify messages in addition to the ClientKeyExchange, ChangeCipherSpec, and Finished messages.

  The Certificate message contains the client's certificate or certificate chain, and the CertificateVerify message is used by the client to prove that it possesses the private key corresponding to the public key contained in its certificate.

  The CertificateVerify message consists of a digital signature (using the private key corresponding to the public key in the certificate). Only the owner of the client's certificate should possess the private key, and so the client is able to prove that it really is the owner of the certificate sent in the certificate message.

So, the client authenticates the VPN gateway using the Certificate that the VPN gateway sends (this is the same as the regular RSA handshake with no client authentication), and the VPN gateway authenticates the client using the Certificate and CertificateVerify messages that the client sends.

Resuming an SSL Session

So far, this chapter has described SSL handshakes that allow the establishment of new SSL sessions. This section describes the resumption of a previous SSL session. Figure 10-13 shows the exchange of SSL messages necessary to resume a previous SSL session.

Figure 10-13. Resuming an SSL Session

You might be wondering why you would want to resume an SSL session instead of just establishing a new one (as described in the previous two sections). The answer is that the public key operations that are necessary when establishing a new session are cryptographically expensive/processor intensive, whereas resuming a previous session does not require these expensive public key operations.

As you can see in Figure 10-13, to resume an SSL session, the follows happens:

Closing an SSL Connection

The final action in an SSL connection is its closure. This is illustrated in Figure 10-14.

Figure 10-14. SSL Connection Closure

In Figure 10-14, the client initiates connection closure by sending a close_notify Alert message to the VPN gateway. The client then sends a TCP FIN message to terminate the underlying TCP connection.

The VPN gateway responds by sending its own close_notify Alert message, followed by a TCP FIN. The SSL connection is now closed.

Although Figure 10-14 shows the client initiating SSL connection closure, it is possible for either the client or the VPN gateway (server) to initiate connection closure.

Note that if a client or VPN gateway receives a TCP FIN before receiving a close_notify Alert, it marks the connection as being unresumablethis prevents truncation attacks, where an attacker inserts a TCP FIN into the traffic stream between the client and VPN gateway.