Creating a Tunnel

Problem

You want to tunnel IP traffic through your network.

Solution

The basic GRE tunnel configuration is simply a matter of defining the source and destination addresses or interfaces on both devices. On the first router, you need to create the tunnel interface and define its source and destination:

Router1#configure terminal Enter configuration commands, one per line. End with CNTL/Z. Router1(config)#interface Tunnel1 Router1(config-if)#ip address 192.168.35.6 255.255.255.252 Router1(config-if)#tunnel source 172.25.1.5 Router1(config-if)#tunnel destination 172.25.1.7 Router1(config-if)#exit Router1(config)#end Router1#

Then, on the other router you must create a tunnel interface with a matching source and destination:

Router5#configure terminal Enter configuration commands, one per line. End with CNTL/Z. Router5(config)#interface Tunnel3 Router5(config-if)#ip address 192.168.35.5 255.255.255.252 Router5(config-if)#tunnel source 172.25.1.7 Router5(config-if)#tunnel destination 172.25.1.5 Router5(config-if)#exit Router5(config)#end Router5#

Discussion

Creating a basic tunnel is very simpleyou just need to define a source and destination on each of two routers. When you do this, as with any other virtual interface such as subinterfaces and loopback interfaces, there is an additional memory requirement on the router. However, the CPU overhead is not as bad as you might initially think. This is because GRE tunnels do work well with Cisco Express Forwarding (CEF). So the main scaling issue in creating tunnels on routers is the memory required to support them.

The only tricky part of configuring a tunnel is making sure that the source of the tunnel on one router matches the destination on the other. In this case, Router1 uses a source IP address of 172.25.1.5, which happens to be its Ethernet port. If you look at the tunnel destination command on the other router, you will see that it matches. Similarly, the destination on the first router is 172.25.1.7, and the source is 172.25.1.5.

You could also use an alternative syntax, specifying the interface name, rather than the IP address:

Router5(config)#interface Tunnel3 Router5(config-if)#tunnel source Ethernet0

This points the tunnel source to the primary IP address on a particular interface on this router. It is crucial that this IP address match the destination address configured on the other router.

If you then look at the new tunnel interface, you will see that it is up:

Router1#show interfaces Tunnel1 Tunnel1 is up, line protocol is up Hardware is Tunnel Internet address is 192.168.35.6/30 MTU 1514 bytes, BW 9 Kbit, DLY 500000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation TUNNEL, loopback not set Keepalive not set Tunnel source 172.25.1.5 (FastEthernet0), destination 172.25.1.7 Tunnel protocol/transport GRE/IP, key disabled, sequencing disabled Checksumming of packets disabled, fast tunneling enabled Last input 00:11:08, output 00:00:08, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/0 (size/max) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 5 packets input, 740 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 73 packets output, 6604 bytes, 0 underruns 0 output errors, 0 collisions, 0 interface resets 0 output buffer failures, 0 output buffers swapped out Router1#

This is deceptive, though. Even if we remove the tunnel configuration from the other router, this interface will still appear to be up. Indeed, this tunnel interface will appear to be up even if you turn off the power on the far end router. In IOS Version 12.2(8)T, Cisco introduced a new keepalive option for GRE tunnels that overcomes this limitation. When you configure a tunnel with this new feature, the interface will go down if there are any connection problems:

Router1(config)#interface Tunnel1 Router1(config-if)#keepalive

By default, this keepalive command sends a packet through the tunnel to check its status once every 10 seconds. If there is no response to three successive polls, the router declares the tunnel interface to be down. So this will change the tunnel's status about 30 seconds after a failure.

You can adjust both the time interval and the number of retries. For example, to send a keepalive packet every 5 seconds, but to keep the default three retry limit, you could use the following command:

Router1(config)#interface Tunnel1 Router1(config-if)#keepalive 5

And if you want to change the number of retries, you can specify the new value after the time interval. The following example will send a keepalive packet every 3 seconds, and will declare the tunnel down if it doesn't hear a response back to two successive keepalive tests:

Router1(config)#interface Tunnel1 Router1(config-if)#keepalive 3 2

If you are concerned about the integrity of tunneled data, you can enable checksums on a GRE tunnel:

Router1(config)#interface Tunnel1 Router1(config-if)#tunnel checksum

When you turn on checksums, the router will verify the checksum of every GRE packet it receives and drop any packets that don't match. A similar feature checks to see if packets are received in the correct order:

Router1(config)#interface Tunnel1 Router1(config-if)#tunnel sequence-datagrams

When you enable the sequence-datagrams option, the router will drop any packets that it receives out of their correct order. These two options can be useful in networks that have a tendency to damage packets, or when there are multiple paths between the tunnel routers. Remember that GRE doesn't use TCP, so these features can help to improve the reliability of a tunnel connection. However, even when you enable these features, the routers will not resend dropped packets as TCP does.

We do suggest using some caution when you enable either checksums or sequencing on a GRE tunnel, because these features do not work with CEF. So as soon as you enable either of them, the router will have to resort to process switching, which could drive up your CPU utilization.

The tunnel used so far in all of the examples in this recipe hasn't specified any particular tunnel protocol, so the routers will use the default GRE protocol. If you prefer to use a different tunnel protocol, change it using the tunnel mode command as follows:

Router1(config)#interface Tunnel1 Router1(config-if)#tunnel mode ipip

Here we have opted to use the IP-in-IP tunnel protocol that we discussed in the introduction to this chapter. There are several other options for tunnel protocols, which we list in Table 12-1.

In the recipe example, the two routers shared an Ethernet segment, so the routing was trivial. But in practice, routing between the tunnel endpoints is often the most difficult thing to get right. The problem, which we will discuss more in Recipe 12.3, is that the tunnel itself is only one routing hop for packets that travel through it, although it might be several physical hops. As a result, the routing protocol will often decide that the best way to get to the tunnel's destination IP address is through the tunnel itself. This is called recursive routing, and it makes the tunnel useless. So when a router notices that it is routing GRE packets for a tunnel destination address through the same tunnel, it will automatically disable the tunnel with the following error message:

Jan 16 12:05:04 EST: %TUN-5-RECURDOWN: Tunnel1 temporarily disabled due to recursive routing Jan 16 12:05:05 EST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to down

Cisco has attempted to reduce this problem by making the default bandwidth for all tunnel interfaces 9 Kbps. For most routing protocols, this means that you have to traverse several hops before the tunnel looks like a better path. But some protocols, most notably RIP, don't look at interface bandwidths. And it is important to bear in mind that no matter what protocol you use, at some point a single hop of 9 Kbps is going to look better than a large number of higher bandwidth hops.

The only way to avoid this problem is to ensure that there is always a good route to the tunnel destination that doesn't use the tunnel itself. This is most easily accomplished by using static routes, although we discuss other techniques in Recipe 12.3.

One of the inherent problems with tunnels is that the entire IP packet is stuffed inside of another IP packet, which effectively means that the maximum size for your packet payload is smaller. For example, a GRE packet has a 24-byte header. So if your network uses the standard 1,500 byte Ethernet MTU, the largest packet that you can put through the tunnel is 1,476 bytes. If the payload packet's Don't Fragment (DF) bit is not set, then the router will simply break up any larger packets before encapsulating the pieces into multiple (usually 2) GRE packets.

This is a problem because the extra overhead due to packet fragmentation and reassembly can cause extra delays. And if one of the fragments of a tunneled TCP packet is lost due to congestion in the network, all of the fragments constituting the original packet must be retransmitted, which makes the congestion problems worse.

For TCP connections you can use the ip tcp path-mtu-discovery global configuration command to tell the router to monitor for the ICMP "fragmentation needed but DF bit set" messages. These ICMP messages tell end devices to adjust their MTU values to match the maximum that the network can transmit end-to-end. However, GRE doesn't use TCP so this approach doesn't work. Fortunately, in 12.0(7)T3, Cisco introduced an equivalent Path MTU Discovery (PMTUD) command for use with GRE and IP-in-IP tunnels:

Router1(config)#interface Tunnel1 Router1(config-if)#tunnel path-mtu-discovery

This command tells the routers to set the DF bit in the headers of GRE and IP-in-IP tunnels, monitor for ICMP messages, and to adjust MTU values accordingly exactly the way it is done for TCP PMTUD.

Note that this is an ongoing process. The routers must check ever packet because there may be multiple paths through the network, each with different MTU restrictions. The larger the effective MTU, the greater the efficiency of the network, so it is important to make the MTU as large as the network can carry. So the PMTUD process allows the routers to periodically try larger packet sizes, just in case the network topology has changed, and the new path can support larger packets. You can adjust the length of time when the router will hold only a particular MTU value before resetting to the maximum by using the age-timer keyword:

Router1(config)#interface Tunnel1 Router1(config-if)#tunnel path-mtu-discovery age-timer 15

This keyword takes a time-out value expressed in minutes. The default is 10 minutes.

Starting in IOS Version 12.2(13)T, you can also specify a minimum MTU value that the tunnel will negotiate down to by using the min-mtu keyword. If the network wants packets smaller than this, then the routers will just fragment:

Router1(config)#interface Tunnel1 Router1(config-if)#tunnel path-mtu-discovery min-mtu 500

The default here is the minimum value of 92 bytes. This command was added because of a clever denial of service attack in which attackers sent repeated ICMP "fragmentation required but DF bit set" packets to routers running tunnel interfaces, compelling them to reduce their MTUs to unfeasibly small values.

See Also

Recipe 12.3