Setting Up Route Reflectors

Problem

Your local AS has a large number of IBGP routers, and you want to reduce the number of IBGP sessions that you need to configure and maintain.

Solution

Configure one of the IBGP routers to be the route reflector for a route reflection cluster:

[edit protocols bgp] aviva@RouterG# set group internal-within-AS65500 cluster 192.168.19.1

Discussion

The configuration in Recipe 13.2, in which all BGP systems within the AS are fully meshed, is a standard IBGP implementation. The full mesh results from listing all IBGP peers in a peering group rather than from having them all be physically connected and from using an IGP within the AS to distribute BGP routes. The full mesh is necessary so that external routing information can be redistributed among all routers within the AS with the help of the IGP running in the AS. As you can imagine, as the number of IBGP routers increases, you have to configure many BGP neighbor commands in each routers configuration, and there is a lot of overhead because a large number of TCP connections need to be maintained for each IBGP peering.

There are two common ways to deal with this scaling issue. One is route reflection, which provides one means of decreasing BGP control traffic and minimizing the number of update messages sent within the AS. The second method is confederations.

With normal BGP route redistribution rules, IBGP peers are not allowed to advertise routes learned from IBGP rules. Route reflection works by bending this rule. Each route reflector system has a set of client peers that are arranged in a cluster. The clients send their routes to the route reflector, and the route reflector advertises these routes to the other clients in the cluster and to other IBGP peers outside of the cluster (nonclient peers). The route reflector is not allowed to change any of the routes attributes, which is one way of preventing routing loops. When the route reflector learns routes from nonclient peers, it advertises them to its clients and to no one else. For routes learned from EBGP, the route reflector follows normal BGP route advertisement rules, advertising these routes to both clients and nonclients.

In this recipe, RouterG is the route reflector (see Figure 13-3). The only configuration needed is to add the cluster identifier to the IBGP group with the set cluster command. The cluster identifier is a 32-bit number. You can use the router identifier or any 32-bit number. When the router reflector forwards a route from one of its client routers to a nonclient router, the cluster identifier is prepended to the cluster list. If a route reflector ever receives a BGP update that contains its own cluster identifier, it ignores the update because it knows a routing loop has occurred.

Heres the complete IBGP group configuration on RouterG:

[edit protocols bgp] aviva@RouterG# show group internal-within-AS65500 { type internal; local-address 192.168.19.1; cluster 192.168.19.1; neighbor 192.168.13.1; neighbor 192.168.12.1; neighbor 192.168.17.1; neighbor 192.168.15.1; neighbor 192.168.16.1; }

You still need to configure all the IBGP neighbors, both those within the cluster (192.168.13.1 and 192.168.12.1) and the three IBGP peers outside the cluster. For the nonclient routers, those outside the cluster, you include the route reflector in the IBGP group but don need to have neighbor statements for the routers within the cluster. On the client routers, you need to configure only one IBGP peer, the route reflector 192.168.19.1:

[edit protocols bgp] aviva@RouterA# show group cluster-within-AS65500 { type internal; local-address 192.168.13.1; neighbor 192.168.19.1; }

This configuration is a lot simpler than having to specify all the IBGP neighbors. Another advantage from the point of view of the routers configurations is if you later add routers to the cluster, you do not need to change the configuration on the existing client routers in the cluster.

The nonclient routers in the AS, RouterE and RouterF, need to establish an IBGP session with RouterG but they don need sessions with RouterA or RouterB. In this small network, this is a savings of four TCP sessions and four IBGP peerings. Again, if you add routers to the cluster, the IBGP group configuration on the nonclient routers doesn need to change.

All the client routers in the cluster and all the nonclient routers are in the same AS, so make sure that you configure them all with the same AS number:

[edit routing-options] aviva@RouterB# show autonomous-system 65500;

Looking at the routes that RouterB has learned, you see that it still knows how to reach routes in AS 65505:

aviva@RouterB> show route protocol bgp inet.0: 21 destinations, 22 routes (21 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 10.0.24.0/24 *[BGP/170] 00:18:38, localpref 100, from 192.168.19.1 AS path: 65505 I > via se-0/0/2.0 10.0.29.0/24 *[BGP/170] 00:18:38, localpref 100, from 192.168.19.1 AS path: 65505 I > via se-0/0/2.0 192.168.11.1/32 *[BGP/170] 00:18:38, MED 1, localpref 100, from 192.168.19.1 AS path: 65505 I > via se-0/0/2.0 192.168.14.1/32 *[BGP/170] 00:18:38, localpref 100, from 192.168.19.1 AS path: 65505 I > via se-0/0/2.0 192.168.18.1/32 *[BGP/170] 00:18:38, MED 65, localpref 100, from 192.168.19.1 AS path: 65505 I > via se-0/0/2.0

A detailed look at one of these routes shows information about the cluster:

aviva@RouterB> show route 10.0.24.0/24 extensive inet.0: 21 destinations, 22 routes (21 active, 0 holddown, 0 hidden) 10.0.24.0/24 (1 entry, 1 announced) TSI: KRT in-kernel 10.0.24.0/24 -> {indirect(291)} *BGP Preference: 170/-101 Next-hop reference count: 15 Source: 192.168.19.1 Next hop: via se-0/0/2.0, selected Protocol next hop: 192.168.16.1 Indirect next hop: 8670300 291 State: <Active Int Ext> Local AS: 65500 Peer AS: 65500 Age: 25:44 Metric2: 90 Task: BGP_65500.192.168.19.1+1106 Announcement bits (2): 0-KRT 4-Resolve tree 1 AS path: 65505 I (Originator) Cluster list: 192.168.19.1 AS path: Originator ID: 192.168.16.1 Localpref: 100 Router ID: 192.168.19.1 Indirect next hops: 1 Protocol next hop: 192.168.16.1 Metric: 90 Indirect next hop: 8670300 291 Indirect path forwarding next hops: 1 Next hop: via se-0/0/2.0 192.168.16.1/32 Originating RIB: inet.0 Metric: 90 Node path count: 1 Forwarding nexthops: 1 Nexthop: via se-0/0/2.0

The Cluster list field shows the cluster ID of the route reflector. This line shows the 32-bit value that we configured on RouterG. The address of the route reflector that originally sent the route is in the Originator ID field.

This recipe illustrates a simple route reflection setup. One problem is that the route reflector is a single point of failure. If it goes down, its two peers lose their connection to the IBGP network. It is a common practice to configure two route reflectors for each cluster to provide backup.

Another less commonly used way to subdivide a large IBGP network into more manageable groups is create sub-ASs within your AS and link them together as a BGP confederation. On each IBGP router in the sub-AS, define the sub-AS number and the confederation group:

[edit routing-options] aviva@RouterG# set autonomous-system 65502 aviva@RouterG# set confederation 65500 members [ 65501 65502 ]

Here, the main AS number is 65500 and the two sub-ASs are 65501 and 65502. This router is in the 65502 sub-AS. The set autonomous-system command defines the sub-AS number, and the set confederation command identifies all the sub-ASs in the main AS.

From BGPs point of view, each sub-AS is a separate AS because each has a different AS number. So, you configure each sub-AS as you would an EBGP connection, setting up border routers between the sub-ASs. On each of these routers, create an external group to connect the two sub-ASs. The following commands configure the EBGP group on the border router in sub-AS 65502:

[edit protocols bgp] aviva@RouterG# set group to-subAS65501 type external aviva@RouterG# set group to-subAS65501 neighbor 192.168.17.1 aviva@RouterG# set group to-subAS65501 peer-as 65501

The set peer-as command identifies the sub-AS number. The border router also has an IBGP group to create peerings with the IBGP routers in the sub-AS:

[edit protocols bgp] aviva@RouterG# show group internal-within-subAS65502 { type internal; export next-hop-self; local-address 192.168.19.1; neighbor 192.168.13.1; neighbor 192.168.12.1; }

Just like with the EBGP sessions between ASs, you need a next-hop-self policy in the IBGP group for the sub-AS border router (see Recipe 13.4).

As with route reflectors, BGP confederations (see Figure 13-4) reduce the number of peering sessions you need to configure and the number of TCP sessions that the routers need to establish to maintain full-mesh connections between all IBGP routers.

See Also

RFC 1965, Autonomous System Confederations for BGP; RFC 1966, BGP Route Reflection: An Alternative to Full-Mesh IBGP; RFC 2796, BGP Route Reflection; RFC 3065, Autonomous System Confederations for BGP