Chapter 23: Failover (Part 2)

Failover Operation

Now that you understand how to cable up the appliances, let’s talk about how failover works: how the appliances communicate with each other, how they discover problems, and when failover occurs.

Failover Communications

As I mentioned earlier in the “Failover Cabling” section, the failover link is either the serial cable used on the PIXs or an Ethernet cable with LBF. The failover link is used to communicate the following information between the paired appliances:

• The state of the appliances: active or standby

• If the appliances are PIXs, their power status

• Failover hello messages

• Network link status of the appliances interfaces

• Exchange of MAC addresses used on the appliance interfaces

• Configuration of the active unit synchronized with the standby unit

When implementing stateful failover, the following information is transferred across the stateful link from the active to the standby unit: xlate table, conn table, VPN sessions (only in active/standby failover), MAC address table (when running in transparent mode), SIP signaling information, and the current date and time. As you can see, not all state information is replicated across the state link. This includes the uauth table used by Cut-through Proxy (CTP), HTTP sessions (by default), the local routing table, the ARP table, DHCP server addresses that are currently leased, SSM card information, and certain WebVPN components (AnyConnect client sessions, clientless sessions, Citrix authentication, and so on).

Tip

HTTP connections in the conn table of the active unit are not by default replicated to the standby unit. Cisco set this as the default since downloading a web page can involve many connections that typically are very short-lived. If most of your traffic is web based, this can create a huge processing burden to replicate all these changes. You have the option of enabling HTTP connection replication, but I typically don’t recommend it. If a failover occurs in the middle of someone opening a web page, the easiest way to fix this problem is to have the users click their refresh button within their web browser application.

Failover Triggers

Many things can trigger failover on an appliance: loss of power, one or more interfaces failing, a card failing, a software problem like memory exhaustion, or someone forcing failover with the failover active command on the standby unit. Based on the amount of time it takes to detect a problem, cutover to the standby unit might not be immediate. Table 23-1 shows the cutover times for ASAs, and Table 23-2 for PIXs.

Table 23-1: ASA Failover Times

Failover Condition	Default Time	Minimum Time	Maximum Time
Active unit loses power or stops normal operation	15 seconds	800 ms	45 seconds
Active unit motherboard interface is down	5 seconds	500 ms	15 seconds
Active unit 4-GE card interface is down	5 seconds	2 seconds	15 seconds
Active unit IPS or CSC card fails	2 seconds	2 seconds	2 seconds
Active unit interface is up, but has connection problems that cause interface testing	25 seconds	5 seconds	75 seconds

Table 23-2: PIX Failover Times

Failover Condition	Default Time	Minimum Time	Maximum Time
Active unit loses power or stops normal operation	15 seconds	800 ms	45 seconds
Active unit interface is up, but has connection problems that cause interface testing	25 seconds	5 seconds	75 seconds

Failover Link Monitoring

Basically two types of interfaces are monitored by the failover pair: failover link and data interfaces. This section will discuss what monitoring on the failover link is, and the next section will discuss monitoring of the data interfaces.

Failover hello messages are generated on the failover link every 15 seconds by default. (I’ll show you how to change this later in the chapter.) If three consecutive hello messages are missed from a failover mate, an ARP is generated on all the appliance interfaces. If no response is received from the mate on any of the interfaces, failover will take place, with the unit promoting itself to an active state, assuming that it was in a standby state. If no response to the ARP is seen on the failover link, but a response is seen on one of the other interfaces (stateful link or data interfaces), then failover will not occur. In this situation, the failover link is marked as “failed.”

Interface Monitoring

Interface monitoring is used to monitor the status of any of the data or stateful link interfaces on the appliance. Failover hello messages are generated on all active interfaces. These are the same messages used on the failover link connection. Up to 250 interfaces can be monitored per appliance. If a hello message from a mate is not seen on a monitored interface for one-half the hold-down period, the appliance will run interface tests on the suspect interface to determine what, if any, problem exists.

The purpose of the interface tests is to determine which unit, if any, has had a failure. Before each test begins, the received packet count statistic is cleared on the interface. At the conclusion of each test, each appliance checks to see if any valid frames/packets were received; and if so, the interface is considered operational. If no traffic is seen for a particular test, then the appliance proceeds to the next test. The four interface tests that the appliance may run include the following:

• Link up/down test The suspect interface is disabled and re-enabled, where normal interface hardware diagnostics are run.

• Network activity test The appliance looks for valid frames coming into the interface for up to 5 seconds.

• ARP test The appliance generates ARP queries for the two most recent entries in the ARP table, where the appliance is looking for any valid frame (not just an ARP reply) coming into the interface for up to 5 seconds.

• Broadcast ping test The appliance generates a broadcast ping, where again, the appliance is looking for any valid frame (not just an ICMP echo reply) coming into the interface for up to 5 seconds.

Table 23-3 lists the possible results that can occur and what will occur based on the results.

Table 23-3: Failover Results and Responses

Failover Result	Failover Response
Both appliances see no valid frames on the interface being tested.	No failover takes place.
Both appliances see a valid frame on the interface being tested.	No failover takes place.
The active unit sees a valid frame on the interface being tested, but the standby unit doesn’t.	No failover takes place.
The active unit doesn’t see a valid frame on the interface being tested, but the standby unit does.	Failover takes place.

Switch Connections

Normally the appliances are connected to switches for layer 2 connectivity. Based on the fact that the appliances generate failover messages on all their active interfaces by default, it is important that nothing disrupts this process, causing inadvertent failovers, or, in a worst-case situation, temporarily having both appliances in an active state.

To greatly reduce the likelihood of unattended failovers, you should make sure that the paired interfaces on the mates can see each other’s hello messages. Therefore, on the switch, you need to make sure that the interfaces are in the same VLAN. (If this is impossible, you can disable monitoring for the unconnected interface, which I discuss later in the chapter.) Next make sure that any disruptions that STP might create won’t cause the appliance interfaces to be placed into a nonforwarding state, like blocking. For Cisco Catalyst switches, you should enable the PortFast feature, which keeps the ports in a forwarding state when changes in STP are occurring. If you forget to do this and your switches are not using rapid STP (RSTP), but are using IEEE’s original implementation (802.1d), then STP recalculations, which could take anywhere from 30 to 45 seconds, will probably cause the appliances to miss three hello messages and cause possible failover problems.

Active/Standby Configuration

The last two sections in this chapter will discuss how to configure failover on the appliances. This section discusses how to set up active/standby failover by using the serial cable on the PIXs or by using LBF. The second section discusses how to set up active/active failover on the appliances.

Active/Standby—PIXs and the Serial Cable

Of all the failover configurations to set up, setting up active/standby on two PIXs using the serial cable for the failover link is definitely the easiest. This and the next few pages will discuss how to set up failover on PIXs using the serial cable. To make it more readable, I’ve broken up the configuration into easily understood steps.

Active/Standby Using the Serial Cable—Step 1

In the first step, make sure the secondary appliance is not connected to the network. If the secondary appliance has any configuration on it, erase it:

secondary# write erase

Then turn the secondary appliance off.

Active/Standby Using the Serial Cable—Step 2

Once the secondary unit is turned off, cable up the appliances, making sure you plug the primary end of the serial cable into the primary appliance, and the secondary end of the cable into the secondary appliance. Also connect the data interface to your switches and the stateful link connection if you’ll be enabling stateful failover.

Note

When cabling up the appliances, if e0 on a PIX or e0/1 on an ASA is connected to the outside on the primary, you must also cable it up this way on the secondary. The cabling of the interfaces of the two appliances must match.

Active/Standby Using the Serial Cable—Step 3

After you have connected the two appliances to the network, you’re ready to begin the configuration of the primary appliance. First, you’ll configure the IP addresses on the data interfaces. This is slightly different than what was discussed in Chapter 3, where each interface had a single IP address. With failover, the interface will have two IP addresses: one address will be used by the active appliance, and the second address by the standby appliance.

Here’s the command to configure the IP addressing on the data interfaces:

primary(config)# interface phy_if_name
primary(config-if)# ip address active_IP_addr net_mask
                       standby standby_IP_addr

Active/Standby Using the Serial Cable—Step 4

After configuring the IP addresses on the data interfaces, you’re ready to enable failover on the primary/active appliance with the failover command:

primary(config)# failover

This command enables failover on the appliance; and since the secondary unit is turned off, you’re guaranteeing that this appliance will take over the active role.

Note

Either before or after this step, you can enable optional failover commands, discussed later in the “Active/Standby: Optional Commands” section.

Active/Standby Using the Serial Cable—Step 5

After enabling failover on the primary appliance, verify the failover process with the show failover command. Here’s an example of verifying failover on the primary appliance when the secondary hasn’t been turned on yet:

primary# show failover
Failover On
Failover unit Primary
Failover LAN Interface: N/A - Serial-based failover enabled
Unit Poll frequency 500 milliseconds, holdtime 6 seconds
Interface Poll frequency 600 milliseconds, holdtime 15 seconds
Interface Policy 1
Monitored Interfaces 2 of 250 maximum
Version: Ours 7.2(1), Mate Unknown
Last Failover at: 13:21:38 UTC Dec 10 2006
       This host: Primary – Active
            Active time: 200 (sec)
            Interface outside (192.168.1.2): Normal (Waiting)
            Interface inside (10.0.1.1): Normal (Waiting)
       Other host: Secondary – Not detected
            Active time: 0 (sec)
            Interface outside (192.168.1.7): Unknown (Waiting)
            Interface inside (10.0.1.7): Unknown (Waiting)

Stateful Failover Logical Update Statistics
       Link : Unconfigured

Notice that at the top of the preceding example, this appliance is using serial-based failover, with the primary end of the cable plugged into it. At the bottom of the example, notice that this appliance is in an active role, and the secondary appliance hasn’t been detected yet (because it is turned off). Also notice at the very bottom of the example that stateful failover hasn’t been configured.

Active/Standby Using the Serial Cable—Step 6

Now that the primary appliance is ready, turn on the secondary appliance. If you happen to be at the console of the primary when the secondary finishes booting up, you see the following message displayed on the primary:

Detected an active mate
Beginning configuration replication to mate.
End configuration replication to mate.

This message indicates that the primary appliance configuration has been successfully replicated to the secondary. If you happen to be on the secondary console when it finishes booting up, the message will read from mate instead of to mate.

Active/Standby Using the Serial Cable—Step 7

Once you have turned on the secondary appliance and the configuration replication takes place, use the show failover command on the primary (or secondary) to verify the operation of failover. Here’s an example of the use of this command on the primary appliance:

primary# show failover
Failover On
Failover unit Primary
Failover LAN Interface: N/A - Serial-based failover enabled
Unit Poll frequency 500 milliseconds, holdtime 6 seconds
Interface Poll frequency 600 milliseconds, holdtime 15 seconds
Interface Policy 1
Monitored Interfaces 3 of 250 maximum
Version: Ours 7.2(1), Mate 7.2(1)
Last Failover at: 13:21:38 UTC Dec 10 2006
       This host: Primary – Active
               Active time: 320 (sec)
               Interface outside (192.168.1.2): Normal
               Interface inside (10.0.1.1): Normal
   Other host: Secondary – Standby Ready
               Active time: 0 (sec)
               Interface outside (192.168.1.7): Normal
               Interface inside (10.0.1.7): Normal
Stateful Failover Logical Update Statistics
       Link : Unconfigured

At the bottom of the display, notice that this appliance is the primary and is performing the active role; the secondary has been detected and is performing the standby role. Two interfaces were configured on the primary, where the configuration was replicated to the secondary: you can see the logical names and IP addresses each is using on its data interfaces, as well as the status of these interfaces.

Active/Standby Using the Serial Cable—Step 8

The last step, which is optional, but highly recommended, is to test your failover configuration. You could turn off the primary appliance, or, if you are at the console of the secondary appliance, execute the failover active command:

secondary(config)# failover active

Then use the show failover on the secondary appliance to make sure it is performing the active role and the primary appliance (assuming it is up) is performing the standby role.

Active/Standby—LBF

In this section I’ll discuss how to implement active/standby failover using LAN-based failover (LBF). The PIXs support this, as well as serial cabling; however, the ASAs support only this approach. In active/standby failover for a PIX or ASA using the LBF cable, the first three steps are the same as those for using a serial failover cable on the PIXs. After this, the configuration is radically different.

Active/Standby Using LBF—Step 1

In the first step, make sure the secondary appliance is not connected to the network. If the secondary appliance has any configuration on it, erase it:

secondary# write erase

Then turn the secondary appliance off. This is the same as Step 1 of a PIX using a serial cable.

Active/Standby Using LBF—Step 2

Once the secondary unit is turned off, then cable up the appliances’ Ethernet interfaces: the data interfaces, the LBF interface, and the stateful interface (if used and if different from the LBF interface). Again this is basically the same as Step 2 of a PIX using a serial cable.

Active/Standby Using LBF—Step 3

After you have connected the two appliances to the network, you’re ready to begin the configuration of the primary appliance. First, you’ll configure the IP addresses on the data interfaces, which was discussed previously in Step 3 of a PIX using a serial cable:

primary(config)# interface phy_if_name
primary(config-if)# ip address active_IP_addr net_mask
                       standby standby_IP_addr

Active/Standby Using LBF—Step 4

The configuration of LBF at this point greatly differs from the PIXs using a serial failover cable. First, you’ll need to determine which interface you’ll be using on the appliances for the failover link—this must be exactly the same on both appliances. Then you need to enable the interface for LBF: this can be a physical interface or a logical subinterface associated with a particular VLAN:

primary(config)# interface physical_LBF_if_name
primary(config-if)# no shutdown

In either case, the failover link cannot be a data interface with a security level (security-level command), logical name (nameif command), or IP address (ip address command).

After you’ve enabled the LBF interface on the primary appliance, you’re ready to configure failover:

primary(config)# failover lan enable
primary(config)# failover lan unit primary
primary(config)# failover lan interface logical_LBF_if_name
                       physical_LBF_if_name
primary(config)# failover interface ip logical_LBF_if_name
                       primary_IP_addr subnet_mask
                       standby secondary_IP_addr
primary(config)# failover key encryption_key
primary(config)# failover
primary# show failover

The failover lan enable command is only applicable to the PIXs: it disables the use of the serial cable interface (this command doesn’t exist on the ASAs). The failover lan unit primary command specifies the role the appliance should perform; in this case the appliance is the primary. The failover lan interface command assigns a logical name of the LBF interface. The failover interface ip command assigns the primary and secondary IP addresses to the LBF connection. When failover occurs, these addresses are not changed on the appliances. Optionally you can encrypt the LBF messages using the failover key command. If you want to encrypt messages, the primary and secondary units must use the same key. Finally, the failover command enables failover on the primary; at this point you should verify the status of the primary appliance with the show failover command.

Active/Standby Using LBF—Step 5

Once the primary has been configured, you are ready to configure the secondary. Unlike serial failover where all you had to do was cable up the secondary and turn it on for the synchronization to take place, LBF communicates with IP, so you’ll need a minimal configuration on the secondary appliance. Actually the configuration done in Step 4 for the primary is almost the same configuration you’ll execute on the secondary ... with one exception: the secondary appliance must have a role of “secondary” so that it knows to use the second IP address in the failover interface ip command. This is accomplished by using the failover lan unit secondary (instead of the failover lan unit primary) command:

secondary(config)# failover lan unit secondary

Active/Standby Using LBF—Step 6

Now you’re ready to verify your failover operation with the show failover command on the paired appliances. Here’s an example of the status of failover of a secondary appliance after Step 5 has been completed:

secondary(config)# show failover
Failover On
Failover unit Secondary
Failover LAN Interface: LANFAIL GigabitEthernet0/2 (up)
Unit Poll frequency 500 milliseconds, holdtime 6 seconds
Interface Poll frequency 600 milliseconds, holdtime 15 seconds
Interface Policy 1
Monitored Interfaces 3 of 250 maximum
Version: Ours 7.2(1), Mate 7.2(1)
Last Failover at: 18:03:38 UTC Dec 12 2006
 This host: Secondary – Standby Ready
   Active time: 0 (sec)
    slot 0: ASA5520 hw/sw rev (1.0/7.2(1)) status (Up Sys)
        Interface outside (192.168.1.7): Normal (Waiting)
        Interface inside (10.0.1.7): Normal (Waiting)
    slot 1: ASA-SSM-10 hw/sw rev (1.0/5.0(2)S152.0) status
             (Up/Up)IPS, 5.0(2)S152.0 Up
Other host: Primary – Active
  Active time: 3795 (sec)
        slot 0: ASA5520 hw/sw rev (1.0/7.2(1)) status (Up Sys)
        Interface outside (192.168.1.2): Normal (Waiting)
<--output omitted-->

Notice that in the preceding example this unit is the secondary, and the other unit, the primary, is performing the active role.

Active/Standby—Optional Commands

Other optional commands you can configure to tune your active/standby configuration are

ciscoasa(config)# failover link logical_if_name
ciscoasa(config)# failover replication http
ciscoasa(config)# [no] monitor-interface logical_if_name
ciscoasa(config)# failover polltime [unit | interface] [msec] time
                       [holdtime time]
ciscoasa(config)# failover interface-policy number[%]
ciscoasa(config)# failover mac address phy_if active_MAC_addr
                       standby_MAC_addr

To enable stateful failover, use the failover link command. If you are using LBF, the logical name for stateful failover can be the LBF logical name. By default HTTP connections are not replicated when you enable stateful failover. If you want to replicate HTTP connections, use the failover replication http command.

Monitoring of physical interfaces is enabled by default: the appliance generates failover keepalives on all active physical interfaces; monitoring of logical interfaces (subinterfaces) is disabled by default. With trunk connections the keepalives are sent in the native VLAN (untagged). You can change this behavior with the monitor-interface command and specify the logical name of the subinterface, causing the appliance to generate keepalives on the VLAN subinterfaces as well. Likewise if two appliances have a monitored interface that is not in the same broadcast domain, you can disable the keep- alive function on the respective interface with the no monitor-interface command.

The failover polltime command determines how often failover hello messages (keepalives) are generated on interfaces—LBF, stateful, and data interfaces. By default this is 15 seconds. The unit parameter applies the keepalive interval to all interfaces, including LBF or the serial, whereas the interface parameter specifies the keepalive interval is for the data interfaces. Valid values for the failover polltime command are from 1 to 15 seconds or, if the msec keyword is used, from 500 to 999 milliseconds. The hold time determines how long it takes from the time a hello packet is missed to when the interface is marked as failed, where valid values are from 2 to 75 seconds. You cannot enter a hold time that is more than five times greater than the poll time interval.

By default a single interface failure will cause a failover. You can increase this value to affect the number of interfaces (or percentage of interfaces) that have to fail before a failover will occur. This is configured with the failover interface-policy command.

When in active/standby mode, the MAC addresses for the primary unit are always associated with the active IP addresses. However, if the secondary unit boots up first and thus assumes the active role, it uses its own burned-in (MAC) addresses (BIAs) for its interfaces. Thus, when the primary unit comes online and when it assumes the active role, the secondary unit will automatically obtain the MAC addresses from the primary unit and change its MAC addresses. Obviously this change can disrupt your user’s traffic, since the active unit MAC addresses changed from the secondary ones to the primary ones. You can configure virtual MAC addresses for each interface with the exception of the failover (LBF) and stateful links, since these don’t change during a failover. To configure virtual MAC addresses, use the failover mac address command.

Active/Standby—Example Configuration

To help you better understand how to configure active/standby on the appliances, let’s look at an example. I’ll use the network shown in Figure 23-6. I’ll use two ASA 5510s, which require LBF for the failover LAN link. In this example, I’ll primarily focus on the failover configuration of the two appliances. To make the example more readable, I’ve broken it into different sections.

Figure 23-6: Active/standby example using LBF

Active/Standby Example—Primary Data Interfaces

I’ll first set up the data interfaces on the primary, which are e0/0 and e0/1:

primary(config)# interface e0/0
primary(config-if)# no shutdown
primary(config-if)# ip address 192.168.11.1 255.255.255.0
                       standby 192.168.11.2
primary(config-if)# nameif outside
primary(config-if)# security-level 0
primary(config-subif)# exit
primary(config)# interface e0/1
primary(config-if)# no shutdown
primary(config-if)# ip address 10.0.1.111 255.255.255.0
                       standby 10.0.1.112
primary(config-if)# nameif inside
primary(config-if)# security-level 100
primary(config-if)# exit

Note

For an existing appliance, you would only have to change the IP addresses on the existing interfaces for failover, specifying the standby unit address for each interface you are using.

Active/Standby Example—Primary Failover Configuration

Next I’ll set up the LBF interface (e0/2) on the primary and enable failover:

primary(config)# interface e0/2
primary(config-if)# no shutdown
primary(config-if)# exit
primary(config)# failover lan unit primary
primary(config)# failover lan interface lanfail e0/2
INFO: Non-failover interface config is cleared on Ethernet0/2
and its sub-interfaces
primary(config)# failover interface ip lanfail
                       172.16.100.1 255.255.255.0 standby 172.16.100.2
primary(config)# failover

Now I’ll verify the failover operation and configuration on the primary:

primary(config)# show failover
Failover On
Cable status: N/A - LAN-based failover enabled
Failover unit Primary
Failover LAN Interface: lanfail Ethernet0/2 (up)
Unit Poll frequency 15 seconds, holdtime 45 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 0 of 250 maximum
Version: Ours 8.0(3), Mate Unknown
Last Failover at: 00:25:08 UTC Jan 1 1993
       This host: Primary – Active
               Active time: 465 (sec)
                 Interface outside (192.168.11.1): Normal
                 Interface inside (10.0.1.111): Normal
       Other host: Secondary - Not Detected
               Active time: 0 (sec)
                 Interface outside (192.168.11.2): Unknown
                 Interface inside (10.0.1.112): Unknown

Stateful Failover Logical Update Statistics
       Link : Unconfigured.

In the preceding output, notice that LBF is enabled, this unit is the primary and is performing the active role, and the LBF interface is called lanfail and is operational. Also notice that the secondary appliance hasn’t been detected (we haven’t configured it yet).

Active/Standby Example—Secondary Failover Configuration

Now that the primary has been configured, I’ll set up the secondary. Before you begin, make sure you have cleared the configuration on the secondary. Here’s the configuration of the secondary appliance:

secondary(config)# interface e0/2
secondary(config-if)# no shutdown
secondary(config-if)# exit
secondary(config)# failover lan unit secondary
secondary(config)# failover lan interface lanfail e0/2
INFO: Non-failover interface config is cleared on Ethernet0/2 and
    its sub-interfaces
secondary(config)# failover interface ip lanfail
                      172.16.100.1 255.255.255.0  standby 172.16.100.2
secondary(config)# failover
Detected an Active mate
Beginning configuration replication from mate.
End configuration replication from mate.
primary(config)#

As you can see, the only difference between this and the primary configuration is the failover lan unit command. One other item to point out is the bottom of the configuration: notice that once the replication from the primary is complete, the prompt on the secondary appliance changed from “secondary” to “primary.” This occurred because the primary replicated its entire configuration, including the hostname command, which the secondary automatically executes.

Let’s go ahead and verify the operation of failover on the secondary appliance:

primary(config)# show failover
Failover On
Cable status: N/A - LAN-based failover enabled
Failover unit Secondary
Failover LAN Interface: lanfail Ethernet0/2 (up)
Unit Poll frequency 15 seconds, holdtime 45 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 0 of 250 maximum
Version: Ours 8.0(3), Mate 8.0(3)
Last Failover at: 20:37:30 UTC Jul 2 2008
       This host: Secondary - Standby Ready
               Active time: 0 (sec)
                 Interface outside (192.168.11.2): Normal
                 Interface inside (10.0.1.112): Normal
       Other host: Primary – Active
               Active time: 1545 (sec)
                  Interface outside (192.168.11.1): Normal
                 Interface inside (10.0.1.111): Normal

Stateful Failover Logical Update Statistics
       Link : Unconfigured.

In the preceding output, notice that the appliance is configured as the secondary and is performing the standby role.

Active/Standby Example—Optional Commands

Once failover is operational, I decide to tune it by enabling stateful failover and changing the hello interval with the following commands:

primary(config)# failover link lanfail
primary(config)# failover polltime msec 500
INFO: Failover unit holdtime is set to 2 seconds

In this example, I’m using the LBF link for both failover communications and replication of the state table.

Note

With few exceptions, all commands must be executed on the active appliance and are then replicated across the failover link to the standby appliance.

I’ll verify my failover configuration with the show failover command on the primary/active unit:

primary(config)# show failover
Failover On
Cable status: N/A - LAN-based failover enabled
Failover unit Secondary
Failover LAN Interface: lanfail Ethernet0/2 (up)
Unit Poll frequency 500 milliseconds, holdtime 2 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 0 of 250 maximum
Version: Ours 8.0(3), Mate 8.0(3)
Last Failover at: 21:42:26 UTC Jul 2 2008
       This host: Primary – Active
               Active time: 452 (sec)
                 Interface outside (192.168.11.1): Normal
                 Interface inside (10.0.1.111): Normal
       Other host: Secondary - Standby Ready
               Active time: 0 (sec)
                 Interface outside (192.168.11.2): Normal
                 Interface inside (10.0.1.112): Normal
Stateful Failover Logical Update Statistics
       Link : lanfail Ethernet2 (up)
       Stateful Obj    xmit       xerr      rcv        rerr
       General         17         0         15         0
       sys cmd         15         0         15         0
       up time         0          0         0          0
       RPC services    0          0         0          0
       TCP conn        0          0         0          0
       UDP conn        0          0         0          0
<--output omitted-->

Notice that the bottom of the display has changed. You can see that the stateful link is “lanfail,” as well as see the count of the different types of information replicated to the standby appliance. Now that you have failover up and running, you can go ahead and complete the configuration on the active appliance, like configuring translation rules, ACLs, MPF, and so on.

Active/Active Configuration

Active/active failover configuration requires that you place the appliances in multiple mode to set up contexts. You’ll need two contexts, as previously described in Figure 23-3. The following sections will lead you through the process of setting up active/active failover, followed by an example.

Active/Active—LBF Configuration

Even though active/active is supported on the PIXs using the serial cable, this part of the chapter will only discuss the use of LBF. Some of the steps are the same as or similar to the steps performed when configuring active/standby.

Active/Active LBF Configuration—Step 1

The first three steps are basically the same as those used with active/standby LBF. In the first step, make sure the secondary appliance is unconnected to the network. If the secondary appliance has any configuration on it, erase it:

secondary# write erase

Then turn the secondary appliance off.

Active/Active LBF Configuration—Step 2

Once the secondary unit is turned off, cable up the appliances Ethernet interfaces: the data interfaces, the LBF interface, and the stateful interface (if used and if different from the LBF interface). Again this is the same as you would do if configuring active/standby.

Active/Active LBF Configuration—Step 3

Next switch the primary appliance to multiple mode (mode multiple command). Then enable the interfaces you will use on the primary appliance. Once this is done, create your two contexts, allocate the interfaces to them, and specify the location of their configuration files. This process was discussed in Chapter 22.

Active/Active LBF Configuration—Step 4

Once you’ve created the two contexts, you’ll need to go into them (changeto context command) and configure the two sets of addresses (active and standby) for each data interface. The command to do this is the same used when configuring active/standby, except that you must be in a context to execute it:

ciscoasa(config)# interface phy_if_name
ciscoasa/context(config-if)# ip address active_IP_addr net_mask
                                 standby standby_IP_addr

Note

Actually creating the contexts and assigning the IP addresses to the data interfaces can be done at any time—before or after the configuration of failover.

Active/Active LBF Configuration—Step 5

Enabling active/active failover using LBF for the failover link is similar to configuring active/standby. Failover is actually configured within the system area—not in the two contexts. In the system area, you’ll need to enable the LBF interface and configure your failover commands:

primary(config)# interface physical_LBF_if_name
primary(config-if)# no shutdown
primary(config-if)# exit
primary(config)# failover lan enable
primary(config)# failover lan unit primary
primary(config)# failover lan interface logical_LBF_if_name
                       physical_LBF_if_name
primary(config)# failover interface ip logical_LBF_if_name
                       primary_IP_addr net_mask
                       standby secondary_IP_addr
primary(config)# failover key encryption_key

As you can see from the preceding, the commands are the same on the primary. Notice that the commands are configured in the system area, not in a particular context. And the LBF interface is assigned IP addressing in the system area—this is one exception to the rule of assigning IP addresses when using contexts. If you’re using a PIX and will be using LBF, you’ll need to execute the failover lan enable command. Also, don’t enable failover at this point; you’ll need to create the failover groups first, which is discussed in the next step.

Active/Active LBF Configuration—Step 6

In this step, you’ll need to create two failover groups. The function of a failover group is to determine what role, primary or secondary, an appliance will perform for an associated context. For example, if you had contexts CTX1 and CTX2, you would make one context a member of failover group 1 and the other a member of group 2, splitting the roles—primary and secondary—between the two contexts. This is only done on the primary unit and will be synchronized with the secondary unit, along with whatever contexts you create, across the failover link (LBF).

Note

The synchronization of the contexts themselves to the secondary is a great feature, since this really reduces the amount of configuration you’ll have to do on the secondary appliance!

To create your failover groups, use the following configuration:

primary(config)# failover group 1
primary(config-fover-group)# primary
primary(config-fover-group)# exit
primary(config)# failover group 2
primary(config-fover-group)# secondary
primary(config-fover-group)# exit

To associate a failover group to a context, so that the appliance knows what role it should play for a context, use the following configuration:

primary(config)# context context_name
primary(confix-ctx)# join-failover-group {1 | 2}
primary(confix-ctx)# exit

Once you’ve associated the failover groups to the correct contexts, you can go ahead and enable failover on the primary:

primary(config)# failover

You can use the show failover command to verify your configuration:

primary# show failover [group group_#]

Optionally you can qualify the output, examining the failover information for a particular failover group (context).

Active/Active LBF Configuration—Step 7

Once the primary LBF configuration is complete, you’re now ready to set up the secondary unit. If you haven’t already done this, you’ll need to switch the secondary to multiple mode (mode multiple command). Once the secondary has switched to multiple mode, from the system area you can set up LBF. The LBF configuration on the secondary is basically the same as step 5 on the primary—the exception is the failover lan unit command.

secondary(config)# interface physical_LBF_if_name
secondary(config-if)# no shutdown
secondary(config-if)# exit
secondary(config)# failover lan enable
secondary(config)# failover lan unit secondary
secondary(config)# failover lan interface logical_LBF_if_name
                       physical_LBF_if_name
secondary(config)# failover interface ip logical_LBF_if_name
                       primary_IP_addr net_mask
                       standby secondary_IP_addr
secondary(config)# failover key encryption_key
secondary(config)# failover

Again note that you are in the system area when setting up LBF. Also notice that you don’t have to configure failover groups or contexts—just enable failover with the failover command. Once you’ve done this, assuming you’ve configured everything correctly, the secondary should connect to the primary, and the two should synchronize, where the secondary will automatically get this information from the primary.

Active/Active—Optional Commands

Many of the optional configurations available in active/standby exist in active/active, as well as a few others—the main difference is where and how the commands are executed. Here are the optional commands you can configure for active/active failover:

standby(config)# [no] failover active group group_#
active(config)# failover link logical_LBF_if_name
active(config)# failover group {1 | 2}
active(config-fover-group)# preempt [seconds]
active(config-fover-group)# replicate http
active(config-fover-group)# polltime interface [msec] time
                                 [holdtime time]
active(config-fover-group)# interface-policy number[%]

The failover active group command is executed on the standby appliance: it forces the standby to promote itself to the active role for the contexts in the specified group; and the currently active appliance will demote itself to a standby role for the group. The failover link command enables stateful failover on the appliance.

The remaining optional commands are configured on a per-failover group basis. Preemption allows a unit to come back online and preempt the other unit if the failed unit was originally performing the active function for a context. To enable preemption, use the preempt command—you can specify the number of seconds to wait before preemption occurs. To change the failover hello interval, use the polltime command. To change how many data interfaces have to fail before failover takes place, use the interface-policy command.

Active/Active—Example Configuration

To help you better understand how to configure active/active failover on the appliances, let’s look at an example. I’ll use the network shown in Figure 23-7. I’ll use two ASA 5510s, which require LBF for the failover LAN link. In this example, I’ll primarily focus on the failover configuration of the two appliances. To make the example more readable, I’ve broken it into different sections.

Figure 23-7: Active/active example using LBF

Active/Active Example—Primary Initial Configuration

First you’ll need to switch the primary appliance to multiple mode and then enable the interfaces, as shown here:

primary(config)# mode multiple
<--output omitted-->
primary(config)# interface e0/0
primary(config-if)# no shutdown
primary(config-if)# exit
primary(config)# interface e0/1
primary(config-if)# no shutdown
primary(config-if)# exit
primary(config)# interface e0/2
primary(config-if)# no shutdown
primary(config-if)# exit

Next I’ll create the subinterfaces that will be used for trunking on e0/0 and e0/1:

primary(config)# interface e0/0.1
primary(config-subif)# vlan 311
primary(config-subif)# exit
primary(config)# interface e0/0.2
primary(config-subif)# vlan 312
primary(config-subif)# exit
primary(config)# interface e0/1.1
primary(config-subif)# vlan 101
primary(config-subif)# exit
primary(config)# interface e0/1.2
primary(config-subif)# vlan 102
primary(config-subif)# exit

E0/0.1 and E0/1.1 will be in the ct1 context, and E0/0.2 and E0/1.2 will be in the ct2 context.

Active/Active Example—Primary Failover Configuration

Next I’ll configure the failover commands for the primary, using E0/2 as the LBF interface:

primary(config)# failover lan interface lanfail e0/2
primary(config)# failover interface ip lanfail
                       172.16.100.1 255.255.255.0 standby 172.16.100.2
primary(config)# failover link lanfail
primary(config)# failover lan unit primary
primary(config)# failover polltime msec 500
primary(config)# failover group 1
primary(config-fover-group)# exit
primary(config)# failover group 2
primary(config-fover-group)# exit
primary(config)# failover
       Group 1 No Response from Mate, Switch to Active
       Group 2 No Response from Mate, Switch to Active

In the preceding, example I enabled LBF and stateful failover, changed the hello interval, and created the two failover groups.

Here’s the failover status of the primary:

primary(config)# show failover
Failover On
Cable status: N/A - LAN-based failover enabled
Failover unit Primary
Failover LAN Interface: lanfail Ethernet0/2 (up)
Unit Poll frequency 500 milliseconds, holdtime 2 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 0 of 250 maximum
Version: Ours 8.0(3), Mate Unknown
Group 1 last failover at: 00:11:14 UTC Jan 1 1993
Group 2 last failover at: 00:11:14 UTC Jan 1 1993

 This host:    Primary
 Group 1       State:          Active
               Active time:    103 (sec)
 Group 2       State:          Active
               Active time:    103 (sec)

 Other host:   Secondary
 Group 1       State:          Not Detected
               Active time:    0 (sec)
 Group 2       State:          Not Detected
               Active time:    0 (sec)
<--output omitted-->

Notice that the primary is in an active state for both groups, since the secondary hasn’t been configured yet.

Active/Active Example—Primary Context Configuration

After completing the failover configuration on the primary, I’ll create and set up the contexts. Again, this is done on the primary appliance. Actually this step can occur before or after the failover configuration. I’ll create two contexts: ct1 and ct2. Here’s the configuration, which I covered in Chapter 22:

primary(config)# admin-context ct1
primary(config)# context ct1
primary(config-ctx)# allocate-interface e0/0.1
primary(config-ctx)# allocate-interface e0/1.2
primary(config-ctx)# config-url flash:/ct1.cfg
primary(config-ctx)# join-failover-group 1
primary(config-ctx)# exit
primary(config)# context ct2
primary(config-ctx)# allocate-interface e0/0.2
primary(config-ctx)# allocate-interface e0/1.2
primary(config-ctx)# config-url flash:/ct2.cfg
primary(config-ctx)# join-failover-group 2
primary(config-ctx)# exit

After creating the contexts, I’ll switch to them and set up the IP addressing on their data interfaces:

primary(config)# changeto context ct1
primary/ct1(config)# interface e0/0.1
primary/ct1(config-if)# nameif outside
primary/ct1(config-if)# security-level 0
primary/ct1(config-if)# ip address 192.168.11.1 255.255.255.0
                            standby 192.168.11.2
primary/ct1(config-if)# no shutdown
primary/ct1(config-if)# exit
primary/ct1(config)# interface e0/1.1
primary/ct1(config-if)# nameif inside
primary/ct1(config-if)# security-level 100
primary/ct1(config-if)# ip address 10.0.1.111 255.255.255.0
                            standby 10.0.1.112
primary/ct1(config-if)# no shutdown
primary/ct1(config-if)# exit
primary/ct1(config)# changeto context ct2
primary/ct2(config)# interface e0/0.2
primary/ct2(config-if)# nameif outside
primary/ct2(config-if)# security-level 0
primary/ct2(config-if)# ip address 192.168.12.2 255.255.255.0
                            standby 192.168.12.1
primary/ct2(config-if)# no shutdown
primary/ct2(config-if)# exit
primary/ct2(config)# interface e0/1.2
primary/ct2(config-if)# nameif inside
primary/ct2(config-if)# security-level 100
primary/ct2(config-if)# ip address 10.0.2.112 255.255.255.0
                            standby 10.0.2.111
primary/ct2(config-if)# no shutdown
primary/ct2(config-if)# exit

Note

I skipped the rest of the configuration for each context, like address translation, ACLs, static routing, and so on—this will still need to be completed.

Active/Active Example—Secondary Configuration

Now that the primary is finished, I’ll connect to the console of the secondary appliance and set it up:

secondary(config)# mode multiple
<--output omitted-->
secondary(config)# interface e0/2
secondary(config-if)# no shutdown
secondary(config-if)# exit
secondary(config)# failover lan interface lanfail e0/2
secondary(config)# failover interface ip lanfail
                       172.16.100.1 255.255.255.0 standby 172.16.100.2
secondary(config)# failover lan unit secondary
secondary(config)# failover
Detected an Active mate
Beginning configuration replication from mate.
WARNING: Unable to delete ct1 context, because it doesn't exist.
INFO: Admin context is required to get the interfaces
Creating context 'ct1'... Done. (1)
WARNING: Skip fetching the URL flash:/ct1.cfg
INFO: Creating context with default config
INFO: ct1 context will take some time to come up .... please wait.
Creating context 'ct2'... Done. (2)
WARNING: Skip fetching the URL flash:/ct2.cfg
INFO: Creating context with default config
       Group 1 Detected Active mate
       Group 2 Detected Active mate

Notice that once failover was enabled, the secondary automatically pulled the two contexts (and their configurations) into itself.

Here’s the status of the secondary appliance at this point:

secondary(config)# show failover
Failover On
Cable status: N/A - LAN-based failover enabled
Failover unit Secondary
Failover LAN Interface: lanfail Ethernet0/2 (up)
Unit Poll frequency 500 milliseconds, holdtime 2 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 0 of 250 maximum
Version: Ours 8.0(3), Mate 8.0(3)
Group 1 last failover at: 00:52:14 UTC Jan 1 1993
Group 2 last failover at: 00:52:14 UTC Jan 1 1993

 This host:    Secondary
 Group 1       State:          Standby Ready
               Active time:    0 (sec)
 Group 2       State:          Standby Ready
               Active time:    0 (sec)
<--output omitted-->

Notice that the secondary is in a standby state for both contexts.

Active/Active Example—Primary Preemption Configuration

Since one of the purposes of using active/active failover is to have both appliances process traffic, you’ll need to configure preemption to get around the issue shown in the secondary status from the previous show failover command. This is done on the primary appliance:

primary(config)# failover group 1
primary(config-fover-group)# primary
primary(config-fover-group)# preempt
primary(config-fover-group)# exit
primary(config)# failover group 2
primary(config-fover-group)# secondary
primary(config-fover-group)# preempt
primary(config-fover-group)# exit

Now when I examine the status of failover on either appliance, I should see that one is active for one context, and the other appliance is active for the second context:

primary(config)# show failover
Failover On
Cable status: N/A - LAN-based failover enabled
Failover unit Primary
Failover LAN Interface: lanfail Ethernet0/2 (up)
Unit Poll frequency 500 milliseconds, holdtime 2 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 0 of 250 maximum
Version: Ours 8.0(3), Mate 8.0(3)
Group 1 last failover at: 01:15:13 UTC Jan 1 1993
Group 2 last failover at: 01:11:19 UTC Jan 1 1993

 This host:    Primary
 Group 1       State:          Active
               Active time:    329 (sec)
 Group 2       State:          Standby Ready
               Active time:    0 (sec)

      ct1 Interface outside (192.168.11.1): Normal (Not-Monitored)
      ct1 Interface inside (10.0.1.111): Normal (Not-Monitored)
      ct2 Interface outside (192.168.12.1): Normal (Not-Monitored)
      ct2 Interface inside (10.0.2.111): Normal (Not-Monitored)

 Other host:   Secondary
 Group 1       State:          Standby Ready
               Active time:    367 (sec)

  Group 2       State:          Active
               Active time:    696 (sec)

      ct1 Interface outside (192.168.11.2): Normal (Not-Monitored)
      ct1 Interface inside (10.0.1.112): Normal (Not-Monitored)
      ct2 Interface outside (192.168.12.2): Normal (Not-Monitored)
      ct2 Interface inside (10.0.2.112): Normal (Not-Monitored)

Stateful Failover Logical Update Statistics
       Link : lanfail Ethernet0/2 (up)
<--output omitted-->

In the preceding example, the primary appliance is active for failover group 1 (ct1 context), and the secondary appliance is active for failover group 2 (ct2 context). Also notice the status of the data interfaces: “Not-Monitored.” Remember that subinterfaces by default are not included in the failover hello process—only the native VLAN on the trunk interface. You can optionally enable this feature, if needed, however.