ISP Redundancy works during testing but fails if cable is unplugged

[JRCNetworks]

I am trying to setup ISP Redundancy in Primary/Backup mode for one of our sites that has frequent internet outages with their current ISP. This firewall has R77.10 Gaia running on it.

When I use the "fw isp_link <link name> up/down" command to test the links everything seems to work. I have verified that the default route is being updated on the firewall. I have verified that I get the right speedtest results on each ISP link. The problem is that if I disconnect the cable for the primary ISP all internet traffic stops. This happens even if I already have that link set to down with the isp_link command. I think this may be a NAT issue but I'm not sure. I have our internal network set to hide behind the gateway as specified in the guide.

We do not have any DMZ or publicly accessible servers to worry about in this case, just the outgoing traffic from inside. My firewall object is using the external IP from ISP1 as its IP address, could this be messing up the NAT? Any help would be appreciated. Thanks.

[EricAnderson]

First, what do your logs tell you? You should see indications of the primary ISP failure, and the outbound traffic should indicate the correct Hide NAT (behind the secondary IP address). Is either not the case?

-E

[ShadowPeak.com]

Are your firewall's licenses bound to the IP address of the interface you are unplugging? Use "cplic print" on the firewall to check.

[JRCNetworks]

I do see messages in the logs when either link goes up or down. The Hide NAT address is not changing. It remains set to the IP address of the interface for the primary ISP.

Yes, the firewall licenses are tied to the IP of the interface used by the primary ISP

[EricAnderson]

I do see messages in the logs when either link goes up or down. The Hide NAT address is not changing. It remains set to the IP address of the interface for the primary ISP.

What's the message? Is it from ISP Redundancy? Is the gateway/hide address coded into the NAT settings, or is it set to "Hide behind gateway"?

-E

[JRCNetworks]

I am attaching screen shots of the messages and an example of one of the down and up message details. They are coming from the ISP Redundancy function. These were generated by unplugging / plugging the backup then primary ISP cables from the firewall interfaces. I see the same messages when I use the isp_link command to change their state. In case this helps, I have set both interfaces to use 8.8.8.8 for up/down testing instead of using the default next hop as the target.

I'm afraid I don't know exactly what you mean by "coded into the NAT settings". I believe the answer is that it is set to Hide Behind Gateway, you can see the hide NAT rule screenshot in my original post.

[ShadowPeak.com]

Yes, the firewall licenses are tied to the IP of the interface used by the primary ISP

When the primary ISP interface goes down, pretty sure all your firewall licenses will become invalid since they are bound to the IP address of that interface and it is now dead. ISP Redundancy is a licensed feature of the Advanced Networking blade which may be why it stops working and does not fail over when you unplug.

Get a 30-day eval license from your reseller bound to the internal IP address of the firewall and apply it. I bet the problem with unplugging that interface will go away and ISP Redundancy will work properly. If this proves out obviously next step would be to reissue all firewall licenses against the firewall's internal IP address. There is a common misconception that the IP address for license binding must be an Internet-routable address which is simply not true. Just needs to be the IP address of a valid configured interface that is up.