failover nat-problem

[veste]

hi all!

i've 2 splat ngx-r60/hfa04 with cluster-xl in ha-mode running here.

my problem is, when failing over to the standby-mashine the nat-rules to the internet aren't working anymore. vpn-access is still working, but this is without nat.

i do manual nat.
have added

echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp
which are my 2 interfaces connected to the internet, each with an own public net.

the mac-addresses to /etc/ethers
and also static routes for the natted ip-addresses.

everything works well on one firewall, but on the other, which is exactly identical (except the mac-entries in ethers-file), i've the problems :-(

seems to me like a arp problem, but i've no idea what else to check.
any ideas?

thx!

[Robby Cauwerts]

Hi,

check your arp setting on the two firewall nodes with:
fw ctl arp

This will show you the static arp settings.


What do you mean with "which are my 2 interfaces connected to the internet, each with an own public net."

Check what's going on on the wire with tcpdump on your external interfaces when you do a manual failover

Further:
echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
This method is not recommended, as SecurePlatform periodically resets this to 0

You should create a file local.arp that contains the arp entries. More/latest info about this: check SecureKnowledge and search for sk25851.

Kr.
Robby

[veste]

Originally Posted by Robby Cauwerts

Hi,
check your arp setting on the two firewall nodes with:
fw ctl arp

This will show you the static arp settings.
What do you mean with "which are my 2 interfaces connected to the internet, each with an own public net."

Check what's going on on the wire with tcpdump on your external interfaces when you do a manual failover

Further:
echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
This method is not recommended, as SecurePlatform periodically resets this to 0

You should create a file local.arp that contains the arp entries. More/latest info about this: check SecureKnowledge and search for sk25851.

Kr.
Robby

thanks,

i thought fw ctl arp relies only to automatic arp. i do it manually.
the output is: No proxy ARP entries

until now SPLAT doesn't overwrite my proxy_arp settings.

isn't local.arp only for windows?
when using the "addarp"-command the mac-address is added to the ethers-file and restored at boot-time.

eth0 and eth1 are connected to the internet with different isps (NO dialup).

with my service-level i don't have access to sk25851 :-(

[northlandboy]

So if you've got different MAC addresses in the proxy ARP configuration on both firewalls, then how does the upstream router know that you've failed over? Won't it still have the primary system's MAC addresses stored in its ARP cache?

If you suspect it's an ARP issue, then have you checked out the routers ARP cache? Looked at traffic on the wire with tcpdump -e?

[veste]

i didn't know about the -e switch from tcpdump, thanks - this helps a lot.

unfortunately i don't have access to the router, but while sniffing the packets exactly what you said happend:
when i do a manual failover, the returning packets are still sent to the previous active firewall.

i'm a little confused now: the mac-address of the cluster-ip gets changed within a second, but what happens with the mac-entries for nat?

please enlighten me! thanks.

[northlandboy]

Normally if you've got a Nokia cluster or VRRP pair, you would use the virtual cluster MAC address for any proxy ARP.

I can't remember what MAC ClusterXL/SPLAT uses for the cluster IP - if it's using the real MAC address of the member, then you've got a bit of a problem with manual NAT, don't you? With automatic proxy ARP, then it will only start publishing those ARPs when the cluster fails over.

I believe ClusterXL also sends gratuitous ARPs, which is why traffic to the cluster IP would fail over. (side note: I think it's poor design, the way systems update their ARP caches with ARP replies that are not in response to a request. But I digress). Looks like it doesn't also send gratuitous ARPs for the proxy ARP entries it has. Hmm. Bit of a problem then.

Either add /32 routes to the upstream router for all your NAT addresses, pointing to the cluster IP, or have a hunt around in the docs/KB to find out what the recommended solution is. If it's not a problem someone else has had/resolved, then there's something very wrong with the design of ClusterXL. Personally I think that using a virtual MAC, ala HSRP, VRRP, et al, is the way to do it.

[northlandboy]

Check out sk30197

Looks like you need to do it with $FWDIR/conf/local.arp, and it will then do gratuitous ARPs for the proxy ARPs as well.

If you're doing multicast MAC active/active, this is not an issue.

[veste]

Originally Posted by northlandboy

Check out sk30197

Looks like you need to do it with $FWDIR/conf/local.arp, and it will then do gratuitous ARPs for the proxy ARPs as well.

If you're doing multicast MAC active/active, this is not an issue.

wow it works - i'm bloody impressed.

northlandboy, you're an über-guru :-)))

thanks!