[northlandboy]

When you install policy, it will install policy on both nodes in the cluster - default is revert the installation on one if it fails on the other. Cluster nodes will start complaining if there is a different policy on each of them.

That is one reason why when you restart a member of a cluster, it pulls policy down from the other member, to check for differences.

ClusterXL is not keeping things like routing and proxy ARP in sync though. My guess is that someone's added stuff to the configuration of one node, and not the other.

You mention that OWA was failing when fw1 was up - what troubleshooting did you do? Where was it failing? Did you trace the traffic? If you've got a consistent failure, this should be trivial to trace.

Similarly with the NAT failures - what was actually happening? Was the firewall not natting the packets? Or were you not receiving any replies? What troubleshooting did you do? A random guess here would be that you've got proxy ARP configured for the NAT addresses on fw1, using the cluster MAC, but you haven't configured it on the secondary. This sort of thing is reasonably easy to troubleshoot - where were you seeing the failures?

A standby member should not handle traffic, unless you have got messed up routing somewhere - that's always a possibility, that you've got routes pointing to real addresses. I've seen that too. Was fw1 actually handling the traffic? Again, what did you see when looking at packets?

What would be the point in building a new machine? What problem are you trying to solve there?