That's almost certain to be a routing loop. Run an fw monitor when you see the problem. I bet you will see a SYN pass through the firewall, then the same SYN hit the firewall on the interface it just...