Problems Creating VPN's Between NG With Application Intelligence and PIX VPN's

[roadrunner]

Problems creating VPNs between NG with Application Intelligence and PIX VPNs
We ran into a problem trying to get a PIX and a FW1 NG AI box to set up a VPN tunnel. We were using pre-shared keys and IKE would happily get through main mode, but we'd kept getting the following error message on PhaseII:

IKE: Quick Mode Received Notification From Peer: no proposal chosen

The really strange thing is that when connections from the PIX side were initiated, everything worked fine. The FW-1 side failed every time on a new connection.

Answer


--------------------------------------------------------------------------------
The Interoperable Device object created to represent the PIX firewall has a checkbox under VPN->Advanced for "Support key exchange for subnets". This is checked by default. The PIX FW does not like this when attempting to negotiate the PhaseII/QuickMode SAs. Disable this and push the rules to the firewall.
Additionally, it's wise to make sure encryption domains are created similarly, ie hosts vs. nets etc.


--------------------------------------------------------------------------------

-- RobertGraham - 07 Jan 2004


FAQForm
FAQs.Class: EncryptionFAQs
FAQs.OS:
FAQs.Version:

[dragec]

I had the same problem when I had several C class networks in VPN domain. I solve it by unchecking "Support Key Exchange for Subnets ". Is there some other solution to this?

Thanks!

PS: I use NG AI R55

[alienbaby]

The problem was because the PIX ACL and the CheckPoint Encryption Domain did not match.

CheckPoint is far more forgiving than the PIX is.

Consider the following example.

Site 1 is a large network and the Administrator has placed network object of 10.0.0.0/255.0.0.0 as the Encyption Domain. Site 1 is the CheckPoint firewall.
Site 2 is a small company (a vendor, credit card processor or something) and uses a PIX or Cisco Router with IPSec VPN feature set. Site 2's VPN device has only been told about a small portion of Site 1's network; ie 10.100.1.0/255.255.255.0.

When Site 2 attempts to initate a VPN, it negociates for 10.100.1.0/255.255.255.0 subnet. Since the checkpoint firewall finds that 10.100.1.0/255.255.255.0 is within 10.0.0.0/255.0.0.0, it allows the erronious subnet negociated VPN. The VPN is setup successfully.
When Site 1 attempts to initiate a VPN to Site 2, the CheckPoint includes it's 10.0.0.0/255.0.0.0 subnet in the subnet negociation. The PIX/Cisco VPN device strictly checks the 10.0.0.0/255.0.0.0 subnet against it's VPN ACL. When it does not match, the PIX/Cisco VPN device disallows the VPN.

The two solutions are to Uncheck the Subnet negociation checkbox on the Interoperable device for the PIX/CIsco; or correct the Encyption Domain on the CheckPoint side. In addition to the 10.0.0.0/255.0.0.0 network object, you can include a network object for 10.100.1.0/255.255.255.0. Even though one is technically inside the other, it will work. You could even change the Cisco's ACL to include the 10.0.0.0/255.0.0.0 instead of 10.100.1.0/255.255.255.0 and use the firewall policy to allow the very few hosts accross the VPN.

Matching the Encyption Domains is the correct solution.

Unchecking the Subnet Negociation checkbox increases the overhead on both Firewalls. Two SAs will be needed for every IP Address pair instead on two SAs for the subnet to subnet VPN.

More than just the Phase 1 and Phase 2 properties have to be corrent for a successful IKE negociation to take place. The Encryption Domain/VPN ACLs have to match as well.

[PuRowdy]

I recently spoke with a Checkpoint engineer about this problem, he pointed me in an interesting direction and I would like to see what you guys were thinking.

He said that this wasn't really a Checkpoint issue so much as a hardware issue, a Nokia issue. We were told that it comes down to the way that Nokia boxes do clustering, which causes the "Invalid SPI" error that I often see. Something like the Cisco receives 2 messages back with the same SPI from different members of the cluster causing the error message.

Is anybody else running into this issue specifically with Nokia hardware?

[Lackie]

I work exclusively with Nokia's and have not heard of this issue in a cluster environment.

[PuRowdy]

We are currently running NGX on Nokia boxes, we had this issue with NG w/AI before as well. Checkpoint's recommendation is to upgrade the software on the Cisco device, and to exchange key's per host instead of per subnet.

I've tried both and so far havn't had any luck but still looking for other options to solve this issue.

[jjonessec]

Originally Posted by PuRowdy

We are currently running NGX on Nokia boxes, we had this issue with NG w/AI before as well. Checkpoint's recommendation is to upgrade the software on the Cisco device, and to exchange key's per host instead of per subnet.

I've tried both and so far havn't had any luck but still looking for other options to solve this issue.

I am having same type of error message on SPLAT, so defiantly dont think it is a Nokia issue

[t0mt0m]

I also have the same error with SPLAT (R55) and a VPN3000.
However, I have another partner with VPN3000 concentrator and not using Key exchange for subnets fixed the problem.

I guess it's an issue with the VPN OS version ...

[thebuffman]

I wanted to thank the participants for such a great thread. I had the problem between our checkpoint and an associate's pix. The pix of my associate had our 10.0.5.0/24 domain as the encryption domain. Our actual E.D. is a lot bigger than that. I added a network object "10.0.5.0/24" to the E.D. and voila! everything started flowing. Pix is extremely pickie about the E.D. Even though the pix could send to us without a problem our checkpoint could not initiate communication until we applied this fix.

Thanks a million!

[cciesec2006]

The other alternative solution is to modify the $FWDIR/lib/user.def file
and list exactly what you have in your local Encryption Domain. You will also
need to disalbe "IKE_largest_possible_subnet" to false via gui-dbedit. That
will prevent checkpoint from suppernetting.

HTH

[jvalenzuela]

Hello

I have a quite similar problem with a vpn. The difference is that we are using public Ip's for the vpns. Eventhough my public ip range is on the domain, i added the server that participate on the vpn but it didn't work.

There's communication from the pix to my FW1 R60 but not on the other direction. I'm receiving the common message:

"encryption fail reason: Packet is dropped because there is no valid SA - please refer to solution sk19423 in SecureKnowledge Database for more information"

I also move to the option "one tunnel per subnet pair" on the interop.device object of the pix.

Any suggestion?

Thanks

Jorge

[chillyjim]

If you have access to the PIX try a:
debug cry isa

and on the VPN-1
vpn debug ikeon (File will be in $FWDIR/logs)

and see if the encryption domains match.

-----------
To clear the tunnel as mentioned below:

clear cry isa sa

[jvalenzuela]

Well, we solved the problem by restarting the vpn on the pix side. A problem I usually have with pix's is that they don't renegociate the key if they think the tunnel is still up. Also, the tunnel may be down for an hour and they still think it's up. So when the fw1 sends a new key, the pix reject it.

However, it's working now. I'll use the command you post to check the domains. I'll need to add more host to this vpn and would have the same problem again.

Thanks

Jorge