VPN-1 UTM Edge Problems

[dantro]

Hi there,

I'd like to discuss some Edge experiences we have made during the last two years.

We are supporting customers with mid-size global networks often secured by VPN-1 UTM Edge appliances. Most customers have a Check Point SmartCenter Server, so we centrally manage their Edges on it. Some customers with just a few Edge appliances have no problems at all. Even when these Edges are just standalone appliances protecting a small office. However we have learned that the policy installation is not always handled and translated correctly by the libsw libraries. Sometimes corrupt policies were installed onto the EdgeConnector. Especially when there were groups with exclusions or objects with special characters. So we always use an extra firewall policy just for the Edges. And we keep policies clean.

But we encountered and still have major problems with customers who maintain a global VPN network, all fully meshed, all protected with VPN-1 UTM Edge Clusters. These clusters are sometimes configured as WAN-HA (the new method with an own external IP for each cluster member) or via the old way where both cluster members share a single external IP. The SmartCenter Server is located in the headquarter with an internal address which is translated via Static NAT to an external IP. To this external IP all Edge-Clusters are connected to the Service Center. On the SmartCenter Server we configured the Edge-Clusters as VPN-1 Edge Gateway object. Even in NGX (R62) it is not possible to create an Edge Cluster object. I don't know if NGX (R65) will implement this feature.

This global VPN network is running on a newly consolidated policy with a clean database. But whenever we change something some Edge Clusters fail completely and require a manual restart. So if we have to change the topology and install the new policy to all Edge appliances some encounter a "failed to install updated policy" error and later on losing all internal and external connections. Sometimes if we click on "Service Center Refresh" locally on an Edge appliance the box stops working or starts rebooting. Sometimes after some days while an Edge is still working all connections are being dropped. Even a reset doesn't help. We need to manually disconnect it from the Service Center and reconnect it again. It's such a mess you can't believe. Sometimes Edges have a strange time setting and are running at 2001 or in 2022. This causes VPN tunnels to fail until we fix the time setting. Since the overall network is very global and several admins are included by a follow-the-sun procedure firmware updates require some time. We are currently using 6.5.43x as a stable firmware version for all Edge Clusters. This is a problem for Check Point Support who only provides help if everything is running on the most recent firmware version. Before we could even update all offices to the recent version a new one is released and we can start all from the beginning. Of course firmware updates are also causing Edges to fail sometimes. We are always getting the Edge Clusters back to work as intended after several restarts and Service Center reconnects, but that is in no way a working scenario a customer is interested in.

What are your experiences? How do you configure Edge Clusters on an SCS? How do you think about the Check Point Support?

Best regards,
Danny Trommer

CCSA / CCSE / CCSE+

[BarryStiefel]

Originally Posted by dantro

Hi there,

I'd like to discuss some Edge experiences we have made during the last two years.

We are supporting customers with mid-size global networks often secured by VPN-1 UTM Edge appliances. Most customers have a Check Point SmartCenter Server, so we centrally manage their Edges on it. Some customers with just a few Edge appliances have no problems at all. Even when these Edges are just standalone appliances protecting a small office. However we have learned that the policy installation is not always handled and translated correctly by the libsw libraries. Sometimes corrupt policies were installed onto the EdgeConnector. Especially when there were groups with exclusions or objects with special characters. So we always use an extra firewall policy just for the Edges. And we keep policies clean.

But we encountered and still have major problems with customers who maintain a global VPN network, all fully meshed, all protected with VPN-1 UTM Edge Clusters. These clusters are sometimes configured as WAN-HA (the new method with an own external IP for each cluster member) or via the old way where both cluster members share a single external IP. The SmartCenter Server is located in the headquarter with an internal address which is translated via Static NAT to an external IP. To this external IP all Edge-Clusters are connected to the Service Center. On the SmartCenter Server we configured the Edge-Clusters as VPN-1 Edge Gateway object. Even in NGX (R62) it is not possible to create an Edge Cluster object. I don't know if NGX (R65) will implement this feature.

This global VPN network is running on a newly consolidated policy with a clean database. But whenever we change something some Edge Clusters fail completely and require a manual restart. So if we have to change the topology and install the new policy to all Edge appliances some encounter a "failed to install updated policy" error and later on losing all internal and external connections. Sometimes if we click on "Service Center Refresh" locally on an Edge appliance the box stops working or starts rebooting. Sometimes after some days while an Edge is still working all connections are being dropped. Even a reset doesn't help. We need to manually disconnect it from the Service Center and reconnect it again. It's such a mess you can't believe. Sometimes Edges have a strange time setting and are running at 2001 or in 2022. This causes VPN tunnels to fail until we fix the time setting. Since the overall network is very global and several admins are included by a follow-the-sun procedure firmware updates require some time. We are currently using 6.5.43x as a stable firmware version for all Edge Clusters. This is a problem for Check Point Support who only provides help if everything is running on the most recent firmware version. Before we could even update all offices to the recent version a new one is released and we can start all from the beginning. Of course firmware updates are also causing Edges to fail sometimes. We are always getting the Edge Clusters back to work as intended after several restarts and Service Center reconnects, but that is in no way a working scenario a customer is interested in.

What are your experiences? How do you configure Edge Clusters on an SCS? How do you think about the Check Point Support?

Best regards,
Danny Trommer

CCSA / CCSE / CCSE+

My experience has been that whatever money you save by buying Edge boxes you more than burn up in the additional labor costs required to install them and maintain them. Check Point seems to have put a lot more effort into getting regular Security Gateways to work properly, so I always recommend that over an Edge box.

Over the long run, Edge boxes always just end up being very expensive in terms of hassle.

[abusharif]

my experience of edge is quite ok actually. Early versions are bug infested but since 6.5 and later its working good. About Libsw, edges with "newer" versions will never download the policy if it was not compiled with correct libsw installed so I am not really sure how you managed to get this into them. Atm libsw is only thing you have to worry about. Luckily it is backward compatible so always install latest one and it will cover any version below its own.

About time settings, yes sometimes for no reason what so ever they change date and time which makes certificates invalid and vpn's dont go up (if smartcenter managed). For that to "solve" you can use NTP sync which came in one of later versions (6.5.x and later i think).

[dantro]

We already use NTP servers because of this behaviour. Also we use the newest libsw libraries when they get released. Make sure the file permissions on newer libsw updates are correctly applied.

Today we opened two Edge appliances because they make more and more trouble. They just have a crappy 3V Li Battery inside which is not rechargeable. So whenever a battery is empty you'll lose the time setting when disconnecting an appliance. Older hardware models have an almost complete different mainboard. Looks like they had to redesign the power handling after some issues. Newer models now accept 9V AC and 12V DC.

Best regards,
Danny Trommer
CCSA/CCSE/CCSE+

[Spoonworker]

My experience is that there is no proper documentation with respect to what you wanna do on a BOX. I have worked on the software version of checkpoint and going to an appliance is not easy. There is nothing there with respect to configuration the appliance. Any know know of any good forum or site to get the information or download docs ?

[dantro]

That's true, especially if you want to do the real pro configurations, like clustering two or more centrally managed VPN-1 Edge appliances. NGX (R67) will have some improvements on these points, also there is now a lot of additional extras available. Like rackmount kits for Edge appliances, improved power connectors and the new Edge industrial model.. more is to come. You'll need some experience for managing these Edges the way you can squeeze out all available options to fit your needs. However, the Edge appliance offers you a powerful firewall and vpn functionality on a professional level for a very low price. Keep in mind that you can only push an Edge to its limits by managing it centrally. By managing it locally you'll find many restrictions (like a maximum of 15 VPN tunnels etc.) that don't show what you can really do with these tiny boxes. I've seen roaming setups with these boxes in really large deployments. But as said above, that needs a lot of experience, good contacts to Check Point and uncounted hours in testlabs.

Best regards,
Danny Trommer
CCSA/CCSE/CCSE+

[kazoinks]

I have to agree with BarryStiefel!

Kaz