CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


 

Results 1 to 6 of 6

Thread: R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

  1. #1
    Join Date
    2014-07-21
    Posts
    54
    Rep Power
    3

    Default R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

    Hi,

    wer are using R77.10 Gateways with GAiA and IPsec Blade. We are using this Gateway for RemoteAccess VPN in Tunnel Mode.
    This Gateway has two Interfaces (VLAN) and both have MTU 1500 set.
    Both IP addresses are private IP-addresses so NAT will be done on another device.

    On the Endpoint Client on the client machines we have a MTU for the virtual Interface configured with MTU 1350.

    I have some questions:

    1.) What MTU size do you recommend on the Client and and on the Gateway in the above szenario? We are using IPv4 only?

    2.) When we sent a big ICMP packet (echo-request with 5000 byte - without DF set) from the Client into the VPN tunnel that the Client will first fragment the clear-text packet and then encrypts it. On the Gateway we see that the echo-reply will first be encrypted and then fragmented. Is there a reason why this is happening differently on the client and the server?

    3.) We see that the echo-reply packets (fragmented) passes the router on the client side (ADSL with PPPoE) and leaves on the LAN interface in direction Endpoint client but on the endpoint client we do not see all the fragments of the UDP encapsulated ESP packets. Has someone else seen this problem and/or could explain?

    4.) We have around 300-500 users connected to a 12.200 Appliance only doing Firewall and IPsec RemoteAccess VPN. The CPU utilization is not higher than 25% on one core when vpnd is running. The throughput on cpview is around 30-70Mbit/s. Our internet pipe has 1.000MBit/s and is used less than 500MBit/s. Unfortunately most of the RemoteAccess VPN users complain that they have bad performance. We tried on an internet access with 400/20 and the speedtest on the client was 32/18. So we could see that "upload" from Remoteaccess Client to our datacenter and to the internet is as fast as the clients connection but download speed is very poor using RemoteAccess VPN. We made sure that all our Firewalls (CheckPoint) and Cisco Routers allow the "ICMP Fragmentation needed" packets but we do not get a "good" performance. Can anyone can give us a good advice on what to check or what to modify to get better performance on the clients?


    Kind regards

  2. #2
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    1,918
    Rep Power
    10

    Default Re: R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

    Quote Originally Posted by Nachtfalke View Post
    Hi,

    wer are using R77.10 Gateways with GAiA and IPsec Blade. We are using this Gateway for RemoteAccess VPN in Tunnel Mode.
    This Gateway has two Interfaces (VLAN) and both have MTU 1500 set.
    Both IP addresses are private IP-addresses so NAT will be done on another device.

    On the Endpoint Client on the client machines we have a MTU for the virtual Interface configured with MTU 1350.

    I have some questions:

    1.) What MTU size do you recommend on the Client and and on the Gateway in the above szenario? We are using IPv4 only?
    Why has the MTU been lowered on the clients? Lowering the gateway's external interface MTU to match is usually a last resort.


    2.) When we sent a big ICMP packet (echo-request with 5000 byte - without DF set) from the Client into the VPN tunnel that the Client will first fragment the clear-text packet and then encrypts it. On the Gateway we see that the echo-reply will first be encrypted and then fragmented. Is there a reason why this is happening differently on the client and the server?
    The client is the one initiating the ping traffic as opposed to the traffic transiting across the firewall which probably explains the difference you are seeing. How are you observing this on the gateway, tcpdump or fw monitor? There are big differences in how these two tools show you fragmentation due to "virtual defragmentation" being performed in Check Point's INSPECT driver.

    3.) We see that the echo-reply packets (fragmented) passes the router on the client side (ADSL with PPPoE) and leaves on the LAN interface in direction Endpoint client but on the endpoint client we do not see all the fragments of the UDP encapsulated ESP packets. Has someone else seen this problem and/or could explain?
    Ah PPPoE with its 1492 MTU is present which is why you had to lower the MTU on the clients. What is the MTU size on the ADSL router interface facing the client? Or is the ADSL device just acting as a bridge?

    4.) We have around 300-500 users connected to a 12.200 Appliance only doing Firewall and IPsec RemoteAccess VPN. The CPU utilization is not higher than 25% on one core when vpnd is running. The throughput on cpview is around 30-70Mbit/s. Our internet pipe has 1.000MBit/s and is used less than 500MBit/s. Unfortunately most of the RemoteAccess VPN users complain that they have bad performance. We tried on an internet access with 400/20 and the speedtest on the client was 32/18. So we could see that "upload" from Remoteaccess Client to our datacenter and to the internet is as fast as the clients connection but download speed is very poor using RemoteAccess VPN. We made sure that all our Firewalls (CheckPoint) and Cisco Routers allow the "ICMP Fragmentation needed" packets but we do not get a "good" performance. Can anyone can give us a good advice on what to check or what to modify to get better performance on the clients?


    Kind regards
    A 12200 has four cores. If you are running top and seeing 25% CPU utilization, one core may be running at or close to 100% which will crimp your VPN performance. Run top and hit "1" to show individual core utilization. It sounds like you have verified that IPSec w/ UDP 4500 NAT Traversal is the transport being used by the client (not SSL). Unfortunately all IPSec VPN and even Remote Access SSL operations can only take place on one core of a R77.10 gateway, which will hurt performance but I imagine the MTU issue is the main problem you are seeing. Multicore SSL VPN (which allows Remote Access VPN SSL processing across multiple cores) is supported starting in R77.20.

    The download speed you are observing is terrible because the gateway is sending full-size (1500 payload) digitally-signed ESP packets that cannot be fragmented (DF set in outer ESP header), which causes initial major packet loss and TCP to start backing off, until the TCP Segment Size shrinks to a level where the overall packet payload data size drops to or below 1492. TCP then sees that packet delivery is reliable at this size and tries to speed back up resulting in loss again, until it backs off and that vicious cycle repeats itself endlessly.

    So it basically sounds like PMTU Discovery is not functioning correctly in your situation. So my recommendations in order are:

    1) Depending on what type of VPN Client software you are using, you may be able to set it to use SSL/TLS as the transport instead of IPSec. SSL/TLS causes slightly higher processing overhead than IPSec but can be fragmented without issue. If feasible this may well solve your problem and you can set the client MTUs back to 1500. The next thing you may run into though is a single core running at 100% on the gateway thus limiting ultimate VPN performance. Multicore IPSec VPN is supported in R77.20 but only with a special patch, and is anticipated to be fully supported in R80.10 gateway.

    2) Since you are in the fortunate scenario of controlling the MTU size on both the gateway and endpoint, lower the gateway's external interface MTU size to match the client's via clish. 1350 is probably a bit low since you are dropping it just to get underneath PPoE's 1492 limitation, I'd try 1450 on both ends. In a situation where you do not control both sides and cannot just lower the MTUs (or don't want to), TCP MSS Clamping (sk61221) is the appropriate solution. I'm not recommending it in your case however because you are running R77.10 and TCP MSS Clamping was significantly enhanced in R77.20 and later to work properly with SecureXL. Trying TCP MSS Clamping with your R77.10 deployment may just cause more problems.

    3) IKEv2 has a built-in mechanism to deal with low MTUs in the network path; unlikely however that your VPN Client software supports it.
    Last edited by ShadowPeak.com; 4 Weeks Ago at 21:19.
    --
    My book "Max Power: Check Point Firewall Performance Optimization"
    now available via http://maxpowerfirewalls.com.

  3. #3
    Join Date
    2014-07-21
    Posts
    54
    Rep Power
    3

    Default Re: R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

    Hi ShadowPeak,

    thank you for taking time to read and answer my questions. I will try to answer yours:

    1.) The MTU on the client was lowered because when we had the MTU of the Virtual Interface (not the (W)LAN) but the one from Endpoint, the users had problems to open network shares and so on. Si 1350 seems to be a good one on the different ADSL accesses with PPPoE.
    We had - for testing purposes only - lowered the MTU on the gateways external interface but it does not improve anything. Here a table with different MTU combinations we tested and the size we could send a ping (no DF bit set) and we tried with SecureXL on and SecureXL of - with different results:

    Code:
    GW-internal	GW-external	VPN-Client-MTU	WLon/Lon/ADSL						ping-size	SecureXL
    1500		1100		1350			WLon(1500)/ADSL(????)				1472		off
    1500		1100		1350			WLon(1500)/ADSL(????)				1072		on	(some pings work, some not, some work...)
    				
    1500		1300		1350			WLon(1500)/ADSL(????)				1472		off
    1500		1300		1350			WLon(1500)/ADSL(????)				1272		on
    				
    1500		1500		1350			WLon(1500)/ADSL(????)				1472		off
    1500		1500		1350			WLon(1500)/ADSL(????)				1394		on
    				
    1500		1700		1350			WLon(1500)/ADSL(????)				1472		off
    1500		1700		1350			WLon(1500)/ADSL(????)				1394		on
    				
    1500		1700		1260			WLon(1500)/ADSL(????)				1394		off
    1500		1700		1260			WLon(1500)/ADSL(????)				1394		on
    				
    1500		1700		1100			WLon(1500)/ADSL(????)				1394		off
    1500		1700		1100			WLon(1500)/ADSL(????)				1394		on
    				
    1500		1500		1100			WLon(1500)/ADSL(????)				1394		off
    1500		1500		1100			WLon(1500)/ADSL(????)				1394		on
    				
    1500		1500		1400			WLon(1500)/ADSL(????)				1322		off
    1500		1500		1400			WLon(1500)/ADSL(????)				1322		on
    				
    1500		1500		1300			WLon(1500)/ADSL(????)				1394		off
    1500		1500		1300			WLon(1500)/ADSL(????)				1394		on

    2.) The captures we did were always with tcpdump. As far as I know tcpdump should see all trafic even if SecureXL is on, right?

    3.) The (W)LAN interface from the router to the clients is 1500 MTU. The part from the router to the ISP with PPPoE is probably 1492. I do not know for sure because the router (AVM FritzBox 7490) will not show the MTU and configures this automatically. Further the FritzBox is doing MSS clamping - but this will not have effect on UDP 4500 and so will not be relevant in our szenario I assume.

    4.) Sorry for misunderstanding. 12200 has 4 cores. I used "top" and hit "1" to see all cores. 1 Core SND, 3 CoreXL. Only one of these 4 cores has 25%, without SecureXL around 40% the other CPUs have less than 5% usage. So I don't think it is a performance issue of the gateway.

    You mentioned "digitally signed ESP packets could not be fragmented" - I configured this "sim_ipsec_dont_fragment=1" from sk98070. So like I understand this SK it should exactly do this - fragment UDP ESP packets, right? But no matter if we have enabled this parameter or not we do not see a difference in speed. Further this parameter should only work if SecureXL is used - if it is disabled it shouldn't do anything I think.


    a) We are using CheckPoint Endpoint Security 7.6.450 on all our clients (This is the reason why we are still using R77.10 and not R77.30 on the VPN gateways because we are not sure if they are compatible. The colleagues maintaining the Endpoint Client are working on E80.6x.)

    When you are talking about TLS/SSL - do you mean Visitor Mode - using port 443/tcp ?


    b) Would like to avoid MSS clamping on R77.10 and ideally don't want to use it on any version.

    c) Not sure if R73 Client can do IKEv2 - perhaps E80.6x can but this will need some time so finding any other solution would be great.


    So any further suggestions would be great.

  4. #4
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    1,918
    Rep Power
    10

    Default Re: R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

    Quote Originally Posted by Nachtfalke View Post
    Hi ShadowPeak,

    thank you for taking time to read and answer my questions. I will try to answer yours:

    1.) The MTU on the client was lowered because when we had the MTU of the Virtual Interface (not the (W)LAN) but the one from Endpoint, the users had problems to open network shares and so on. Si 1350 seems to be a good one on the different ADSL accesses with PPPoE.
    We had - for testing purposes only - lowered the MTU on the gateways external interface but it does not improve anything. Here a table with different MTU combinations we tested and the size we could send a ping (no DF bit set) and we tried with SecureXL on and SecureXL of - with different results:
    Lowering the Gateway external interface MTU didn't help? That doesn't sound right. One easy way to disable SecureXL VPN functions while leaving the rest of it on is the command "sim vpn off". I'm wondering a bit if when SecureXL was off you experienced a different performance issue due to a processor bottleneck.


    2.) The captures we did were always with tcpdump. As far as I know tcpdump should see all trafic even if SecureXL is on, right?
    Yes unless you are using hardware-based acceleration such as a SAM or ADP card. tcpdump also shows the true state of fragments entering/leaving the gateway.

    3.) The (W)LAN interface from the router to the clients is 1500 MTU. The part from the router to the ISP with PPPoE is probably 1492. I do not know for sure because the router (AVM FritzBox 7490) will not show the MTU and configures this automatically. Further the FritzBox is doing MSS clamping - but this will not have effect on UDP 4500 and so will not be relevant in our szenario I assume.

    4.) Sorry for misunderstanding. 12200 has 4 cores. I used "top" and hit "1" to see all cores. 1 Core SND, 3 CoreXL. Only one of these 4 cores has 25%, without SecureXL around 40% the other CPUs have less than 5% usage. So I don't think it is a performance issue of the gateway.

    You mentioned "digitally signed ESP packets could not be fragmented" - I configured this "sim_ipsec_dont_fragment=1" from sk98070. So like I understand this SK it should exactly do this - fragment UDP ESP packets, right? But no matter if we have enabled this parameter or not we do not see a difference in speed. Further this parameter should only work if SecureXL is used - if it is disabled it shouldn't do anything I think.
    Should have been "digitally signed ESP packets could not be fragmented by an intervening device". The endpoint performing the encryption and digital signing can most definitely fragment the original packets and then sign the fragments. An intervening device can't do that.

    a) We are using CheckPoint Endpoint Security 7.6.450 on all our clients (This is the reason why we are still using R77.10 and not R77.30 on the VPN gateways because we are not sure if they are compatible. The colleagues maintaining the Endpoint Client are working on E80.6x.)

    When you are talking about TLS/SSL - do you mean Visitor Mode - using port 443/tcp ?
    Pretty sure that version of EndPoint Security can only use IPSec for transport and not SSL/TLS. Visitor mode is just IKE and IPSEC running over TCP 443 to avoid an intervening device filtering UDP/500 and/or ESP, I'm pretty sure it is not really SSL/TLS. However you mentioned that you have some other device doing MSS clamping. I think it would be very interesting to make sure Visitor Mode support is enabled on the gateway (it is not by default) and then switching on Visitor Mode in the client. The change of Layer 4 transport to TCP might yield some positive results in your situation.


    b) Would like to avoid MSS clamping on R77.10 and ideally don't want to use it on any version.

    c) Not sure if R73 Client can do IKEv2 - perhaps E80.6x can but this will need some time so finding any other solution would be great.


    So any further suggestions would be great.
    Yeah looks like there is no IKEv2 support for Remote Access clients as of R77.30. Bummer. Still think Visitor Mode is your best next step.
    --
    My book "Max Power: Check Point Firewall Performance Optimization"
    now available via http://maxpowerfirewalls.com.

  5. #5
    Join Date
    2014-07-21
    Posts
    54
    Rep Power
    3

    Default Re: R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

    Hi,

    thank you for response. Will talk to the Endpoint Colleagues to make sure theys have enabled "visitor Mode" which is already enabled on Gateway.

    Can you give me some SKs or links to SSL/TLS ans transport mode?
    Perhaps we can test this on a development RA-VPN Cluster Running R77.30 and E80.6x Endpoint Client.

    Regards

  6. #6
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    1,918
    Rep Power
    10

    Default Re: R77.10 GAiA GW with IPsec RemoteAccess - private IP - Best practice MTU?

    Quote Originally Posted by Nachtfalke View Post
    Hi,

    thank you for response. Will talk to the Endpoint Colleagues to make sure theys have enabled "visitor Mode" which is already enabled on Gateway.

    Can you give me some SKs or links to SSL/TLS ans transport mode?
    Perhaps we can test this on a development RA-VPN Cluster Running R77.30 and E80.6x Endpoint Client.

    Regards
    The Endpoint Security client has to use IKE/IPSec on the native ports or on port TCP 443 via Visitor Mode, after investigating I found it doesn't support SSL/TLS natively. Only the SSL Network Extender (which is part of the Mobile Access Blade) uses SSL/TLS natively as a transport from what I can tell.

    Your older VPN client version should have an option for enabling and/or forcing Visitor Mode as it has been around a very long time. Try creating a new site in Endpoint Security like this: "SiteIP:443" instead of just the firewall's external IP or resolvable name to force the use of Visitor Mode. This assumes of course that support for Visitor Mode has been enabled on the gateway.
    --
    My book "Max Power: Check Point Firewall Performance Optimization"
    now available via http://maxpowerfirewalls.com.

Similar Threads

  1. RemoteAccess - SecuRemote
    By bhavinjbhatt in forum R77.30
    Replies: 0
    Last Post: 2016-02-09, 20:10
  2. Replies: 5
    Last Post: 2015-04-02, 08:56
  3. RemoteAccess VPN wrong default Gateway
    By kevinc in forum IPsec VPN Blade (Virtual Private Networks)
    Replies: 0
    Last Post: 2010-07-09, 18:43
  4. RemoteAccess from internal LAN
    By ktm628 in forum SecureClient/SecuRemote
    Replies: 6
    Last Post: 2009-05-30, 02:12
  5. Unable to route between VPN site2site and RemoteAccess
    By Wullum in forum IPsec VPN Blade (Virtual Private Networks)
    Replies: 2
    Last Post: 2007-10-17, 09:22

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •