CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it again! He has just released the 2nd edition of "Max Power".
Rather than get into details here, I urge you to check out this announcement post.
It's a massive upgrade, and well worth checking out. -E

 

Results 1 to 17 of 17

Thread: High CPU problem on checkpoint gateway

  1. #1
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default High CPU problem on checkpoint gateway

    A pair of R77.30 with HFA_216 clusterXL in Active/Standby on Dell PowerEdge R710 with 405 license. Only fw blade is enabled as confirmed with "enabled_blades" output.

    I have a 10G interfaces on bond0 and 10G interface on eth1. My encrypted Oracle traffics traverse the firewall on the first rule in the security policy. No NAT involved, just routing. SecureXL is ON. Dynamics dispatching is also ON. This Oracle traffic rule is at the top of security policy. The Oracle traffic enters interface bond0 and out of interface eth1

    To ensure that the Oracle traffic is not fragmented, I set the SDU paramater in Oracle to 1400 to make sure it fits the MTU on the interface which is 1500.

    As soon as the traffics get to 2Gbps, I see one of the CPU is maxed out at 100% utilization. In my case, it was cpu 0. When I run top, I see cpu 0 is pegged with 100% si, like this:

    top - 13:08:58 up 501 days, 12:14, 1 user, load average: 0.00, 0.02, 0.00
    Tasks: 162 total, 1 running, 161 sleeping, 0 stopped, 0 zombie
    Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 03.0%id, 0.0%wa, 0.0%hi, 95.0%si, 0.0%st


    When I check "sim affinity -l", I see this:

    [Expert@gw1:0]# sim affinity -l
    eth0 : 0
    eth1 : 0
    eth11 : 0
    eth13 : 0
    eth3 : 0
    eth4 : 0
    [Expert@gw1:0]#

    When I check with 'fwaccel conns | grep "x.x.x.x" | grep F', all of the Oracle traffic between the source and destination are in accelerated mode.

    How do I go about fixing the high CPU issue? Is this a "known" issue with Checkpoint firewalls? Does it mean I have to manually do the "sim affinity"? What is the downside of manually assigning individual CPUs to interfaces?

    I notice that the same problem with R80.20 management that manages R80.20 gateways as well.

    How do you about fixing this issue?

  2. #2
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    still looking for suggestions on this.

  3. #3
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    A pair of R77.30 with HFA_216 clusterXL in Active/Standby on Dell PowerEdge R710 with 405 license. Only fw blade is enabled as confirmed with "enabled_blades" output.

    I have a 10G interfaces on bond0 and 10G interface on eth1. My encrypted Oracle traffics traverse the firewall on the first rule in the security policy. No NAT involved, just routing. SecureXL is ON. Dynamics dispatching is also ON. This Oracle traffic rule is at the top of security policy. The Oracle traffic enters interface bond0 and out of interface eth1

    To ensure that the Oracle traffic is not fragmented, I set the SDU paramater in Oracle to 1400 to make sure it fits the MTU on the interface which is 1500.

    As soon as the traffics get to 2Gbps, I see one of the CPU is maxed out at 100% utilization. In my case, it was cpu 0. When I run top, I see cpu 0 is pegged with 100% si, like this:

    top - 13:08:58 up 501 days, 12:14, 1 user, load average: 0.00, 0.02, 0.00
    Tasks: 162 total, 1 running, 161 sleeping, 0 stopped, 0 zombie
    Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 03.0%id, 0.0%wa, 0.0%hi, 95.0%si, 0.0%st


    When I check "sim affinity -l", I see this:

    [Expert@gw1:0]# sim affinity -l
    eth0 : 0
    eth1 : 0
    eth11 : 0
    eth13 : 0
    eth3 : 0
    eth4 : 0
    [Expert@gw1:0]#

    When I check with 'fwaccel conns | grep "x.x.x.x" | grep F', all of the Oracle traffic between the source and destination are in accelerated mode.

    How do I go about fixing the high CPU issue? Is this a "known" issue with Checkpoint firewalls? Does it mean I have to manually do the "sim affinity"? What is the downside of manually assigning individual CPUs to interfaces?

    I notice that the same problem with R80.20 management that manages R80.20 gateways as well.

    How do you about fixing this issue?
    Because you are licensed for only 4 cores, you probably have the default 1/3 split of SND/IRQ cores to Firewall Worker cores. Please provide output of fw ctl affinity -l -r and fwaccel stats -s to confirm.

    Since it sounds like almost all traffic is accelerated, all of it is only being handled by the one SND/IRQ core. The Dynamic Dispatcher will not help in this case since its job is to balance load among Firewall Worker cores, not SND/IRQ cores. Would suggest reducing the number of Firewall Workers (kernel instances) from 3 to 2 via cpconfig so that there will be a split of 2 SND/IRQ cores vs. 2 Firewall Worker cores; this will double the SND/IRQ processing resources available to handle the accelerated Oracle traffic.
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  4. #4
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    Because you are licensed for only 4 cores, you probably have the default 1/3 split of SND/IRQ cores to Firewall Worker cores. Please provide output of fw ctl affinity -l -r and fwaccel stats -s to confirm.

    Since it sounds like almost all traffic is accelerated, all of it is only being handled by the one SND/IRQ core. The Dynamic Dispatcher will not help in this case since its job is to balance load among Firewall Worker cores, not SND/IRQ cores. Would suggest reducing the number of Firewall Workers (kernel instances) from 3 to 2 via cpconfig so that there will be a split of 2 SND/IRQ cores vs. 2 Firewall Worker cores; this will double the SND/IRQ processing resources available to handle the accelerated Oracle traffic.

    [Expert@gw-1:0]# fw ctl affinity -l -r
    CPU 0: eth3 eth4 eth11 eth13 eth0 eth1
    CPU 1: fw_2
    CPU 2: fw_1
    CPU 3: fw_0
    CPU 4:
    CPU 5:
    CPU 6:
    CPU 7:
    All: rtmd fwd in.ahclientd mpdaemon in.aclientd lpd cprid cpd
    The current license permits the use of CPUs 0, 1, 2, 3 only.
    [Expert@gw-1:0]# fwaccel stats -s
    Accelerated conns/Total conns : 5199/5228 (99%)
    Accelerated pkts/Total pkts : 10750376389/10862974478 (98%)
    F2Fed pkts/Total pkts : 78461822/10862974478 (0%)
    PXL pkts/Total pkts : 34136267/10862974478 (0%)
    QXL pkts/Total pkts : 0/10862974478 (0%)
    [Expert@gw-1:0]#

  5. #5
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    [Expert@gw-1:0]# fw ctl affinity -l -r
    CPU 0: eth3 eth4 eth11 eth13 eth0 eth1
    CPU 1: fw_2
    CPU 2: fw_1
    CPU 3: fw_0
    CPU 4:
    CPU 5:
    CPU 6:
    CPU 7:
    All: rtmd fwd in.ahclientd mpdaemon in.aclientd lpd cprid cpd
    The current license permits the use of CPUs 0, 1, 2, 3 only.
    [Expert@gw-1:0]# fwaccel stats -s
    Accelerated conns/Total conns : 5199/5228 (99%)
    Accelerated pkts/Total pkts : 10750376389/10862974478 (98%)
    F2Fed pkts/Total pkts : 78461822/10862974478 (0%)
    PXL pkts/Total pkts : 34136267/10862974478 (0%)
    QXL pkts/Total pkts : 0/10862974478 (0%)
    [Expert@gw-1:0]#
    Yup definitely decrease kernel instances from 3 to 2 with cpconfig. Will help a lot.
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  6. #6
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    Yup definitely decrease kernel instances from 3 to 2 with cpconfig. Will help a lot.
    Actually I just did and it made the problem worse. Now everything is 50% slower :-(. Had to revert back my change.

  7. #7
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    Actually I just did and it made the problem worse. Now everything is 50% slower :-(. Had to revert back my change.
    Huh? That makes no sense, please define what "50% slower" means. If you have a cluster changing the number of kernel instances needs to be handled the same way as code upgrade.

    You may have something else going on, please post the output of these commands:

    fwaccel stat
    grep -c ^processor /proc/cpuinfo
    /sbin/cpuinfo
    fw ctl affinity -l -r
    sim affinity -l
    netstat -ni
    fw ctl multik stat
    free -m
    enabled_blades
    fw ver
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  8. #8
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    Huh? That makes no sense, please define what "50% slower" means. If you have a cluster changing the number of kernel instances needs to be handled the same way as code upgrade.

    You may have something else going on, please post the output of these commands:

    fwaccel stat
    grep -c ^processor /proc/cpuinfo
    /sbin/cpuinfo
    fw ctl affinity -l -r
    sim affinity -l
    netstat -ni
    fw ctl multik stat
    free -m
    enabled_blades
    fw ver

    YES, I know how to change it. I made the change and shutdown both firewalls at the same time. I wait for gw-1 to fully come up and then power up gw-2. The box has plenty of memory 32GB RAM

    here is the info you requested:


    [Expert@gw-1:0]# fwaccel stat
    Accelerator Status : on
    Accept Templates : disabled by Firewall
    disabled from rule #223
    Drop Templates : disabled
    NAT Templates : disabled by user

    Accelerator Features : Accounting, NAT, Cryptography, Routing,
    HasClock, Templates, Synchronous, IdleDetection,
    Sequencing, TcpStateDetect, AutoExpire,
    DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
    WireMode, DropTemplates, NatTemplates,
    Streaming, MultiFW, AntiSpoofing, Nac,
    ViolationStats, AsychronicNotif, ERDOS,
    NAT64, GTPAcceleration, SCTPAcceleration,
    McastRoutingV2
    Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
    3DES, DES, CAST, CAST-40, AES-128, AES-256,
    ESP, LinkSelection, DynamicVPN, NatTraversal,
    EncRouting, AES-XCBC, SHA256
    [Expert@gw-1:0]# grep -c ^processor /proc/cpuinfo
    8
    [Expert@gw-1:0]# /sbin/cpuinfo
    HyperThreading=disabled
    [Expert@gw-1:0]# fw ctl affinity -l -r
    CPU 0: eth3 eth4 eth11 eth13 eth0 eth1
    CPU 1: fw_2
    CPU 2: fw_1
    CPU 3: fw_0
    CPU 4:
    CPU 5:
    CPU 6:
    CPU 7:
    All: rtmd fwd in.ahclientd mpdaemon in.aclientd lpd cprid cpd
    The current license permits the use of CPUs 0, 1, 2, 3 only.
    [Expert@gw-1:0]# sim affinity -l
    eth0 : 0
    eth1 : 0
    eth11 : 0
    eth13 : 0
    eth3 : 0
    eth4 : 0
    [Expert@gw-1:0]# netstat -ni
    Kernel Interface table
    Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
    eth0 1500 0 303091082306 0 1073511 0 178517031230 0 0 0 BMRU
    eth0.240 1500 0 299392985198 0 0 0 167860770115 0 0 0 BMRU
    eth0.252 1500 0 3695903681 0 0 0 10669617915 0 0 0 BMRU
    eth1 1500 0 110231138042 7 627374 0 250957302115 0 0 0 BMRU
    eth3 1500 0 9818223697 0 3241 0 13910956679 0 0 0 BMRU
    eth4 1500 0 30137368467 0 12253 0 10829584051 0 0 0 BMRU
    eth11 1500 0 29952040488 0 160 0 24767949784 0 0 0 BMRU
    eth13 1500 0 1656847496 0 0 0 4012111094 0 0 0 BMRU
    lo 16436 0 18992915 0 0 0 18992915 0 0 0 LRU
    [Expert@gw-1:0]# fw ctl multik stat
    ID | Active | CPU | Connections | Peak
    ----------------------------------------------
    0 | Yes | 3 | 1755 | 47832
    1 | Yes | 2 | 1839 | 46804
    2 | Yes | 1 | 1792 | 48461
    [Expert@gw-1:0]# free -m.148.
    total used free shared buffers cached
    Mem: 32044 12916 19127 0 464 2898
    -/+ buffers/cache: 9553 22491
    Swap: 33549 0 33549
    [Expert@gw-1:0]# enabled_blades
    fw
    [Expert@gw-1:0]# fw ver
    This is Check Point's software version R77.30 - Build 048
    [Expert@gw-1:0]#


    I am also seeing this in CPVIEW:

    |---------------------------------------------------------------------------------------------------------------------|
    | CPVIEW.Overview 19Sep2018 12:29:31 |
    |---------------------------------------------------------------------------------------------------------------------|
    | Overview SysInfo Network CPU Software-blades Advanced |
    |---------------------------------------------------------------------------------------------------------------------|
    | CPU: |
    | |
    | Num of CPUs: 8 |
    | |
    | CPU Used |
    | 0 92% |
    | 1 0% |
    | 2 0% |
    | ------------------------------------------------------------------------------------------------------------------- |
    | Memory: |
    | |
    | Total MB Used MB Free MB |
    | Physical 32,044 9,571 22,473 |
    | FW Kernel 23,937 1,135 22,801 |
    | Swap 33,549 0 33,549 |
    | ------------------------------------------------------------------------------------------------------------------- |
    | Network: |
    | |
    | Bits/sec 2,022M |
    | Packets/sec 244K |
    | Connections/sec 40 |
    | Concurrent connections 4,669 |
    | ------------------------------------------------------------------------------------------------------------------- |

    in top:

    top - 12:31:36 up 508 days, 11:37, 1 user, load average: 0.25, 0.23, 0.19
    Tasks: 162 total, 1 running, 161 sleeping, 0 stopped, 0 zombie
    Cpu0 : 2.0%us, 0.0%sy, 0.0%ni, 7.8%id, 0.0%wa, 0.0%hi, 90.2%si, 0.0%st
    Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu4 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu5 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Mem: 32813684k total, 13244112k used, 19569572k free, 476144k buffers
    Swap: 34354992k total, 0k used, 34354992k free, 2965380k cached
    Last edited by cciesec2006; 4 Days Ago at 08:32.

  9. #9
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    You are getting some RX-DRPs which confirms that the lone SND/IRQ core is getting killed due to the high percentage of fully-accelerated traffic. As mentioned you need to drop the number of kernel instances from 3 to 2 via cpconfig. Please run all those commands again once you have made that change and are passing traffic, when you claim that everything is "50% slower". I'm not seeing any other issues given the output you provided, and the "50% slower" behavior after such a change makes no sense. You may want to leave one cluster member down or completely off during the test to ensure ClusterXL is not somehow getting into a split-brain state which will massively screw up performance.
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  10. #10
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    You are getting some RX-DRPs which confirms that the lone SND/IRQ core is getting killed due to the high percentage of fully-accelerated traffic. As mentioned you need to drop the number of kernel instances from 3 to 2 via cpconfig. Please run all those commands again once you have made that change and are passing traffic, when you claim that everything is "50% slower". I'm not seeing any other issues given the output you provided, and the "50% slower" behavior after such a change makes no sense. You may want to leave one cluster member down or completely off during the test to ensure ClusterXL is not somehow getting into a split-brain state which will massively screw up performance.
    1- I used cpconfig to change the number of cores from 3 to 2
    2- reboot both gateways at the same time
    3- run cphaprob state on both gateways and confirmed active/standby

    Now instead of getting 2Gbps throughput on Oracle, I am getting 1Gbps throughput.

  11. #11
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    1- I used cpconfig to change the number of cores from 3 to 2
    2- reboot both gateways at the same time
    3- run cphaprob state on both gateways and confirmed active/standby

    Now instead of getting 2Gbps throughput on Oracle, I am getting 1Gbps throughput.
    Run all commands again in this configuration please.
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  12. #12
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    Run all commands again in this configuration please.
    Here is the info you asked when I use cpconfig to change the CPU from 3 to 2:

    [Expert@gw-1:0]# fwaccel stats -s
    Accelerated conns/Total conns : 4794/4821 (99%)
    Accelerated pkts/Total pkts : 2949199867/2972450416 (99%)
    F2Fed pkts/Total pkts : 12563479/2972450416 (0%)
    PXL pkts/Total pkts : 10687070/2972450416 (0%)
    QXL pkts/Total pkts : 0/2972450416 (0%)
    [Expert@gw-1:0]#
    [Expert@gw-1:0]# fwaccel stat
    Accelerator Status : on
    Accept Templates : disabled by Firewall
    disabled from rule #223
    Drop Templates : disabled
    NAT Templates : disabled by user

    Accelerator Features : Accounting, NAT, Cryptography, Routing,
    HasClock, Templates, Synchronous, IdleDetection,
    Sequencing, TcpStateDetect, AutoExpire,
    DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
    WireMode, DropTemplates, NatTemplates,
    Streaming, MultiFW, AntiSpoofing, Nac,
    ViolationStats, AsychronicNotif, ERDOS,
    NAT64, GTPAcceleration, SCTPAcceleration,
    McastRoutingV2
    Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
    3DES, DES, CAST, CAST-40, AES-128, AES-256,
    ESP, LinkSelection, DynamicVPN, NatTraversal,
    EncRouting, AES-XCBC, SHA256
    [Expert@gw-1:0]# grep -c ^processor /proc/cpuinfo
    8
    [Expert@gw-1:0]# /sbin/cpuinfo
    HyperThreading=disabled
    [Expert@gw-1:0]# fw ctl affinity -l -r
    CPU 0: eth3 eth4 eth1
    CPU 1: fw_2
    CPU 2: fw_1
    CPU 3: eth13 eth0 eth11
    CPU 4:
    CPU 5:
    CPU 6:
    CPU 7:
    All: rtmd fwd in.ahclientd mpdaemon in.aclientd lpd cprid cpd
    The current license permits the use of CPUs 0, 1, 2, 3 only.
    [Expert@gw-1:0]# fw ctl affinity -l
    eth3: CPU 0
    eth4: CPU 0
    eth11: CPU 0
    eth13: CPU 0
    eth0: CPU 0
    eth1: CPU 0
    fw_0: CPU 2
    fw_1: CPU 2
    fw_2: CPU 1
    The current license permits the use of CPUs 0, 1, 2, 3 only.
    [Expert@gw-1:0]# netstat -ni
    Kernel Interface table
    Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
    eth0 1500 0 305435057259 0 1333108 0 179490904268 0 0 0 BMRU
    eth0.240 1500 0 301736346480 0 0 0 168834007877 0 0 0 BMRU
    eth0.252 1500 0 3696513322 0 0 0 10670249960 0 0 0 BMRU
    eth1 1500 0 111153789530 7 627374 0 253248476828 0 0 0 BMRU
    eth3 1500 0 9825713475 0 3241 0 13920923759 0 0 0 BMRU
    eth4 1500 0 30153976923 0 12253 0 10841966237 0 0 0 BMRU
    eth11 1500 0 29986516403 0 160 0 24801337764 0 0 0 BMRU
    eth13 1500 0 1659844256 0 0 0 4036960952 0 0 0 BMRU
    lo 16436 0 19027739 0 0 0 19027739 0 0 0 LRU
    [Expert@gw-1:0]# fw ctl multik stat
    ID | Active | CPU | Connections | Peak
    ----------------------------------------------
    0 | Yes | 2 | 1701 | 47832
    1 | Yes | 2 | 1715 | 46804
    2 | Yes | 1 | 1768 | 48461
    [Expert@gw-1:0]# free -m
    total used free shared buffers cached
    Mem: 32044 12929 19115 0 464 2897
    -/+ buffers/cache: 9566 22477
    Swap: 33549 0 33549
    [Expert@gw-1:0]# enabled_blades
    fw
    [Expert@gw-1:0]# fw ver
    This is Check Point's software version R77.30 - Build 048
    [Expert@gw-1:0]#

  13. #13
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    [Expert@gw-1:0]# fw ctl affinity -l -r
    CPU 0: eth3 eth4 eth1
    CPU 1: fw_2
    CPU 2: fw_1
    CPU 3: eth13 eth0 eth11
    CPU 4:
    CPU 5:
    CPU 6:
    CPU 7:
    All: rtmd fwd in.ahclientd mpdaemon in.aclientd lpd cprid cpd
    The current license permits the use of CPUs 0, 1, 2, 3 only.

    [Expert@gw-1:0]# fw ctl affinity -l
    eth3: CPU 0
    eth4: CPU 0
    eth11: CPU 0
    eth13: CPU 0
    eth0: CPU 0
    eth1: CPU 0
    fw_0: CPU 2
    fw_1: CPU 2
    fw_2: CPU 1
    The current license permits the use of CPUs 0, 1, 2, 3 only.


    [Expert@gw-1:0]# fw ctl multik stat
    ID | Active | CPU | Connections | Peak
    ----------------------------------------------
    0 | Yes | 2 | 1701 | 47832
    1 | Yes | 2 | 1715 | 46804
    2 | Yes | 1 | 1768 | 48461
    [
    Are there three kernel instances or only two? Output from the commands above is conflicting. The first output is missing fw_0, the second shows three kernel instances fighting for 2 CPU's and the third shows 3 kernel instances.
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  14. #14
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    Are there three kernel instances or only two? Output from the commands above is conflicting. The first output is missing fw_0, the second shows three kernel instances fighting for 2 CPU's and the third shows 3 kernel instances.
    Sorry, that's what happen when you have cut and paste. There is no fw_2, only fw_0 and fw_1

  15. #15
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    Sorry, that's what happen when you have cut and paste. There is no fw_2, only fw_0 and fw_1
    So interface affinity is spread between Cores 0 & 3 while fw_0 and fw_1 are running on CPUs 1 and 2 respectively? How does the CPU load distribution look via top when things are "50% slower"?
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  16. #16
    Join Date
    2006-09-26
    Posts
    3,150
    Rep Power
    16

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by ShadowPeak.com View Post
    So interface affinity is spread between Cores 0 & 3 while fw_0 and fw_1 are running on CPUs 1 and 2 respectively? How does the CPU load distribution look via top when things are "50% slower"?
    Tasks: 162 total, 1 running, 161 sleeping, 0 stopped, 0 zombie
    Cpu0 : 2.0%us, 0.0%sy, 0.0%ni, 7.8%id, 0.0%wa, 0.0%hi, 90.2%si, 0.0%st
    Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu3 : 3.0%us, 0.0%sy, 0.0%ni, 6.7%id, 0.0%wa, 0.0%hi, 88.9%si, 0.0%st

  17. #17
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,219
    Rep Power
    13

    Default Re: High CPU problem on checkpoint gateway

    Quote Originally Posted by cciesec2006 View Post
    Tasks: 162 total, 1 running, 161 sleeping, 0 stopped, 0 zombie
    Cpu0 : 2.0%us, 0.0%sy, 0.0%ni, 7.8%id, 0.0%wa, 0.0%hi, 90.2%si, 0.0%st
    Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu3 : 3.0%us, 0.0%sy, 0.0%ni, 6.7%id, 0.0%wa, 0.0%hi, 88.9%si, 0.0%st
    Well this is a first for me, adding more SND/IRQ cores actually reduces the performance of fully-accelerated (SXL path) traffic? Beyond engaging Check Point TAC the only explanation could be your slightly unusual situation of having more physical cores (8) than licensed cores (4). Might be interesting to load up an eval license and set six kernel instances (2 SND cores) and see if that improves things when all physical cores are licensed and used.
    --
    Second Edition of my "Max Power" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

Similar Threads

  1. IPSEC VPN tunnel problem between checkpoint and Juniper Gateway
    By tofke in forum IPsec VPN Blade (Virtual Private Networks)
    Replies: 6
    Last Post: 2014-04-17, 03:51
  2. node gateway vs checkpoint gateway
    By jgarzam in forum SmartDashboard
    Replies: 1
    Last Post: 2013-05-13, 08:51
  3. Is Checkpoint TAC high on crack?
    By cciesec2006 in forum Installing And Upgrading
    Replies: 7
    Last Post: 2010-03-08, 12:06
  4. problem with a SC high availability
    By jvalenzuela in forum Licensing
    Replies: 8
    Last Post: 2009-04-29, 19:25
  5. High CPU, Strange alerts problem
    By ggts2008 in forum Check Point UTM-1 Appliances
    Replies: 1
    Last Post: 2008-07-21, 16:08

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •