CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


CPUG Challenge 2018?? We will be holding another CPUG Challenge for 2018.
The plan is to time it around CPX again (earlier this year), but not necessarily limit it to those in attendance.
I'll provide more details as we get a bit closer, but be ready! -E

 

Page 2 of 2 FirstFirst 12
Results 21 to 29 of 29

Thread: Java Process Consuming High CPU in R80

  1. #21
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,088
    Rep Power
    12

    Default Re: Java Process Consuming High CPU in R80

    Quote Originally Posted by venkata View Post
    Hello ShadowPeak

    Can I run this tool on R80 SMS as well? I dont think we are doing smart event on the SMS. Can you please provide instructions as well? Also, do I need to run this command during off hours (Will this create any drastic load on the server resources?)

    Thanks.
    This tool was included in R77.30, so it should be in R80 as well, just run CPLogInvestigator. It will cause a bit of load, but only on the Security Management Server which won't affect the gateways. You can run it during regular hours.
    --
    My Book "Max Power: Check Point Firewall Performance Optimization"
    Second Edition Coming Soon

  2. #22
    Join Date
    2016-10-19
    Posts
    24
    Rep Power
    0

    Default Re: Java Process Consuming High CPU in R80

    Hello ShadowPeak

    This is what I saw in sk98767

    The SmartEvent Sizing Tool is not suitable for R80.
    The above Sizing Table applies to R80 version as well (the numbers were received based on estimations and testing in the lab).

    Another reason why I think this LogInvestigator tool cannot be used for R80 is when I ran the tool on my Mgmt server, I used the sk you've mentioned and saw that I need Smart-1 205 appliance which I dont think is enough to deal with 75GB of logs each day (logging + indexing). I could be wrong in this approach.

    Just want to know your thoughts on this.

    Thanks.

  3. #23
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,088
    Rep Power
    12

    Default Re: Java Process Consuming High CPU in R80

    Quote Originally Posted by venkata View Post
    Hello ShadowPeak

    This is what I saw in sk98767

    The SmartEvent Sizing Tool is not suitable for R80.
    The above Sizing Table applies to R80 version as well (the numbers were received based on estimations and testing in the lab).

    Another reason why I think this LogInvestigator tool cannot be used for R80 is when I ran the tool on my Mgmt server, I used the sk you've mentioned and saw that I need Smart-1 205 appliance which I dont think is enough to deal with 75GB of logs each day (logging + indexing). I could be wrong in this approach.

    Just want to know your thoughts on this.

    Thanks.
    Personally I wouldn't want to do R80+ management on anything lower than a Smart-1 225 which has 4 cores and 16GB of RAM. It will certainly work on a Smart-1 205 or 210 but the performance will not be good as these boxes only have 2 cores and 4GB RAM (205) or 8GB RAM (210). I guess you could try loading up a 205/210 with RAM to help out but the processor will still be a major bottleneck. Not recommended.
    --
    My Book "Max Power: Check Point Firewall Performance Optimization"
    Second Edition Coming Soon

  4. #24
    Join Date
    2005-08-14
    Location
    Gig Harbor, WA, USA
    Posts
    2,388
    Rep Power
    15

    Default Re: Java Process Consuming High CPU in R80

    The Smart-1 205 was definitely not designed for that scale of logs (75GB/day), even in R77.30.
    http://phoneboy.org
    Unless otherwise noted, views expressed are my own

  5. #25
    Join Date
    2016-10-19
    Posts
    24
    Rep Power
    0

    Default Re: Java Process Consuming High CPU in R80

    Hello Team

    Can someone shed some light on my situation?

    I have an Open server running Checkpoint SMS + Logging and here are the stats of current environment:

    Number of clusters: 7
    Logs per second: 5500-7000
    Indexed logs/sec: 500-700
    Log size per day: 60GB - 75GB max (45GB - 55GB : real time )
    LEA Connections: 1 (Splunk)
    SSH connections: 1 (Algosec)

    Specs of Open Server:
    8 core Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz
    128G Memory
    OS Drive = Raid 1 (Mirrored)
    Data Drive = RAID 5 ( 3 drives)

    We are experiencing performance issue with the Management server while using smart console (response time is very slow at times), some admins sessions get disconnected. When looking at "top" output, I can see that java, log_indexer are the top consumers of CPU. So we decided to offload either Logging or Policy Management from the Open server and buy a 3050 Appliance (Not that this will solve all the issues, but want to offload the load)

    Specs of 3050: CPAP-NGSM3050

    indexed logs/sec: 26000 / 16000 for single domain
    event logs/sec: 3000
    events per day: 4000000
    log size per day: 40GB
    Raid type: 5,10

    My Question: Is running logging on the Open server and running policy management on 3050 preferred or other way around is preferred? Will load of "java" process be split between Open server and 3050 once I separate logging and Policy Mgmt?

    As the CPU of the Open server is little higher than the 3050, I am tending to use logging on the Open server as we also do Log Indexing.

  6. #26
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,088
    Rep Power
    12

    Default Re: Java Process Consuming High CPU in R80

    Quote Originally Posted by venkata View Post
    Hello Team

    Can someone shed some light on my situation?

    I have an Open server running Checkpoint SMS + Logging and here are the stats of current environment:

    Number of clusters: 7
    Logs per second: 5500-7000
    Indexed logs/sec: 500-700
    Log size per day: 60GB - 75GB max (45GB - 55GB : real time )
    LEA Connections: 1 (Splunk)
    SSH connections: 1 (Algosec)

    Specs of Open Server:
    8 core Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz
    128G Memory
    OS Drive = Raid 1 (Mirrored)
    Data Drive = RAID 5 ( 3 drives)

    We are experiencing performance issue with the Management server while using smart console (response time is very slow at times), some admins sessions get disconnected. When looking at "top" output, I can see that java, log_indexer are the top consumers of CPU. So we decided to offload either Logging or Policy Management from the Open server and buy a 3050 Appliance (Not that this will solve all the issues, but want to offload the load)

    Specs of 3050: CPAP-NGSM3050

    indexed logs/sec: 26000 / 16000 for single domain
    event logs/sec: 3000
    events per day: 4000000
    log size per day: 40GB
    Raid type: 5,10

    My Question: Is running logging on the Open server and running policy management on 3050 preferred or other way around is preferred? Will load of "java" process be split between Open server and 3050 once I separate logging and Policy Mgmt?

    As the CPU of the Open server is little higher than the 3050, I am tending to use logging on the Open server as we also do Log Indexing.
    Please provide output of following (ideally while access is slow):

    free -m
    mpstat 2 5
    iostat 2 5
    /sbin/cpuinfo
    --
    My Book "Max Power: Check Point Firewall Performance Optimization"
    Second Edition Coming Soon

  7. #27
    Join Date
    2016-10-19
    Posts
    24
    Rep Power
    0

    Default Re: Java Process Consuming High CPU in R80

    Hello, here is the output of the commands (Not during the access is slow, I will try and get that later next week)

    [Expert@xxxx]# free -m
    total used free shared buffers cached
    Mem: 128729 126102 2627 0 1719 71654
    -/+ buffers/cache: 52728 76001
    Swap: 8189 0 8188
    [Expert@xxxx]#
    [Expert@xxxx]#
    [Expert@xxxx]# mpstat 2 5
    Linux 2.6.18-92cpx86_64 (xxxx) 12/08/17

    21:10:53 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    21:10:55 all 23.72 44.44 3.43 0.06 0.06 0.62 0.00 27.65 2468.34
    21:10:57 all 21.31 49.62 3.12 0.06 0.00 0.31 0.00 25.56 1467.50
    21:10:59 all 22.92 43.66 3.06 0.19 0.00 0.44 0.00 29.73 2891.50
    21:11:01 all 18.95 42.09 2.56 0.00 0.06 0.25 0.00 36.09 1486.07
    21:11:03 all 23.61 38.23 12.18 0.06 0.00 0.81 0.00 25.11 2088.50
    Average: all 22.10 43.61 4.87 0.07 0.02 0.49 0.00 28.83 2079.40
    [Expert@xxxx]#
    [Expert@xxxx]#
    [Expert@xxxx]# iostat 2 5
    Linux 2.6.18-92cpx86_64 (xxxx) 12/08/17

    avg-cpu: %user %nice %system %iowait %steal %idle
    17.44 34.60 4.30 0.15 0.00 43.52

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 54.93 28.79 7689.83 385513966 102962977947
    cciss/c0d1 288.83 590.88 36687.55 7911562490 491227713104
    dm-0 158.82 6.31 1269.50 84456346 16998006000
    dm-1 6.37 611.60 43107.67 8188972530 577189925768
    sda 0.00 0.00 0.00 41820 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    34.69 41.69 2.50 0.00 0.00 21.12

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 5.53 8.04 168.84 16 336
    cciss/c0d1 13.07 1672.36 0.00 3328 0
    dm-0 21.61 8.04 168.84 16 336
    dm-1 13.07 1672.36 0.00 3328 0
    sda 0.00 0.00 0.00 0 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    31.31 37.56 4.69 0.12 0.00 26.31

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 216.50 0.00 5488.00 0 10976
    cciss/c0d1 939.50 1540.00 151604.00 3080 303208
    dm-0 70.00 0.00 560.00 0 1120
    dm-1 18.50 1540.00 156532.00 3080 313064
    sda 0.00 0.00 0.00 0 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    29.48 39.16 2.87 0.06 0.00 28.42

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 7.04 0.00 329.65 0 656
    cciss/c0d1 12.06 1543.72 0.00 3072 0
    dm-0 41.21 0.00 329.65 0 656
    dm-1 12.06 1543.72 0.00 3072 0
    sda 0.00 0.00 0.00 0 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    32.46 40.76 2.43 0.00 0.00 24.34

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 7.46 0.00 175.12 0 352
    cciss/c0d1 12.94 1655.72 0.00 3328 0
    dm-0 21.89 0.00 175.12 0 352
    dm-1 12.94 1655.72 0.00 3328 0
    sda 0.00 0.00 0.00 0 0

    [Expert@xxxx]# /sbin/cpuinfo
    HyperThreading=disabled

    Thanks.

  8. #28
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,088
    Rep Power
    12

    Default Re: Java Process Consuming High CPU in R80

    Quote Originally Posted by venkata View Post
    Hello, here is the output of the commands (Not during the access is slow, I will try and get that later next week)

    [Expert@xxxx]# free -m
    total used free shared buffers cached
    Mem: 128729 126102 2627 0 1719 71654
    -/+ buffers/cache: 52728 76001
    Swap: 8189 0 8188
    Plenty of RAM, no swap space usage. This assumes of course that the Smart-1 has not been rebooted since the last slow period(s).

    [Expert@xxxx]#
    [Expert@xxxx]#
    [Expert@xxxx]# mpstat 2 5
    Linux 2.6.18-92cpx86_64 (xxxx) 12/08/17

    21:10:53 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    21:10:55 all 23.72 44.44 3.43 0.06 0.06 0.62 0.00 27.65 2468.34
    21:10:57 all 21.31 49.62 3.12 0.06 0.00 0.31 0.00 25.56 1467.50
    21:10:59 all 22.92 43.66 3.06 0.19 0.00 0.44 0.00 29.73 2891.50
    21:11:01 all 18.95 42.09 2.56 0.00 0.06 0.25 0.00 36.09 1486.07
    21:11:03 all 23.61 38.23 12.18 0.06 0.00 0.81 0.00 25.11 2088.50
    Average: all 22.10 43.61 4.87 0.07 0.02 0.49 0.00 28.83 2079.40
    [Expert@xxxx]#
    [Expert@xxxx]#
    A total of 43.61% CPU time is nice'd (has a lower priority) in process space which is SOLR doing log indexing. No excessive waiting for I/O.

    [Expert@xxxx]# iostat 2 5
    Linux 2.6.18-92cpx86_64 (xxxx) 12/08/17

    avg-cpu: %user %nice %system %iowait %steal %idle
    17.44 34.60 4.30 0.15 0.00 43.52

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 54.93 28.79 7689.83 385513966 102962977947
    cciss/c0d1 288.83 590.88 36687.55 7911562490 491227713104
    dm-0 158.82 6.31 1269.50 84456346 16998006000
    dm-1 6.37 611.60 43107.67 8188972530 577189925768
    sda 0.00 0.00 0.00 41820 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    34.69 41.69 2.50 0.00 0.00 21.12

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 5.53 8.04 168.84 16 336
    cciss/c0d1 13.07 1672.36 0.00 3328 0
    dm-0 21.61 8.04 168.84 16 336
    dm-1 13.07 1672.36 0.00 3328 0
    sda 0.00 0.00 0.00 0 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    31.31 37.56 4.69 0.12 0.00 26.31

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 216.50 0.00 5488.00 0 10976
    cciss/c0d1 939.50 1540.00 151604.00 3080 303208
    dm-0 70.00 0.00 560.00 0 1120
    dm-1 18.50 1540.00 156532.00 3080 313064
    sda 0.00 0.00 0.00 0 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    29.48 39.16 2.87 0.06 0.00 28.42

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 7.04 0.00 329.65 0 656
    cciss/c0d1 12.06 1543.72 0.00 3072 0
    dm-0 41.21 0.00 329.65 0 656
    dm-1 12.06 1543.72 0.00 3072 0
    sda 0.00 0.00 0.00 0 0

    avg-cpu: %user %nice %system %iowait %steal %idle
    32.46 40.76 2.43 0.00 0.00 24.34

    Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
    cciss/c0d0 7.46 0.00 175.12 0 352
    cciss/c0d1 12.94 1655.72 0.00 3328 0
    dm-0 21.89 0.00 175.12 0 352
    dm-1 12.94 1655.72 0.00 3328 0
    sda 0.00 0.00 0.00 0 0

    [Expert@xxxx]# /sbin/cpuinfo
    HyperThreading=disabled

    Thanks.
    These statistics were not taken during a slow period, yet the total CPU is only 28% idle. The box just looks very busy from a CPU perspective, my guess is during the slow periods the CPU is running at 100%, even if quite a bit of process space time is nice'd there can still be contention on the hard drive as well. I guess we will find out when you capture some stats during a slow period.

    Please provide output of raidconfig status to make sure your RAID setup is healthy, although based on the very low wio percentage I doubt your RAID setup is degraded.

    Finally you can run cpview in historical mode with the -t option and step through minute-by-minute a period where the access was known to be slow. I'd recommend using cpview and look at its CPU screen during a past slow period, see sk101878 for how to invoke cpview in historical mode.
    Last edited by ShadowPeak.com; 4 Days Ago at 09:43.
    --
    My Book "Max Power: Check Point Firewall Performance Optimization"
    Second Edition Coming Soon

  9. #29
    Join Date
    2016-10-19
    Posts
    24
    Rep Power
    0

    Default Re: Java Process Consuming High CPU in R80

    Hello ShadowPeak

    Looks like historic mode is not available for SMS versions R77 and above - "history mode is not supported" (Found this in Supported Deployements part of the sk). Our SMS is R80 Take 76.

    Any thoughts?

    Thanks.

Page 2 of 2 FirstFirst 12

Similar Threads

  1. high cpu on the fw process of the standby firewall
    By cciesec2006 in forum Miscellaneous
    Replies: 18
    Last Post: 4 Weeks Ago, 20:23
  2. Web Applications and Active-x (java code)
    By Kiwi_wgtn in forum Mobile Access Blade (Formerly Connectra)
    Replies: 0
    Last Post: 2012-11-18, 18:52
  3. java.net.SocketException on FW1
    By sgaiotti in forum Firewall Blade
    Replies: 0
    Last Post: 2012-08-10, 06:09
  4. High CPU - FWM and CPD process (R71.30)
    By DntBrnDPig in forum Check Point UTM-1 Appliances
    Replies: 3
    Last Post: 2011-10-18, 04:03
  5. Several UTM's running high CPD process...
    By boldin in forum Check Point UTM-1 Appliances
    Replies: 0
    Last Post: 2009-02-07, 16:21

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •