CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it yet again - That's right, the 3rd edition is here!
You can read his announcement post here.
It's a massive upgrade focusing on current versions, and well worth checking out. -E

 

Results 1 to 7 of 7

Thread: Appliances 5900 R80.10 and bonding interfaces limited throughput

  1. #1
    Join Date
    2018-10-06
    Posts
    6
    Rep Power
    0

    Default Appliances 5900 R80.10 and bonding interfaces limited throughput

    Hi guys,

    I have a clusterxl with two appliances 5900 and Gaia OS R80.10.
    Network is configured with 2 bond interfaces 802.3ad layer 3+4, each bond has 4 ethernet aggregated, each bond has more vlan interfaces configured, about 10 in internal bond and 3 on external bond. All ethernet interfaces are connected to cisco switch with LACP configured, 2 groups.
    No QoS is configured, no IPS is configured only security policies with about 50 rules.
    My test is simple I trying to transfer data between vlan on internal bond and my speed is about 3Gbps and somethings more, the same test on external bond I got 3Gbps.
    When I trying to transfer data between vlan on internal bond and vlan on external bond, I reach the limit of ethernet interface 1Gbps. Why?
    In my test I have used different source address and destination address so also TCP ports all different
    While my test is executing I can see in cpview the throughput on each interfaces, I can see the traffic is balanced between they but the sum don't go over 1Gpbs when I use vlans on both bond interfaces, but when the vlan is on same bond interface I see more interfaces reach the limit of 1Gbps so the sum on bond interface is over the 1Gbps and I got 3Gbps and more of throughput.

    Suggestions?
    Thanks

  2. #2
    Join Date
    2006-09-26
    Posts
    3,194
    Rep Power
    17

    Default Re: Appliances 5900 R80.10 and bonding interfaces limited throughput

    Quote Originally Posted by juvann View Post
    Hi guys,

    I have a clusterxl with two appliances 5900 and Gaia OS R80.10.
    Network is configured with 2 bond interfaces 802.3ad layer 3+4, each bond has 4 ethernet aggregated, each bond has more vlan interfaces configured, about 10 in internal bond and 3 on external bond. All ethernet interfaces are connected to cisco switch with LACP configured, 2 groups.
    No QoS is configured, no IPS is configured only security policies with about 50 rules.
    My test is simple I trying to transfer data between vlan on internal bond and my speed is about 3Gbps and somethings more, the same test on external bond I got 3Gbps.
    When I trying to transfer data between vlan on internal bond and vlan on external bond, I reach the limit of ethernet interface 1Gbps. Why?
    In my test I have used different source address and destination address so also TCP ports all different
    While my test is executing I can see in cpview the throughput on each interfaces, I can see the traffic is balanced between they but the sum don't go over 1Gpbs when I use vlans on both bond interfaces, but when the vlan is on same bond interface I see more interfaces reach the limit of 1Gbps so the sum on bond interface is over the 1Gbps and I got 3Gbps and more of throughput.

    Suggestions?
    Thanks
    A few questions for you:

    1- Is SecureXL enabled?
    2- How do you perform the test? Are you using Iperf to do this test?


    My guess is that you might have a sim affinity issue based on the symptom you described.

  3. #3
    Join Date
    2018-10-06
    Posts
    6
    Rep Power
    0

    Default Re: Appliances 5900 R80.10 and bonding interfaces limited throughput

    Yes SecureXL is enabled and yes I have used iperf for test.
    With cpview I can't see none CPU with 100% usage.

  4. #4
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,252
    Rep Power
    14

    Default Re: Appliances 5900 R80.10 and bonding interfaces limited throughput

    Quote Originally Posted by juvann View Post
    Yes SecureXL is enabled and yes I have used iperf for test.
    With cpview I can't see none CPU with 100% usage.
    Please provide output of the following commands run on the firewall from expert mode:

    fwaccel stat
    fwaccel stats -s
    grep -c ^processor /proc/cpuinfo
    /sbin/cpuinfo
    fw ctl affinity -l -r
    sim affinity -l
    netstat -ni
    fw ctl multik stat
    fw ctl multik get_mode
    cpstat os -f multi_cpu -o 1
    free -m
    enabled_blades
    --
    Third Edition of my "Max Power 2020" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

  5. #5
    Join Date
    2006-09-26
    Posts
    3,194
    Rep Power
    17

    Default Re: Appliances 5900 R80.10 and bonding interfaces limited throughput

    Quote Originally Posted by juvann View Post
    Yes SecureXL is enabled and yes I have used iperf for test.
    With cpview I can't see none CPU with 100% usage.
    I assume that you also have this command on your Cisco switches:

    port-channel load-balance src-dst-ip

    On Cisco newer switches, you also see this:

    port-channel load-balance vlan-src-dst-mixed-ip-port

  6. #6
    Join Date
    2018-10-06
    Posts
    6
    Rep Power
    0

    Default Re: Appliances 5900 R80.10 and bonding interfaces limited throughput

    Thank you to all for suggestions! I solved my issue.
    I enabled multi queue on ethernet in bond interfaces, and now I got all 2Gpbs with multi connections.

    I enabled 4 queue for driver igb
    cpmq set rx_num igb 4

    Enabled queue for interfaces
    cpmq set -f

    and after reboot I have this situation:

    Code:
    # cpmq get -v
    
    Active igb interfaces:
    Mgmt [Off]
    Sync [Off]
    eth1 [On]
    eth1-01 [On]
    eth1-05 [On]
    eth5 [On]
    eth8 [Off]
    
    The rx_num for igb is: 4
    
    multi-queue affinity for igb interfaces:
    CPU | TX | Vector                      | RX Bytes
    -------------------------------------------------------------
    0   | 0  | eth1-01-TxRx-0 (186)        | 30661660675
        |    | eth1-05-TxRx-0 (91)         |
        |    | eth1-TxRx-0 (179)           |
        |    | eth5-TxRx-0 (227)           |
    1   | 2  | eth1-01-TxRx-2 (202)        | 19214905571
        |    | eth1-05-TxRx-2 (107)        |
        |    | eth1-TxRx-2 (195)           |
        |    | eth5-TxRx-2 (52)            |
    2   | 0  |                             |
    3   | 2  |                             |
    4   | 0  |                             |
    5   | 2  |                             |
    6   | 0  |                             |
    7   | 2  |                             |
    8   | 1  | eth1-01-TxRx-1 (194)        | 26115216389
        |    | eth1-05-TxRx-1 (99)         |
        |    | eth1-TxRx-1 (187)           |
        |    | eth5-TxRx-1 (235)           |
    9   | 3  | eth1-01-TxRx-3 (210)        | 683483583031
        |    | eth1-05-TxRx-3 (115)        |
        |    | eth1-TxRx-3 (203)           |
        |    | eth5-TxRx-3 (60)            |
    10  | 1  |                             |
    11  | 3  |                             |
    12  | 1  |                             |
    13  | 3  |                             |
    14  | 1  |                             |
    15  | 3  |                             |

  7. #7
    Join Date
    2009-04-30
    Location
    Colorado, USA
    Posts
    2,252
    Rep Power
    14

    Default Re: Appliances 5900 R80.10 and bonding interfaces limited throughput

    Quote Originally Posted by juvann View Post
    Thank you to all for suggestions! I solved my issue.
    I enabled multi queue on ethernet in bond interfaces, and now I got all 2Gpbs with multi connections.

    I enabled 4 queue for driver igb
    cpmq set rx_num igb 4

    Enabled queue for interfaces
    cpmq set -f

    and after reboot I have this situation:

    Code:
    # cpmq get -v
    
    Active igb interfaces:
    Mgmt [Off]
    Sync [Off]
    eth1 [On]
    eth1-01 [On]
    eth1-05 [On]
    eth5 [On]
    eth8 [Off]
    
    The rx_num for igb is: 4
    
    multi-queue affinity for igb interfaces:
    CPU | TX | Vector                      | RX Bytes
    -------------------------------------------------------------
    0   | 0  | eth1-01-TxRx-0 (186)        | 30661660675
        |    | eth1-05-TxRx-0 (91)         |
        |    | eth1-TxRx-0 (179)           |
        |    | eth5-TxRx-0 (227)           |
    1   | 2  | eth1-01-TxRx-2 (202)        | 19214905571
        |    | eth1-05-TxRx-2 (107)        |
        |    | eth1-TxRx-2 (195)           |
        |    | eth5-TxRx-2 (52)            |
    2   | 0  |                             |
    3   | 2  |                             |
    4   | 0  |                             |
    5   | 2  |                             |
    6   | 0  |                             |
    7   | 2  |                             |
    8   | 1  | eth1-01-TxRx-1 (194)        | 26115216389
        |    | eth1-05-TxRx-1 (99)         |
        |    | eth1-TxRx-1 (187)           |
        |    | eth5-TxRx-1 (235)           |
    9   | 3  | eth1-01-TxRx-3 (210)        | 683483583031
        |    | eth1-05-TxRx-3 (115)        |
        |    | eth1-TxRx-3 (203)           |
        |    | eth5-TxRx-3 (60)            |
    10  | 1  |                             |
    11  | 3  |                             |
    12  | 1  |                             |
    13  | 3  |                             |
    14  | 1  |                             |
    15  | 3  |                             |
    Great, thanks for the follow-up.
    --
    Third Edition of my "Max Power 2020" Firewall Book
    Now Available at http://www.maxpowerfirewalls.com

Similar Threads

  1. How many CPU cores 5900 has?
    By blason in forum Firewall Blade
    Replies: 3
    Last Post: 2018-05-11, 09:05
  2. Bonding interfaces with 2 Switch HP 5406ZL
    By networkingkool in forum Clustering (Security Gateway HA and ClusterXL)
    Replies: 2
    Last Post: 2013-06-20, 02:38
  3. using bonding interfaces in Virtual Systems
    By zenitt in forum Check Point VSX/VSX-1 Appliances
    Replies: 1
    Last Post: 2012-10-04, 05:53
  4. Interface Bonding on UTM Appliances not supported!
    By Yasushi Kono in forum Check Point UTM-1 Appliances
    Replies: 12
    Last Post: 2012-09-08, 08:28
  5. Power-1 11067 2x10Gb interfaces loadbalancing (8021ad) bonding
    By pekka.kovesjarvi in forum Check Point Power-1 Appliances
    Replies: 4
    Last Post: 2012-03-27, 09:30

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •