PDA

View Full Version : SecureXL - All traffic goto Medium path



vincent.tai
2015-06-09, 04:21
I am wondering why all my traffic go to Medium Path (71%) instead of Accelerated Path (22%) when we have only FW blade turn on:

*This is the stats for the FW running for 1 month:

[Expert@fw:0]# fwaccel stats -s
Accelerated conns/Total conns : 14394/15924 (90%)
Accelerated pkts/Total pkts : 961767706/4279074477 (22%)
F2Fed pkts/Total pkts : 240266805/4279074477 (5%)
PXL pkts/Total pkts : 3077039966/4279074477 (71%)
QXL pkts/Total pkts : 0/4279074477 (0%)

*Here is the stats after reset:

[Expert@fw:0]# fwaccel stats -s
Accelerated conns/Total conns : 13455/14892 (90%)
Accelerated pkts/Total pkts : 38969893/124838184 (31%)
F2Fed pkts/Total pkts : 6249717/124838184 (5%)
PXL pkts/Total pkts : 79618574/124838184 (63%)
QXL pkts/Total pkts : 0/124838184 (0%)

[Expert@fw:0]# enabled_blades
fw

*fwaccel working till the bottom of the rule bases

[Expert@fw:0]# fwaccel stat
Accelerator Status : on
Accept Templates : disabled by Firewall
disabled from rule #779
Drop Templates : disabled
NAT Templates : disabled by user

Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, ViolationStats,
Nac, AsychronicNotif, ERDOS, McastRoutingV2
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256


[Expert@fw:0]# fwaccel stats
Name Value Name Value
-------------------- --------------- -------------------- ---------------

Accelerated Path
------------------------------------------------------------------------------
accel packets 39230840 accel bytes 17187865539
conns created 63359 conns deleted 95219
C total conns 15110 C templates 2730
C TCP conns 12668 C delayed TCP conns 0
C non TCP conns 2442 C delayed nonTCP con 0
conns from templates 583131 temporary conns 434861
nat conns 4 dropped packets 57
dropped bytes 5496 nat templates 0
port alloc templates 0 conns from nat tmpl 0
port alloc conns 0 conns auto expired 536499

Accelerated VPN Path
------------------------------------------------------------------------------
C crypt conns 0 enc bytes 0
dec bytes 0 ESP enc pkts 0
ESP enc err 0 ESP dec pkts 0
ESP dec err 0 ESP other err 0
AH enc pkts 0 AH enc err 0
AH dec pkts 0 AH dec err 0
AH other err 0 espudp enc pkts 0
espudp enc err 0 espudp dec pkts 0
espudp dec err 0 espudp other err 0

Medium Path
------------------------------------------------------------------------------
PXL packets 80124843 PXL async packets 80127035
PXL bytes 31583067669 C PXL conns 233
C PXL templates 18446744073709551545

Accelerated QoS Path
------------------------------------------------------------------------------
QXL packets 0 QXL async packets 0
QXL bytes 0 C QXL conns 0
C QXL templates 0

Firewall Path
------------------------------------------------------------------------------
F2F packets 6283399 F2F bytes 1488030265
C F2F conns 1139 TCP violations 39382
C partial conns 0 C anticipated conns 0
port alloc f2f 0

General
------------------------------------------------------------------------------
memory used 0 free memory 0
C used templates 2040 pxl tmpl conns 880
C conns from tmpl 10148 C non TCP F2F conns 269
C tcp handshake conn 24 C tcp established co 6562
C tcp closed conns 6082 C tcp f2f handshake 0
C tcp f2f establishe 739 C tcp f2f closed con 131
C tcp pxl handshake 2 C tcp pxl establishe 215
C tcp pxl closed con 16 outbound packets 39230840
outbound pxl packets 80124843 outbound f2f packets 6509911
outbound bytes 17811061913 outbound pxl bytes 32771395550
outbound f2f bytes 2461898435

*We don't have performance issue as we are running on SG12600 HPP with the below CoreXL tuning. I just want to find out why all the traffic will go thru medium path of SecureXL.

[Expert@fw:0]# fw ctl affinity -l
eth1-06: CPU 5
eth1-07: CPU 3
eth1-08: CPU 2
eth1-02: CPU 1
eth1-03: CPU 3
eth1-04: CPU 1
eth2-02: CPU 4
eth2-03: CPU 2
Sync: CPU 0
Mgmt: CPU 0
fw_0: CPU 11
fw_1: CPU 10
fw_2: CPU 9
fw_3: CPU 8
fw_4: CPU 7
fw_5: CPU 6
Interface eth1-05: has multi queue enabled
Interface eth1-01: has multi queue enabled
Interface eth2-01: has multi queue enabled
Interface eth2-04: has multi queue enabled

[Expert@fw:0]# cpstat -f multi_cpu os



Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 2| 18| 82| ?| 0|
| 2| 1| 9| 10| 90| ?| 0|
| 3| 0| 7| 13| 87| ?| 0|
| 4| 0| 7| 13| 87| ?| 0|
| 5| 0| 2| 18| 82| ?| 0|
| 6| 0| 12| 8| 92| ?| 0|
| 7| 0| 8| 12| 88| ?| 0|
| 8| 0| 9| 11| 89| ?| 0|
| 9| 0| 7| 13| 87| ?| 0|
| 10| 0| 4| 16| 84| ?| 0|
| 11| 0| 7| 13| 87| ?| 0|
| 12| 0| 7| 13| 87| ?| 0|
---------------------------------------------------------------------------------

taganrog
2015-06-09, 04:25
Could you please add 'fw ctl pstat' from the GW?

vincent.tai
2015-06-09, 04:28
Could you please add 'fw ctl pstat' from the GW?

Sure :-)

[Expert@fw:0]# fw ctl pstat

System Capacity Summary:
Memory used: 7% (674 MB out of 8803 MB) - below watermark
Concurrent Connections: 14992 (Unlimited)
Aggressive Aging is disabled

Hash kernel memory (hmem) statistics:
Total memory allocated: 922746880 bytes in 225280 (4096 bytes) blocks using 1 pool
Total memory bytes used: 56685600 unused: 866061280 (93.86%) peak: 139084060
Total memory blocks used: 18750 unused: 206530 (91%) peak: 36455
Allocations: 3973309732 alloc, 0 failed alloc, 3972816256 free

System kernel memory (smem) statistics:
Total memory bytes used: 1359306348 peak: 1377294872
Total memory bytes wasted: 4668923
Blocking memory bytes used: 6188612 peak: 6688184
Non-Blocking memory bytes used: 1353117736 peak: 1370606688
Allocations: 12076015 alloc, 0 failed alloc, 12072156 free, 0 failed free
vmalloc bytes used: 1343492324 expensive: no

Kernel memory (kmem) statistics:
Total memory bytes used: 491150228 peak: 568179768
Allocations: 3985345593 alloc, 0 failed alloc
3984850371 free, 0 failed free
External Allocations: 64512 for packets, 93942515 for SXL

Cookies:
3857168534 total, 0 alloc, 0 free,
14403 dup, 79558363 get, 671996326 put,
1874878110 len, 4814000 cached len, 0 chain alloc,
0 chain free

Connections:
771939250 total, 581787229 TCP, 170815157 UDP, 19208455 ICMP,
128409 other, 316027 anticipated, 61765 recovered, 14992 concurrent,
48203 peak concurrent

Fragments:
7417657 fragments, 3700840 packets, 2577 expired, 0 short,
0 large, 0 duplicates, 0 failures

NAT:
24862616/0 forw, 19815489/0 bckw, 157869 tcpudp,
42551126 icmp, 3617919-1102430 alloc

Sync:
Version: new
Status: Able to Send/Receive sync packets
Sync packets sent:
total : 2303905619, retransmitted : 729, retrans reqs : 1003, acks : 7719
Sync packets received:
total : 3277269129, were queued : 15519865, dropped by net : 1387
retrans reqs : 585, received 56289 acks
retrans reqs for illegal seq : 0
dropped updates as a result of sync overload: 2749

Irek_Romaniuk
2015-06-09, 09:13
Maybe http inspection is in use ?

ShadowPeak.com
2015-06-09, 09:22
Please post output of "fwaccel stats -p" which will show SecureXL violation counters that indicate why traffic could not be handled in the accelerated path.

vincent.tai
2015-06-10, 02:11
Please post output of "fwaccel stats -p" which will show SecureXL violation counters that indicate why traffic could not be handled in the accelerated path.

Here is the output. Thanks.

[Expert@fw:0]# fwaccel stats -s
Accelerated conns/Total conns : 11536/13019 (88%)
Accelerated pkts/Total pkts : 1082788768/4794406750 (22%)
F2Fed pkts/Total pkts : 281341619/4794406750 (5%)
PXL pkts/Total pkts : 3430276363/4794406750 (71%)
QXL pkts/Total pkts : 0/4794406750 (0%)

[Expert@fw:0]# fwaccel stats -p
F2F packets:
--------------
Violation Packets Violation Packets
-------------------- --------------- -------------------- ---------------
pkt is a fragment 119683 pkt has IP options 2591
ICMP miss conn 1952761 TCP-SYN miss conn 7393223
TCP-other miss conn 139882 UDP miss conn 29838677
other miss conn 26 VPN returned F2F 0
ICMP conn is F2Fed 3250273 TCP conn is F2Fed 469465574
UDP conn is F2Fed 1097758 other conn is F2Fed 28
uni-directional viol 0 possible spoof viol 0
TCP state viol 14787834 out if not def/accl 1
bridge, src=dst 0 routing decision err 0
sanity checks failed 0 temp conn expired 434
fwd to non-pivot 0 broadcast/multicast 0
cluster message 0 partial conn 0
PXL returned F2F 97622 cluster forward 0
chain forwarding 742394 general reason 0

vincent.tai
2015-06-10, 02:16
Maybe http inspection is in use ?

No, since we only have FW blade enable.

ShadowPeak.com
2015-06-10, 10:54
Here is the output. Thanks.

[Expert@fw:0]# fwaccel stats -s
Accelerated conns/Total conns : 11536/13019 (88%)
Accelerated pkts/Total pkts : 1082788768/4794406750 (22%)
F2Fed pkts/Total pkts : 281341619/4794406750 (5%)
PXL pkts/Total pkts : 3430276363/4794406750 (71%)
QXL pkts/Total pkts : 0/4794406750 (0%)

[Expert@fw:0]# fwaccel stats -p
F2F packets:
--------------
Violation Packets Violation Packets
-------------------- --------------- -------------------- ---------------
pkt is a fragment 119683 pkt has IP options 2591
ICMP miss conn 1952761 TCP-SYN miss conn 7393223
TCP-other miss conn 139882 UDP miss conn 29838677
other miss conn 26 VPN returned F2F 0
ICMP conn is F2Fed 3250273 TCP conn is F2Fed 469465574
UDP conn is F2Fed 1097758 other conn is F2Fed 28
uni-directional viol 0 possible spoof viol 0
TCP state viol 14787834 out if not def/accl 1
bridge, src=dst 0 routing decision err 0
sanity checks failed 0 temp conn expired 434
fwd to non-pivot 0 broadcast/multicast 0
cluster message 0 partial conn 0
PXL returned F2F 97622 cluster forward 0
chain forwarding 742394 general reason 0

Hmm, nothing is jumping out at me there. You'll have to run a debug to determine what is going on, will probably want to do this during a non-busy time for your firewall. Procedure:

1) Find out the IP address of an Internet website you can test with. Hopefully it will be a site that only has one IP address returned for a DNS lookup. Let's suppose its IP is 129.82.102.32 and the internal workstation you will be initiating test traffic from is 192.168.1.100.

2) Set a filter for the debug, format is sip,sport,dip,dport,proto:

sim dbg -f 192.168.1.100,*,129.82.102.32,80,6

3) Run "sim dbg list". At the end of the output you should see the filter displayed. MAKE SURE it is displaying completely and correctly; messing up a debug in the SecureXL driver can be very bad from a firewall stability and performance perspective.

4) Start the debug for 15 seconds like this:

sim dbg -m pkt + pxl + f2f ; sleep 15 ; sim dbg resetall

5) During that 15 seconds initiate HTTP traffic to the 129.82.102.32 website from your testing workstation. Note that the command provided will automatically terminate the debug after 15 seconds as a safety measure, you can adjust this time as desired. This test traffic must be a stateful protocol like TCP, specifically ICMP/ping is never accelerated and thus will not give us the results we need.

6) Manually run "sim dbg resetall" again just to be sure.

7) Check /var/log/messages and you should see something resembling this:

[fw4_0];get_conn_flags: APCL is set on for the connection -> PXL;

If you don't see anything in the log, that connection was accelerated by SecureXL and did not go Medium Path. You'll have to run "fwaccel conns" (DO NOT run this command on a firewall running a release earlier than R77 due to the situation described in sk97772) and look for Medium Path connections which are indicated by both the F and S flags being present. Adjust your filter to catch traffic from that specific connection(s) and try again.

torone
2017-02-07, 02:34
have you controlled that you are compliant with exception list in SecureXL?
https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk32578#Acceleration of packets

wayne0206
2017-02-07, 15:21
1) it seems traffic is not accelerated from rule #779. perhaps your high volume traffic is below this rule?
2) please post the output "cpstat blades" and "enabled_blades"
3) there are many conditions could contribute to this. read sk32578

cciesec2006
2017-02-07, 15:28
1) it seems traffic is not accelerated from rule #779. perhaps your high volume traffic is below this rule?
2) please post the output "cpstat blades" and "enabled_blades"
3) there are many conditions could contribute to this. read sk32578

what type of traffics?

I have multiple tickets opened with Checkpoint and Microsoft DFS-R will NOT be accelerated by the firewalls, known issue and I think if you have lot of fragmented packets, it will not be accelerated either. No fix anytime soon.

jdmoore0883
2017-02-07, 15:29
1) it seems traffic is not accelerated from rule #779. perhaps your high volume traffic is below this rule?

Acceleration will continue throughout the entire rulebase, past any rule that you see here; it isn't ACCELERATION that stops at rule 779, but TEMPLATING.

wayne0206
2017-02-07, 16:09
SexureXL is a good marketing feature. you may puke after reading sk32578

jflemingeds
2017-02-07, 16:40
SexureXL is a good marketing feature. you may puke after reading sk32578

you guys know this is a super old thread right?

ShadowPeak.com
2017-02-07, 17:14
Acceleration will continue throughout the entire rulebase, past any rule that you see here; it isn't ACCELERATION that stops at rule 779, but TEMPLATING.

Absolutely correct and a common source of confusion. Acceleration is the ability to process in a more efficient path than F2F, whereas templating is the ability to "cache" rulebase lookups and avoid a full top-down first-fit evaluation of the security policy. In fact the output of "fwaccel stat" in R80.10 has been significantly clarified due to this common misconception.