CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it yet again - That's right, the 3rd edition is here!
You can read his announcement post here.
It's a massive upgrade focusing on current versions, and well worth checking out. -E

 

Results 1 to 4 of 4

Thread: Connections drops same time every day!

  1. #1
    Join Date
    2017-11-01
    Posts
    37
    Rep Power
    0

    Default Connections drops same time every day!

    Hi,

    Our external 3rd party agencies that connect via a firewall in a VSX cluster go offline for 1 minute - ish, every day at 15:23.

    Interestingly, this has now moved to 14:23 when the clocks went back from BST to GMT.

    A fw ctl zdebug shows up the following

    ;[vs_0];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_0];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=547883185, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_0];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_0];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=547883185);
    ;[vs_0];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=547883185, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_1];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_1];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=547837520, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_1];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_1];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=547837520);
    ;[vs_1];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=547837520, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_2];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_2];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=546980469, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_2];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_2];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=546980469);
    ;[vs_2];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=546980469, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_3];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_3];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=546965855, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_3];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_3];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=546965855);
    ;[vs_3];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=546965855, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_4];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_4];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=546962636, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_4];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_4];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=546962636);
    ;[vs_4];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=546962636, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_5];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_5];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=546971276, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_5];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_5];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=546971276);
    ;[vs_5];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=546971276, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_6];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_6];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=546981184, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);


    Im aware that it looks like some policy installation is occuring. There are no scheduled policy installations, and no blades are updating either. Infact, this VSX cluster has no additional blades attached, just firewall.

    I have searched everywhere and I cant not find what would be triggering the policy push - if it is a policy push that is?

    I have checked chrontab and no jobs appear to show.

    Anyone shed any light to what could be happening?

    Judging by the logs, it appears to start with the gateway - vs_ 0 and cascade its way down.

    Any help would be greatly appreciated.

  2. #2
    Join Date
    2006-09-26
    Posts
    3,194
    Rep Power
    17

    Default Re: Connections drops same time every day!

    Quote Originally Posted by JPYDX View Post
    Hi,

    Our external 3rd party agencies that connect via a firewall in a VSX cluster go offline for 1 minute - ish, every day at 15:23.

    Interestingly, this has now moved to 14:23 when the clocks went back from BST to GMT.

    A fw ctl zdebug shows up the following

    ;[vs_0];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_0];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=547883185, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE, freeze_timeout=300, freeze_event_timeout=150);
    ;[vs_0];[tid_1];[fw4_0];FW-1: monitor filter loaded;
    ;[vs_0];[tid_1];[fw4_0];fwioctl: Policy has ended. Continuing extending dead timouts (fwha_cul_policy_done_time=547883185);
    ;[vs_0];[tid_1];[fw4_0];FW-1: [CUL - Member] Policy Freeze mechanism disabled, Enabling state machine at 4 (time=547883185, caller=fwioctl: FWHA_CUL_POLICY_STATE_FREEZE);
    ;[vs_1];[tid_1];[fw4_0];fwioctl: Policy has started. Extending dead timeouts;
    ;[vs_1];[tid_1];[fw4_0];FW-1: [cul_policy_freeze][CUL - Member] fwha_cul_policy_freeze_state_change: set Policy Freeze [ON], FREEZING state machine at ACTIVE (time=547837520, caller=fwioctl:

    Any help would be greatly appreciated.
    I've run into issues like this with checkpoint last year and it caused me many nights of sleep. It has nothing to do with policy push.

    Look like the firewalls had to process a lot of traffics during that time and it just freeze up. I see this quite often in R75.47. have not seen it yet in R77.30 or higher

  3. #3
    Join Date
    2006-03-08
    Location
    Lausanne
    Posts
    1,030
    Rep Power
    15

    Default Re: Connections drops same time every day!

    You are correct, there is a policy push at this time. Please make sure you do not have scheduled IPS update at this time. Automated IPS updates may cause policy push, depending on the settings.

    Also, if you are in MDSM environment, global policy application can also cause the symptoms.
    Generally speaking, check what's going on at this time on your management. Also, check your GWs do not have cron actions with "fw load" commands.
    -------------

    Valeri Loukine
    CCMA, CCSM, CCSI
    http://checkpoint-master-architect.blogspot.com/

  4. #4
    Join Date
    2006-03-08
    Location
    Lausanne
    Posts
    1,030
    Rep Power
    15

    Default Re: Connections drops same time every day!

    it seems your VSX cluster is under load at this time. CUL refers to "cluster under load".

    I have had a similar symptoms when an Internet facing VS was scanned. Scans were dropped on a clean-up rule, but since it was deep down in the rulebase, it caused a CPU spike.

    In you case, it seems VS0 and VS1 are using the same FW core. You may want to adjust affinity and probably increase amount of instances on VS1. I assume you are using DMI config, where VS0 is not forwarding any production traffic.
    -------------

    Valeri Loukine
    CCMA, CCSM, CCSI
    http://checkpoint-master-architect.blogspot.com/

Similar Threads

  1. R77.10 drops RPC over TCP
    By cpguy in forum R77.10
    Replies: 2
    Last Post: 2014-05-08, 09:23
  2. Real time monitoring of connections?
    By Spacetrucker in forum Check Point SecurePlatform (SPLAT)
    Replies: 1
    Last Post: 2008-09-04, 22:30
  3. Replies: 16
    Last Post: 2008-01-04, 07:26
  4. NATted SIP connections time out on NGX R65
    By easel in forum NAT (Network Address Translation)
    Replies: 0
    Last Post: 2007-07-18, 23:48
  5. Disconnecting Connections at a Specific Time
    By Barry J. Stiefel in forum Miscellaneous
    Replies: 0
    Last Post: 2005-08-13, 01:08

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •