CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it again! He has just released the 2nd edition of "Max Power".
Rather than get into details here, I urge you to check out this announcement post.
It's a massive upgrade, and well worth checking out. -E

 

Results 1 to 20 of 20

Thread: Management Server HA two different data centers?

  1. #1
    Join Date
    2018-04-18
    Posts
    40
    Rep Power
    0

    Default Management Server HA two different data centers?

    Good Morning,

    My organization currently has two data center, one active, one standby. We currently have only one Check Point SMS running on VMware and we are looking to add a secondary SMS on VMware at our standby site. In the event there is a site down at our active site we want to be able to start using the secondary management server at the secondary / standby site as seamlessly as possible, with SIC already initialized and the security policies up to date.

    We have brought this up many times with your Check Point rep and he seems to keep steering us away from this. We were told the only recommended way for SMS HA is having both servers at the same site and also a VMware snapshot will suffice, however if we went the snapshot route the IP of the management server would have to change in a site down DR event.

    Can anyone provide any insight on what we are looking at doing is actually feasible? I know we would need to purchase another SMS license.

    Currently our SMS is also the log server. We are currently on R77.30 and would be looking to do this when we upgrade to R80.10 later this year.

    Thank you.

  2. #2
    Join Date
    2006-03-21
    Posts
    87
    Rep Power
    13

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    Good Morning,

    My organization currently has two data center, one active, one standby. We currently have only one Check Point SMS running on VMware and we are looking to add a secondary SMS on VMware at our standby site. In the event there is a site down at our active site we want to be able to start using the secondary management server at the secondary / standby site as seamlessly as possible, with SIC already initialized and the security policies up to date.

    We have brought this up many times with your Check Point rep and he seems to keep steering us away from this. We were told the only recommended way for SMS HA is having both servers at the same site and also a VMware snapshot will suffice, however if we went the snapshot route the IP of the management server would have to change in a site down DR event.

    Can anyone provide any insight on what we are looking at doing is actually feasible? I know we would need to purchase another SMS license.

    Currently our SMS is also the log server. We are currently on R77.30 and would be looking to do this when we upgrade to R80.10 later this year.

    Thank you.
    Hi there,

    As long as:
    1. There is a decent bandwidth between the primary and secondary SMS
    2. The secondary SMS is able to reach all the gateways

    There shouldn't be any problem with building a standby SMS. In R77.30 the database synchronization between the primary and secondary SMS can be set to manual or automatic.

    Management high availability does not have as many requirements as clusterXL.

    All management operations such as editing and installing the Security Policy and modifying users and objects, are done by the Active SMS. If the Active SMS is down, and any of the aforementioned operations need to be performed, one of the Standby SMSs should be made active by the system administrator. This transition from Standby to Active should be initiated manually.

  3. #3
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    272
    Rep Power
    12

    Default Re: Management Server HA two different data centers?

    This is absolutely workable.

    Be sure to test flipping your managements occasionally. Assume any DR plan you haven't tested is no good.

    Throughput requirements depends mostly on your log volume. Keep in mind, the firewalls will log locally if they can't get to any log servers. This may be acceptable or it may not, depending on your environment. Management database sync sounds big, but it only happens when you make changes. Unless you're doing hundreds of changes per day, I doubt that would even be 1% of the throughput needed for logs in a DR situation.

    Also keep in mind this is not the same as backup. It's more like RAID. It protects you from a device failure. It does not protect you from a data failure. If you delete something, that deletion will be propagated to the secondary SmartCenter.
    Zimmie

  4. #4
    Join Date
    2018-04-18
    Posts
    40
    Rep Power
    0

    Default Re: Management Server HA two different data centers?

    Thank you for the feedback. With the primary SMS being the log server, will logs continually replicate to the secondary SMS?

  5. #5
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    272
    Rep Power
    12

    Default Re: Management Server HA two different data centers?

    I believe they can if you set them to. I don't typically do that. SmartLog can connect to multiple log servers at once.
    Zimmie

  6. #6
    Join Date
    2006-03-21
    Posts
    87
    Rep Power
    13

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    Thank you for the feedback. With the primary SMS being the log server, will logs continually replicate to the secondary SMS?
    It all depends on your settings in the firewall object under:

    Click image for larger version. 

Name:	Image 2.png 
Views:	13 
Size:	22.0 KB 
ID:	1407

    If secondary SMS is not configured as log server, firewalls will store logs locally until the primary SMS becomes available.

    You can also configure the secondary SMS to forward logs to the primary in scheduled times, so that logs are always available in the primary.

    Click image for larger version. 

Name:	Image 4.png 
Views:	12 
Size:	19.0 KB 
ID:	1408

  7. #7
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    272
    Rep Power
    12

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by eduardoxmunoz View Post
    You can also configure the secondary SMS to forward logs to the primary in scheduled times, so that logs are always available in the primary.
    The real question is how this handles failures. If you have the secondary set to forward logs at midnight, and your primary is down for 48 hours, what happens when you get the primary back up? Does the secondary know the log forwarding attempt failed, or does it only forward logs since the last forwarding interval?

    I don't remember the answer, and I've never tested it. Anyone depending on that should test it first.
    Zimmie

  8. #8
    Join Date
    2006-03-21
    Posts
    87
    Rep Power
    13

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by Bob_Zimmerman View Post
    I don't remember the answer, and I've never tested it. Anyone depending on that should test it first.
    Agree, Check Point documentation is not very clear about how exactly the schedule log forwarding mechanism works. It is my understanding that any object with this setting enable will forward local log files, so my guessing is that if the connection fails, it will try to send them again in the next attempt.

    I would recommend to test it in advance if logs is a critical matter in the network.

  9. #9
    Join Date
    2006-09-26
    Posts
    3,172
    Rep Power
    16

    Default Re: Management Server HA two different data centers?

    Here is my opinion about Checkpoint Management High Availability.

    It is a piece of junk. I first experienced with it in 2004/2005 with NG AI and it is nothing but trouble. Even when I had the Active/Standby Provider-1 on the same subnet, they sometime go into collision mode. It is nothing but trouble. It didn't get better with NGX R60/65. I've not used it since but I wouldn't be surprised that parts of it is still broken.

    Here is a better idea for your DR scenario. You configure your SMS in a network that is "floating" in either the Active or Standby Data Center. For example, let say your SMS management network is 192.168.1.0/24. You can setup this network in your Active DC. Configure your SMS for upgrade_export and move the files to an external location where it can be accessed from both the Active and Standby DR.

    In case your Active DC is down, all you have to do is bring up the Standby DC and "swing" the network 192.168.1.0/24 over to the Standby DC. Once you have that network in place, just bring up your VM there and perform an upgrade_import and you will have the original SMS.

    As far as storing log on the SMS, it is NOT advised. You will be better off with a decidate Customer Log Module (CLM) or Stand-alone log server. Your SMS will become slow and unstable if you have too much log on the SMS.

    Under my scenario, you will save money on both Checkpoint licensing, support and less headache in the long run.

    Best of luck,

  10. #10
    Join Date
    2006-07-28
    Location
    San Francisco, USA
    Posts
    2,492
    Rep Power
    15

    Default Re: Management Server HA two different data centers?

    Generally agree with @cciesec2006. Using Check Point HA tends to be more hassle than it's worth. Due to the nature of the separation between enforcement & management, it's fine to have a short delay while you bring up a new mgmt server. _Most_ people don't need to switch over within minutes.

    Also agreed with separating out logs to a separate system (and using SmartLog or whatever it's called these days)

  11. #11
    Join Date
    2018-04-18
    Posts
    40
    Rep Power
    0

    Default Re: Management Server HA two different data centers?

    Hello,

    Moving the network the SMS is on in the active data center to the DR data center in an emergency isn't feasible. We plan on getting a dedicated log server in the future.

    I was able to setup a secondary management server (R77.30) in our second data center and the sync works perfect with the primary. The failover from standby to active also seems to work great. The only issue that i haven't been able to resolve is that two different HA pairs show the standby gateway as disconnected on the secondary management server. Communication with the primary HA member seems fine. From the gateways that are showing disconnected I am able to ping the management server from them and I don't see anything being blocked in SmartView Tracker.

    I have attached a screen shot.

    I don't understand how the managementClick image for larger version. 

Name:	disconnected.PNG 
Views:	16 
Size:	17.3 KB 
ID:	1409 server can talk to one of the HA members but not the other?

  12. #12
    Join Date
    2006-07-28
    Location
    San Francisco, USA
    Posts
    2,492
    Rep Power
    15

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    I don't understand how the management server can talk to one of the HA members but not the other?
    Trace the traffic through your network. When I've seen that behavior in the past, it was because traffic was arriving via an unexpected interface, and anti-spoofing was kicking in.

    Traffic from your new standby server to firewall B is going to a specific IP on that firewall. That IP might be on the "other side" of the cluster, relative to the standby server. So when your network routes traffic to that IP, it sends it to the closest firewall VIP. That then gets forwarded out the other interface, and to your standby firewall. But your standby firewall doesn't expect to receive traffic with that source via that interface - it expects to receive it on the interface closest to the standby management server.

  13. #13
    Join Date
    2006-03-21
    Posts
    87
    Rep Power
    13

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    i haven't been able to resolve is that two different HA pairs show the standby gateway as disconnected on the secondary management server. Communication with the primary HA member seems fine. From the gateways that are showing disconnected I am able to ping the management server from them and I don't see anything being blocked in SmartView Tracker.
    Make sure you can open connection to all the relevant ports from/to the secondary SMS. In this link there is a good diagram of all the required ports.

    https://community.checkpoint.com/doc...-communication

    Also, check that the standby is not hidden behind the virtual IP, have a look at sk31832. Perhaps it works with the primary SMS because they are in the same management VLAN?

    if any of the above doesn't help, run fw ctl zdebug + drops in both gateways and seek for any drops.

    Good luck!
    Last edited by eduardoxmunoz; 2018-08-24 at 07:44. Reason: spelling

  14. #14
    Join Date
    2018-04-18
    Posts
    40
    Rep Power
    0

    Default Re: Management Server HA two different data centers?

    Hello All,

    I have traffic from the secondary management server successfully passing to all security gateways after adjusting anti spoofing, except for two gateways.

    I have a internal HA pair of security gateways in each data center and only gateway A in each cluster will work with the secondary management server. Gateway's B will not work. After looking through logs it seems as if, during a policy push for example, traffic intended for security gateways B first hit security gateways A and never leave A. The traffic seems to disappear there.

    I have tried fw monitor on the B's and don't see any traffic from the secondary mgmt server and also ctrl zdbug + drops on both a and b gateways and I don't see any drops.

    I can't figure out where the heck this traffic is going or what gateways A are doing with it once they receive it.

  15. #15
    Join Date
    2006-07-28
    Location
    San Francisco, USA
    Posts
    2,492
    Rep Power
    15

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    I can't figure out where the heck this traffic is going or what gateways A are doing with it once they receive it.
    Can you draw a simple diagram showing the traffic flows here?

    What firewall IP address is the management server trying to use, and where is that IP relative to the management server? Is your routing set up such that the traffic needs to go through firewall A, and come in the other side of firewall B?

  16. #16
    Join Date
    2018-04-18
    Posts
    40
    Rep Power
    0

    Default Re: Management Server HA two different data centers?

    It turned out to be a routing issue for the passive members in each HA cluster.

    The second management server sent traffic to it's default gateway 10.x.x.1 (The VIP of the cluster), in this case gateway A is the active member, instead of routing the traffic to passive gateway B, A would route it directly back to itself and a zdbug showed it being dropped.

    The default route on the secondary management server was the VIP of the internal cluster. I added routes specific for each gateway member of the cluster and now the traffic is sent destined to each individual member and not the VIP.

  17. #17
    Join Date
    2006-07-28
    Location
    San Francisco, USA
    Posts
    2,492
    Rep Power
    15

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    I added routes specific for each gateway member of the cluster and now the traffic is sent destined to each individual member and not the VIP.
    That was the sort of thing I was going to suggest. Basically the problem is the traffic coming in the primary firewall, and going around to the secondary. Gets things a bit confused.

  18. #18
    Join Date
    2018-04-18
    Posts
    40
    Rep Power
    0

    Default Re: Management Server HA two different data centers?

    I have a question regarding the two management servers in a real DR event.

    To fail over to the secondary server, per the Check Point documentation, I am to first login to the active server and set that to standby, then log in to the secondary and make it active.

    In a real live DR where site A goes down where the primary management server lives what do I do when i am unable to login to the primary and switch it to standby? Go right to the secondary and make it active?

    After the DR is over can I simply bring the primary management at site A back online and not have it over right a newer config on the secondary (now active)?

  19. #19
    Join Date
    2005-08-14
    Location
    Gig Harbor, WA, USA
    Posts
    2,488
    Rep Power
    16

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by cciesec2006 View Post
    It is a piece of junk. I first experienced with it in 2004/2005 with NG AI and it is nothing but trouble. Even when I had the Active/Standby Provider-1 on the same subnet, they sometime go into collision mode. It is nothing but trouble. It didn't get better with NGX R60/65. I've not used it since but I wouldn't be surprised that parts of it is still broken.
    FYI, in R80.x, this got a major overhaul due to the other changes in management architecture.
    http://phoneboy.org
    Unless otherwise noted, views expressed are my own

  20. #20
    Join Date
    2006-03-21
    Posts
    87
    Rep Power
    13

    Default Re: Management Server HA two different data centers?

    Quote Originally Posted by mjensen View Post
    I have a question regarding the two management servers in a real DR event.

    To fail over to the secondary server, per the Check Point documentation, I am to first login to the active server and set that to standby, then log in to the secondary and make it active.

    In a real live DR where site A goes down where the primary management server lives what do I do when i am unable to login to the primary and switch it to standby? Go right to the secondary and make it active?
    Yes, that is correct. The process is all manual. When you login into the Standby SMS in pre-R80 it would ask you whether you want to make it Active. R80 and later will let you login in read-only and then there is an option to switch to Active.

    After the DR is over can I simply bring the primary management at site A back online and not have it over right a newer config on the secondary (now active)?
    Yes, once the primary is up, you basically do the reverse process. If you have made any policy changes when the Standby was Active, you can synchronize the database from the Standby to the Active.

    As PhoneBoy mentioned above, in my experience too, R80.x works better than R70.X

    Regards

Similar Threads

  1. Replies: 1
    Last Post: 2017-02-10, 09:53
  2. Migration from R65 Window management server to Smart-1 (SPLAT) management server
    By nick_bar in forum Security Management Server (Formerly SmartCenter Server ((Formerly Management Server))
    Replies: 3
    Last Post: 2011-08-24, 02:22
  3. Training Centers and Seminars
    By amani in forum Off-Topic
    Replies: 1
    Last Post: 2010-09-14, 21:22
  4. Set Firewall to save log data to SQL server
    By SteveS in forum SmartView Tracker
    Replies: 7
    Last Post: 2010-06-24, 08:24
  5. Merging the data frm Two Mgmnt server into a Single Mgmtn Server
    By gladiatorsword in forum Security Management Server (Formerly SmartCenter Server ((Formerly Management Server))
    Replies: 4
    Last Post: 2007-07-23, 03:41

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •