CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it again! He has just released the 2nd edition of "Max Power".
Rather than get into details here, I urge you to check out this announcement post.
It's a massive upgrade, and well worth checking out. -E

 

Results 1 to 5 of 5

Thread: Geo Protection IP Database updates for VSX

  1. #1
    Join Date
    2016-09-17
    Posts
    1
    Rep Power
    0

    Default Geo Protection IP Database updates for VSX

    Intro:
    Since I did not find much on the net I thought I would share my discoveries.
    Some details have been removed to exclude any restricted Check Point article information.
    If CP notice anything on here that needs removing please let me know.

    Please make a copy of the following file before starting any work:
    $FWDIR/tmp/geo_location_tmp/updates/IpToCountry.csv



    Glossary of terms

    Check Point VSX
    VSX (Virtual System Extension) is a physical box loaded with Check Point software using VSX configuration. This allows you to run multiple contexts of virtual firewalls on a single box that has separate network configuration and logical separation. Management and log servers also can be virtualised
    A VSX gateway is a physical machine that serves as a container for Virtual Systems and other virtual network components
    Ref: https://sc1.checkpoint.com/documents...l_frameset.htm

    Context 0 or VS0
    Context 0 or VS0 is the VSX physical gateway or container. Context 1 or VS1 and above are virtual gateways running on the VSX gateway.

    MDS
    Multi Domain Security Management server which can manage independent security management environment with a separate database, log server and its own set of security policies.
    Ref: https://www.checkpoint.com/downloads...-datasheet.pdf

    EPOCH time
    Unix time (also known as POSIX time) is a system for describing instants in time, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, [note 1] not counting leap seconds.
    Ref: https://en.wikipedia.org/wiki/Unix_time
    To convert EPOCH time to actual time use an online converter e.g.
    http://www.esqsoft.com/javascript_ex...e-to-epoch.htm

    Linux Soft links
    Linux Soft link or symbolic link is a small file acting as a pointer to another file

    Background and Reasons for investigation.

    We needed virtual firewalls GeoProtection IP database to update as legitimate IP addresses assigned to the allowed countries were being blocked. Checking the IP database file it showed that the IP addresses in question were assigned to other countries in the past. Checking further we had noticed the IP database was a few years old.
    Further investigation found that the Manager did not update the IP Database with IPS updates and that the gateways were designed to run the updates themselves. Also that the virtual firewalls did not update the database instead they had a Linux Soft link to context 0 (VS0) files.

    Example view of the file listed in VS1:
    /var/log/opt/CPsuite-R77/fw1/CTX/CTX00001/tmp/geo_location_tmp/updates/
    IpToCountry.csv -> /opt/CPsuite-R77/fw1/tmp/geo_location_tmp/updates/IpToCountry.csv

    Our VSX gateways (VS0 aka context 0 i.e. the physical VSX Gateways) were isolated from the internet. We applied the proxy settings to the VSX cluster configuration via the SmartDashBoard GUI and updates continued to fail.

    Another issue arose where manually copying the IP Database file onto the VSX gateway started causing some problems with previously allowed traffic. Check Point TAC confirmed we had setup Geo Protection as a White list. This was not supported and TAC sited the SK article sk110683 to prove it.

    My understanding:
    White Lists: Countries configured as Allow in the first section “Policy for Specific Countries “ and then configured to Block all other countries in the second section “Policy for Other Countries”.
    In this configuration the traffic will be allowed through the first section “Policy for Specific Countries” but then blocked by the second section “Policy for Other Countries”
    Black Lists: Block all unwanted countries in the first section “Policy for Specific Countries “, then in ‘Policy for Other Countries’ set to Allow.
    Blacklist example:
    If you want to block the world but allow New Zealand and Australia then under the first section “Policy for Specific Countries “ you must add individually (approx. 250), all countries except NZ and AU one at a time.
    Then set Policy for Other Countries to Allow.

    Licencing and Contracts
    IPS blade licence is needed as the profile needs to be attached to a gateway in IPS. in.geod process will not show unless enabled and pushed to the gateway.
    Contract: I don’t know if you need it for Geo Protect updates maybe someone can assist with this.

    Blocked countries show as ‘Allowed’ on Maps
    Note: Some countries are set to blocked but still show green (allow) on the map.
    As long as they are showing blocked in the first section then it will be ok.
    To fix the maps you need to use GuiDBedit util to edit the management database directly, the aim is to change the display name of problem countries in 'Policy for Specific Countries' to match the map names.

    Example of the process can be viewed in sk109366.
    View sk45003 for other specific issues with North Korea.

    Steps:
    Close any GUI windows.
    Open GuiDBedit.exe found in the check Point install directory of your management machine where the GUI clients are installed.
    Connect to the management server IP of the CMA – this is important as I was using DNS when connecting in the Lab which was the MDS server and not seeing the changes in the GUI of my CMA.
    Browse to:
    Table -> Other -> Countries
    For the country with the map issue compare the string country_display_name to the name you get when you hover the mouse over the country in the map view in Geo Protect.
    Replace the country_display_name with the Map name.
    Note: The references for the countries are done by the country code so changing this display name shouldn’t hurt anything – you’ll have to test further to be 100% sure.
    When Finished, File -> Save All
    Start the GUI client and check that the names have changed in Policy for Specific Countries and if blocked it should show as red in the maps.

    Countries in this version:
    (Replace country_display_name with Map name)

    Countries Object: KOREA_DEMOCRATIC_PEOPLES_REPUBLIC_OF
    country_display_name: Korea, Democratic People's Republic of
    Map: North Korea

    Countries Object: KOREA_REPUBLIC_OF
    country_display_name: Korea, Republic Of
    Map: Republic Of Korea

    Countries Object: TANZANIA_UNITED_REPUBLIC_OF
    country_display_name: Tanzania, United Republic of
    Map: United Republic Of Tanzania

    Countries Object: MACEDONIA_THE_FORMER_YUGOSLAV_REPUBLIC_OF
    country_display_name: Macedonia, The Former Yugoslav Republic Of
    Map: The Former Yugoslav Republic Of Macedonia

    Countries Object: MOLDOVA_REPUBLIC_OF
    country_display_name: Moldova, Republic Of
    Map: Republic Of Moldova

    Ideas to try if you are having issues:
    Make sure the correct CMA was used when connecting GuiDBedit.
    Install database
    Close GUI clients and GuiDBedit
    Restart the MDS CMA
    mdsstop_customer <CMA Name>
    mdsstart_customer <CMA Name>
    Reboot the whole MDS server

    Geo Protect configuration
    To save me re-writing documentation please view the following link:
    https://sc1.checkpoint.com/documents...min/103526.htm
    There are a number of errors in this document the main one for this article is in Geo Protection setup to choose ‘ACTION’ Block or Allow.
    Ignore this and use the Blacklist method for more reliable results. I.e. sk110683.
    Make sure you donot block any public IP’s that are used for management communications to the gateways as the policy pushes will stop working.
    Also exclude the IP addresses for the following:
    sc1.checkpoint.com, crl.verisign.com, updates.checkpoint.com and your DNS server. These are the Geo Protection updates connections.

    Geo Protect IP database.
    There are numerous files involved.
    The one we will be concentrating on is $FWDIR/tmp/geo_location_tmp/updates/IPtoCountry.csv . This is the IP database that matches IP addresses to countries.
    The following is from sk92823. I’ve found that the solution in this article for testing and updating the IP database are not good enough for VSX and possibly non VSX systems as well. The method in this document is more reliable.
    The IPtoCountry.csv file, which resides in $FWDIR/conf, comes with installation, and will not be changed ever.
    The file is downloaded from the download center every interval (24 hours) to $FWDIR/tmp/geo_location_tmp/updates, and loaded into the kernel. The file in $FWDIR/conf is used only in case there is not file in $FWDIR/tmp/geo_location_tmp/updates.
    In case the file cannot be downloaded, the existing file (in $FWDIR/tmp/geo_location_tmp/updates) is used.
    The file in $FWDIR/tmp/geo_location_tmp/updates will always be newer than the one in in $FWDIR/conf (unless no file was downloaded).

    IP Database Updates
    Note: I found for VSX changing the ‘update_countries_list = true’ to false using GuiDBedit, saving, restarting MDS with mdsrestart then pushing policy to the gateways did not stop the database from updating.

    Updates by Direct Connection to the internet
    Updates happen directly from the gateway every 24hrs, and not the management servers like scheduled IPS updates.
    The actual update time each day is the time it was successful on its last update or when the in.geod process first starts. This time can be found in the $FWDIR/tmp/geo_location_tmp/updates/timestamp file in EPOCH format. Use an online convertor to view this time in a readable format.
    E.g. http://www.esqsoft.com/javascript_ex...e-to-epoch.htm

    In VSX the update is only done by VSX0. The virtual servers do not do the updates. They have Linux Soft links to the VSX0 files.
    In the VSX gateway policy, and any other firewalls routers etc in the path of the VS0 reaching the internet,.rules, NAT’s, routes will be needed to allow your external IP (or NAT IP of the VSX0) to reach the internet.
    There needs to be a default route pointing to the internet on the VSX0 configuration or alternatively four static routes to sc1.checkpoint.com, crl.verisign.com, updates.checkpoint.com and your DNS server(s).
    Your gateways DNS server will need to resolve these domains.

    Updates using a proxy.
    Adding proxy setting into CLISH did not help and was removed.
    I’ve seen lab results differ to production in other tests. In the lab the proxy would only work if the cluster and physical server objects had the proxy set via GUI.
    I noticed once working the proxy setting could be removed from the physical server objects and left on the cluster object and it would still use the proxy. A reboot was not tested and that may have reset this.
    I.e. In VSX policy (policy for context 0)
    double click (or edit) cluster -> topology -> proxy -> enter IP and port
    double click (or edit) all physical VSX gateway objects -> topology -> proxy -> enter IP and port.
    Always push policy after proxy changes even though saving the config looks like it pushes the changes to the GW's.

    Automatic Updates:
    This happens every 24hrs depending on the $FWDIR/tmp/geo_location_tmp/updates/timestamp file contents.
    e.g. if the timestamps file has 1473719743 which is September 13 2016 10:35:43
    The next update will be September 14 2016 10:35. It drops the milliseconds.
    If it fails it will try at 10:35 the next day.
    You can change the update time by editing the timestamp file with a new EPOCH time from the previous day with the time that you want.
    Use and online convertor e.g. http://www.esqsoft.com/javascript_ex...e-to-epoch.htm

    Manual Updates:
    You cannot get a successful update by changing the gateways date to the next day as this will fail the security certificate check on the verisign site – you will get a failed down load message in tracker.
    Change the EPOCH time in the $FWDIR/tmp/geo_location_tmp/updates/timestamp file to the previous day.
    Use and online convertor
    e.g. http://www.esqsoft.com/javascript_ex...e-to-epoch.htm
    Once saved kill the in.geod process
    e.g.
    ps -ef | grep geo
    Will show two processes. One from your grep, the other is the geo update process in.geod
    If it is not there Geo Protect may have never been configured. You may need to configure a detect mode on geo Protection on the VSX0 policy and push. I.e. in SmartDashBoard -> IPS tab -> geo protection change the profile action to detect and push the policy.

    [Expert@FWG01:0]# ps -ef | grep geo
    admin 3566 3487 0 22:59 pts/2 00:00:00 grep geo
    admin 28235 13803 0 16:14 ? 00:00:01 in.geod 0

    Use kill to kill the process.
    E.g. [Expert@FWG01:0]#kill 28235
    The in.geod process will re-spawn immediately with a new process ID.

    Backup IP database daily and delete files older than 14 days
    #This script is to backup the IP to Countries IP address database
    #every day so if there is an update with a corrupt file then it can be manually copied back.
    #these files are kept for two weeks
    #then deleted

    #set up the Check Point environment so that $FWDIR will work
    #
    CPDIR=/opt/CPshrd-R77
    . $CPDIR/tmp/.CPprofile.sh

    #Debug logging
    #
    set -x
    exec 2>$FWDIR/tmp/geo_location_tmp/updates/iptocountrybackup.log

    #Variables
    #
    var_database=$FWDIR/tmp/geo_location_tmp/updates/IpToCountry.csv
    var_dblocation=$FWDIR/tmp/geo_location_tmp/updates/
    var_backupdir=$FWDIR/tmp/geo_location_tmp/updates/backup/
    var_date2=`/bin/date +%Y%m%d`

    #Backup Database
    cp $var_database $var_backupdir/IpToCountry.$var_date2

    #Remove databases older than 14 days
    /usr/bin/find $var_backupdir -type f -ctime +14 -exec /bin/rm {} \;


    Save to fav script directory.
    set file permissions to execute chmod +x /usr/local/scripts/geodbbackup.sh
    Add to cron to run before the first update (so you have a copy. ...or copy manually). I used the CLISH command:
    add cron job GeoDBbackup command "/usr/local/scripts/geodbbackup.sh" recurrence daily time 07:00

    Updates via scripts:
    Posting this I realised I had a script from Check Point Article sk108425. I can't publish this and will need to write my own and add here at a later stage.
    This article has more on this script creation and scheduling.
    Use these two lines to add a proxy for http and https into your script:
    export http_proxy=http://<proxy IP address>:<proxy port>/
    export https_proxy=$http_proxy
    Make sure your proxy server will allow the connections.

    I'm not a master scripter or anything so just giving ideas here.
    I'm trying not to remember the Check Point script to avoid any trouble from them :-) - If they approve then I will add it.

    **Update to come as this is work in progress**

    # Environment Variables
    CPDIR=/opt/CPshrd-R77
    . $CPDIR/tmp/.CPprofile.sh

    #Debug logging Remove Remarks if wanted
    # set -x
    # exec 2>$FWDIR/tmp/geo_location_tmp/updates/backup/iptocountryupdate.log

    # Other variables
    var_database=$FWDIR/tmp/geo_location_tmp/updates/IpToCountry.csv
    var_dblocation=$FWDIR/tmp/geo_location_tmp/updates/
    var_backupdir=$FWDIR/tmp/geo_location_tmp/updates/backup/
    var_date2=`/bin/date +%Y%m%d`
    var_updatelog=$FWDIR/tmp/geo_location_tmp/updates/backup/iptocountryupdate.log

    # Set Proxy if needed and Remove Remarks #>> $var_updatelog
    # export http_proxy=http://<proxy IP address>:<proxy port>/ #>> $var_updatelog
    # export https_proxy=$http_proxy #>> $var_updatelog

    # remove the current compressed database files, if they exists. #>> $var_updatelog

    # download the database from Check Point server using wget or curl #>> $var_updatelog
    cd $FWDIR/tmp/geo_location_tmp/updates/ #>> $var_updatelog
    /sysimg/CPwrapper/linux/MiniWrapper/Actions/wget https://sc1.checkpoint.com/freud/IpToCountry.csv.gz #>> $var_updatelog

    # If decompressed file exists then check backup folder if ok remove current #>> $var_updatelog

    # Decompress #>> $var_updatelog

    # Change Permissions #>> $var_updatelog

    # Check if file exists if not copy latest from backup. #>> $var_updatelog



    Viewing Connections during the update:
    Use netstat to list the IP's as they connect.
    In a separate console connection set watch to every second and grep the resolved ip's for sc1.checkpoint.com, crl.verisign.com, updates.checkpoint.com.
    For me they resolved to the following which are added into the netstat command.
    watch -n1 "netstat -aon | grep '184.27.86.9\|23.37.133.163\|209.87.209.87'"

    This is what I caught over 30 seconds, not all showed at the same time so I missed some established connections:
    tcp 0 0 172.16.16.142:49802 209.87.209.87:443 ESTABLISHED off (0.00/0/0)
    tcp 0 0 172.16.16.142:51034 184.27.86.9:443 ESTABLISHED off (0.00/0/0)
    tcp 0 0 172.16.16.142:49802 209.87.209.87:443 TIME_WAIT timewait (53.20/0/0)
    tcp 0 0 172.16.16.142:60685 23.37.133.163:80 TIME_WAIT timewait (52.66/0/0)
    tcp 0 0 172.16.16.142:60684 23.37.133.163:80 TIME_WAIT timewait (52.52/0/0)

    If using a proxy it will be proxy IP you'll need to watch:
    So you should see similar connections.
    watch -n1 "netstat -aon | grep <proxy IP>"


    Viewing update status in tracker.
    In tracker for your VSX0 policy filter on ‘Control’ under the ‘Type’ column.
    Example of success:
    Number: 35238
    Date: 14Sep2016
    Time: 10:33:35
    Interface: geod daemon
    Origin: FWCLX01
    Type: Control
    Action:
    Product: IPS Software Blade
    Description: Countries DB download has succeeded
    Product Family: Network
    Policy Info: Policy Name: VSX


    Example of fail:
    This will happen even if date of GW is wrong and will not even check internet connection.
    Number: 77
    Date: 4Aug2016
    Time: 15:25:55
    Interface: geod daemon
    Origin: FWCLX01
    Type: Control
    Action:
    Product: IPS Software Blade
    Description: Countries DB download has failed. Please verify internet connectivity or contact technical support
    Product Family: Network
    Policy Info: Policy Name: VSX


    IP checks: Find which IP is assigned to a country
    There is a restricted Check Point KB on this sk94364
    Publicly available is http://dev.maxmind.com/geoip/legacy/csv/
    I converted a blocked IP to 2523638785 this nunber was changed slightly to hide actual address.
    I.e.
    <ip Address> Removed for privacy
    4th octet + (3rd octet * 256) + (2nd octet * 65536) + (1st octet * 16777216)
    = 2523638785

    Using grep to list the countries with the first 5 digits of 2523638785
    2523638785 falls into "2523594752","2523660287","iana","410227200","EU", "EUR","Europe"
    In a later version of the file it falls under New Zealand.

    [Expert@FWG01]# cat /opt/CPsuite-R77/fw1/tmp/geo_location_tmp/updates/IpToCountry.csv | grep 25236
    "1121252360","1121252367","iana","410227200","ZA", "ZAF","South Africa"
    "1121252368","1121252375","iana","410227200","GG", "GGA","Guernsey"
    "1296252360","1296252367","iana","410227200","FR", "FRA","France"
    "1296252368","1296252383","iana","410227200","DE", "DEU","Germany"
    "2523594752","2523660287","iana","410227200","EU", "EUR","Europe"
    "2523660288","2524119039","iana","410227200","US", "USA","United States"
    "2954825236","2954825239","iana","410227200","PL", "POL","Poland"
    "3252359168","3252362239","iana","410227200","DK", "DNK","Denmark"
    "3252362240","3252362495","iana","410227200","PL", "POL","Poland"
    "3252362496","3252362751","iana","410227200","IT", "ITA","Italy"
    "3252362752","3252363263","iana","410227200","DK", "DNK","Denmark"
    "3252363264","3252379647","iana","410227200","LT", "LTU","Lithuania"
    "3642519552","3642523647","iana","410227200","IT", "ITA","Italy"
    "3642523648","3642527743","iana","410227200","GB", "GBR","United Kingdom"

    Troubleshooting:
    cat $FWDIR/log/geod.elg
    cat /var/log/messages

    References:

    • sk79360 - How to check if Geo Protections have been updated
    • sk92823 - Problems With Geo Protection Updating
    • sk108425 - IPS Geo Protection does not perform daily update
    • sk95976 - How GEO protection country file is getting updated
    http://www.esqsoft.com/javascript_ex...e-to-epoch.htm
    https://sc1.checkpoint.com/documents...l_frameset.htm
    • Ref: https://en.wikipedia.org/wiki/Unix_time
    https://www.checkpoint.com/downloads...-datasheet.pdf
    http://dev.maxmind.com/geoip/legacy/csv/
    • Roshans Trusty lab
    • Brain
    Last edited by Damian_Barlow; 2016-12-13 at 14:53. Reason: Add netstat command and Backup script. GuiDBedit Update.

  2. #2
    Join Date
    2014-09-02
    Posts
    349
    Rep Power
    10

    Default Re: Geo Protection IP Database updates for VSX

    Pretty solid first post! Good stuff.

    -E

  3. #3
    Join Date
    2015-05-07
    Posts
    4
    Rep Power
    0

    Default Re: Geo Protection IP Database updates for VSX

    Thanks so much for this! I couldn't figure out why my VSX firewalls were not getting updates and in.geod was not running under VSX0. I have not seen anywhere in the documentation that you are required to run IPS with geo protection enabled on VSX0 in order to get updates.

  4. #4
    Join Date
    2006-03-08
    Location
    Lausanne
    Posts
    1,030
    Rep Power
    15

    Default Re: Geo Protection IP Database updates for VSX

    Unfortunately, it is not only about GP on VSX, but some other features too, when there are all kinds of unclear dependencies between VS0 and production VSs. For example, I have seen something completely dumb when using HTTP proxy on a VS.

    Let's say one of VSs has a proxy running on it, so it has to resolve URLs and hostnames properly. The issue is, it needs DNS servers defined. The only DNS servers you can have are defined in VS0 content. They will be used, with a proxy VS trying to send DNS requests to those servers.

    If you remember, in DMI VSX deployment VS0 is used for MGMT communication only, and it is likely one has internal DNS defined there. It is also likely proxy VS not to have connectivity to internal DNS servers and/or DMI network segment.

    In such situation, proxy fails because it cannot resolve URLs. There could be two different ways to resolve this:

    1. allow proxy VS connectivity to internal DNS servers
    2. define external DNS servers on VS0

    Just sharing :-)
    -------------

    Valeri Loukine
    CCMA, CCSM, CCSI
    http://checkpoint-master-architect.blogspot.com/

  5. #5
    Join Date
    2015-05-07
    Posts
    4
    Rep Power
    0

    Default Re: Geo Protection IP Database updates for VSX

    FWIW I just did some R77.30 to R80.20 upgrades, and found that the in.geod was not running in the VSX0. I had to change the geo policy in order to jumpstart getting it running. I'm not sure why that is...

Similar Threads

  1. Gaia Software Updates vs. CLI Updates
    By Neilharrison_253 in forum R75.40 (GAiA)
    Replies: 1
    Last Post: 2013-09-17, 06:51
  2. Database revision control vs user database
    By jgarzam in forum Security Management Server (Formerly SmartCenter Server ((Formerly Management Server))
    Replies: 1
    Last Post: 2013-03-01, 02:24
  3. Geo Protection Updates.....how often?
    By Eros_G in forum Geo Protection
    Replies: 4
    Last Post: 2011-09-19, 08:37
  4. Need clarification regarding IPS updates
    By jonta in forum IPS Blade (Formerly SmartDefense)
    Replies: 2
    Last Post: 2011-08-22, 11:19
  5. CI: AV Database Updates Failing
    By jaandrade3rd in forum Content Security/Security Servers/CVP/UFP
    Replies: 2
    Last Post: 2009-04-22, 12:27

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •