Geo Protection IP Database updates for VSX

[Damian_Barlow]

Intro:
Since I did not find much on the net I thought I would share my discoveries.
Some details have been removed to exclude any restricted Check Point article information.
If CP notice anything on here that needs removing please let me know.

Please make a copy of the following file before starting any work:
$FWDIR/tmp/geo_location_tmp/updates/IpToCountry.csv


Glossary of terms

Check Point VSX
VSX (Virtual System Extension) is a physical box loaded with Check Point software using VSX configuration. This allows you to run multiple contexts of virtual firewalls on a single box that has separate network configuration and logical separation. Management and log servers also can be virtualised
A VSX gateway is a physical machine that serves as a container for Virtual Systems and other virtual network components
Ref: https://sc1.checkpoint.com/documents...l_frameset.htm

Context 0 or VS0
Context 0 or VS0 is the VSX physical gateway or container. Context 1 or VS1 and above are virtual gateways running on the VSX gateway.

MDS
Multi Domain Security Management server which can manage independent security management environment with a separate database, log server and its own set of security policies.
Ref: https://www.checkpoint.com/downloads...-datasheet.pdf

EPOCH time
Unix time (also known as POSIX time) is a system for describing instants in time, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, [note 1] not counting leap seconds.
Ref: https://en.wikipedia.org/wiki/Unix_time
To convert EPOCH time to actual time use an online converter e.g.
http://www.esqsoft.com/javascript_ex...e-to-epoch.htm

Linux Soft links
Linux Soft link or symbolic link is a small file acting as a pointer to another file
 
Background and Reasons for investigation.

We needed virtual firewalls GeoProtection IP database to update as legitimate IP addresses assigned to the allowed countries were being blocked. Checking the IP database file it showed that the IP addresses in question were assigned to other countries in the past. Checking further we had noticed the IP database was a few years old.
Further investigation found that the Manager did not update the IP Database with IPS updates and that the gateways were designed to run the updates themselves. Also that the virtual firewalls did not update the database instead they had a Linux Soft link to context 0 (VS0) files.

Example view of the file listed in VS1:
/var/log/opt/CPsuite-R77/fw1/CTX/CTX00001/tmp/geo_location_tmp/updates/
IpToCountry.csv -> /opt/CPsuite-R77/fw1/tmp/geo_location_tmp/updates/IpToCountry.csv

Our VSX gateways (VS0 aka context 0 i.e. the physical VSX Gateways) were isolated from the internet. We applied the proxy settings to the VSX cluster configuration via the SmartDashBoard GUI and updates continued to fail.

Another issue arose where manually copying the IP Database file onto the VSX gateway started causing some problems with previously allowed traffic. Check Point TAC confirmed we had setup Geo Protection as a White list. This was not supported and TAC sited the SK article sk110683 to prove it.

My understanding:
White Lists: Countries configured as Allow in the first section “Policy for Specific Countries “ and then configured to Block all other countries in the second section “Policy for Other Countries”.
In this configuration the traffic will be allowed through the first section “Policy for Specific Countries” but then blocked by the second section “Policy for Other Countries”
Black Lists: Block all unwanted countries in the first section “Policy for Specific Countries “, then in ‘Policy for Other Countries’ set to Allow.
Blacklist example:
If you want to block the world but allow New Zealand and Australia then under the first section “Policy for Specific Countries “ you must add individually (approx. 250), all countries except NZ and AU one at a time.
Then set Policy for Other Countries to Allow.

Licencing and Contracts
IPS blade licence is needed as the profile needs to be attached to a gateway in IPS. in.geod process will not show unless enabled and pushed to the gateway.
Contract: I don’t know if you need it for Geo Protect updates maybe someone can assist with this.

Blocked countries show as ‘Allowed’ on Maps
Note: Some countries are set to blocked but still show green (allow) on the map.
As long as they are showing blocked in the first section then it will be ok.
To fix the maps you need to use GuiDBedit util to edit the management database directly, the aim is to change the display name of problem countries in 'Policy for Specific Countries' to match the map names.

Example of the process can be viewed in sk109366.
View sk45003 for other specific issues with North Korea.

Steps:
Close any GUI windows.
Open GuiDBedit.exe found in the check Point install directory of your management machine where the GUI clients are installed.
Connect to the management server IP of the CMA – this is important as I was using DNS when connecting in the Lab which was the MDS server and not seeing the changes in the GUI of my CMA.
Browse to:
Table -> Other -> Countries
For the country with the map issue compare the string country_display_name to the name you get when you hover the mouse over the country in the map view in Geo Protect.
Replace the country_display_name with the Map name.
Note: The references for the countries are done by the country code so changing this display name shouldn’t hurt anything – you’ll have to test further to be 100% sure.
When Finished, File -> Save All
Start the GUI client and check that the names have changed in Policy for Specific Countries and if blocked it should show as red in the maps.

Countries in this version:
(Replace country_display_name with Map name)

Countries Object: KOREA_DEMOCRATIC_PEOPLES_REPUBLIC_OF
country_display_name: Korea, Democratic People's Republic of
Map: North Korea

Countries Object: KOREA_REPUBLIC_OF
country_display_name: Korea, Republic Of
Map: Republic Of Korea

Countries Object: TANZANIA_UNITED_REPUBLIC_OF
country_display_name: Tanzania, United Republic of
Map: United Republic Of Tanzania

Countries Object: MACEDONIA_THE_FORMER_YUGOSLAV_REPUBLIC_OF
country_display_name: Macedonia, The Former Yugoslav Republic Of
Map: The Former Yugoslav Republic Of Macedonia

Countries Object: MOLDOVA_REPUBLIC_OF
country_display_name: Moldova, Republic Of
Map: Republic Of Moldova

Ideas to try if you are having issues:
Make sure the correct CMA was used when connecting GuiDBedit.
Install database
Close GUI clients and GuiDBedit
Restart the MDS CMA
mdsstop_customer <CMA Name>
mdsstart_customer <CMA Name>
Reboot the whole MDS server

Geo Protect configuration
To save me re-writing documentation please view the following link:
https://sc1.checkpoint.com/documents...min/103526.htm
There are a number of errors in this document the main one for this article is in Geo Protection setup to choose ‘ACTION’ Block or Allow.
Ignore this and use the Blacklist method for more reliable results. I.e. sk110683.
Make sure you donot block any public IP’s that are used for management communications to the gateways as the policy pushes will stop working.
Also exclude the IP addresses for the following:
sc1.checkpoint.com, crl.verisign.com, updates.checkpoint.com and your DNS server. These are the Geo Protection updates connections.

Geo Protect IP database.
There are numerous files involved.
The one we will be concentrating on is $FWDIR/tmp/geo_location_tmp/updates/IPtoCountry.csv . This is the IP database that matches IP addresses to countries.
The following is from sk92823. I’ve found that the solution in this article for testing and updating the IP database are not good enough for VSX and possibly non VSX systems as well. The method in this document is more reliable.
The IPtoCountry.csv file, which resides in $FWDIR/conf, comes with installation, and will not be changed ever.
The file is downloaded from the download center every interval (24 hours) to $FWDIR/tmp/geo_location_tmp/updates, and loaded into the kernel. The file in $FWDIR/conf is used only in case there is not file in $FWDIR/tmp/geo_location_tmp/updates.
In case the file cannot be downloaded, the existing file (in $FWDIR/tmp/geo_location_tmp/updates) is used.
The file in $FWDIR/tmp/geo_location_tmp/updates will always be newer than the one in in $FWDIR/conf (unless no file was downloaded).

IP Database Updates
Note: I found for VSX changing the ‘update_countries_list = true’ to false using GuiDBedit, saving, restarting MDS with mdsrestart then pushing policy to the gateways did not stop the database from updating.

Updates by Direct Connection to the internet
Updates happen directly from the gateway every 24hrs, and not the management servers like scheduled IPS updates.
The actual update time each day is the time it was successful on its last update or when the in.geod process first starts. This time can be found in the $FWDIR/tmp/geo_location_tmp/updates/timestamp file in EPOCH format. Use an online convertor to view this time in a readable format.
E.g. http://www.esqsoft.com/javascript_ex...e-to-epoch.htm

In VSX the update is only done by VSX0. The virtual servers do not do the updates. They have Linux Soft links to the VSX0 files.
In the VSX gateway policy, and any other firewalls routers etc in the path of the VS0 reaching the internet,.rules, NAT’s, routes will be needed to allow your external IP (or NAT IP of the VSX0) to reach the internet.
There needs to be a default route pointing to the internet on the VSX0 configuration or alternatively four static routes to sc1.checkpoint.com, crl.verisign.com, updates.checkpoint.com and your DNS server(s).
Your gateways DNS server will need to resolve these domains.

Updates using a proxy.
Adding proxy setting into CLISH did not help and was removed.
I’ve seen lab results differ to production in other tests. In the lab the proxy would only work if the cluster and physical server objects had the proxy set via GUI.
I noticed once working the proxy setting could be removed from the physical server objects and left on the cluster object and it would still use the proxy. A reboot was not tested and that may have reset this.
I.e. In VSX policy (policy for context 0)
double click (or edit) cluster -> topology -> proxy -> enter IP and port
double click (or edit) all physical VSX gateway objects -> topology -> proxy -> enter IP and port.
Always push policy after proxy changes even though saving the config looks like it pushes the changes to the GW's.

Automatic Updates:
This happens every 24hrs depending on the $FWDIR/tmp/geo_location_tmp/updates/timestamp file contents.
e.g. if the timestamps file has 1473719743 which is September 13 2016 10:35:43
The next update will be September 14 2016 10:35. It drops the milliseconds.
If it fails it will try at 10:35 the next day.
You can change the update time by editing the timestamp file with a new EPOCH time from the previous day with the time that you want.
Use and online convertor e.g. http://www.esqsoft.com/javascript_ex...e-to-epoch.htm

Manual Updates:
You cannot get a successful update by changing the gateways date to the next day as this will fail the security certificate check on the verisign site – you will get a failed down load message in tracker.
Change the EPOCH time in the $FWDIR/tmp/geo_location_tmp/updates/timestamp file to the previous day.
Use and online convertor
e.g. http://www.esqsoft.com/javascript_ex...e-to-epoch.htm
Once saved kill the in.geod process
e.g.
ps -ef | grep geo
Will show two processes. One from your grep, the other is the geo update process in.geod
If it is not there Geo Protect may have never been configured. You may need to configure a detect mode on geo Protection on the VSX0 policy and push. I.e. in SmartDashBoard -> IPS tab -> geo protection change the profile action to detect and push the policy.

[Expert@FWG01:0]# ps -ef | grep geo
admin 3566 3487 0 22:59 pts/2 00:00:00 grep geo
admin 28235 13803 0 16:14 ? 00:00:01 in.geod 0

Use kill to kill the process.
E.g. [Expert@FWG01:0]#kill 28235
The in.geod process will re-spawn immediately with a new process ID.

Backup IP database daily and delete files older than 14 days
#This script is to backup the IP to Countries IP address database
#every day so if there is an update with a corrupt file then it can be manually copied back.
#these files are kept for two weeks
#then deleted

#set up the Check Point environment so that $FWDIR will work
#
CPDIR=/opt/CPshrd-R77
. $CPDIR/tmp/.CPprofile.sh

#Debug logging
#
set -x
exec 2>$FWDIR/tmp/geo_location_tmp/updates/iptocountrybackup.log

#Variables
#
var_database=$FWDIR/tmp/geo_location_tmp/updates/IpToCountry.csv
var_dblocation=$FWDIR/tmp/geo_location_tmp/updates/
var_backupdir=$FWDIR/tmp/geo_location_tmp/updates/backup/
var_date2=`/bin/date +%Y%m%d`

#Backup Database
cp $var_database $var_backupdir/IpToCountry.$var_date2

#Remove databases older than 14 days
/usr/bin/find $var_backupdir -type f -ctime +14 -exec /bin/rm {} \;

Save to fav script directory.
set file permissions to execute chmod +x /usr/local/scripts/geodbbackup.sh
Add to cron to run before the first update (so you have a copy. ...or copy manually). I used the CLISH command:
add cron job GeoDBbackup command "/usr/local/scripts/geodbbackup.sh" recurrence daily time 07:00

Updates via scripts:
Posting this I realised I had a script from Check Point Article sk108425. I can't publish this and will need to write my own and add here at a later stage.
This article has more on this script creation and scheduling.
Use these two lines to add a proxy for http and https into your script:
export http_proxy=http://<proxy IP address>:<proxy port>/
export https_proxy=$http_proxy
Make sure your proxy server will allow the connections.

I'm not a master scripter or anything so just giving ideas here.
I'm trying not to remember the Check Point script to avoid any trouble from them :-) - If they approve then I will add it.

**Update to come as this is work in progress**

# Environment Variables
CPDIR=/opt/CPshrd-R77
. $CPDIR/tmp/.CPprofile.sh

#Debug logging Remove Remarks if wanted
# set -x
# exec 2>$FWDIR/tmp/geo_location_tmp/updates/backup/iptocountryupdate.log

# Other variables
var_database=$FWDIR/tmp/geo_location_tmp/updates/IpToCountry.csv
var_dblocation=$FWDIR/tmp/geo_location_tmp/updates/
var_backupdir=$FWDIR/tmp/geo_location_tmp/updates/backup/
var_date2=`/bin/date +%Y%m%d`
var_updatelog=$FWDIR/tmp/geo_location_tmp/updates/backup/iptocountryupdate.log

# Set Proxy if needed and Remove Remarks #>> $var_updatelog
# export http_proxy=http://<proxy IP address>:<proxy port>/ #>> $var_updatelog
# export https_proxy=$http_proxy #>> $var_updatelog

# remove the current compressed database files, if they exists. #>> $var_updatelog

# download the database from Check Point server using wget or curl #>> $var_updatelog
cd $FWDIR/tmp/geo_location_tmp/updates/ #>> $var_updatelog
/sysimg/CPwrapper/linux/MiniWrapper/Actions/wget https://sc1.checkpoint.com/freud/IpToCountry.csv.gz #>> $var_updatelog

# If decompressed file exists then check backup folder if ok remove current #>> $var_updatelog

# Decompress #>> $var_updatelog

# Change Permissions #>> $var_updatelog

# Check if file exists if not copy latest from backup. #>> $var_updatelog



Viewing Connections during the update:
Use netstat to list the IP's as they connect.
In a separate console connection set watch to every second and grep the resolved ip's for sc1.checkpoint.com, crl.verisign.com, updates.checkpoint.com.
For me they resolved to the following which are added into the netstat command.
watch -n1 "netstat -aon | grep '184.27.86.9\|23.37.133.163\|209.87.209.87'"

This is what I caught over 30 seconds, not all showed at the same time so I missed some established connections:
tcp 0 0 172.16.16.142:49802 209.87.209.87:443 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.16.16.142:51034 184.27.86.9:443 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.16.16.142:49802 209.87.209.87:443 TIME_WAIT timewait (53.20/0/0)
tcp 0 0 172.16.16.142:60685 23.37.133.163:80 TIME_WAIT timewait (52.66/0/0)
tcp 0 0 172.16.16.142:60684 23.37.133.163:80 TIME_WAIT timewait (52.52/0/0)

If using a proxy it will be proxy IP you'll need to watch:
So you should see similar connections.
watch -n1 "netstat -aon | grep <proxy IP>"


Viewing update status in tracker.
In tracker for your VSX0 policy filter on ‘Control’ under the ‘Type’ column.
Example of success:
Number: 35238
Date: 14Sep2016
Time: 10:33:35
Interface: geod daemon
Origin: FWCLX01
Type: Control
Action:
Product: IPS Software Blade
Description: Countries DB download has succeeded
Product Family: Network
Policy Info: Policy Name: VSX


Example of fail:
This will happen even if date of GW is wrong and will not even check internet connection.
Number: 77
Date: 4Aug2016
Time: 15:25:55
Interface: geod daemon
Origin: FWCLX01
Type: Control
Action:
Product: IPS Software Blade
Description: Countries DB download has failed. Please verify internet connectivity or contact technical support
Product Family: Network
Policy Info: Policy Name: VSX


IP checks: Find which IP is assigned to a country
There is a restricted Check Point KB on this sk94364
Publicly available is http://dev.maxmind.com/geoip/legacy/csv/
I converted a blocked IP to 2523638785 this nunber was changed slightly to hide actual address.
I.e.
<ip Address> Removed for privacy
4th octet + (3rd octet * 256) + (2nd octet * 65536) + (1st octet * 16777216)
= 2523638785

Using grep to list the countries with the first 5 digits of 2523638785
2523638785 falls into "2523594752","2523660287","iana","410227200","EU", "EUR","Europe"
In a later version of the file it falls under New Zealand.

[Expert@FWG01]# cat /opt/CPsuite-R77/fw1/tmp/geo_location_tmp/updates/IpToCountry.csv | grep 25236
"1121252360","1121252367","iana","410227200","ZA", "ZAF","South Africa"
"1121252368","1121252375","iana","410227200","GG", "GGA","Guernsey"
"1296252360","1296252367","iana","410227200","FR", "FRA","France"
"1296252368","1296252383","iana","410227200","DE", "DEU","Germany"
"2523594752","2523660287","iana","410227200","EU", "EUR","Europe"
"2523660288","2524119039","iana","410227200","US", "USA","United States"
"2954825236","2954825239","iana","410227200","PL", "POL","Poland"
"3252359168","3252362239","iana","410227200","DK", "DNK","Denmark"
"3252362240","3252362495","iana","410227200","PL", "POL","Poland"
"3252362496","3252362751","iana","410227200","IT", "ITA","Italy"
"3252362752","3252363263","iana","410227200","DK", "DNK","Denmark"
"3252363264","3252379647","iana","410227200","LT", "LTU","Lithuania"
"3642519552","3642523647","iana","410227200","IT", "ITA","Italy"
"3642523648","3642527743","iana","410227200","GB", "GBR","United Kingdom"

Troubleshooting:
cat $FWDIR/log/geod.elg
cat /var/log/messages

References:

• sk79360 - How to check if Geo Protections have been updated
• sk92823 - Problems With Geo Protection Updating
• sk108425 - IPS Geo Protection does not perform daily update
• sk95976 - How GEO protection country file is getting updated
• http://www.esqsoft.com/javascript_ex...e-to-epoch.htm
• https://sc1.checkpoint.com/documents...l_frameset.htm
• Ref: https://en.wikipedia.org/wiki/Unix_time
• https://www.checkpoint.com/downloads...-datasheet.pdf
• http://dev.maxmind.com/geoip/legacy/csv/
• Roshans Trusty lab
• Brain

[EricAnderson]

Pretty solid first post! Good stuff.

-E

[jawillia]

Thanks so much for this! I couldn't figure out why my VSX firewalls were not getting updates and in.geod was not running under VSX0. I have not seen anywhere in the documentation that you are required to run IPS with geo protection enabled on VSX0 in order to get updates.

[varera]

Unfortunately, it is not only about GP on VSX, but some other features too, when there are all kinds of unclear dependencies between VS0 and production VSs. For example, I have seen something completely dumb when using HTTP proxy on a VS.

Let's say one of VSs has a proxy running on it, so it has to resolve URLs and hostnames properly. The issue is, it needs DNS servers defined. The only DNS servers you can have are defined in VS0 content. They will be used, with a proxy VS trying to send DNS requests to those servers.

If you remember, in DMI VSX deployment VS0 is used for MGMT communication only, and it is likely one has internal DNS defined there. It is also likely proxy VS not to have connectivity to internal DNS servers and/or DMI network segment.

In such situation, proxy fails because it cannot resolve URLs. There could be two different ways to resolve this:

1. allow proxy VS connectivity to internal DNS servers
2. define external DNS servers on VS0

Just sharing :-)

[jawillia]

FWIW I just did some R77.30 to R80.20 upgrades, and found that the in.geod was not running in the VSX0. I had to change the geo policy in order to jumpstart getting it running. I'm not sure why that is...