CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it yet again - That's right, the 3rd edition is here!
You can read his announcement post here.
It's a massive upgrade focusing on current versions, and well worth checking out. -E

 

Results 1 to 7 of 7

Thread: Importing old version txt logs

  1. #1
    Join Date
    2012-09-03
    Posts
    15
    Rep Power
    0

    Default Importing old version txt logs

    I have a request to look at some traffic from a few months ago, which typically happened before I upgraded my firewalls from R75.30 SPLAT to R76 GAIA. I have the imported logs as txt files, but at 2.5gb they're seemingly impossible to open in excel and filter that way, which was what I hoped I could do.

    Is there a way I can import files from a previous version into my current version, seeing as they are fully converted text files?

  2. #2
    Join Date
    2010-01-12
    Posts
    40
    Rep Power
    0

    Default Re: Importing old version txt logs

    I'm not aware of any import function from SmartView Tracker or using the FW Log/fwm logexport commands. So I don't think you will get this back into Checkpoint for analysis.
    You can have a look on the Internet and find some kind of program that supports opening large files and provides filtering etc or you could loom at importing the text file into a SQL database, there are tools for doing this based on headings ect.
    You can try command line tools such as grep or awk to search the file based on search strings.

    However personally I'd look at writing a script to handle this. For instance Python is good at handling large files as it can read one line at a time (no temporary files or large amounts or RAM required).
    For instance the script below will read through the existing log, pick out all lines with a value of 192.168.1.1 and save this to a new file (newlog.txt). As the new file contains only a subset of the logs, it should be much smaller and should open in Excel.

    #!/usr/bin/env python

    # Value to filter on
    value = '192.168.1.1'

    # New log file
    outfile = open('newlog.txt','a')

    #Open log and read line by line, saving intersting lines to new log
    with open('log.txt') as infile:
    for line in infile:
    if value in line:
    outfile.write(line)

    # Close New Log
    outfile.close()

  3. #3
    Join Date
    2012-09-03
    Posts
    15
    Rep Power
    0

    Default Re: Importing old version txt logs

    That's excellent. I have literally no experience scripting with Python, so thanks for the basic layout. I'll give that a go.

  4. #4
    Join Date
    2010-01-12
    Posts
    40
    Rep Power
    0

    Default Re: Importing old version txt logs

    Just noticed the formatting is messed up and python is quite picky about whitespace

    Code:
    #!/usr/bin/env python
    
    # Value to filter on
    value = '192.168.1.1'
    
    # New log file
    outfile = open('newlog.txt','a')
    
    #Open log and read line by line, saving intersting lines to new log
    with open('log.txt') as infile:
        for line in infile:
            if value in line:
                outfile.write(line)
    
    # Close New Log
    outfile.close()

  5. #5
    Join Date
    2006-04-30
    Location
    Europe, Germany
    Posts
    433
    Rep Power
    14

    Default Re: Importing old version txt logs

    Quote Originally Posted by Andim View Post
    I have a request to look at some traffic from a few months ago, which typically happened before I upgraded my firewalls from R75.30 SPLAT to R76 GAIA. I have the imported logs as txt files, but at 2.5gb they're seemingly impossible to open in excel and filter that way, which was what I hoped I could do.

    Is there a way I can import files from a previous version into my current version, seeing as they are fully converted text files?
    Excel is really not the right tool ...
    Install a PgSQL / MySQL / MSSQL and import the data there.
    Create a ODBC connect from Excel or better from Access to the DB, this way they can write filters and handle the data even the size is 100GB

  6. #6
    Join Date
    2006-04-30
    Location
    Europe, Germany
    Posts
    433
    Rep Power
    14

    Default Re: Importing old version txt logs

    Quote Originally Posted by Andim View Post
    I have the imported logs as txt files, but at 2.5gb they're seemingly impossible to open in excel and filter that way, which was what I hoped I could do.
    I totally forgot on *NIX machines you can use the command split

    Say they are able to open a file with 100.000 lines in Excel you can use the command split to chop the log into parts with $num_lines
    On a *NIX machine use the command:
    Code:
    > mkdir $space/split_log
    > cd $space/split_log
    
    # Notice: the '.' in $split_log. separates the filename from the file split suffix
    > split -l $num_lines -a 5 - $split_log.  < $big_log
    
    # or if the log is gzip'd
    > gzcat $big_log | split -l $num_lines -a 5 - $split_log.

  7. #7
    Join Date
    2012-09-03
    Posts
    15
    Rep Power
    0

    Default Re: Importing old version txt logs

    Thanks for your help everyone, I'm going to try and create an MSSQL database and import the data there.

Similar Threads

  1. Importing logs into an Oracle Database
    By Barry J. Stiefel in forum SmartView Tracker
    Replies: 3
    Last Post: 2008-03-26, 11:18
  2. Import Logs from Previous Version
    By NickBrandson in forum SmartView Tracker
    Replies: 3
    Last Post: 2007-02-13, 06:26
  3. Importing old logs into SmartView Reporter
    By pop_alex in forum Eventia Analyzer/Reporter/SmartView Reporter
    Replies: 3
    Last Post: 2006-05-17, 06:39
  4. Replies: 0
    Last Post: 2006-02-08, 00:00
  5. Importing settings/logs from temporary SmartCenter
    By pop_alex in forum Miscellaneous
    Replies: 1
    Last Post: 2006-01-26, 14:18

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •