CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


Tim Hall has done it again! He has just released the 2nd edition of "Max Power".
Rather than get into details here, I urge you to check out this announcement post.
It's a massive upgrade, and well worth checking out. -E

 

Results 1 to 12 of 12

Thread: SmartLog via CLI

  1. #1
    Join Date
    2008-01-25
    Location
    Karlsruhe / Germany
    Posts
    15
    Rep Power
    0

    Default SmartLog via CLI

    Hi all,

    is there a way to get access to the SmartLog database via CLI?
    I am looking for a special logs over an long time period (2 month). The logfile is switch nearly every hour.
    So using "fw log" or SmartTracker is not so much fun.

    When I use SmartLog, I can search a longer time period, but there are a lot of matches (5-10k) for my query and so
    I have to view all matches before you can export all the logs into a seperate CSV-file.

    Thanks for your support!

    Regards
    Chili

  2. #2
    Join Date
    2011-08-02
    Location
    http://spikefishsolutions.com
    Posts
    1,654
    Rep Power
    10

    Default Re: SmartLog via CLI

    Quote Originally Posted by Chili View Post
    Hi all,

    is there a way to get access to the SmartLog database via CLI?
    I am looking for a special logs over an long time period (2 month). The logfile is switch nearly every hour.
    So using "fw log" or SmartTracker is not so much fun.

    When I use SmartLog, I can search a longer time period, but there are a lot of matches (5-10k) for my query and so
    I have to view all matches before you can export all the logs into a seperate CSV-file.

    Thanks for your support!

    Regards
    Chili
    I don't know of any cli tools. Smartlogd (the system process) is basically a webserver from what i've seen. Smartlog (the log verview) i think it just sending http requests with a speical cookie for auth. if you can figure out how to authinicate then i think you should be able to hack up some http calls in your fav scripting language.

    2nd option: I think its using a google database on the back end but i'm not sure.

    Let us know what you find if anything. I think this will be something everyone would like to know.

  3. #3
    Join Date
    2005-08-14
    Location
    Gig Harbor, WA, USA
    Posts
    2,494
    Rep Power
    17

    Default Re: SmartLog via CLI

    At the moment, there are no CLI tools for SmartLog.
    This issue with exporting all matched records is familiar, let me check with my contacts in R&D.
    http://phoneboy.org
    Unless otherwise noted, views expressed are my own

  4. #4
    Join Date
    2010-11-11
    Posts
    57
    Rep Power
    10

    Default Re: SmartLog via CLI

    Necro an old thread.
    Has there ever been a solution to this problem?

  5. #5
    Join Date
    2011-08-02
    Location
    http://spikefishsolutions.com
    Posts
    1,654
    Rep Power
    10

    Default Re: SmartLog via CLI

    Quote Originally Posted by Christoph View Post
    Necro an old thread.
    Has there ever been a solution to this problem?
    I don't know why i thought it was some google database. Seems to be apache solr after looking at it again. I never did figure out how to hack a http session, but this was many moons ago.

    Not really a very useful follow up on my part. :D

    night!

  6. #6
    Join Date
    2010-11-11
    Posts
    57
    Rep Power
    10

    Default Re: SmartLog via CLI

    Quote Originally Posted by jflemingeds View Post
    I don't know why i thought it was some google database. Seems to be apache solr after looking at it again. I never did figure out how to hack a http session, but this was many moons ago.

    Not really a very useful follow up on my part. :D

    night!
    Thank you. I took a quick look:

    # curl_cli -b "CPToken=XYZ123XYZ123XYZ123DUMMY" "http://localhost:18242/search_tops?query=blade:%22Threat%20Emulation%22&c ount=10&time_effort=10&sub_session_action=update&s ub_session=0"

    This produces valid output.

    EDIT: But you're right. I should have read your earlier comment better. I have no idea how long this Token is valid, so it may change.
    Last edited by Christoph; 2016-08-17 at 05:02.

  7. #7
    Join Date
    2011-08-02
    Location
    http://spikefishsolutions.com
    Posts
    1,654
    Rep Power
    10

    Default Re: SmartLog via CLI

    Quote Originally Posted by Christoph View Post
    Thank you. I took a quick look:

    # curl_cli -b "CPToken=XYZ123XYZ123XYZ123DUMMY" "http://localhost:18242/search_tops?query=blade:%22Threat%20Emulation%22&c ount=10&time_effort=10&sub_session_action=update&s ub_session=0"

    This produces valid output.

    EDIT: But you're right. I should have read your earlier comment better. I have no idea how long this Token is valid, so it may change.
    Interesting, where did you find the cookie? I remember seeing urls flying around when i was debugging but i didn't break them down any further.

    Is the data returned in json format?

  8. #8
    Join Date
    2010-11-11
    Posts
    57
    Rep Power
    10

    Default Re: SmartLog via CLI

    Quote Originally Posted by jflemingeds View Post
    Interesting, where did you find the cookie? I remember seeing urls flying around when i was debugging but i didn't break them down any further.
    The cookie was in the first http request when the initial or any subsequent request by smartlog. Not sure when the cookie was set and how persistent it is.

    tcpdump -nnvXSs 0 -i any port 18242 -w x.pcap
    Personally I found Microsoft Message Analyzer in this case more convenient than Wireshark to find the relevant part.

    Is the data returned in json format?
    Not 100% sure as I am home right now. From my memory I cannot tell if it was XML (iirc the packet capture had some XML labels) or JSON, but I just skimmed over the data as I just wanted to have a valid query to begin with.

    Maybe I find some time over the weekend to check the persistence of the cookie, the data format returned and some more meaningful queries.

  9. #9
    Join Date
    2011-08-02
    Location
    http://spikefishsolutions.com
    Posts
    1,654
    Rep Power
    10

    Default Re: SmartLog via CLI

    Quote Originally Posted by Christoph View Post
    The cookie was in the first http request when the initial or any subsequent request by smartlog. Not sure when the cookie was set and how persistent it is.

    tcpdump -nnvXSs 0 -i any port 18242 -w x.pcap
    Personally I found Microsoft Message Analyzer in this case more convenient than Wireshark to find the relevant part.


    Not 100% sure as I am home right now. From my memory I cannot tell if it was XML (iirc the packet capture had some XML labels) or JSON, but I just skimmed over the data as I just wanted to have a valid query to begin with.

    Maybe I find some time over the weekend to check the persistence of the cookie, the data format returned and some more meaningful queries.
    aah -i any. I didn't think about that. I think i tried -i lo but again its been a while.

    Silly question, have you looked through the R80 API docs? This might be covered in there in some manner.

  10. #10
    Join Date
    2008-01-25
    Location
    Karlsruhe / Germany
    Posts
    15
    Rep Power
    0

    Default Re: SmartLog via CLI

    Hi all,

    regarding to my initial question there was some Kind of workaround in newer versions.
    Check Point added a new field in the export form where you can define the number of logs to export.
    The number is limited to a million logs. This should handle nearly all requests even if I would prefer "export all logs <from date> <to date>".

    I already took a look at the R80 api reference guide.
    It does not show any possibilities for gettings logs.
    I would say right now it can only handle actions that are performed by SmartDashboard.

    Cheers,
    Sven

  11. #11
    Join Date
    2010-11-11
    Posts
    57
    Rep Power
    10

    Default Re: SmartLog via CLI

    Looked a bit further into it. You can gather all the relevant information in XML form from the smartlog database via i.e. curl.
    Default offset is a start at line 1 from your request with an end of 50. If you scroll the offset moves (1-50, 50-100..). So a good way for the cli to gather data is by using a large offset in a defined time peroid.

    The only problem left is the session cookie, which expires at the moment you close smart log.
    The generation of the session cookie takes place when you start smart log with a:

    QUERY:
    POST /login HTTP/1.0
    Content-Length: 283

    <login>
    <user><![CDATA[username]]></user>
    <magic_number><![CDATA[CP_Etude_2055]]></magic_number>
    <password><![CDATA[]]></password>
    <sso_token><![CDATA[1234567890ABCDEF]]></sso_token>
    <get_all_columns_def />
    </login>


    REPLY
    HTTP/1.1 200 OK
    Content-Type: text/plain
    Cache-Control: no-cache
    Connection: keep-alive

    <?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
    <root><read_only>false</read_only>...<token>Z1234567890ABCDEFZ</token>...


    I will try to use the </password> field. Without it one would have to have a read only smartlog active all the time use its credentials.

    EDIT: Ok this works, remove the sso token, enter a cleartext password and you get you token. The token looks to stay valid for a reasonable time. No idea when it expires serverwise, but if you fetch a token for every query one should be fine.
    This should be sufficient to automatically query smartlog.
    Last edited by Christoph; 2016-08-29 at 11:30.

  12. #12
    Join Date
    2011-08-02
    Location
    http://spikefishsolutions.com
    Posts
    1,654
    Rep Power
    10

    Default Re: SmartLog via CLI

    nice bit of hacking there.

Similar Threads

  1. Smartlog displaying username & user
    By torenhof in forum SmartLog
    Replies: 0
    Last Post: 2014-04-01, 02:34
  2. SmartLog disk space location
    By HRattink in forum Miscellaneous
    Replies: 2
    Last Post: 2013-08-23, 15:18
  3. how to adjust smartlog retention time
    By eduardw in forum SmartLog
    Replies: 3
    Last Post: 2013-04-17, 17:29
  4. SmartLog Gui Auto refresh?
    By abc150781 in forum SmartLog
    Replies: 1
    Last Post: 2013-03-11, 09:25
  5. R76 GAIA / SmartLog
    By Jejerod in forum SmartLog
    Replies: 3
    Last Post: 2013-03-02, 13:05

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •