CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


First, I hope you're all well and staying safe.
Second, I want to give a "heads up" that you should see more activity here shortly, and maybe a few cosmetic changes.
I'll post more details to the "Announcements" forum soon, so be on the lookout. -E

 

Results 1 to 13 of 13

Thread: automated MDS backup

  1. #1
    Join Date
    2019-09-16
    Posts
    6
    Rep Power
    0

    Default automated MDS backup

    Hello team
    hopefully this is in correct section of forum
    I need to do specific task in job, which is do monthly backup and send to storage server.

    Because im not a linux a guy was thinking about script which would look like this and add to cron:

    mdsstop && cd /local/backups/ && mds_backup -g -l && mdsstart

    Later probably I'll add tar command to compress files into single one and then send via scp to storage server then delete it.


    Does it make sense to do it like this? The point is that it needs to be automated fully, so there are no questions during execution of scrit like "press y to logout users". etc.

  2. #2
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    363
    Rep Power
    14

    Default Re: automated MDS backup

    I normally use mds_backup -b -i -l. The b sets batch mode, which doesn't prompt for anything. The i includes the rule hit counts. The l (lowercase L) excludes logs (I have separate MLMs, so this is just a belt-and-suspenders to keep the size tolerable).

    I've been working on a script to fully automate this, complete with email notification. I don't have SCP automated yet, but the email, creation of the backup, and the removal of old backups past the retention period all works:

    ========================================
    THIS SCRIPT IS OBSOLETE.
    Leaving it here for educational purposes.
    A new version has been posted below.
    ========================================

    Code:
    #!/bin/env bash
    MTA="<mail.example.com>"
    RECIPIENTS="<someAddress@example.com; secondAddress@example.com>"
    SUBJECT_DATE="$(date --iso-8601)"
    WORKING_DIR="/var/log/mds_backup"
    NUMBER_TO_KEEP=1
    
    . /etc/profile.d/CP.sh
    ############################################################
    ## Send a notice the backup is starting, then wait five
    ## minutes to give people a chance to see it.
    printf "From: root@$(hostname)
    To: ${RECIPIENTS}
    Subject: MDS backup for ${SUBJECT_DATE}
    mds_backup beginning on $(hostname) in five minutes, at $(date -d '+5 minutes' --iso-8601=seconds)." \
    | /sbin/sendmail --host="${MTA}" --read-envelope-from -t
    sleep 300s
    
    ############################################################
    ## Run the backup.
    if [ ! -d ${WORKING_DIR} ];then mkdir ${WORKING_DIR};fi
    touch markerFile
    MDS_ERROR=$( { mds_backup -b -i -l -d ${WORKING_DIR}>/dev/null; } 2>&1 )
    MDS_EXIT=$?
    cd ${WORKING_DIR}
    for fileName in $(ls *.mdsbk.tgz | grep -v $(hostname)); do
    	dateStamp=$(echo $fileName | cut -d. -f1)
    	mv ${fileName} ${dateStamp}.$(hostname).mdsbk.tgz
    	done
    cd "$OLDPWD"
    
    ############################################################
    ## SCP code here.
    NEW_BACKUP="$(find ${WORKING_DIR} -newer markerFile -name '*.mdsbk.tgz')"
    ## SCP_EXIT=$?
    SCP_EXIT=0
    
    ############################################################
    ## If the SCP worked, clean up old files to leave only the
    ## number specified in FILES_TO_KEEP. Older files are
    ## removed first.
    FILES_REMOVED=""
    if [ ${SCP_EXIT} -eq 0 ];then
    	NUMBER_TO_REMOVE=$(($(ls -t ${WORKING_DIR}/*.mdsbk.tgz | wc -l)-${NUMBER_TO_KEEP}))
    	if [ ${NUMBER_TO_REMOVE} -gt 0 ];then
    		FILES_REMOVED=$(ls -t ${WORKING_DIR}/*.mdsbk.tgz | tail -n ${NUMBER_TO_REMOVE})
    		echo "${FILES_REMOVED}" | xargs -L 1 /bin/rm
    		fi
    	fi
    /bin/rm markerFile
    
    ############################################################
    ## Report backup status to the admins.
    printf "From: root@$(hostname)
    To: ${RECIPIENTS}
    Subject: MDS backup for ${SUBJECT_DATE}
    mds_backup finished on $(hostname) at $(date --iso-8601=seconds).
    
    mds_backup exit code: ${MDS_EXIT}
    SCP exit code: ${SCP_EXIT}
    
    mds_backup STDERR:
    ############################################################
    ${MDS_ERROR}
    ############################################################
    
    Files removed:
    ${FILES_REMOVED}" \
    | /sbin/sendmail --host="${MTA}" --read-envelope-from -t
    ========================================
    THIS SCRIPT IS OBSOLETE.
    Leaving it here for educational purposes.
    A new version has been posted below.
    ========================================


    To use this, you need to replace <mail.example.com> and <someAddress@example.com; secondAddress@example.com> with values for your environment. You can put the script anywhere, and run it with cron.

    The removal happens after the SCP, so you can set NUMBER_TO_KEEP to 0 if you don't want to retain any local copies. I like to retain at least one local in case I need to import without rebuilding the whole MDS.

    It puts the files in /var/log/mds_backup by default. If the directory doesn't exist, the script will create it.

    It renames the files to include the hostname of the MDS. My organization's fileservers are managed by a separate team, so I include this to help them find me if they have questions about the enormous files.
    Last edited by Bob_Zimmerman; 2020-07-06 at 14:35.

  3. #3
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    363
    Rep Power
    14

    Default Re: automated MDS backup

    I just updated my MDS past the versions in sk163300, which changed mds_backup to no longer gzip the final tar file. That broke my file renaming logic. Testing a fix.

    Edited to add: This should work for versions which emit .mdsbk.tar files instead of .mdsbk.tgz. Also fixed a stupid ordering issue with the markerFile. The marker was created after running the mds backup:

    Code:
    #!/bin/env bash
    MTA="<mail.example.com>"
    RECIPIENTS="<someAddress@example.com; secondAddress@example.com>"
    WORKING_DIR="/var/log/mds_backup"
    NUMBER_TO_KEEP=1
    
    . /etc/profile.d/CP.sh
    ############################################################
    ## Send a notice the backup is starting, then wait five
    ## minutes to give people a chance to see it.
    printf "From: root@$(hostname)
    To: ${RECIPIENTS}
    Subject: MDS backup for $(date --iso-8601)
    mds_backup beginning on $(hostname) in five minutes, at $(date -d '+5 minutes' --iso-8601=seconds)." \
    | /sbin/sendmail --host="${MTA}" --read-envelope-from -t
    sleep 300s
    
    ############################################################
    ## Run the backup.
    if [ ! -d ${WORKING_DIR} ];then mkdir ${WORKING_DIR};fi
    touch ${WORKING_DIR}/markerFile
    MDS_ERROR=$( { mds_backup -b -i -l -d ${WORKING_DIR}>/dev/null; } 2>&1 )
    MDS_EXIT=$?
    cd ${WORKING_DIR}
    for fileName in $(ls *.mdsbk.tgz | grep -v $(hostname)); do
    	dateStamp=$(echo $fileName | cut -d. -f1)
    	mv ${fileName} ${dateStamp}.$(hostname).mdsbk.tgz
    	done
    for fileName in $(ls *.mdsbk.tar | grep -v $(hostname)); do
    	dateStamp=$(echo $fileName | cut -d. -f1)
    	mv ${fileName} ${dateStamp}.$(hostname).mdsbk.tar
    	done
    cd "$OLDPWD"
    
    ############################################################
    ## Check the API afterwards. Sometimes it seems to die
    ## during an mds_backup. If dead, restart it.
    api status>/dev/null
    API_EXIT=$?
    if [ ${API_EXIT} -ne 0 ];then api start>/dev/null;fi
    
    ############################################################
    ## SCP code here.
    NEW_BACKUP="$(find ${WORKING_DIR} -newer markerFile -name '*.mdsbk.*')"
    ## SCP_EXIT=$?
    SCP_EXIT=0
    
    ############################################################
    ## If the SCP worked, clean up old files to leave only the
    ## number specified in FILES_TO_KEEP. Older files are
    ## removed first.
    FILES_REMOVED=""
    if [ ${SCP_EXIT} -eq 0 ];then
    	NUMBER_TO_REMOVE=$(($(ls -t ${WORKING_DIR}/*.mdsbk.* | wc -l)-${NUMBER_TO_KEEP}))
    	if [ ${NUMBER_TO_REMOVE} -gt 0 ];then
    		FILES_REMOVED=$(ls -t ${WORKING_DIR}/*.mdsbk.* | tail -n ${NUMBER_TO_REMOVE})
    		echo "${FILES_REMOVED}" | xargs -L 1 /bin/rm
    		fi
    	fi
    /bin/rm markerFile
    
    ############################################################
    ## Report backup status to the admins.
    printf "From: root@$(hostname)
    To: ${RECIPIENTS}
    Subject: MDS backup for $(date --iso-8601)
    mds_backup finished on $(hostname) at $(date --iso-8601=seconds).
    
    mds_backup exit code: ${MDS_EXIT}
    After backup, API was $(if [ ${API_EXIT} -ne 0 ];then printf "not running. Restarted.";else printf "running.";fi)
    SCP exit code: ${SCP_EXIT}
    
    mds_backup STDERR:
    ############################################################
    ${MDS_ERROR}
    ############################################################
    
    Files removed:
    ${FILES_REMOVED}" \
    | /sbin/sendmail --host="${MTA}" --read-envelope-from -t
    Last edited by Bob_Zimmerman; 2020-10-09 at 11:19.

  4. #4
    Join Date
    2014-09-02
    Posts
    374
    Rep Power
    10

    Default Re: automated MDS backup

    Nice work, Zimmie (as always).

    -E

  5. #5
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    363
    Rep Power
    14

    Default Re: automated MDS backup

    Thanks for the comment! I'm never sure if anybody else cares about this kind of thing.

  6. #6
    Join Date
    2019-09-16
    Posts
    6
    Rep Power
    0

    Default Re: automated MDS backup

    This is amazing what you did Bob. Big thanks on it.

    I did so far something basic script before i found out someone responded to this thread. I've added this to be executed by cron.
    cat script.sh
    source /etc/profile.d/CP.sh
    touch /home/cronuser/$(date +%y-%m-%d)mds_backup.log
    date >> /home/cronuser/$(date +%y-%m-%d)mds_backup.log
    echo test OK >> /home/cronuser/$(date +%y-%m-%d)mds_backup.log
    mds_backup -l -b -d /home/admin/MDSbackup >> /home/cronuser/$(date +%y-%m-%d)mds_backup.log


    The point is that overall the mds_backup was performed, but it didnt log anything. do you know why? Script even didnt created file (touch /home/cronuser/$(date +%y-%m-%d)mds_backup.log).
    Is it because im using CP.sh as source? Or did i mess something?

  7. #7
    Join Date
    2019-09-16
    Posts
    6
    Rep Power
    0

    Default Re: automated MDS backup

    Quote Originally Posted by Wiktor View Post
    The point is that overall the mds_backup was performed, but it didnt log anything. do you know why? Script even didnt created file (touch /home/cronuser/$(date +%y-%m-%d)mds_backup.log).
    Is it because im using CP.sh as source? Or did i mess something?
    Edit:
    I've checked today files on MDS and i see that log was created, but according to my worker when he did ls /home/cronuser | grep *.log file was not there during backup process. I somehow doubt that file was not there during backup and was created after finishing backup.
    I clearly see in log, that echo which is executed in my script right before backup was done properly, so im not sure if its neither my worker is lying or maybe indeed file was not visible during backup process

    cat 20-07-12backup.log | less
    Sun Jul 12 20:00:02 BST 2020
    test OK

    Multi-Domain Server Backup
    =======================================
    <output omitted>

  8. #8
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    363
    Rep Power
    14

    Default Re: automated MDS backup

    The file should be created as soon as you touch it, and it should have contents as soon as the >> is run. My bet would be time zone confusion (maybe he checked before the script had run?) or node confusion (maybe he checked on the secondary MDS?).

    That said, the way your script is written, it runs '$(date +%y-%m-%d)' several separate times. It would probably be a better idea to run that once and store the result in a variable, which you then use each of the other times. As it is, if your script runs at 23:59 of some day, it could end up writing its output to two different files.

  9. #9
    Join Date
    2019-09-16
    Posts
    6
    Rep Power
    0

    Default Re: automated MDS backup

    Thats what I though so, but still he was really confident about it. I'll verify backup this week by myself.

    Anyways Bob, you are totally right about the date part in my script. I just never did any scripting on real work this is early draft which I'll fix soon. Thanks on this.

    Finally i have last question.
    I want to do SCP automated authentciaton with ssh key.
    I suppose that i need to run ssh-keygen -b 4096 command on MDS and then copy key to remote server right?
    Is there any chance that generating pub key on MDS may break anything? Like connectivity between MDS and other domains? I suppose it will not, but want to be 100% sure

  10. #10
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    363
    Rep Power
    14

    Default Re: automated MDS backup

    SSH keys are a user-level thing. Check Point doesn't use them directly for anything, and they won't interfere with anything Check Point does.

    I'm working on SCP stuff myself (specifically, still building the file server to push the backups to), but it's really easy in general. You just specify the identity file you want to use (probably /home/admin/.ssh/id_rsa), and specify what you want to copy to where. The exit code should be 0 for success or non-0 for failure. My code already tests for success before removing any files, but I currently hard-code the variable to 0. Just remove 'SCP_EXIT=0', uncomment '## SCP_EXIT=$?', and add your SCP code immediately above that line. 'NEW_BACKUP' will contain the path to the file which was just created.

  11. #11
    Join Date
    2019-09-16
    Posts
    6
    Rep Power
    0

    Default Re: automated MDS backup

    Hi Bob

    few things
    1) I have watched cronbackup this week and i know why my coworker didnt see backup log. grep *.log seems to not work as i thougt, however file was created from the beggining and was visible when doing ls command
    2) when mds_backup was locking databases i've seen there was one failure
    Code:
    Failed to connect to group04, server is down
    CPmds-R77/customers/Group04/
    CPmds-R77/customers/Group04/CPSmartLog-R77/
    CPmds-R77/customers/Group04/CPSmartLog-R77/tmp/
    CPmds-R77/customers/Group04/opsec_pull_cert
    CPmds-R77/customers/Group04/CPSmartLog-R77/smartlog_server
    CPmds-R77/customers/Group04/CPSmartLog-R77/conf
    <ommited rest of output>
    Does it means that Group 04 was backuped but MDS didnt manage to lockdatabase?
    What can be the reason that MDS coudnt connect to database?
    3) Im trying to configure sending mails with your script, but im getting followed error
    Code:
    ./sendmail.sh
    sendmail: cannot locate host <my_company.com>: Name or service not known
    sendmail: could not send mail
    Doest it means im using wrong MTA or what?

    Sorry for all these questions. I never used linux services at this level

  12. #12
    Join Date
    2019-09-16
    Posts
    6
    Rep Power
    0

    Default Re: automated MDS backup

    Quote Originally Posted by Wiktor View Post
    3) Im trying to configure sending mails with your script, but im getting followed error

    Doest it means im using wrong MTA or what?

    Sorry for all these questions. I never used linux services at this level
    Please ignore. I didnt realise i had to remove "<>" from script to make it work.

    So if possible please answer to my second question and i guess this is all

  13. #13
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    363
    Rep Power
    14

    Default Re: automated MDS backup

    Ah. Yeah. By convention, brackets indicate optional arguments in UNIX/Linux, and less-than and greater-than indicate mandatory arguments. In both cases, the enclosing characters need to be removed as well.

    As for the "Failed to connect to group04", that's a good question. I don't know the answer, unfortunately. I supported Provider-1 in the TAC back in 2010 or so, but didn't touch it again for about eight years after I left. Just now getting back into it.

Similar Threads

  1. automated FW push scripts
    By bingdude in forum R80
    Replies: 1
    Last Post: 2017-08-30, 08:11
  2. SUPPORTED automated log archive procedure
    By boldin in forum SmartView Tracker
    Replies: 5
    Last Post: 2010-05-06, 21:41
  3. Automated Upgrade Export
    By Felix001 in forum Miscellaneous
    Replies: 0
    Last Post: 2009-11-05, 06:46
  4. Automated\graceful shutdown when power is lost?
    By Spacetrucker in forum Check Point SecurePlatform (SPLAT)
    Replies: 7
    Last Post: 2009-07-09, 10:55
  5. how can i do automated failover of vpn tunnel
    By venkatnarayana in forum IPsec VPN Blade (Virtual Private Networks)
    Replies: 1
    Last Post: 2007-09-28, 02:36

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •