CPUG: The Check Point User Group

Resources for the Check Point Community, by the Check Point Community.


First, I hope you're all well and staying safe.
Second, I want to give a "heads up" that you should see more activity here shortly, and maybe a few cosmetic changes.
I'll post more details to the "Announcements" forum soon, so be on the lookout. -E

 

Results 1 to 2 of 2

Thread: API target for development

  1. #1
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    403
    Rep Power
    15

    Default API target for development

    Most of my development work so far has been against a 2200 which I personally own. It has a perpetual license, but it's sometimes a little unpredictable. The API service sometimes crashes. It has a lot of unique state from all the changes I've made on it while building stuff. This makes it okay for human testing, but really not great for automated testing (it's hard to write a unit test if the API target can move).

    Clearly the 15-day plug-n-play eval is well-suited for this. Just spin up a new VM with a known config, and run the automated tests against that, right? Well, it turns out the 15-day eval starts counting down when you first boot GAiA. If you do stuff like installing patches or adding your SSH keys, you're out of luck. The image is only good for 15 days. And the installation is kind of hard to automate. You have to type stuff like the password you want for admin, and the initial IP address.

    Blink is the solution to this, right? Well ... not really. It lets you automate some of the installation, but you still end up with a different web UI certificate and SSH host keys every time you reimage the system. Automated tests against a new certificate every time? Pretty gross. Key change prompt every time you connect via SSH? Even worse.



    I ended up playing a little trick on GAiA. I ran the R80.40 installer, then at the end when it asks to reboot, I instead powered off the VM. Next, I booted it with a Linux LiveCD (I happened to use Devuan. Anything which understands LVM and XFS should be fine.) and mounted the LVM volumes. This let me make changes to the filesystem before the first boot. Specifically, I made these changes:

    1. Copied /web/conf/server.key and /web/conf/server.crt from a working VM to this new one
    2. Copied /etc/ssh/ssh_host* from a working VM to this new one
    3. Made /home/admin/.ssh/authorized_keys and added my workstation's public key to it


    Note that /home/admin/.ssh must be mode 700, and /home/admin/.ssh/authorized_keys must be mode 600.

    With the files in place, I shut down the VM, removed the LiveCD from its optical drive, and took another snapshot. Booted the VM, and checked the eval expiration date via 'show installer status'. Using the hypervisor's CLI tool, I powered the VM off, restored it to the snapshot, and powered it on again. Checked the eval expiration date again, and it was different! Success!

    Note that when you restore to the snapshot, it is a completely unconfigured GAiA system. If you give it your SSH key via the LiveCD as I described above, you can easily copy over a config_system file and use that to run the initial wizard. From there, you can easily also copy over the latest CPUSE, jumbo, and so on, and install them. I am further using some mgmt_cli commands to insert some initial data for development purposes.



    Kind of a headache, but it's the best way I've found so far to get disposable SmartCenters or MDSs without a huge amount of manual interaction.

  2. #2
    Join Date
    2007-03-30
    Location
    DFW, TX
    Posts
    403
    Rep Power
    15

    Default Re: API target for development

    I eventually decided using snapshots for this is too slow. I have a ludicrously powerful desktop (2x Xeon X5675 [3.06 GHz, 6 cores plus hyperthreading], 96 GB of RAM), and it was still taking over 20 minutes to run config_system and another 20 to install the jumbo. I flattened the snapshot and cloned the pre-first-boot VM. Now I have a VM named "R80.40" which is a base R80.40 installation with the modifications I made above. It has never been booted. I then have my build script clone this VM into a new working VM. config_system is down to 7:30 m:ss, and installing the jumbo is down to a hair over ten minutes. I'm currently using a SATA SSD, which is clearly the next bottleneck (waiting for a good deal on an Intel P4618, but if somebody wanted to donate an Optane card, I wouldn't say no!). Still, cutting build time by ~45% is nothing to sneeze at.

    Here's my basic build script for the moment:

    Code:
    #!/bin/zsh
    sourceImage="R80.40"
    systemIPAddress="10.0.3.240"
    localBuildDir=~/OneDrive/CPMDS_Build
    systemName="TestMDS"
    
    success() {
    echo " [\e[0;92m\e[1mDone at $(date "+%H:%M:%S")\e[0m]"
    }
    
    sshCmd() {
    ssh admin@"${systemIPAddress}" "$argv" >>/tmp/"${systemName}".output 2>>/tmp/"${systemName}".output
    }
    
    scpFile() {
    for file in "$argv[@]";do
    scp "$file" admin@"${systemIPAddress}": >>/tmp/"${systemName}".output 2>>/tmp/"${systemName}".output
    done
    }
    
    jumboName="$(cd "${localBuildDir}";ls *.tgz | grep JUMBO)"
    
    echo -n "" > /tmp/"${systemName}".output
    echo "Starting at $(date "+%H:%M:%S")"
    
    echo -n "Resetting VM to the cleanly installed state ..."
    vboxmanage controlvm "${systemName}" poweroff >>/tmp/"${systemName}".output 2>>/tmp/"${systemName}".output
    vboxmanage unregistervm "${systemName}" --delete >>/tmp/"${systemName}".output 2>>/tmp/"${systemName}".output
    vboxmanage clonevm "${sourceImage}" --mode=machine --name="${systemName}" --register >>/tmp/"${systemName}".output 2>>/tmp/"${systemName}".output
    false;while [ $? -ne 0 ];do
    vboxmanage startvm "${systemName}" >>/tmp/"${systemName}".output 2>>/tmp/"${systemName}".output
    done
    success
    
    echo -n "Waiting for the system to finish booting ..."
    ( exit 255 );while [ $? -gt 1 ];do
    sleep 1
    sshCmd lock database override
    done
    sshCmd set user admin shell /bin/bash
    sshCmd clish -c \"save config\"
    success
    
    echo -n "Running first-time config wizard. This takes a while ..."
    scpFile "${localBuildDir}"/"${systemName}".config_system.txt
    sshCmd config_system -f "${systemName}".config_system.txt
    systemIPAddress="$(grep ipaddr_v4 "${localBuildDir}"/"${systemName}".config_system.txt | cut -d'=' -f2)"
    sshCmd rm "${systemName}".config_system.txt
    success
    
    echo -n "Copying CPUSE and jumbo to the system ..."
    scpFile "${localBuildDir}"/*.tgz
    success
    
    echo -n "Updating CPUSE ..."
    sshCmd clish -c \"lock database override\"
    sshCmd clish -c \"installer agent install $(cd "${localBuildDir}";ls *.tgz | grep DeploymentAgent)\"
    sshCmd rm DeploymentAgent_\*
    success
    
    echo -n "Sleeping five minutes to give services a chance to quiesce ..."
    sleep 300
    success
    
    echo -n "Installing jumbo ..."
    sshCmd clish -c \"lock database override\"
    sshCmd clish -c \"installer import local $jumboName\"
    echo "y" | sshCmd clish -c \"installer install $jumboName\"
    sshCmd rm $jumboName
    sshCmd 'while true;do sleep 1;done'
    success
    
    echo -n "Running API script ..."
    false;while [ $? -ne 0 ];do
    sleep 60
    sshCmd mgmt_cli -r true show hosts limit 1
    done
    sshCmd mgmt_cli -r true set api-settings accepted-api-calls-from \"all ip addresses that can be used for gui clients\" automatic-start true
    sshCmd api restart
    sshCmd mgmt_cli -r true add domain name FirstCMA servers.0.name CMA1 servers.0.ipv4-address 10.0.3.241 servers.0.multi-domain-server "${systemName}" servers.0.active true servers.0.type \"management server\"
    success
    The important variables are at the top. "sourceImage" needs to match the name of the template VM in VirtualBox. systemIPAddress needs to be the IP you gave it in the GAiA installation wizard. localBuildDir is the directory which contains your config_system file, deployment agent, and jumbo. Finally, systemName is the name you want the new VM to get. This name is also used to find the appropriate config_system file in the localBuildDir. In my specific example, the config_system file is named "TestMDS.config_system.txt".

    Towards the middle (right after config_system runs), you can see it searches the config_system file for the ipaddr_v4 field and sets systemIPAddress to that value. VMs cloned from a template will always have the original address you specified, but this allows multiple VMs to be built from that single template (e.g., if you want to try management HA, or MDSM/MDSC separation). If you need to move the VM to a different network, that's the time to do it. Note that all VMs produced from a given template will have the same SSH keys.

    All output from the remote system is logged to /tmp/<systemName>.output. If something doesn't work, or takes longer than expected, check there.
    Last edited by Bob_Zimmerman; 2021-03-21 at 17:07.

Similar Threads

  1. Using license on a device in our development lab
    By agnetsolutions in forum Licensing
    Replies: 2
    Last Post: 2012-02-20, 20:58
  2. USB CD emulating flash drive under development.
    By alienbaby in forum Check Point SecurePlatform (SPLAT)
    Replies: 1
    Last Post: 2011-07-27, 23:23
  3. IPSO development environment
    By mltv1 in forum Check Point IP Appliances and IPSO (Formerly Sold By Nokia)
    Replies: 1
    Last Post: 2006-05-06, 16:43

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •