Backgroound Image

Tag: automation

Automated ThousandEyes Raspberry Pi image customization

Posted on by mytechgnome

If you haven’t caught this yet, I am a huge fan of ThousandEyes! I was working on a project where we were planning to deploy ThousandEyes agents to hundreds of sites, and to keep costs down we were going to use the Raspberry Pi 4 Model B. Having to image hundreds of SD cards was going to be a nightmare, so I wanted to automate the ThousandEyes Raspberry Pi agent image customization process to simplify the deployment.

Full disclosure: I started this project just as the supply chain issues were hitting the Pi market, and I left that job a few months later. I was never able to run this at scale, but I was able to use this for the few units I was able to get my hands on.

I also have a series on getting ThousandEyes deployed in a lab environment if you want to read more on getting ThousandEyes running: https://www.mytechgnome.com/2022/03/28/thousandeyes-walkthrough-part-1-what/

Hardware/Software required

Configure Pi for imaging

I used a Raspberry Pi as the system for building the images. I found it makes the process much easier, but this isn’t a hard requirement. The main thing is you will need to be able to mount a Linux file system, and it seemed to work much better using a native Ubuntu instance instead of trying to map an SD card to a VM or WSL instance.

  1. Using your preferred tool, flash the Ubuntu image onto one of the microSD cards – these are both good options:
  2. Boot a Pi using the newly flashed Ubuntu image and complete the setup

Automated and Manual Processes

I have an automated process available at my GitHub page: https://github.com/mytechgnome/TE-Imaging
The automation uses a JSON file with the device-specific information. You could automate the process of filling out the hostname and IP info if you have that info in an IPAM or similar platform.

How to use the vars.json file:

  • Token Value – Group token from ThousandEyes portal
  • Image name – Shouldn’t need to do anything with this, unless ThousandEyes changes the image file name
  • SSH key – Leave blank if you aren’t adding a key, otherwise paste in the contents of the public key file
  • Devices – Duplicate the device objects as needed
    • Hostname – The name of the specific agent
    • IP – Assign static IP or for DHCP, leave it blank- “”
    • Subnet mask and gateway – self explanatory
    • DNS – Also self explanatory. If you’re only using one DNS server, leave DNS2 blank- “”
vars.json

Below is the manual process. The overall process is the same as the automated script, but with less automated goodness. If you want to understand how the automated process works, you can read through the manual process to get a better understanding.

Mount the ThousandEyes image

Before we can customize the image we need to mount it. Run the following steps from a terminal on the imaging Pi created previously.

  1. Download the ThousandEyes image
    wget https://app.thousandeyes.com/install/downloads/appliance/thousandeyes-appliance.rpi4.img.xz
  2. Decompress the image
    unxz -k thousandeyes-appliance.rpi4.img.xz
  3. Create a mount directory
    mkdir /tmp/temount
  4. Mount the image
    sudo mount -o loop,offset=269484032 thousandeyes-appliance.rpi4.img /tmp/temount
    • Note: Because there are two partitions, we need to mount the second partition
      • If this doesn’t work it could be caused by a change in the partition layout, use these steps to get the correct offset
        fdisk -l thousandeyes-appliance.rpi4.img
      • Here’s an example output:
        Disk thousandeyes-appliance.rpi4.img: 4.07 GiB, 4367262720 bytes, 8529810 sectors
        Units: sectors of 1 * 512 = 512 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disklabel type: dos
        Disk identifier: 0x354db354

        Device Boot Start End Sectors Size Id Type
        thousandeyes-appliance.rpi4.img1 * 2048 526335 524288 256M c W95 FAT32 (LBA)
        thousandeyes-appliance.rpi4.img2 526336 8529809 8003474 3.8G 83 Linux
      • Multiply the sector size (512) by the start block (526336) and use that value in the offset.
  5. Verify the image mounted
    ls /tmp/temount/
    • There should be several folders listed. If not, check the offset mentioned in step 4.

Customize the image

There are three files that need to be modified to get the image ready.

  1. Apply the ThousandEyes Account Token
    sudo sed -i 's/<account-token>/$TOKEN/g' /tmp/temount/etc/te-agent.cfg
    • Note: replace $TOKEN with your token value.
    • Follow these steps to get your account token:
      • Open a web browser and navigate to https://www.thousandeyes.com/
      • Log into your account
      • Click the Hamburger icon in the top left
      • Expand Cloud & Enterprise Agents
      • Click Agent Settings
      • Click the Add New Enterprise Agent button
      • Click the eye button to show the token, or the copy button to store it on the clipboard
  2. Set the hostname
    sudo sed -i 's/tepi/$HOSTNAME/g' /tmp/temount/etc/hostname
    • Replace $HOSTNAME with your desired hostname
  3. Configure a static IP (Optional – by default the appliance will use DHCP)
    sudo sed -i 's/dhcp/static/g' /tmp/temount/etc/network/interfaces
    sudo echo address $IP >> /tmp/temount/etc/network/interfaces
    sudo echo netmask $MASK >> /tmp/temount/etc/network/interfaces
    sudo echo broadcast $BROADCAST >> /tmp/temount/etc/network/interfaces
    sudo echo gateway $GW >> /tmp/temount/etc/network/interfaces
    sudo sed -i 's/#DNS=/DNS= $DNS/g' /tmp/temount/etc/systemd/resolved.conf
    • These values will need to be updated accordingly: $IP $MASK $BROADCAST $GW $DNS
    • A second DNS server can be added by simply adding both addresses with a space between them
  4. Add an SSH key (Optional)
    sudo echo $SSH >> /tmp/temount/etc/ssh/keys/thousandeyes/authorized_keys
    • You can generate an SSH key with the following command: ssh-keygen -b 2048 -t rsa
    • Copy the contents of the key file and put it in place of $SSH
  5. Unmount the image
    sudo umount /tmp/temount

Flash the image to a microSD card

This is the easiest part, but also the most time consuming. After plugging in the SD card, find the device path for it by running this:

sudo fdisk -l | grep sd

If the SD card is already has partitions on it the output will show each partition, so if you see /dev/sda1 and /dev/sda2 that’s showing the partitions. The image file contains its own partitions, so it needs to be written directly to the card, not into an existing partition. To do that just ignore the numbers shown, so the destination would be /dev/sda.


To start writing the SD card run this command with the correct destination location.

dd if=thousandeyes-appliance.rpi4.img of=/dev/sda status=progress

Make sure the correct destination is selected, otherwise you might overwrite something you don’t want to lose. It usually takes about 10 minutes to write the SD card. When it’s finished, try booting a Pi using the SD card and it should come up with the correct hostname, IP, and account token.

All set!

After booting the Pi you’ll want to log in and change the local admin username and password

Default username: admin
Default password: welcome

After that, you can double check the setup wizard to make sure everything is good to go. The ThousandEyes agent will reach out to their registration servers and it will use the account token to get assigned to your account. The agent should be online in the portal within a few minutes.

BONUS Fun!

I’m not going to go through all the possible code examples of what you can do with this, but I thought I’d throw out a few things to think about.

  • Make API calls into an IPAM to get the site name and IP info, then populate the vars.json file automatically with that data.
  • Flip the script – this process builds the site-specific data into the image. Another option would be to use a generic image (still embedding the token and SSH key) and use DHCP addressing. Then, query the ThousandEyes API to find new agents named “tepi” and then remotely update the IP and hostname that way
  • A unique SSH key can be added to each image by moving the SSH key value under the devices in the vars.json and moving the SSH section of the script under the per device section, along with changing the variable to use the correct location from vars.json.
  • Similarly, you could also create new, unique SSH keys by including the keygen process in the script. Just make sure you keep a copy of the keys somewhere safe!

That’s it! Overall, the process is pretty easy, and far easier than finding Raspberry Pi 4s in stock anywhere… If you use this process to automate the ThousandEyes Raspberry Pi agent image customization, I’d love to hear how it worked for you and what, if any, changes you made to the process. You can add a comment here,

Intro to Automation – Learning while having fun with Autonauts!

Posted on by Dan Kelcher

I have heard too many network engineers push back on learning how to automate things because they “don’t want to be a software developer.” There’s a slight problem with that argument, though. You don’t need to be a “software developer” to get into automation. Automation is surprisingly painless, and it can even be fun!

 One recommendation I would give people looking for a fun and easy approach to learning automation would be to look at games like Autonauts (https://www.humblebundle.com/store/autonauts) or if you want to add in some combat Autonauts Vs. Piratebots (https://www.humblebundle.com/store/autonauts-vs-piratebots)

Both games are surprisingly fun to play. You start with some simple resources and need to craft and build your way to greatness. Luckily, you have bots at your disposal that you can train to do pretty much anything. Chop down trees. Collect materials. Craft tools. Build. Autonauts vs. Piratebots adds in attack and defense. The best thing that the bots can do… build more bots!

Bot script

This is an example of a relatively simple script for a bot. It uses a drag-and-drop interface and can learn tasks by recording the player’s character’s actions.

As can be guessed by its name, this bot is a stone miner. To mine stone in both Autonauts games, you need a pick. During a game, hundreds, if not thousands, of units of stone are required. Why mine all of that manually when you can have a bot do it?

This script uses a repeat loop, with a condition of “forever!” meaning the bot will always be running this task.

Inside that first Repeat loop is another Repeat loop, and this one has an exit condition when the bot’s hands are full. When the bot isn’t holding anything, it will perform the action of moving to the Crude Pick Storage location and then the action of removing a pick from storage.

Now that the bot is holding a pick, it exits the repeat loop and moves on to the next step in the script. The next Repeat loop will run continuously until the tool breaks, and the bot’s hands are empty. During this loop, the bot will look for a stone deposit, move to it, and then use the pick to mine the stone.

The final script results in a bot that will mine until its pick breaks, and when that happens, it will get a replacement pick and return to mining. The bot will continue producing stone as long as there are picks in the storage location and open stone deposits.

Of course, the work doesn’t stop with just this one bot. The crafting and storing of picks will require automation. All that stone that was mined will need to be moved to a storage location.

This script could also be improved upon. The bot can move tools to a backpack and retrieve them. The mining location could be tied to a spot indicated by a sign. The script can be saved and then linked to other bots.  

As the player progresses through the game, the tasks become far more complex. Each bot has limited memory in its “bot brain,” which limits each script’s length. A difficult task requires efficient scripting or multiple bots doing parts of the script.

Sure, the game doesn’t specifically cover anything about network automation. However, that’s where I think the value is. The best part of this game is how it makes you think of different ways to build out the various tasks. There are several mechanisms beyond the repeat loops that can be used. It shifts from a linear focus for each task to a more systematic approach. 

Traditional CLI configuration is linear. Log into a device, move through the different config layers, and apply the config change. Network automation can use that same linear methodology, but the real power of it is when logic is layered on. Using if/then/else statements, while and for loops, variables, functions, etc., can all increase the power of a script significantly.

The beautiful part about network automation is much like a script in Autonauts; it can be improved iteratively. As new techniques are learned, they can be applied to increase the efficiency of the scripts. Those scripts can be reused later or modified to suit changing needs.