If you haven’t caught this yet, I am a huge fan of ThousandEyes! I was working on a project where we were planning to deploy ThousandEyes agents to hundreds of sites, and to keep costs down we were going to use the Raspberry Pi 4 Model B. Having to image hundreds of SD cards was going to be a nightmare, so I wanted to automate the ThousandEyes Raspberry Pi agent image customization process to simplify the deployment.
Full disclosure: I started this project just as the supply chain issues were hitting the Pi market, and I left that job a few months later. I was never able to run this at scale, but I was able to use this for the few units I was able to get my hands on.
I also have a series on getting ThousandEyes deployed in a lab environment if you want to read more on getting ThousandEyes running: https://www.mytechgnome.com/2022/03/28/thousandeyes-walkthrough-part-1-what/
Hardware/Software required
- Raspberry Pi 4 Model B – 4GB+ RAM
- (Optional) Second Raspberry Pi (doesn’t need to be a 4, but needs a USB port and Ubuntu desktop support)
- 2x 32GB micro SD card (the following are recommended by ThousandEyes)
- Samsung EVO Plus (32GB minimum required)
- SanDisk Extreme (32 GB minimum required)
- Ubuntu Desktop OS image https://ubuntu.com/download/raspberry-pi/thank-you?version=22.04.2&architecture=desktop-arm64+raspi
- ThousandEyes account – you can sign up for trial here: https://www.thousandeyes.com/signup/
- USB microSD card adapter
Configure Pi for imaging
I used a Raspberry Pi as the system for building the images. I found it makes the process much easier, but this isn’t a hard requirement. The main thing is you will need to be able to mount a Linux file system, and it seemed to work much better using a native Ubuntu instance instead of trying to map an SD card to a VM or WSL instance.
- Using your preferred tool, flash the Ubuntu image onto one of the microSD cards – these are both good options:
- Boot a Pi using the newly flashed Ubuntu image and complete the setup
Automated and Manual Processes
I have an automated process available at my GitHub page: https://github.com/mytechgnome/TE-Imaging
The automation uses a JSON file with the device-specific information. You could automate the process of filling out the hostname and IP info if you have that info in an IPAM or similar platform.
How to use the vars.json file:
- Token Value – Group token from ThousandEyes portal
- Image name – Shouldn’t need to do anything with this, unless ThousandEyes changes the image file name
- SSH key – Leave blank if you aren’t adding a key, otherwise paste in the contents of the public key file
- Devices – Duplicate the device objects as needed
- Hostname – The name of the specific agent
- IP – Assign static IP or for DHCP, leave it blank- “”
- Subnet mask and gateway – self explanatory
- DNS – Also self explanatory. If you’re only using one DNS server, leave DNS2 blank- “”
Below is the manual process. The overall process is the same as the automated script, but with less automated goodness. If you want to understand how the automated process works, you can read through the manual process to get a better understanding.
Mount the ThousandEyes image
Before we can customize the image we need to mount it. Run the following steps from a terminal on the imaging Pi created previously.
- Download the ThousandEyes image
wget https://app.thousandeyes.com/install/downloads/appliance/thousandeyes-appliance.rpi4.img.xz
- Note: The download path might change. The download link can be accessed from the ThousandEyes Add New Enterprise Agent page: https://app.thousandeyes.com/settings/agents/enterprise/?section=agents&add-agent
- Decompress the image
unxz -k thousandeyes-appliance.rpi4.img.xz
- Create a mount directory
mkdir /tmp/temount
- Mount the image
sudo mount -o loop,offset=269484032 thousandeyes-appliance.rpi4.img /tmp/temount
- Note: Because there are two partitions, we need to mount the second partition
- If this doesn’t work it could be caused by a change in the partition layout, use these steps to get the correct offset
fdisk -l thousandeyes-appliance.rpi4.img
- Here’s an example output:
Disk thousandeyes-appliance.rpi4.img: 4.07 GiB, 4367262720 bytes, 8529810 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x354db354
Device Boot Start End Sectors Size Id Type
thousandeyes-appliance.rpi4.img1 * 2048 526335 524288 256M c W95 FAT32 (LBA)
thousandeyes-appliance.rpi4.img2 526336 8529809 8003474 3.8G 83 Linux - Multiply the sector size (512) by the start block (526336) and use that value in the offset.
- If this doesn’t work it could be caused by a change in the partition layout, use these steps to get the correct offset
- Note: Because there are two partitions, we need to mount the second partition
- Verify the image mounted
ls /tmp/temount/
- There should be several folders listed. If not, check the offset mentioned in step 4.
Customize the image
There are three files that need to be modified to get the image ready.
- Apply the ThousandEyes Account Token
sudo sed -i 's/<account-token>/$TOKEN/g' /tmp/temount/etc/te-agent.cfg
- Note: replace $TOKEN with your token value.
- Follow these steps to get your account token:
- Open a web browser and navigate to https://www.thousandeyes.com/
- Log into your account
- Click the Hamburger icon in the top left
- Expand Cloud & Enterprise Agents
- Click Agent Settings
- Click the Add New Enterprise Agent button
- Click the eye button to show the token, or the copy button to store it on the clipboard
- Set the hostname
sudo sed -i 's/tepi/$HOSTNAME/g' /tmp/temount/etc/hostname
- Replace $HOSTNAME with your desired hostname
- Configure a static IP (Optional – by default the appliance will use DHCP)
sudo sed -i 's/dhcp/static/g' /tmp/temount/etc/network/interfaces
sudo echo address $IP >> /tmp/temount/etc/network/interfaces
sudo echo netmask $MASK >> /tmp/temount/etc/network/interfaces
sudo echo broadcast $BROADCAST >> /tmp/temount/etc/network/interfaces
sudo echo gateway $GW >> /tmp/temount/etc/network/interfacessudo sed -i 's/#DNS=/DNS= $DNS/g' /tmp/temount/etc/systemd/resolved.conf
- These values will need to be updated accordingly: $IP $MASK $BROADCAST $GW $DNS
- A second DNS server can be added by simply adding both addresses with a space between them
- Add an SSH key (Optional)
sudo echo $SSH >> /tmp/temount/etc/ssh/keys/thousandeyes/authorized_keys
- You can generate an SSH key with the following command:
ssh-keygen -b 2048 -t rsa
- Copy the contents of the key file and put it in place of $SSH
- You can generate an SSH key with the following command:
- Unmount the image
sudo umount /tmp/temount
Flash the image to a microSD card
This is the easiest part, but also the most time consuming. After plugging in the SD card, find the device path for it by running this:
sudo fdisk -l | grep sd
If the SD card is already has partitions on it the output will show each partition, so if you see /dev/sda1 and /dev/sda2 that’s showing the partitions. The image file contains its own partitions, so it needs to be written directly to the card, not into an existing partition. To do that just ignore the numbers shown, so the destination would be /dev/sda.
To start writing the SD card run this command with the correct destination location.
dd if=thousandeyes-appliance.rpi4.img of=/dev/sda status=progress
Make sure the correct destination is selected, otherwise you might overwrite something you don’t want to lose. It usually takes about 10 minutes to write the SD card. When it’s finished, try booting a Pi using the SD card and it should come up with the correct hostname, IP, and account token.
All set!
After booting the Pi you’ll want to log in and change the local admin username and password
Default username: admin
Default password: welcome
After that, you can double check the setup wizard to make sure everything is good to go. The ThousandEyes agent will reach out to their registration servers and it will use the account token to get assigned to your account. The agent should be online in the portal within a few minutes.
BONUS Fun!
I’m not going to go through all the possible code examples of what you can do with this, but I thought I’d throw out a few things to think about.
- Make API calls into an IPAM to get the site name and IP info, then populate the vars.json file automatically with that data.
- Flip the script – this process builds the site-specific data into the image. Another option would be to use a generic image (still embedding the token and SSH key) and use DHCP addressing. Then, query the ThousandEyes API to find new agents named “tepi” and then remotely update the IP and hostname that way
- A unique SSH key can be added to each image by moving the SSH key value under the devices in the vars.json and moving the SSH section of the script under the per device section, along with changing the variable to use the correct location from vars.json.
- Similarly, you could also create new, unique SSH keys by including the keygen process in the script. Just make sure you keep a copy of the keys somewhere safe!
That’s it! Overall, the process is pretty easy, and far easier than finding Raspberry Pi 4s in stock anywhere… If you use this process to automate the ThousandEyes Raspberry Pi agent image customization, I’d love to hear how it worked for you and what, if any, changes you made to the process. You can add a comment here,