Author: mytechgnome

If you haven’t caught this yet, I am a huge fan of ThousandEyes! I was working on a project where we were planning to deploy ThousandEyes agents to hundreds of sites, and to keep costs down we were going to use the Raspberry Pi 4 Model B. Having to image hundreds of SD cards was going to be a nightmare, so I wanted to automate the ThousandEyes Raspberry Pi agent image customization process to simplify the deployment.

Full disclosure: I started this project just as the supply chain issues were hitting the Pi market, and I left that job a few months later. I was never able to run this at scale, but I was able to use this for the few units I was able to get my hands on.

I also have a series on getting ThousandEyes deployed in a lab environment if you want to read more on getting ThousandEyes running: https://www.mytechgnome.com/2022/03/28/thousandeyes-walkthrough-part-1-what/

Hardware/Software required

• Raspberry Pi 4 Model B – 4GB+ RAM
• (Optional) Second Raspberry Pi (doesn’t need to be a 4, but needs a USB port and Ubuntu desktop support)
• 2x 32GB micro SD card (the following are recommended by ThousandEyes)
  • Samsung EVO Plus (32GB minimum required)
  • SanDisk Extreme (32 GB minimum required)
• Ubuntu Desktop OS image https://ubuntu.com/download/raspberry-pi/thank-you?version=22.04.2&architecture=desktop-arm64+raspi
• ThousandEyes account – you can sign up for trial here: https://www.thousandeyes.com/signup/
• USB microSD card adapter

Configure Pi for imaging

I used a Raspberry Pi as the system for building the images. I found it makes the process much easier, but this isn’t a hard requirement. The main thing is you will need to be able to mount a Linux file system, and it seemed to work much better using a native Ubuntu instance instead of trying to map an SD card to a VM or WSL instance.

1. Using your preferred tool, flash the Ubuntu image onto one of the microSD cards – these are both good options:
   • Raspberry Pi Imager
   • balenaEtcher
2. Boot a Pi using the newly flashed Ubuntu image and complete the setup

Automated and Manual Processes

I have an automated process available at my GitHub page: https://github.com/mytechgnome/TE-Imaging
The automation uses a JSON file with the device-specific information. You could automate the process of filling out the hostname and IP info if you have that info in an IPAM or similar platform.

How to use the vars.json file:

Below is the manual process. The overall process is the same as the automated script, but with less automated goodness. If you want to understand how the automated process works, you can read through the manual process to get a better understanding.

Mount the ThousandEyes image

Before we can customize the image we need to mount it. Run the following steps from a terminal on the imaging Pi created previously.

Customize the image

There are three files that need to be modified to get the image ready.

1. Apply the ThousandEyes Account Token
   sudo sed -i 's/<account-token>/$TOKEN/g' /tmp/temount/etc/te-agent.cfg
   • Note: replace $TOKEN with your token value.
   • Follow these steps to get your account token:
     • Open a web browser and navigate to https://www.thousandeyes.com/
     • Log into your account
     • Click the Hamburger icon in the top left
     • Expand Cloud & Enterprise Agents
     • Click Agent Settings
     • Click the Add New Enterprise Agent button
     • Click the eye button to show the token, or the copy button to store it on the clipboard
2. Set the hostname
   sudo sed -i 's/tepi/$HOSTNAME/g' /tmp/temount/etc/hostname
   • Replace $HOSTNAME with your desired hostname
3. Configure a static IP (Optional – by default the appliance will use DHCP)
   sudo sed -i 's/dhcp/static/g' /tmp/temount/etc/network/interfaces
sudo echo address $IP >> /tmp/temount/etc/network/interfaces
sudo echo netmask $MASK >> /tmp/temount/etc/network/interfaces
sudo echo broadcast $BROADCAST >> /tmp/temount/etc/network/interfaces
sudo echo gateway $GW >> /tmp/temount/etc/network/interfaces
sudo sed -i 's/#DNS=/DNS= $DNS/g' /tmp/temount/etc/systemd/resolved.conf
   • These values will need to be updated accordingly: $IP $MASK $BROADCAST $GW $DNS
   • A second DNS server can be added by simply adding both addresses with a space between them
4. Add an SSH key (Optional)
   sudo echo $SSH >> /tmp/temount/etc/ssh/keys/thousandeyes/authorized_keys
   • You can generate an SSH key with the following command: ssh-keygen -b 2048 -t rsa
   • Copy the contents of the key file and put it in place of $SSH
5. Unmount the image
   sudo umount /tmp/temount

Flash the image to a microSD card

This is the easiest part, but also the most time consuming. After plugging in the SD card, find the device path for it by running this:

sudo fdisk -l | grep sd

If the SD card is already has partitions on it the output will show each partition, so if you see /dev/sda1 and /dev/sda2 that’s showing the partitions. The image file contains its own partitions, so it needs to be written directly to the card, not into an existing partition. To do that just ignore the numbers shown, so the destination would be /dev/sda.

To start writing the SD card run this command with the correct destination location.

dd if=thousandeyes-appliance.rpi4.img of=/dev/sda status=progress

Make sure the correct destination is selected, otherwise you might overwrite something you don’t want to lose. It usually takes about 10 minutes to write the SD card. When it’s finished, try booting a Pi using the SD card and it should come up with the correct hostname, IP, and account token.

All set!

After booting the Pi you’ll want to log in and change the local admin username and password

Default username: admin
Default password: welcome

After that, you can double check the setup wizard to make sure everything is good to go. The ThousandEyes agent will reach out to their registration servers and it will use the account token to get assigned to your account. The agent should be online in the portal within a few minutes.

BONUS Fun!

I’m not going to go through all the possible code examples of what you can do with this, but I thought I’d throw out a few things to think about.

• Make API calls into an IPAM to get the site name and IP info, then populate the vars.json file automatically with that data.
• Flip the script – this process builds the site-specific data into the image. Another option would be to use a generic image (still embedding the token and SSH key) and use DHCP addressing. Then, query the ThousandEyes API to find new agents named “tepi” and then remotely update the IP and hostname that way
• A unique SSH key can be added to each image by moving the SSH key value under the devices in the vars.json and moving the SSH section of the script under the per device section, along with changing the variable to use the correct location from vars.json.
• Similarly, you could also create new, unique SSH keys by including the keygen process in the script. Just make sure you keep a copy of the keys somewhere safe!

That’s it! Overall, the process is pretty easy, and far easier than finding Raspberry Pi 4s in stock anywhere… If you use this process to automate the ThousandEyes Raspberry Pi agent image customization, I’d love to hear how it worked for you and what, if any, changes you made to the process. You can add a comment here,