Backgroound Image

Month: June 2023

Automated ThousandEyes Raspberry Pi image customization

Posted on by mytechgnome

If you haven’t caught this yet, I am a huge fan of ThousandEyes! I was working on a project where we were planning to deploy ThousandEyes agents to hundreds of sites, and to keep costs down we were going to use the Raspberry Pi 4 Model B. Having to image hundreds of SD cards was going to be a nightmare, so I wanted to automate the ThousandEyes Raspberry Pi agent image customization process to simplify the deployment.

Full disclosure: I started this project just as the supply chain issues were hitting the Pi market, and I left that job a few months later. I was never able to run this at scale, but I was able to use this for the few units I was able to get my hands on.

I also have a series on getting ThousandEyes deployed in a lab environment if you want to read more on getting ThousandEyes running: https://www.mytechgnome.com/2022/03/28/thousandeyes-walkthrough-part-1-what/

Hardware/Software required

Configure Pi for imaging

I used a Raspberry Pi as the system for building the images. I found it makes the process much easier, but this isn’t a hard requirement. The main thing is you will need to be able to mount a Linux file system, and it seemed to work much better using a native Ubuntu instance instead of trying to map an SD card to a VM or WSL instance.

  1. Using your preferred tool, flash the Ubuntu image onto one of the microSD cards – these are both good options:
  2. Boot a Pi using the newly flashed Ubuntu image and complete the setup

Automated and Manual Processes

I have an automated process available at my GitHub page: https://github.com/mytechgnome/TE-Imaging
The automation uses a JSON file with the device-specific information. You could automate the process of filling out the hostname and IP info if you have that info in an IPAM or similar platform.

How to use the vars.json file:

  • Token Value – Group token from ThousandEyes portal
  • Image name – Shouldn’t need to do anything with this, unless ThousandEyes changes the image file name
  • SSH key – Leave blank if you aren’t adding a key, otherwise paste in the contents of the public key file
  • Devices – Duplicate the device objects as needed
    • Hostname – The name of the specific agent
    • IP – Assign static IP or for DHCP, leave it blank- “”
    • Subnet mask and gateway – self explanatory
    • DNS – Also self explanatory. If you’re only using one DNS server, leave DNS2 blank- “”
vars.json

Below is the manual process. The overall process is the same as the automated script, but with less automated goodness. If you want to understand how the automated process works, you can read through the manual process to get a better understanding.

Mount the ThousandEyes image

Before we can customize the image we need to mount it. Run the following steps from a terminal on the imaging Pi created previously.

  1. Download the ThousandEyes image
    wget https://app.thousandeyes.com/install/downloads/appliance/thousandeyes-appliance.rpi4.img.xz
  2. Decompress the image
    unxz -k thousandeyes-appliance.rpi4.img.xz
  3. Create a mount directory
    mkdir /tmp/temount
  4. Mount the image
    sudo mount -o loop,offset=269484032 thousandeyes-appliance.rpi4.img /tmp/temount
    • Note: Because there are two partitions, we need to mount the second partition
      • If this doesn’t work it could be caused by a change in the partition layout, use these steps to get the correct offset
        fdisk -l thousandeyes-appliance.rpi4.img
      • Here’s an example output:
        Disk thousandeyes-appliance.rpi4.img: 4.07 GiB, 4367262720 bytes, 8529810 sectors
        Units: sectors of 1 * 512 = 512 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disklabel type: dos
        Disk identifier: 0x354db354

        Device Boot Start End Sectors Size Id Type
        thousandeyes-appliance.rpi4.img1 * 2048 526335 524288 256M c W95 FAT32 (LBA)
        thousandeyes-appliance.rpi4.img2 526336 8529809 8003474 3.8G 83 Linux
      • Multiply the sector size (512) by the start block (526336) and use that value in the offset.
  5. Verify the image mounted
    ls /tmp/temount/
    • There should be several folders listed. If not, check the offset mentioned in step 4.

Customize the image

There are three files that need to be modified to get the image ready.

  1. Apply the ThousandEyes Account Token
    sudo sed -i 's/<account-token>/$TOKEN/g' /tmp/temount/etc/te-agent.cfg
    • Note: replace $TOKEN with your token value.
    • Follow these steps to get your account token:
      • Open a web browser and navigate to https://www.thousandeyes.com/
      • Log into your account
      • Click the Hamburger icon in the top left
      • Expand Cloud & Enterprise Agents
      • Click Agent Settings
      • Click the Add New Enterprise Agent button
      • Click the eye button to show the token, or the copy button to store it on the clipboard
  2. Set the hostname
    sudo sed -i 's/tepi/$HOSTNAME/g' /tmp/temount/etc/hostname
    • Replace $HOSTNAME with your desired hostname
  3. Configure a static IP (Optional – by default the appliance will use DHCP)
    sudo sed -i 's/dhcp/static/g' /tmp/temount/etc/network/interfaces
    sudo echo address $IP >> /tmp/temount/etc/network/interfaces
    sudo echo netmask $MASK >> /tmp/temount/etc/network/interfaces
    sudo echo broadcast $BROADCAST >> /tmp/temount/etc/network/interfaces
    sudo echo gateway $GW >> /tmp/temount/etc/network/interfaces
    sudo sed -i 's/#DNS=/DNS= $DNS/g' /tmp/temount/etc/systemd/resolved.conf
    • These values will need to be updated accordingly: $IP $MASK $BROADCAST $GW $DNS
    • A second DNS server can be added by simply adding both addresses with a space between them
  4. Add an SSH key (Optional)
    sudo echo $SSH >> /tmp/temount/etc/ssh/keys/thousandeyes/authorized_keys
    • You can generate an SSH key with the following command: ssh-keygen -b 2048 -t rsa
    • Copy the contents of the key file and put it in place of $SSH
  5. Unmount the image
    sudo umount /tmp/temount

Flash the image to a microSD card

This is the easiest part, but also the most time consuming. After plugging in the SD card, find the device path for it by running this:

sudo fdisk -l | grep sd

If the SD card is already has partitions on it the output will show each partition, so if you see /dev/sda1 and /dev/sda2 that’s showing the partitions. The image file contains its own partitions, so it needs to be written directly to the card, not into an existing partition. To do that just ignore the numbers shown, so the destination would be /dev/sda.


To start writing the SD card run this command with the correct destination location.

dd if=thousandeyes-appliance.rpi4.img of=/dev/sda status=progress

Make sure the correct destination is selected, otherwise you might overwrite something you don’t want to lose. It usually takes about 10 minutes to write the SD card. When it’s finished, try booting a Pi using the SD card and it should come up with the correct hostname, IP, and account token.

All set!

After booting the Pi you’ll want to log in and change the local admin username and password

Default username: admin
Default password: welcome

After that, you can double check the setup wizard to make sure everything is good to go. The ThousandEyes agent will reach out to their registration servers and it will use the account token to get assigned to your account. The agent should be online in the portal within a few minutes.

BONUS Fun!

I’m not going to go through all the possible code examples of what you can do with this, but I thought I’d throw out a few things to think about.

  • Make API calls into an IPAM to get the site name and IP info, then populate the vars.json file automatically with that data.
  • Flip the script – this process builds the site-specific data into the image. Another option would be to use a generic image (still embedding the token and SSH key) and use DHCP addressing. Then, query the ThousandEyes API to find new agents named “tepi” and then remotely update the IP and hostname that way
  • A unique SSH key can be added to each image by moving the SSH key value under the devices in the vars.json and moving the SSH section of the script under the per device section, along with changing the variable to use the correct location from vars.json.
  • Similarly, you could also create new, unique SSH keys by including the keygen process in the script. Just make sure you keep a copy of the keys somewhere safe!

That’s it! Overall, the process is pretty easy, and far easier than finding Raspberry Pi 4s in stock anywhere… If you use this process to automate the ThousandEyes Raspberry Pi agent image customization, I’d love to hear how it worked for you and what, if any, changes you made to the process. You can add a comment here,

Cisco Live 2023

Posted on by Dan Kelcher

I was lucky enough to win a trip to Cisco Live this year through the Cisco Insider Advocates program – https://insideradvocates.cisco.com/. The event was amazing and I wanted to share my experience.

The Numbers

In true geek fashion, I thought it’d be fun to share some of my Cisco Live stats.

  • 2,598 miles flown from MSP to LAS and back (yay for direct flights)
  • 6 days (Satuday-Thursday) in Las Vegas
  • 55,000 steps (roughly 27 miles)
  • 3 books signed – CCDE OCG signed by Zig Zsiga, The Art of Network Design signed by Denise Donohue, and DevNet Associate OCG signed by Jason Gooley
  • 30 selfies taken (with too many awesome people to list here!)
  • 1 exam taken
  • 1 exam failed (but I was close and now I know what I need to study)
  • 1 award received – Cisco Advocates Transformation Trailblazer
  • 3 content presentations, a presentation in the Content Corner, a session with Itential at the IP Fabric booth, and an interview with David Bombal
  • 2 MTE sessions – Jason Gooley and John Capobianco
  • 6 ribbons collected

The Event

The event itself was a great time! Plenty (ok, maybe too many) sessions to pick, tons of vendor booths, and more activities than you could shake a stick at.

I wrote about Cisco Live 2022 previously https://www.mytechgnome.com/2022/09/23/cisco-live-las-vegas-2022-recap-good/ so I won’t rewrite all the same stuff. Instead, I thought it’d be fun to review my previous post and see how much of it still applies.

The first big difference – last year I was a delegate of TFDx, and this year I wasn’t because I’ve crossed over to the dark side. I now work for a vendor, which means to avoid any conflict of interest or bias, I can’t participate as a delegate. However, since I work for a vendor that sponsored Cisco Live, I had the opportunity to hang out at our booth and talk to people.

The Good

The people. That part is absolutely 100% accurate. The people are by far the best part of Cisco Live! This year was even better because I’ve had the opportunity to meet people at the last Cisco Live, as well as meeting more people through the Cisco Insider programs.

I did the CCDE techtorial again this year, and it, again, was an awesome session!

The Bad

I tried to take my own advice this time. I didn’t attend some of the parties, and I was far more selective on the sessions I attended. I tried to focus my time on the things that were the most valuable to me, and that was mainly taking the time to talk to people. There was still a lot of walking, but so it goes.

I wasn’t planning to take an exam this year, but I was able to grab a spot Sunday afternoon, so I went for it. This year the test center was in Mandalay Bay, so at least there wasn’t the 20-minute walk to and from Luxor for the exam.

The Tech

This year I wasn’t blown away with the tech announcements. I will admit, I am extremely interested in the Full Stack Observability platform. I didn’t get a chance to look at it much, but it’s on my radar to dig into as I get time.

New Thoughts from 2023

There are a few things that are new for me this year that I wanted to dig into.

Cisco Insider program

Though I touched on the programs in my last post, the Insider team had a lot of new stuff that happened this year. As I mentioned, I won the trip to Cisco Live through the Insider Advocates program. In addition to the trip, I was surprised to see my picture in the Insiders Rockstar Hall of Fame!

I was also shortlisted for two awards in the Cisco Global Advocates Awards, and I ended up winning the Transformation Trailblazer award! I was interviewed after winning the award, and I participated in a panel interview with David Bombal. For a guy that can talk a lot, I don’t have the words to convey how awesome that was!

Moving past the ego boost, the best part of the Insiders program is the community. Through the Advocates and Champions program I had the opportunity to meet tons of amazing people. If you’re not already part of the community I strongly urge you to join.

Booth Duty

Though I wasn’t officially on booth duty, I did stop by the IP Fabric booth a few times to hang out. Again, the best part of Cisco Live is the conversations, and I was able to have some really interesting conversations with people. I’ve found there are two topics that universally resonate with network engineers – STP issues and MTU issues. Any time I asked people if they’ve had either issue (I know I’ve had my share) I could watch their soul die. Then I’d show the STP topology in IP Fabric, complete with root bridge and blocked ports, and I could watch the souls spring back to life. I’m not trying to make a sales pitch here, though I probably should. This just another example of how important the conversations are.

Recommendations for Future Attendees

Yes – I just copied this entire section from my previous post because it’s still accurate.

  1. Wear good shoes. It’s a lot of walking! I think I calculated something like 30+ miles of walking during the week.
  2. Bring a water bottle and stay hydrated. With all the walking, the Vegas heat, and the overall dryness, it’s easy to get dehydrated. Add in air travel and perhaps some alcohol consumption, and that’s a recipe for disaster. There are plenty of water coolers, but sometimes it was challenging to find one that wasn’t empty. Bring a water bottle, fill it when you can, and make sure to drink enough water.

Now that the basic human needs are covered, on to the actual conference recommendations.

  1. Make time to talk to people. Sessions fill fast, making it feel like you need to register for as many as possible. Don’t fall into that trap. Sign up for the sessions you really want to attend, and then use the open time to talk to people. Most sessions are recorded, but the chance to talk to people isn’t.
  2. Don’t be afraid to talk to someone. See your favorite blogger, podcaster, and beard? Go ahead and say “hi” to me. And if I’m not your favorite, that’s fine. Say “hi” to me, and them too. Did you hear someone ask a question in a session, and it sounded like they might be in a similar position to you? Talk to them. Maybe they’ve solved a problem you’re working on. Maybe you have some advice you could give them.
  3. Don’t focus on the parties. Sure, they can be fun, but after an 8+ hour day of talking and tech sessions, if you need some downtime, take it. Maybe doing back-to-back-to-back 16-hour days for a week is something you can do, and if so, go for it. If not, that’s cool. The parties, swag, and all of that are great, but if you risk burning yourself out, make sure to pace yourself.
  4. Swag is great. Prizes are also great. Neither should be the focus of a trip to Cisco Live. The cost of the conference pass, hotel, and airfare far outweigh the value of the swag. There’s always joking about finding the vendors with the best swag, but really, look for the vendors that can help you. Talk to them. Talk to vendors you’ve never heard of. Maybe they have a product that can solve a problem you have, and you didn’t even know it existed. If it ends up not being a good fit, move on. There are plenty of vendors for a participant to talk to and plenty of participants for vendors to talk to, so if there’s no value, then it’s better for both of you to move on.
  5. If you need approval to make the trip, highlight how Cisco Live is a lot more than sales demos and swag. You have the opportunity to meet a lot of people and learn about what they are doing. The odds are pretty good that you can find people that have solved whatever challenge you might be facing or at least people that could provide useful information.
  6. (New recommendation) Book MTE sessions! I didn’t book any in 2022, and I have realized that was a huge mistake! I booked two sessions with people I’ve followed for years (Jason Gooley and John Capobianco) and I am very happy I did! They are both quite popular, and being able to have them alone in a room for 45 minutes… uh… in a non-creepy way… is fantastic! Again, the people and conversations are the best part of Cisco Live, and being able to get 1-on-1 time with people that you follow is massively beneficial! You can jump straight into whatever topic you want, and there’s no sales presentation. Just a couple geeks chatting about geek stuff.

TLDR: Go to Cisco Live. Talk to people. Enjoy.