Backgroound Image

ThousandEyes Walkthrough Part 1 – The What and the Why

This post will go over what ThousandEyes is, and why you should be interested in learning how to use it. To see all the posts in this series expand the box below.

ThousandEyes Walkthrough Table of Contents

There are some behind-the-scenes posts that go into more detail on how and why I took the approach that I did. Those can be found here:

 What is ThousandEyes?

I’m not in marketing, so I’m going to avoid all the “founded in” type stuff (if you want to read that stuff check out the ThousandEyes site: https://www.thousandeyes.com/about/) Instead, let’s talk about what it means to IT professionals, and more specifically network engineers.  ThousandEyes is a monitoring tool (I know, one of many, but hear me out) that takes a different approach to monitoring.  We’re all familiar with SNMP monitoring.  Links go up, links go down.  The problem with this sort of monitoring is … well, it sucks for actual performance monitoring.  Sure, I can see the packet rate of a port.  I can use Netflow to look at what type of traffic it is.  None of this actually tells me how that link, or more importantly the service that uses that link, is performing.  More importantly still, what the end-user experience is of that service using that link.

I’ll get more into how ThousandEyes operates shortly, but before that let’s take a look at why we care about it.

Why ThousandEyes? 

“It’s slow” 

I think it’s safe to say those two words are possibly the most annoying words to hear as an engineer.  They are subjective, and often backed with little data.  I can’t look for “slow” in SNMP logs.  These types of issues typically result in spending hours looking at different interfaces, running tests, and often end with a shrug of the shoulders and either saying it’s a transient issue, or it’s on the other side.

“It’s a network problem”

There’s a phrase that can instantly raise the blood pressure of any network engineer.  Again, this statement is often followed with no useful information.  After that phrase is uttered the full weight of a Priority 1 outage is squarely focused on the network team, and now they shoulder the burden of proof before anything else happens.  I’ve had issues drag on for months because people believed, without evidence, there was a network problem, and no matter what I provided, it wasn’t enough.

I’ve often referred to the Internet as the Wild West.  Once traffic leaves the network I manage I lose visibility over it.  Tools like Netflow and SNMP no longer help.  I can’t leverage things like QoS to prioritize my traffic.  Instead, I leave it to the magic of TCP to make sure the traffic gets to the destination.  I’ve lost count of the number of calls where I’ve said “I see the traffic egress our perimeter, and it looked fine.” and similar statements.

I could go on, and on, and on.  I’d wager most network engineers have had similar experiences.

Enter ThousandEyes

With ThousandEyes we have a tool that helps quickly determine if something is slow, and where that might be occurring.  This moves the conversation from the realm of subjective user experience and wild accusations to objective, proactive detection of potential issues.  This is done through the use of Agents and Tests (more on those in a future post).  By running tests we can see hop-by-hop what is happening with that traffic, and most importantly, we can see it through networks that we don’t own.  

What’s the objective of this blog series?   

 The target audience is primarily network engineers, but application developers, server administrators, and countless other people in the IT field would benefit from knowing what this tool can do.

I’ll be building out a virtual lab topology and running ThousandEyes inside it to show what the tool is capable of.

My goal is to show that the tool is incredibly easy to use and powerful.  Over the years I’ve had plenty of vendors talk about how great their product is.  Every vendor thinks whatever their product is will be the greatest product ever.  I’ve watched sales reps move from vendor to vendor, and each new place happens to have the best widgets and gizmos.  No sales pitch here.  Just an IT guy that actually thinks this is an awesome tool, and it would be a great addition to most environments.

What’s Next?

In the next installment of this ThousandEyes Walkthrough series I’ll be detailing the lab environment that I will be using for testing.  Everything will be done using, VMware Workstation, CML, Windows and Ubuntu guests, and a Raspberry Pi for fun.  I’ll provide full configs so you can build out a similar environment.  The lab will include BGP, DNS, and web servers to allow different types of ThousandEyes tests to be configured.
-Spoiler Alert-
Here’s what I’m working on for the lab build:

Windows 11 VM deployment in VMware Workstation with TPM enabled

Windows 11 requires TPM, which adds some complexity to deploying it in VMware Workstation.  The good news is adding TPM to a VM is pretty simple.  Here’s how to get it working:

  1. Download the ISO from Microsoft: https://www.microsoft.com/software-download/windows11
    1. Go through the steps under the “Download Windows 11 Disk Image (ISO)” section
  2. In VMware Workstation create a new VM
    1. Select the Typical option
      1.  
      2. Click Next
    2. Select “Installer disc image file (iso)” and then browse to the Windows 11 ISO file
      1. Click Next
    3. Select the OS
      1. Select Microsoft Windows, and in the Version select Windows 10 and later x64
      2. Click Next
    4. Enter the VM Name and specify the location if not the default
      1. Click Next
    5. Set the hard drive configuration
      1. Windows 11 requires at least 64GB.  Make sure the minimum is set to at least 64GB
      2. Click Next
    6. Click Finish
    7. Enable TPM
      1. Right click on the VM and select Settings
      2. Click the Options tab at the top of the Settings window
      3. Click the “Access Control” option, and then click Encrypt
        1. In the window that pops up enter and confirm an encryption password
          1. Click Encrypt
        2. Click the Hardware tab to return to the hardware settings
        3. Adjust the CPU and NIC settings if needed
        4. The memory must be set to at least 4GB RAM
        5. Click the Add button
        6. Click “Trusted Platform Module”
          1. Click Finish
          2. Click OK
        7. Power up the VM
  3. Follow the on-screen instructions to complete the OS installation.
That’s it.  Windows 11 is now running as a VM with TPM enabled.

CCNP Data Center – DCIT 300-615 Exam Experience

 To finish off my CCNP: Data Center certification I successfully passed the DCIT exam.  While preparing for the exam I found there is very little preparation material outside of the official Cisco course.

I recently posted about my experience with the DCCOR exam, and in that covered my background and some of the general information on this set of exams.  You can read more about it here: https://www.mytechgnome.com/2022/02/ccnp-data-center-dccor-350-601-exam.html

About the Exam

The exam topics can be found here: https://learningnetwork.cisco.com/s/dcit-exam-topics

I went over the specific technologies in the DCCOR exam review, but I’ll quickly touch on them here as well.  The exam is specifically focused on troubleshooting the various technologies in each of these areas

Network (25%)

This section covers L2/L3 technologies including vPC, LACP, STP, OSPF, BGP, PIM, and FHRP (mainly HSRP).  Overlay protocols are also included, primarily VXLAN, but OTV is in there as well.  Of course ACI is also covered here.

Compute (25%)

Under the Compute category it’s exactly what you’d expect to see.  Lots of UCS, and though it doesn’t specifically call out HyperFlex or Intersight I think it’s fair to assume that those would be in scope.  Remember, Cisco states in the exam objectives that “The following topics are general guidelines for the content likely to be included on the exam. However, other related topics may also appear on any specific delivery of the exam.” 

Storage (15%)

Again, nothing unexpected here.  There’s fibre channel and FCoE, and you need to be able to troubleshoot issues that can come up with those protocols.  One specific call out here though is storage is 15% of the exam, but only has one major subtopic.  That leads me to a presumption that there will be a significant number of FC questions compared to something like vPC.

Automation (15%)

Like the Storage section, there’s only two subtopics for 15% of the exam.  It seems sensible to be very familiar with EEM, scheduling, and automation tools.

Management and Operations (20%)

In the final section the topics include the firmware management and security topics that were covered in DCCOR under different parent topics.  This topic feels a little more like a grab bag, being worth 20% and having the most subtopics.

Exam Prep

The biggest resource used for this was the preparation work I did for the DCCOR exam.  Both exams cover the same technology areas, but the DCIT is strictly on troubleshooting, instead of understanding how to use each piece.
As mentioned earlier, the only resource I could find on the exam was the Cisco training course.  Well, that’s not entirely true.  Google was more that willing to provide links to brain dump sites, and I found a set of practice tests on Udemy.  Unfortunately the tests on Udemy appear to be from the previous DCIT v6 300-180 exam, with questions on topics that are no longer in the exam objectives.  I suspect the author just took the same exam they wrote for the 300-180 and renamed it for 300-615.
I ended up purchasing the e-learning bundle from Cisco Learning Network: https://learningnetworkstore.cisco.com/on-demand-e-learning/troubleshooting-cisco-data-center-infrastructure-dcit-v7.0/ELT-DCIT-V7-023907.html mainly for the labs offered.  There are 30 guided labs included, with access to ACI, UCS, MDS, and Nexus hardware.  I found this much more useful than working with the limitations from the UCS Emulator, ACI Simulator, CML, dCloud, and DevNet.  Plus it was less expensive than looking at rack rentals, and it was cheaper and easier than searching eBay for all the hardware to build my own lab.
The video training in the course was pretty good.  Much of it was review from the DCCOR material, but that’s not a bad thing.  It helps to reinforce that learning and to use that understanding to help in the troubleshooting process.
The labs were decent.  I appreciated that they were broken out with your objective, and had a button to see the steps to work through getting the correct solution.  Often I found I took a different path to get to the answer, but I was usually able to figure it out on my own.  I would have liked to have seen more potential scenarios though.  Usually each topic area only had a few misconfigurations.  With some topics there wasn’t a lab at all.  There were no labs for anything under the Automation or Management and Operations topics.  
Within the lab you have full access to the hardware, which allows you to create your own scenarios to some extent.  One example would be EEM, a topic clearly listed in the exam topics, but there’s not a lab for it.  Since there’s not a scenario for it I tried to create my own.  It’s very difficult to build a troubleshooting problem for yourself because you likely know where the issue is when you set it up.  What I did instead was to create a scenario that I knew would not work, but then I would look at different show commands to see how it worked.  Sticking with the EEM example, here some of the questions I worked off of:
  • What happens if you create multiple events, but make a mistake in the tagging?
  • What happens if you don’t add event-default when you need it, or do add it when you don’t?
  • What happens if the event is based on CLI entries, but a user doesn’t type the full command, or uses an alias?
The idea was to work in reverse.  I know the problem, but I am learning how that problem presents.  This can be fun coming up with all the different scenarios of how something could break, and then seeing how that shows up.
The same process can be used for other areas too.  Here’s a few questions to work with on the storage side:
  • What’s the output of show flogi database or show fcns database if
    • a zoneset isn’t active?
    • a zone isn’t part of the active zoneset?
    • a port is assigned to the wrong VSAN?
    • a port mode is incorrect?
    • a WWN is incorrect in a zone config?
    • the FCoE VSAN to VLAN mapping is incorrect?
  • Using the above examples, are there other show commands that would be useful?
The more you know about how problems present themselves the easier the exam will be.  Of course, these are only a few examples what you could encounter on the exam.  The more you build out the better off you’ll be.

Exam Experience

Unfortunately Cisco has removed the simulation questions from the exams.  The TSHOOT exam was one of my favorite tests because it was heavy in the simulation side.  The sims had their share of issues, but I prefer them over the rote memorization of command syntax.  That said, the DCIT exam is the typical multiple choice type exam.  There might be some multi-select or drag-and-drop for some added flair, but no more sims.
Overall I thought the exam wasn’t bad.  Most of the questions were clear, but there were some that had me scratching my head.  I had a few questions that waded into the rote memorization world, and some that seemed very unlikely scenarios.  There were a few questions that were fairly complex, and took some work to find the right answer.
Since the exam is focused on troubleshooting I found it to be easier to work with than the DCCOR exam.    The DCCOR topic list is longer, and it includes verbs like Describe, Apply, Analyze, Implement, Evaluate, and Explain.  The DCIT had one verb, Troubleshoot.

Final Thoughts

I wish the exam had sim questions, but even without, I prefer the scenario-based questions.  I’d like to see less rote memorization and edge case questions, and more focus on things that are more realistic to face in the real world.  Given that though, I think the exam was fair.  If you have a good understanding of how these technologies work, and what happens when they are misconfigured then this exam shouldn’t be too bad.

CCNP Data Center – DCCOR 350-601 Exam Experience

 I recently passed the DCCOR exam, and since it’s a difficult test I thought I’d share my experience with it.

My Background

Before getting into my preparation and exam experience I’m going to provide some background about the knowledge I had coming into this.  Everyone has different skills and experiences, and that’s going to have an impact on how they prepare.  The exam is broken down to five topic domains, so I’ll use that format.

Network

I’ve worked in the IT for about 20 years, and I already have my CCNP:EN (converted from the old CCNP:RS).  The L2 and L3 topics were mostly review, but the overlay technologies were new for me.  I also came in with very little practical ACI knowledge.  I’ve worked in environments where ACI was deployed, but I wasn’t responsible for maintaining it.

Compute

This is an area I felt fairly comfortable with initially.  I’ve worked with UCS off and on for over seven years.  I’ve deployed and managed blades and rack servers, as well as multiple HyperFlex environments.  I’ve also worked a bit with Intersight.

Storage

For a relatively brief period in my career I did a lot of SAN deployments, so I was somewhat familiar with the overall storage concepts.  I stopped working with fibre channel maybe five years ago and had worked mainly with iSCSI connectivity or virtual SAN environments since then.

Automation

I have the DevNet Associate certification, so I’m at least passably familiar with automation.  However, it’s not something I do day-to-day.

Security

In IT Security is part of everything we do.  I’m well versed in the overall security concepts, but I rarely go in and deal with RBAC or similar settings.  Usually that was something where the roles were configured when a platform was deployed, and I could go months without needing to make any changes aside from adding users to the correct groups.

About the Exam

The exam topics list can be found here: https://learningnetwork.cisco.com/s/dccor-exam-topics
First off, I want to say that this exam is massive.  If you compare it to what the CCNP:RS exams were I think that can illustrate my point.  In the old 300-101 ROUTE exam 40% of the exam was on routing.  The remaining 60% was split between five categories.  The 300-115 SWITCH exam had 65% of the exam focused on L2 technologies, with the remaining 35% split between two other categories.
In contrast, the 350-601 DCCOR has 25% of the overall exam on Networking.  In that 25% it includes OSPF and BGP (covered in the previous ROUTE exam), and L2 topics like LACP, and STP.  However, the DCCOR also includes PIM, vPC, overlay protocols (OTV and VXLAN), and ACI.  Just the breadth of topics in the Network topic makes this exam feel bigger, and harder than the previous CCNP:RS.
The remaining 75% of the exam is comparatively simple.  The Network portion is 25% of the exam, but contains 10 subtopics.  The remain 3/4 of the exam is a total of 16 topics.  By topic area, I would have expected the Network portion to be closer to 40% of the exam.
The Compute portion of the exam covers the UCS servers, HyperFlex, Intersight, and general management of a compute environment.  From a topic standpoint nothing unexpected.
Moving to the Storage section of the exam it is, like the Compute section, pretty straight forward.  I will admit that I was surprised the topics didn’t mention iSCSI at all.  Aside from that the Storage is what you’d expect.  It covers how fibre channel works, zoning, NPV/NPIV, VSAN, etc.
Automation is another topic that seems easy, but can get really broad quickly.  Though the topic list doesn’t go into the details I think it’s fair to assume that when it lists things like REST API that means you will need an understanding of how an API would be used to manage any of the hardware covered in the exam.
Lastly, the Security is focused heavily on the AAA and RBAC configurations for the different technologies covered in the exam.  Additionally, there are some more specific subtopics for each technology type.
One final note on the exam topics – Cisco has a line on that exam topics page that I think is important and often overlooked by test takers “The following topics are general guidelines for the content likely to be included on the exam.” Which means that things like iSCSI could actually appear on the exam.  Even though the topic isn’t specifically called out, it is a storage networking protocol that has seen wide adoption, and that means it could appear on the exam. 

Exam Prep

I started a blog post about the resources I used for the exam here: https://www.mytechgnome.com/2021/12/ccnp-datacenter-journey-dccor-350-601.html
I started with the Cisco Press Official Certification Guide, and I think it’s an awesome resource.  Unfortunately, I struggled with reading it and I literally found this book was putting me to sleep.  (That’s more a me problem than a book problem).  I found it worked better for me to use it as a reference for specific topics instead of trying to read it cover-to-cover.  One other thing I realized early on is the DCCOR exam replaced the CCIE written exam.  I found myself questioning if the depth covered in the book was for the benefit of the DCCOR exam, or as a resource for the CCIE lab (spoiler alert – it is for the DCCOR).
When I was having difficulty getting through the book I switched to the INE material.  Their video training is great, but incomplete.  At least at the time I went through it the HyperFlex and Automation sections are listed as “Coming Soon”.  It also seems like the Storage section was just duplicated from the previous CCIE:DC training.  I thought it was very hands-on like what I’d expect when preparing for a CCIE lab, but it seemed to focus more on the configuration requirements than I would have expected would be needed for the DCCOR.
I switched over to the CBT Nuggets training to compliment what INE covered, and to fill the gaps in what INE hadn’t published.  There were a number of things covered in the CBT Nuggets material that wasn’t covered in the INE material, so I was happy I went through it as well.

Labs

Books and video training are great, and they have their place, but the fun part of learning new stuff is to actually do it!  To prepare for the exam I used a few different tools provided by Cisco.
For the Networking topics CML is nearly perfect.  With CML you can create labs to cover most of the network topics.  I created maybe ten different lab environments and built out a bunch of different configurations:
  • Switching: STP, vPC, LACP, HSRP, VRRP
  • Routing: OSPF, BGP (iBGP and eBGP), PIM
  • Overlay: OTV (CML can’t do OTV on the data plane so you can’t actually pass traffic), VXLAN
I also did combination labs.  For example, create a BGP and OSPF “provider” network, and run VXLAN over it.  Then in the “provider” environment I would enable or disable multicast and adjust the VXLAN deployment accordingly.  I would also configure anycast gateways in each network.  It’s also worth pointing out that you can enable the NX-API in CML, and run automation tools against it.  That’s a great way to practice both the automation skills as well as the networking skills.
To get hands-on with ACI I primarily used the ACI simulator.  With the simulator I was able to do a large deployment with three APICs, as well as two spine and two leaf switches.  I was able to go through and build out EPGs, contracts, domains, etc.  One thing that I found particularly cool with the ACI simulator was that I could SSH to spine and leaf switches.  This allowed me to get hands-on with the CLI of the underlying hardware.
On the Compute side, I used the UCS emulator.  This allows you to run UCS manager and create virtual UCS hardware and policies.  Like the previous tools, the UCS emulator also allows CLI access to the virtual FIs.
The storage portion of the exam was the most difficult to actually get hands-on labs for.  There are some labs available through the DevNet Sandbox and dCloud that have MDS switches, but the labs are limited in what they can do.  The good news is that comparatively the commands for storage are far less than what you need to know for other topics.
With both Automation and Security all of the above resources can be used.  DevNet also has a lot of good resources for automation.  The security side is mainly just being familiar with how accounts and roles are created and the configuration of RADIUS/TACACS connections.

Exam Experience

I took the exam remotely, which generally has been a positive experience.  There are a few of tips that I can provide for the online exam.  First would be to go through the pre-check to make sure everything works as expected.  I used a laptop with an external monitor, and I found that it defaulted to the laptop webcam (with the laptop lid closed this didn’t work), but there’s a dropdown to select the webcam so it was an easy fix.  Make sure you have good audio.  The proctors often do PC-based calls, and I found them difficult to hear using my monitor speakers.  Switching to the laptop speakers solved that problem.  Also I recommend installing the Pearson app when checking in.  I had numerous issues using the web-based tool.  It seemed like when trying to upload images they were blurry in the web app, and when I switched to the mobile app the issues were resolved.
Cisco has an NDA all test takers have to accept, so I’m going to be very vague in what I say about the actual exam.  The first thing is I will reiterate that this exam is broad.  For each topic area (ACI, Nexus, UCS, MDS, etc.) be familiar with all of the different tools used to manage them.  That means GUI, CLI, and API.  Make sure you are paying close attention to the exam topics, and you have a firm grasp on each and every topic item.
I did get some questions that seemed to be poorly worded, and a couple left me confused on what specifically was being asked.  In some cases it was just be the way that I read the question, and taking a moment then rereading it and it made more sense.  In others, it didn’t matter how many times I read it, it just didn’t make sense.
One other trend that I’ve found with exams is the tendency to find the most obscure question possible.  As engineers we have access to context sensitive help.  Preparing for the exam I watched CCIEs extensively use the “?” to find the syntax that was needed, or look for a config option in the wrong area.  I find it frustrating when on an exam there are syntax questions that are needlessly specific.  An example would be something like if something is measured in bits, kilobits, bytes, kilobytes, etc.  Another example would be if a timer is configured in seconds or milliseconds.  If these situations ever came up, I’d have access to tools without needing to memorize the plethora of commands and options.  To make matters worse, often when these questions come up they are on commands that are rarely used.  This exam is no different.  Make sure you are committing the exact syntax of commands to memory.
On my first attempt I failed, and looking at the score report I’m guessing it was close.  The passing score and received score aren’t provided anymore, but the percentage per topic is displayed. Not surprisingly, the areas I performed the worst in were Storage, Automation, and Security.  Those areas were where I had much more general knowledge initially, and it showed.  However, after having actually taken the exam I knew where I was weak, and the level of depth on the exam.  I went back and reviewed the Network and Compute topics, and spent more time digging in to the areas I needed to improve.  When I retook the exam I was able to pass the exam.

Final Thoughts

This exam is tough.  I can confidently say it’s one of the most difficult exams I’ve ever taken.  It’s a mile wide, and it can also get quite deep.  There are plenty of challenging questions that really test your knowledge.  With enough preparation and practice it is something that can be accomplished.

CCNP Datacenter Journey – DCCOR 350-601 and DCIT 300-615 – Resources

 

I’m working towards attaining the CCNP: Data Center certification.  I’ll be compiling a list of resources and tips that I used along the way.

Software

VMware Workstation Pro – https://store-us.vmware.com/workstation_buy_dual_new

Cisco Modeling Labs – https://learningnetworkstore.cisco.com/cisco-modeling-labs-personal/cisco-modeling-labs-personal-plus/CML-PERSONAL-PLUS.html

UCS Emulator – https://software.cisco.com/download/beta/1850014776

ACI Simulator – https://software.cisco.com/download/home/286283149/type/286283168/release/5.2(1g) (If you don’t have access to the software you can work with a Cisco SE to grant you access)

Use this command in Linux to merge the downloaded files into a single OVA to import into VMware Workstation

cat acisim-5.2-1g_part1.ova acisim-5.2-1g_part2.ova acisim-5.2-1g_part3.ova acisim-5.2-1g_part4.ova acisim-5.2-1g_part5.ova acisim-5.2-1g_part6.ova > acisim-5.2-1g.ova

Material

Cisco Press Official Cert Guide
INE CCNP:DC training course
CBT Nuggets CCNP:DC training course

I Passed the Cisco DevNet Associate exam and Joined DevNet Class of 2020!

 I’m excited to announce that I passed the DevNet Associate (200-901) exam, and with that I’ve joined the DevNet Class of 2020!

 To start with, for those that don’t know, DevNet is the Cisco Developer Network, focused around developing solutions in the network space.  It focuses heavily on programability and automation of numerous Cisco products.  The DevNet Class of 2020 includes everyone that passes a DevNet exam during the inaugural year of the program.  Originally, the program was slated to end December 31st of 2020, but it was extended to February 24th, 2021.

I found this exam to be simultaneously one of the most challenging and fun certifications I’ve attempted.  With a near 20-year career in IT I’ve never really done much programming.  I’ve made a few HTML sites over the years, and the odd batch or PowerShells script, but never anything more than that.  In many ways this exam broke into a lot of new areas for me.  For network engineers looking to get into automation I thought this was a great way to start, and for people new to IT this is a great way to get into the automation and programmability mindset early on.

How I prepared

First things first.  Learning Python.  Coming from the network background this took some work, but it really wasn’t too bad.  I started with some YouTube videos and books.  One site specifically that I used a lot was automatetheboringstuff.com, as well as the YouTube videos from the same author.

I found the repetition of the labs got boring after a while, so I started to look for beginner projects.  One project that I worked with was a Python clone of the classic Pong game.  However, instead of just duplicating the code I worked on adding additional functionality.  Players could enter their names, and select their paddle colors, as well as set the game speed and score limit.  I added some input validation to make sure the entries didn’t cause the game to crash.  For me, it was important to actually work with the code and play with the options instead of simply copying what someone else said.

Once I felt I had a decent handle on Python I started reading the DEVASC 200-901 Official Cert Guide, which of course hit a lot of the same Python info I was working with already, but added depth.  The book goes into a lot of other things like Git and API configs.  Which, of course, meant getting Git set up and testing committing, branching, and merging code.

The Cisco DevNet site has access to sandboxes that can be used to test out API calls.  Since I don’t have DNA Center, ACI, Meraki, Webex, FMC, etc. all running in my basement it was really good to have access to the sandbox.  I worked through learning the API methods via curl, Postman, Python, and SDK.  This meant a lot of repetition.  The authorization methods between the Cisco platforms changes, and that means the way you interact with the API needs to change.

Looking back, I wish that I had merged the Git exercises more with the API work.  I could have built out a repository of all the tests I was working with.  So, as a recommendation, use Git early, and get in the habit of using it.

I also watched the Pluralsight videos by Nick Russo.  Personally, I found those difficult to follow.  Coming in to programming fresh, there was a lot that I felt was skimmed.  This meant I spent a lot of time pausing videos to duplicate scripts.  There’s a bunch of files attached to the courses, but I felt it was important to actually write the code.

In addition to Python, Git, and APIs, you also need to know the different data formats.  The main ones would be YAML, JSON, and XML.  Again, not coming from a programming background this was another stumbling block for me.  The different sources I used all covered this, but it took some work to really understand it.  It came down to just going over the formatting and syntax a few times until it really made sense.

If that wasn’t enough to learn, there’s also the automation frameworks.  Things like NETCONF, RESTCONF, YANG, Ansible, Puppet, Chef, NSO, etc.  More to learn.  More terms.  More syntax.

But wait, there’s more! Docker and VIRL/CML.  Learning about the tools to build environments programmatically, and how to make them work.  Yet more terms and syntax.

The exam also covers the software design methodologies.  Things like Agile, Lean, Waterfall, etc.  The DevOps ideas.  Testing methods.  Luckily, no syntax, but more terms.

The final topics were (for me) the easiest.  Basic network security and network operations.  Things like attack types and remediation mechanisms, subnetting, and other layer 2/3 functionality.  Since this is stuff I’ve spend years working with these topics were a breeze.  However, for someone new to network operations the process of learning the layer 2 and layer 3 configuration can be a bit more complicated.  Luckily, this isn’t the CCNA.  You don’t need to configure STP or OSPF.  You just need to know what the terms mean.  If you can articulate what a switch and router do, what the OSI model is, and you understand how subnets work then you should be OK.

Thoughts on the exam

As I’d mentioned at the beginning, I found this exam to be both fun and challenging.  There were questions where I stared at the screen slack-jawed trying to understand what was being asked.  Often, those questions were ones I was overthinking, and after a brief befuddlement I figured it out.  This isn’t because the questions were poorly worded.  In fact, it was the opposite.  I thought the questions were well written, but since some of the concepts are still new to me it took a moment to really wrap my head around it.
I think the exam was fair, and it asked good questions.  I didn’t feel like there were any trick questions, or things that were intentionally misleading.  There were definitely some challenging questions, but they seemed fair and I felt like I should have known the answers.  Without giving anything away, I’ll just say that since this is a technical exam knowing the terms, acronyms, and syntax for all of the topics is important.

Final notes

I want to reiterate that I thought this was a great exam, with really good content.  Whether we network engineers want it or not, network programmability is going to be a thing.  Think back to the people that wanted to maintain a PBX instead of moving to VoIP, or the adoption of virtualization.  These shifts take time, but they are happening.  As difficult as some of this was for me to learn, I’m glad I did.  Comparing this to many of my other certifications, this one really feels like there’s a ton of value and I gained some useful skills preparing for it.

vSphere Lab Build Out – The ESXi Server Deployment

Finally, after getting the domain controller and client built now it’s on to actually deploying ESXi!

You can download eval copies of VMware software directly from VMware (after creating an account) here: https://my.vmware.com/web/vmware/downloads/info/slug/datacenter_cloud_infrastructure/vmware_vsphere/7_0

You can also register for the VMware User Group’s VMUG Advatage program and get access to 365-day trial licenses.  More info on the VMUG Advantage program can be found here: https://www.vmug.com/membership/vmug-advantage-membership

After getting the ESXi ISO downloaded we can start the VM builds

VM Creation

  1. In VMware Workstation press CTRL+N to open the New Virtual Machine Wizard, and make sure Typical is selected, then click Next
  2. Select the ISO that was downloaded and click Next
    1. I’m still using Workstation 15.5, and installing ESXi 7.0, so it doesn’t autodetect the OS
  3. Select VMware ESX, and in the drop down select VMware ESXi 6.x, then click Next
  4. Name the VM, and change the location (if needed) then click Next
  5. The default storage of 40GB is more than enough.  The hosts will be configured to connect to an iSCSI target, so leave the default and click Next.
  6. Click Customize Hardware
  7. Configure the hardware.  
    1. Since these will be the virtualization hosts allocate as much CPU and RAM as you can. 
    2. When setting the CPU count verify the box is checked for Virtualize Intel VT-x/EPT or AMD-V/RVI
    3. Set the NIC to Bridged
    4. Click Close
  8. Click Finish

ESX installation

  1. Power on the VM
  2. The VM should automatically boot from the ISO, and it will take a moment to load everything
  3. When prompted to start the install press Enter to continue.
  4. Make sure to read the EULA in its entirety, and if you accept it press F11 to move on.
  5. By default the hard drive created with the VM will be present and selected.  Press Enter again to continue.
  6. Select the correct keyboard layout, and again press Enter to continue.
  7. Type in a password, and then press Tab to re-enter the password to confirm it.  Then press Enter to continue.
  8. Finally, press F11 to confirm the configuration and begin the installation.
  9. The install is fairly quick.  When it is done press Enter to reboot.
  10. It will take a little while for the VM to reboot and come back up.
Initial ESX configuration
  1. At the main screen press F2 to Customize System.
  2. Enter the credentials and press Enter 
    1. Remember, the username is: root
    2. The password is what was entered during the initial set up
  3. In the System Customization window use the Down Arrow to highlight (or is it blacklight? lowlight?) Configure Management Network and press Enter.
  4. Select IPv4 Configuration and press Enter
  5. Select DNS Configuration and press Enter to open the DNS options.
  6. Use the Down Arrow to highlight “Use the following DNS Server…” and press Space to select that, then use the Down Arrow to enter the DNS server address information.  Use the IP of the DNS server that was created for the lab.  Also, type in the the FQDN that matches the DNS record created.  Press Enter to submit these changes.
  7. Again, use the down arrow to select Custom DNS Suffixes (I’m not adding a screenshot since there’s already two for the same page)
  8. Enter the lab domain name and press Enter.
  9. Press Escape to exit the configuration menu, then press Y to accept the changes and restart the management network.
  10. (Optional) Test the management network to verify things work.  It should return OK if everything works as expected.
This completes the configuration of the first ESXi host.  This process can be repeated to create a second host, which will allow the creation of an HA cluster.

vSphere Lab Build Out – The Domain Controller Configuration

For the Domain Controller build the entire process is much easier and quicker when working from PowerShell instead of the GUI.  It also makes it more repeatable, which is awesome for labs.

The first steps are the basic config of the server.  Below is each command needed, with the variables in red.  Change what you need, then paste the commands into PowerShell.

Set the computer name: 

Rename-Computer LabDC

Enable Remote Desktop access (optional)

Enable-NetFirewallRule -DisplayGroup “Remote Desktop”

Set-ItemProperty -Path ‘HKLM:SystemCurrentControlSetControlTerminal Server’ -name “fDenyTSConnections” -value 0 

Disable DHCP, set the IP address, DNS, and default route:

Set-NetIPInterface -InterfaceAlias Ethernet0 -AddressFamily IPv4 -Dhcp Disabled 

New-NetIPAddress -InterfaceAlias Ethernet0 -AddressFamily IPv4 -IPAddress 192.168.1.210 -PrefixLength 24 

Set-DnsClientServerAddress -InterfaceAlias Ethernet0 -AddressFamily IPv4 -ServerAddresses 8.8.8.8 

New-NetRoute -AddressFamily IPv4 -InterfaceAlias Ethernet0 -DestinationPrefix 0.0.0.0/0 -NextHop 192.168.1.1

Install the AD, DNS, iSCSI, and Remote Server Admin Tools.
Install-WindowsFeature -name AD-Domain-Services,DNS,FS-iSCSITarget-Server,RSAT-ADDS

Reboot to apply the name change:

shutdown -r -t 0

Log into the server again, and create the domain:

Install-ADDSForest -DomainName Lab.local -InstallDNS

When prompted for the AD Restore Mode password enter the password, and then confirm it.  After that, accept the prompt by pressing the “A” key and hitting Enter.  Wait, while the new domain is configured.  When the process completes the server will automatically reboot.

The final task will be getting DNS configured with a reverse DNS zone, and records created for the various devices that will be deployed.

Add-DnsServerPrimaryZone -NetworkID “192.168.1.0/24” -ReplicationScope “Forest” 

Add-DnsServerResourceRecordA -Name “ESX1” -ZoneName “Lab.local” -IPv4Address “192.168.1.211” -CreatePtr 

Add-DnsServerResourceRecordA -Name “ESX2” -ZoneName “Lab.local” -IPv4Address “192.168.1.212” -CreatePtr 

Add-DnsServerResourceRecordA -Name “vCenter” -ZoneName “Lab.local” -IPv4Address “192.168.1.213” -CreatePtr 

Add-DnsServerResourceRecordA -Name “vRO” -ZoneName “Lab.local” -IPv4Address “192.168.1.214” -CreatePtr 

Add-DnsServerResourceRecordA -Name “vLCM” -ZoneName “Lab.local” -IPv4Address “192.168.1.215” -CreatePtr

That concludes the initial DC config for the environment.

vSphere Lab Build Out – The Client PC Peployment and Config

In the VMware lab it’s nice to have a client OS to work from.  This client can be joined to the domain, and pointed to the lab DNS, which streamlines some of the config tasks. 

Get the Win10 ISO

This process is a little more involved, and it requires the use of the Media Creation Tool.  You can get that here: https://www.microsoft.com/en-us/software-download/windows10

  1. After the download completes run the tool
  2. Click Accept (after you read the full terms and conditions of course)
  3. Select “Create installation media…” and click Next
  4. Verify the options say English, Windows 10, and 64-bit
    1. If not, uncheck the box and select those options and click Next
    2. If so, click Next
  5. Select ISO file and click Next
  6. Select your download location and click Save.
The downloads might take some time to complete.

Create the Client VM

  1. In VMware Workstation press CTRL+N to open the New Virtual Machine Wizard, and make sure Typical is selected, then click Next
  2. Select the option for Installer Disc Image File, and browse to the location you downloaded the Windows 10 ISO to then click Next
  3. Enter the name for the client and select the location
  4. Use the default hard drive size of 60GB (another drive will be added later for the iSCSI target storage), and click Next
  5. Click Customize Hardware
  6. Adjust the CPU and RAM as needed for your environment (2 vCPUs 4-8GB RAM would be recommended), and change the Network Adapter from NAT to Bridged
  7. Click close, verify the box is checked for “Power on this virtual machine after creation” and click finish.

Deploy the Client OS

NOTE: While in the VM you will need to press Ctrl+Alt to release the cursor to get to your desktop
  1. While the VM is booting you might see a prompt to press a key to boot from CD.  If that happens click into the window and press a key.
  2. Select the language, and keyboard settings
  3. Click Install Now
  4. At the Activate Windows screen click “I don’t have a product key”
  5. Select Windows 10 Pro and click Next
  6. Read through all of the licenses terms, and if you accept the terms check the box to accept them and click Next
  7. Select the Custom install option
  8. By default it should already select Drive 0, which is the 60GB drive initially created.  Click next.  The OS install will start, so just let that process run.

OS Initial Config

Windows 10 has a number of steps to go through to get the OS configured before actually loading to a desktop.
  1. Select your regions and Click Yes
  2. Select your keyboard layout
  3. Skip adding the additional keyboard
  4. Wait a moment for it to progress to the account creation screen, then select “Set up for personal use” and click Next
  5. Microsoft is going to try to link to an online account, but since this is for a temporary lab PC click on “Offline account” in the bottom left.
  6. Microsoft really tries to push the online account, so again look in the bottom left corner and select “Limited experience”
  7. Enter a username and click Next
  8. Create a password and click Next
    1. The next screen will ask to confirm the password.  Reenter the password and click Next
  9. When prompted for the three security questions I just select the first three options and enter random characters.  This is lab, and if I happen to forget the password I can easily recreate the VM.  Click Next
    1. Repeat the process for the other two questions.
  10. For the privacy settings this really doesn’t matter, as it’s a lab machine that won’t exist for long.  Everything can be left enabled by default, or it can be disabled.  After applying the settings click Accept.
  11. The install will prompt to enable activity history.  Again, as a lab machine this isn’t needed, so select No.
  12. Cortana… Microsoft really wants people to enable all their stuff.  Click “Not now” to move on.
  13. Success! The post-install prompts are done.  Now wait for the configuration to complete.

Client OS config

To configure the OS there’s only three tasks that are going to be performed.  
  • Install VMware Tools
  • Configure DNS
  • Join the domain

Install VMware Tools

    1. Log into the VM using the password set previously
    2. Right click on the VM in the Library an select Install VMware Tools
    3. Navigate to the D: drive and double click it.  That should kick off the Autorun for the installer.
    4. Follow the defaults for the install.  Next > Next > Install > Finish and then click Yes when prompted for a reboot.
Configure DNS
  1. Open Powershell as admin
    1. Press the Windows key and type powershell
    2. Press Crtl+Shift+Enter to run as admin
  2. Run this command (replace the IP in red if needed):
Set-DnsClientServerAddress -InterfaceAlias Ethernet0 -ServerAddresses 192.168.1.210

Join the domain
  1. Open Powershell as admin
    1. Press the Windows key and type powershell
    2. Press Crtl+Shift+Enter to run as admin
  2. Run this command (replace the IP in red if needed), and enter the password when prompted
Add-Computer -Credential labadministrator -DomainName lab.local -Force -Restart

This completed the Client PC configuration for the lab.

vSphere Lab Build Out – The Domain Controller Deployment

When building out a lab the first thing I do is build out a Domain Controller and DNS server. I can then use AD for credential management, and the DNS functionality is helpful as well.  I also use that server to create an iSCSI target for my hosts.

1. Virtual Environment

The first step is to have your virtualization environment ready to go.  It’s easy enough to next-next-finish your way through the VMware Workstation install, so I won’t detail out those steps.

2. Download Windows ISOs

You can download the Server 2019 ISO here: https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2019

Select ISO, fill out the info required, and then hit continue.  Select your language, and then start the download.

3. Create the Lab Domain Controller VM

  1. In VMware Workstation press CTRL+N to open the New Virtual Machine Wizard, and make sure Typical is selected, then click Next
  2. Select the option for Installer Disc Image File, and browse to the location you downloaded the Server 2019 ISO to then click Next
  3. Since this will be using the evaluation license leave the product key blank, enter a name and password, and then click Next.
  4. Accept the prompt about not having a product key
  5. Enter the name and location for the VM, and click Next again
  6. Use the default hard drive size of 60GB (another drive will be added later for the iSCSI target storage), and click Next
  7. Click Customize Hardware…
  8. Set the VM hardware
    1. Set the CPU and RAM to what you’d like.  I used 2 vCPUs and 8GB RAM on my VM.
    2. Change the Network Adapter to Bridged
    3. Click Close
  9. Uncheck the box for Power on this virtual machine after creation and click finish.
  10. Now to add a the hard drive for the iSCSI target and remove the floppy drive.  In the library view right-click on the VM and click Settings
    1. Find the Floppy drive and click Remove (NOTE: If you don’t remove the floppy drive the OS install will encounter an error and fail), then click Add
      1. Select Hard Drive and click Next
      2. Leave the default drive (mine happens to be NVMe) and click Next
      3. Leave the default option to create a new drive and click Next
      4. Enter the size for the drive (I used 750GB) and click Next
      5. Leave the default file name and click Finish
    2. Click OK to finish the hardware changes
  11. Power on the VM

4. Install the OS to the Lab DC

NOTE: While in the VM you will need to press Ctrl+Alt to release the cursor to get to your desktop
  1. While the VM is booting you might see a prompt to press a key to boot from CD.  If that happens click into the window and press a key.
  2. Select the language, and keyboard settings
  3. Click Install Now
  4. When prompted to select the OS choose Windows Server 2019 Datacenter Evaluation (Desktop Experience) because we like graphical interfaces, and click Next
  5. Read through all of the licenses terms, and if you accept the terms check the box to accept them and click Next
  6. Select the Custom install option
  7. Select Drive 0, this should be the 60GB drive, and click Next
  8. Wait for the install to complete.  This might take some time.
  9. When the install is complete it will prompt for a password.  Set that and click Finish.
  10. The last thing to do for the VM deployment is to install VMware Tools.
    1. Log into the VM using the password set previously
    2. Right click on the VM in the Library an select Install VMware Tools
    3. Navigate to the D: drive and double click it.  That should kick off the Autorun for the installer.
    4. Follow the defaults for the install.  Next > Next > Install > Finish and then click Yes when prompted for a reboot.
The DC configuration will be detailed out in another posting in this series.