Saturday, February 1, 2020

Why Automate, Part 1: Network Config Templating in Jinja2

Let's answer the big question: "What's the answer to the ultimate question of life, the universe, and everything?"

Kidding, it's easier to cover the question: "Why automate?"

So let's get started! Here I'm going to start a few easy and quick ways to benefit from automation, with a slight networking bias...

File Templating

Have you ever deployed a single-config device (doesn't have to be a router or switch) and encountered copy-paste errors, adding old VLAN names, from some master config  (ideally) or other devices (not ideally)? 

As it turns out, so many developers ran into this issue that they created a parsing language specifically for purposes like this - Jinja2.

When visiting their website, the API documentation can be a bit overwhelming. There are many features for single-file templating, but if your goal is to cookie-cutter generate device configurations, you don't need to learn all that much of it, as Ansible takes care of the vast majority of the coding required. That's right - no coding required.

The Basics

Jinja2 file templates emphasize the use of variables, and escape them with double curly brackets, for example:
hostname {{ hostname }}

As a language, it also supports a hierarchy of variables:
hostname {{ global.hostname }}-{{ global.switchid }}

This is pretty simple, right? The first step I'd recommend here is to go through any configuration standards you have and highlight all of the variables in it.

Now to add a little bit more difficulty - it's time to define the variables in a document, to eventually combine together with the Jinja template we're creating. This is incredibly difficult to do in a vacuum, as you need a good way to name/organize the variables. So let's take that highlighted document, and start attaching names / organizing them at the same time. I'd recommend using a text editor that supports multi-document editing, putting your variable list on one side, and your Jinja template on the other. Here's how I did it in Visual Studio Code:
As you can see, on the left I have used YAML to define attributes of a leaf switch, while adding the names into the template itself. I'll keep this brief, as there's one important aspect to automation here:

YOU are automating YOUR OWN, EXISTING, expertise on a platform. This is not replacing YOU, nor is it making YOUR SKILLS IRRELEVANT. Those skills are still absolutely necessary. YOU will still have to hand-configure and explore equipment like you always have. The biggest change YOU will see is that you'll have more time to test configurations and making them more reliable, instead of performing some of the more boring tasks like editing text files.

For this reason, I'm not going to get very prescriptive on the what or the how from here, as this is an exploration exercise that will vary greatly based on the use case. Here are some quick guidelines while trying it out:
  • Keep it organized! The Jinja document's supporting YAML file is there for YOU to read. Make it easy to do so.
  • If you think you'll need it, add it. You man not have a use case for making MTU a variable currently, but it's seeing widespread adoption in the data center and campus networks - if you think you may change it someday in the future, add it into the documents.
  • Use with extreme prejudice against your configuration templates!
Now that the vast majority of the work here is done, let's focus on the no-code way to combine these files. For this, all you need is python and Ansible, and pretty much any version works. To achieve this, Ansible has a pre-installed module called template.

- hosts: localhost
    - name: Import Vars...
        file: example-ios-switch-dictionary.yml
    - name: Combine IOS Stackable Leaf...
        src: templates/example-ios-stackable-leaf.j2
        dest: example-ios-stackable-leaf.conf

..and that's it. Run it with the command ansible-playbook, and it will create a new file. Unfortunately, this requires one playbook per configuration, as the include_vars module doesn't unload anything from the YAML file.

Usage At Scale

This method scales extremely well - I have provided an example on Github ( which leaves some standardized framework for keeping things organized, like using roles per device configuration, so it should be pretty easy to fork and expand to encompass multiple switches and multiple configuration standards, all in one repo.

In the real world, I use several Git repositories - the sheer quantity of templates and roles just gets out of control otherwise, and collaboration like using Git Pull Requests for continuous review and improvement (It's amazing what you can do with the saved time!) is much easier with that separation.

I've also generated an entire datacenter fabric configuration in seconds this way. Once you get your repositories organized, that's not even that big of a deal.

Demystifying CI/CD and Automation in General

You're already using automation. If you use Pull Requests to improve templates, you're simply formalizing previous practices you already did, but you also (probably) accidentally did CI/CD and network automation here.

A lot of DevOps gurus tend to treat automation work like it's the technological equivalent to inventing the wheel, and a lot of that is more to advance and protect the profession, and less a play to establish dominance / a place of power. Unfortunately, this tends to create a bit of a rift between them and the people they are there to help, but I've never seen that be intentional with DevOps engineers. They're developers, just like other ones, with a fiery burning passion for reducing boring, repetitive tasks for you, and making sure that the methods to do so are well-organized, and want to share those experiences. You don't need to give them a hug, but ask how they do stuff, it's probably the quickest way for you and them to learn something.


  1. Great post! ...lots of info here, but I think the best part is the commentary on how netdevops engineers are automating their own expertise...and that it shouldn't be a source of paranoia related to job security. Such an important point. /tbohn


Using VM Templates and NSX-T for Repeatable Virtual Network Deployments

So far, we've provided the infrastructure for continuous delivery / continuous integration, but it's been for those other guys . Is ...