Thanks to @mierdin for point this out. It looks like the wordpress format is causing some strange word-wrap issues. For a better view please click here to see the full post without presentation issues.
Using GITHub to build our Network Configs
As I wrote in this post, one of my goals for this year is to be able to compltely automate the build of my lab environment programatically.
In the last couple of jinja posts, I wrote about the basics of Jinja2 templates and how they can be applied to building network configurations.
In this post, I’m going to take the next step and move those files from my local hard drive out to…
duh duh dahhhhhhhhhh
The cloud.
Before we get started…
We’re going to go over some basics on the tools we’re using to make sure everyone’s on the same page. cool?
What’s GIT?
Git is a widely-used source code management system for software development. It is a distributed revision control system with an emphasis on speed, data integrity, and support for distributed, non-linear workflows. wikipedia
Huh?
GIT is a piece of software that allows you to track changes to files over time.
So what’s GITHub?
“Where software is built Powerful collaboration, code review, and code management for open source and private projects. Public projects are always free. “Github.com
GITHub is like facebook for developers. It’s a place where you can sync your local GIT repository to a central location, and then sync that central location to other local repositories.
Different people can connect to the same repository allowing multiple people to work on the same project.
What’s a repository?
A repository is essentially a collection of files that make up a project. You could think of it like a folder or directory. That analogy is not exact as it’s possible for a repository to have multiple sub-folders or directories, but it’s close enough for our purposes.
Is GIT only for Code?
GIT was definitely designed for software developers to as a versioning control system while developing software, but you can use it for tracking changes to things other than
You could use it for anything text format that you want to track changes to over time. For example
- grocery lists
- contact list
- tracking your weight
There are a lot of interesting uses for GIT, one of those that we’re going to use today is looking at storing our Jinja2 templates on a public GIT repository and loading them directly into our python script as part of the code.
Import Required Libraries
Unles you’ve already got them, you’ll need to pip install jinj2 and pip install requests these two libraries before loading them into your running environment.
import requests
import yaml
import githubuser
from jinja2 import Environment, FileSystemLoader, Template
Loading Templates from GITHub
Like with most things in python, if it’s useful enough, chances are there’s probably someone else who already put a library together for that. In our case, we’re going to use the python request library to handle loading files directly from our Github repository.
The first thing we’ll do is load the HPE comware switch template from that we used in this post. If you wanted to take a look at this directly on github, it can be found here. All we have to do is to copy and paste the URL from our browser directly into the first input of the requests.get function.
note: The requests function will return a whole object that has various attributes. the ” .text ” at the end of this tells the function to just give us the contents of the file, not of the other information, like the HTTP status_code.
Simple, right?
comware_template = requests.get('https://github.com/netmanchris/Jinja2-Network-Configurations-Scripts/blob/master/simple_comware.j2').text
Looking at the output
So now that we’ve loaded the contents of the simple_comware.j2 template directly from the Github site into the comware_template variable. Let’s take a look to make sure that we have what we need.
print (comware_template)
Hmmmmm. That’s not right?
The requests library is reaching out and grabbing whatever we put into that first variable. If we look at the print contents, we can see the first line is<!DOCTYPE html> . So it looks like we’re grabbing the rendered webpage, not just the contents of the file. Thankfully, looking at the GITHub website, there’s an option to look at any of your files in raw mode. So let’s grab that URL and try this again, ok?
comware_template = requests.get('https://raw.githubusercontent.com/netmanchris/Jinja2-Network-Configurations-Scripts/master/simple_comware.j2').text
print (comware_template)
Ahhhh… That’s better.
Loading Network Specific Values from GITHub
Now we’re going to load our network specific values which were stored in the YAML file in this post. But this time, we’re going to load them directly from a private github repository.
The free GITHub accounts allow you to have public repositories, which means everyone can see what you’re doing, but if you have a paid version, you can get private repositories for as little as five dollars a month.
The private repositories are secured and can only be accessed by someone with a GIThub username and password who has explicitly been given access to this repository.
I would say that it’s probably a bad idea for us to keep any secure information like usernames, passwords, or SNMP strings in a online repository. But for my purposes, I don’t have anythng of value in this lab environment so I’m not too worried about it.
note: Before you put any sensitive data into an online repository of any kind, be sure to check with your companies data policies to see if you’re breaking any corporate rules.
Creating an Auth Object
First, I’m going to create an auth object, which is basically a single object that represents the username and password for my github account. In my case, I’ve got a file on my local hard drive that will automatically create the auth object for me.
In case you’re interested, the file is called githubuser.py and contains the following code.
from requests.auth import HTTPBasicAuth
def gitcreds(): auth = HTTPBasicAuth('netmanchris', 'my_secret_password') return auth
auth = githubuser.gitcreds() #you didn't think I was going to give you my password did you?
Loading simple.yaml
We’ll now load the simple.yaml file like we did in this post but instead of opening it from a local file, we’re going to load it directly from the raw version of the file on github. I’d give you the link but it’s in a private repository, so you won’t be able to access it anyways.
Thigs I want to point out
- yaml.load: takes the response and processes the yaml content directly into a python data structure ( dictionary )
- .text: takes the “.text” attribute from the requests object which is the content of the page.
- auth = auth: takes the auth object we created above and passes it as the username and password during the HTTP request.
Make sense?
simple = yaml.load(requests.get('https://raw.githubusercontent.com/netmanchris/PrivateRepo/master/simple_config.yaml', auth=auth).text)
simple
Putting it all together
So looking at our list
- download simple_comware.j2 template from Github public repo: **Check!**
- download simple.yaml values file from Github private repo: **Check!**
- rendered templates: **Nope**
So I guess we know what comes next, right?
Rendering the final config
We use the Template function to create a jinja2 template object and then we use the simple variable we created during the yaml section as input into the cw_template object.
cw_template = Template(comware_template)
type(cw_template)
print (cw_template.render(simple=simple))
Writing the Config to Disk
So far we’ve only been rendering and printing configurations, but it would be kinda nice to be able to have these on disk so that we can open them in our favorite editor before we cut and paste them into a telnet session to our network device.
The next two commands simply write the rendered template to disk with the filename comware.cfg and then we open and print the file to screen just to make sure it worked.
with open('comware.cfg', "w") as file:
file.write(cw_template.render(simple=simple))
with open('comware.cfg') as file:
print (file.read())
What’s next?
So far, we’ve come pretty far. We’ve written a couple of jinja templates, we’ve figure out how to store those files in a centralized control versioning system, but we’re still cut’ing and past’ing those configurations ourselves which is not ideal.
In the next post, we’ll look at using APIs to push the configuraiton directly to a configuraiton management tool.
Questions or comments? Feel free to post below!
kontrolissues 