Shedding the Lights on Operations: REST, a NMS and a Lightbulb

It’s obvious I’ve caught the automation bug. Beyond just automating the network I’ve finally started to dip my toes in the home automation pool as well.

The latest addition to the home project was the Philipps hue light bulbs. Basically, I just wanted a new toy, but imagine my delight when I found that there’s a full REST API available. 

I’ve got a REST API and a light bulb and suddenly I was inspired!

The Project

Network Management Systems have long suffered from information overload.

Notifications have to be tuned and if you’re really good you can eventually get the stream down to a dull roar. Unfortunately, the notification process is still broken in that the notifications are generally dumped into your email which if you are anything like me…

NewImage

Yes. That’s really my number as of this writing

One of the ways of dealing with the deluge is to use a different medium to deliver the message. Many NMS systems, including HPE IMC, has the capability of issuing audio alarms, but let’s be honest. That can get pretty annoying as well and it’s pretty easy to mute them.

I decided that I would use the REST interfaces of the HPE IMC NMS and the Phillips Hue lightbulbs to provide a visual indication of the general state of the system.Yes, there’s a valid business justifiable reason for doing this. But c’mon, we’re friends?  The real reason I worked on this was because they both have REST APIs and I was bored. So why not, right?

The other great thing about this is that you don’t need to spend your day looking at a NOC screen. You can login when the light goes to whatever color you decide is bad.

Getting Started with Philipps Hue API

The Philipps SDK getting started was actually really easy to work through. As well, there’s an embedded HTML interface that allows you to play around with the REST API directly on the hue bridge.

Once you’ve setup your initial authentication to the bridge ( see the getting started guide ) you can login to the bridge at http://ip_address/debug/clip.html

From there it’s all fun and games. For instance, if you wanted to see the state of light number 14, you would navigate to api/%app_name%/lights/14 and you would get back the following in nice easy to read JSON.

http://ipaddress/debug/clip.html/

NewImage

From here, it would be fairly easy to use a http library like REQUESTS to start issuing HTTP commands at the bridge but, as I’m sure you’re aware by now, there’s very little unread territory in the land of python.

PHUE library

Of course someone has been here before me and has written a nice library that works with both python 2 and python 3.  You can see the library source code here, or you can simple

>>> pip install phue

From your terminal.

The Proof of Concept

You can check out the code for the proof of concept here. Or you can watch the video below.

Breaking down the code

1) Grab Current Alarm List

2) Iterate over the Alarms and find the one with the most severe alarm state

3) Create a function to correlate the alarm state to the color of the Philipps Hue lightbulb.

4) Wait for things to move away from green.

Lessons Learned

The biggest lesson here was that colours on a screen and colours on a light bulb don’t translate very well. The green and the yellow lights weren’t far enough apart to be useful as a visual indicator of the health of the network, at least not IMHO.

The other thing I learned is that you can waste a lot of time working on aesthetics. Because I was leveraging the PHUE library and the PYHPEIMC library, 99% of the code was already written. The project probably took me less than 10 minutes to get the logic together and more than a few hours playing around with different colour combinations to get something that I was at least somewhat ok with. I imagine the setting and the ambient light would very much effect whether or not this looks good in your place of business.If you use my code, you’ll want to tinker with it.

Where to Next

We see IoT devices all over in our personal lives, but it’s interesting to me that I could set up a visual indicator for a NOC environment on network health state for less than 100$.  Just thinking about some of the possibilities here

  • Connect each NOC agents ticket queue with the light color. Once they are assigned a ticket, they go orange for DO-NOT-DISTURB
  • Connect the APP to a Clearpass authentication API and Flash the bulbs blue when the boss walks in the building. Always good to know when you should be shutting down solitaire and look like you’re doing something useful, right?
  • Connect the APP to a Meridian location API and turn all the lights green when the boss walks on the floor.

Now I’m not advocating you should hide things from your boss, but imagine how much faster network outages would get fixed if we didn’t have to stop fixing them to explain to our boss what was happening and what we were going to be doing to fix them, right?

Hopefully, this will have inspired someone to take the leap and try something out,

Comments, questions?

@netmanchris

Auto Network Diagram with Graphviz

One of the most useful and least updated pieces of network documentation is the network diagram. We all know this, and yet we still don’t have/make time to update this until something catastrophic happens and then we says to ourselves

Wow. I wish I had updated this sooner…

Graphviz

According to the website 

Graphviz is open source graph visualization software. Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. It has important applications in networking, bioinformatics,  software engineering, database and web design, machine learning, and in visual interfaces for other technical domains.

note: Lots of great examples and docs there BTW.  Definitely check it out.

Getting started

So you’re going to have to first install graphviz from their website. Go ahead… I’l wait here.

Install the graphviz python binding

This should be easy assuming you’ve already got python and pip installed. I’m assuming that you do.

>>> pip install graphviz

Getting LLDP Neighbors from Arista Devices

You can use the Arista pyeapi library, also installable through pip as well.  There’s a blog which introduces you to the basics here which you can check out. Essentially I followed that blog and then substituted the “show lldp neighbors” command to get the output I was looking for.

Creating a Simple Network Diagram

The code for this is available here

Essentially, I’m just parsing the JSON output from the Arista eAPI and creating a DOTfile which is used to generate the diagram.

Pros: It’s automated

Cons: It’s not very pretty at all.

SimpleTopo.png

 

Prettying it up a Bit

Code for this is available here

So with a little bit of work using the .attr methods we can pretty this up a bit.  For example the

dot.attr('node', shape='box')

method turns the node shape from an ellipse into a box shape. The other transformations are pretty obvious as well.

Notice that we changed the shape of the shape, the style of the arrows a bit and also shaded in the box.  There are a lots of other modifications we can make, but I’ll leave you to check out the docs for that. 

SimplePrettierTopo.png

 

 

Adding your own graphics

Code for this is available here

Getting a bit closer to what I want, but still I think we can do a bit better. For this example, I used mspaint to create a simple PNG file with a switch-ish image on it. From what I can tell, there’s no reason you couldn’t just use the vendor icons for whatever devices you’re using, but for now, just playing with something quick and dirty.

Once the file is created and placed somewhere in the path, you can use this method

dot.attr('node', image="./images/switch1.png")

to get the right image.  You’ll also notice I used

dot.attr('edge', arrowhead='none')

to remove the arrow heads. ( I actually removed another command, can you spot it? )

SimplePrettierGraphicTopo.png

 

Straighter Lines

Code for this is available here

So looking at this image, one thing I don’t like is the curved lines. This is where Graphviz beat me for the day. I did find that I was able to apply the

dot.graph_attr['splines'] = "ortho"

attribute to the dot object to get me the straight lines I wanted, but when I did that, I got a great message that told me that I would need to use xlables instead of standard labels.

SimplePrettierGraphicOrthoTopo.png

Next Steps

Code for this is available here

For this next step, it was time to get the info live from the device, and also to attempt to stitch multiple devices together into a single topology. Some things I noticed is that the name of the node MUST match the hostname of the device, otherwise you end up with multiple nodes.  You can see there’s still a lot of work to do to clean this up, but I think it’s worth sharing. Hopefully you do too.

MultiTopo.png

 

Thoughts

Pros: Graphviz is definitely cool. I can see a lot of time spent in drawing network diagrams here. The fact that you could automatically run this every X period to ensure you have a up to date network diagram at all times is pretty awesome. It’s customizable which is nice, and multi-vendor would be pretty easy to implement. Worse case scenario, you could just poll the LLDP MIB with SNMP and dump the data into the appropriate bucket. Not elegant, but definitely functional.

Cons:  The link labels are a pain. In the short time I was playing with it, I wasn’t able to google or documentation my way into what I want, which is a label on each end of the link which would tell me what interface on which device. Not the glob of data in the middle that makes me wonder which end is which.

The other thing I don’t like is the curvy lines. I want straight lines. Whether that’s an issue with the graphviz python library that I’m using or it’s actually a problem with the whole graphviz framework isn’t clear to me yet. Considering the time saved, I could probably live with this as is, but I’d also like to do better.

If anyone has figured out how to get past these minor issues, please drop me a line!  @netmanchris on twitter or comment on the blog.

As always, comments and fixes are always appreciated!

@netmanchris

Jinja2 and… Powershell? Automation(ish) Microsoft DHCP

Most of us have home labs, right?

I’m in the middle of doing some zero touch provisioning testing, and I had the need to create a bunch of DHCP scopes and reservations, some with scope specific options, and some with client specific options. As often as I’ve had to create a Microsoft DHCP server in the lab and set up some custom scopes, I decided I was going to figure out how to automate this as much as I could with a little effort as possible.

After taking a quick look around for a python library to help me out, python being my weapon of choice, I realized that I was going to have to get into some Powershell scripting. I’ve dabbled before, but I’ve never really take the time to learn much about Powershell control structures ( loops, conditionals, pipes, etc…).  I really didn’t want to spend the time getting up to speed on a new language, so I instead decided I was going to use the python skills I had to auto generate the scripts using a little jinja2 and some google-technician skills.

Figuring out the Powershell Syntax

This was the easy part actually, Microsoft has some pretty great documentation for Powershell CmdLets and there was more than a couple of blogs out there with examples, Unfortunately, I didn’t take notes on all the posts I went through… yeah, I suck, but I offer thanks to everyone

Creating the Scopes

The Jinja Template for Creating Scopes

Once I figured out the specific syntax that I needed to generate the DHCP scopes with the proper scope options, I dropped the syntax into a Jinja template using the For loop to run over multiple scopes as defined in the YAML file ( see the next GIST ).

The YAML file to define the Scopes

I chose to use YAML to define the inputs because well, that’s what I felt like working in at the time and it also allowed me to separate out the global Values from those specific to each scope. As I move forward in my full home lab automation project, I’m thinking I might use a single globals values YAML file to hold all the global values for everything in the entire infrastructure, but for now, I decided to keep things simple and just include it in the same YAML file.

If you take a look at the GIST below, you should be able to easily identify what each of the different elements are for.

The Python Script to Generate the Powershell Script

Nothing too complicated here, I load the variables, pass them into the jinga library and spit out a file with a PS1 extension.

Creating the Reservations

For my specific project, I need to set different DHCP option 67s for some of my clients. Although I could have manually created these as well, I decided that I would just take the same approach and template the whole thing.

The Jinja Template for Creating DHCP Reservations

Very similar to the approach above, I figured out the syntax for one, and then I created a Jinja template using a For loop.

The CSV file to define the DHCP Reservations

In this case, since I didn’t have to deal with anything more than the reservations, I decided on using a CSV file as the input format. Although YAML is what all the cool kids are doing, using a CSV file allows me to edit this in Excel which I found to be easier for this specific project. There are only a coupe of reservations in here right now, but I’ve got another 30 or so devices which I will need to perform this same step for, so having the ability to quickly add reservations into a CSV file is a good thing in the long run.

The Python Script to Generate the Powershell Script

Wrap up

To be honest, it’s a bit lazy and I wish I had more time to learn more things, but sometimes, you just use what you know to address a problem in a quick and dirty way. Hopefully, someone else will find these useful as well.

At the beginning of the year, I wrote a blog that said my major goal was to be able to automate the configuration of my entire lab with as little effort as possible. Considering how many times I’ve had to manually create DHCP Scopes and Reservations over the years, I think this one will be something that will definitely come in handy. Hopefully someone else will thing so to!

Questions, Comments? Feel free to post below!

@netmanchris

Devops for Networking Forum in Santa Clara

Normally, I would be writing this a few weeks ago, but sometimes the world just takes the luxury of time away from you.  In this case, I couldn’t be happier though as I’m about to part of something that I believe is going to be really really amazing.  This event is really a testimony to Brent Salisbury and John Willis’s commitment to community and their relentless pursuit of trying to evolve the whole industry, bringing along as many of the friends they’ve made along the way as possible. 

Given the speaker list, I don’t believe there’s been any event in recent ( or long term!) memory that has such an amazing list of speakers. The most amazing part is that this event was really put together in the last month!!!! 

If you’re in the bay area, you should definitely be there. If you’re not in the area, you should buy a plane ticket as you might not ever get a chance like this again. 

 

DevOps Forum for Networking

From the website

 

previously known as DevOps4Networks is an event started in 2014 by John Willis and Brent Salisbury to begin a discussion on what Devops and Networking will look like over the next five years. The goal is to create a conversation for change similar to what CloudCamp did for Cloud adoption and DevopsDays for Devops.

 

When and Where

You can register here

DevOps Networking Forum 2016

Monday, March 14, 2016 9:00 AM – 5:00 PM (Pacific Time)

Santa Clara Convention Center
5001 Great America Pkwy
Santa ClaraCalifornia 95054
United States
Questions? Contact us at events@linuxfoundation.org

 Who

You can hit the actual speakers page here, but the here’s the short list

  • Kelsey Hightower, Google,
  • Kenneth Duda, Arista
  • Dave Meyer, Brocade
  • Anees Shaikh, Google
  • Chris Young, HPE
  • Leslie Carr, SFMIX
  • Dinesh Dutt, Cumulus
  • Petr Lapukhov, Facebook
  • Matt Oswalt, keepingitclasseless 
  • Scott Lowe, VMware

I’ve also heard that other of a few industry notables who will be wandering the hallways as ONS starts to spin up for the week. 

Yup. What an amazing list and for the low low price of $100, you can join us as well!

OMG

Im absolutely honoured and, to be honest, a little intimidated to be sharing a spot with some of the industry luminaries who have been guiding lights personally for me in the last five years. I’m hoping to be a little education, a little entertaining, and other than that, I’ll be in the front row with a box of popcorn soaking up as much as I can from the rest of the speakers.  

Hope to see you there!

 

@netmanchris

 

Implenting Idempotency using HPE IMC

 

Try saying that five times fast.

 

What if those VLANS already exist?

There’s a concept called idempotency which can be loosely explained as

Make sure it’s like this. If it’s not like this, make it like this. If it’s already like this. Don’t do anything

Essentially, it’s a way to declare the desired configuration state of whatever it is you’re trying to configure. If the configuration state of that server, or switch or router is already in that state, than just leave it alone.

It’s a way to ensure that configuration drift doesn’t happen.

So if there’s some rabbid network administrator with a console cable running around laughing maniacly as they randomly changes things… this will help you keep them in check.

jack photo

 

Idempotent VLANs

So we’re going to look at the last example here where we did the following:

  • grabbed the jinja template for vlans directly from a GIThub repository
  • grabbed the desired vlans file directly from a GIThub repository
  • renderd the Jinja template using the values from the vlan file to get our final config
  • used the pyhpeimc library to push the commands through the executecmd RESTful API
 

Import Libraries

You know the drill here, right? Like in all the other examples, and pretty much every useful python script on the planet, we need to first import the specific libraries that we need to help us achieve whatever outcome it is that we want to perform.

In [2]:
import requests
import yaml
import time
from pyhpeimc.auth import *
from pyhpeimc.plat.device import *
from pyhpeimc.plat.icc import *
from pyhpeimc.plat.vlanm import *
auth = IMCAuth("http://", "10.101.0.203", "8080", "admin", "admin")
#auth = IMCAuth("http://", "kontrolissues.thruhere.net", "8086", "admin", "admin")
 

Download VLANs list from Github

Just like in the last blog post, we’re going to download the VLAN’s directly from the GIThub account. This ensures that we’ve got control versioning in place, as well as all the collaborative multi-user goodness that GIThub gives us. If you’re not already using it for SOMETHING. You should be asking yourself “why”?

In [3]:
desired_vlan_list = yaml.load(requests.get('https://raw.githubusercontent.com/netmanchris/Jinja2-Network-Configurations-Scripts/master/vlans.yaml' ).text)
 

As we’re just starting to play around with this, it’s always good to ensure that what we THINK we’ve got is what we’ve actually got. We’re going to now print out the contents of the GITHub file to make sure we know exactly what VLANs are actually in there.

In [4]:
print (yaml.dump(desired_vlan_list['vlans'], indent = 4))
 
- {vlanId: '1', vlanName: default, vlanStatus: '1'}
- {vlanId: '2', vlanName: TenantABC, vlanStatus: '1'}
- {vlanId: '3', vlanName: management, vlanStatus: '1'}
- {vlanId: '10', vlanName: mgmt, vlanStatus: '1'}

 

Gather just the VLAN IDs

If this was my production network, I’d probably be doing more than just checking the VLAN ID, but for our purposes, I’d like to do a quick and dirty “Does a VLAN with this ID exist or not on the device I’m looking at” check.

I’m not currently doing 802.1x identify based networking usng the VLAN name as the deployment key, so this is going to work just fine for me.

I’m going to do a list comprehension to pull out just the VLAN IDs from the YAML file above and store them in the variable called desired vlans_ids. This will setup the list of things VLAN IDs I want to compare the current state to. Make sense?

In a nutshell, this new list will let us compare the desired VLAN IDs to the existing VLAN IDs fairly easily.

In [14]:
desired_vlan_ids = [vlan['vlanId'] for vlan in desired_vlan_list['vlans']]
desired_vlan_ids
Out[14]:
['1', '2', '3', '10']
 

Get Current VLANs on Target Device

Now that we’ve got the desired list, we need to figure out the existing list of VLANs on the target device. This is a two step process

  • get the device ID of the target device using the get_dev_details function and look at the value in the id key.
  • run the get_dev_vlans function usng the devid from step one as the inut value to designate the target device.
In [15]:
devid = get_dev_details('10.20.10.10', auth.creds, auth.url)['id']
dev_vlan_list = get_dev_vlans(devid, auth.creds, auth.url)
 

What do we have here?

As with the other steps, we’ll stop here and take a look to see exactly what’s currently on the device to make sure that our code is working as desired. In a production environment, we would have to trust that this was all working properly, and make sure that we had all the appropriate tests built into our code to make sure that the trust was well deserved.

In [16]:
print (yaml.dump(dev_vlan_list, indent = 4))
 
- {vlanId: '1', vlanName: default, vlanStatus: '1'}
- {vlanId: '5', vlanName: DoesntBelong, vlanStatus: '1'}

 

Add Desired VLANs to Target Device

Now that we’ve got the current and desired state of the VLANs on the device. We need to figure out how to make them match.

For the first step, we will need to figure out how to create and any of the missing VLANs and push them to the target device.

Thankfully, there’s a create_dev_vlan function in the pyhpeimc library that allows us to push VLANs to the device directly using an API without having to use the CLI. No CLI commands is a good thing here, right?

This means that we will not have to worry about vendor specific syntax and can focus on what really matters which is the VLAN IDs, names, and descriptions. Everything else is just details.

In [17]:
help (create_dev_vlan)
 
Help on function create_dev_vlan in module pyhpeimc.plat.vlanm:

create_dev_vlan(devid, vlanid, vlan_name, auth, url)
    function takes devid and vlanid vlan_name of specific device and 802.1q VLAN tag and issues a RESTFUL call to add the
    specified VLAN from the target device. VLAN Name MUST be valid on target device.
    :param devid: int or str value of the target device
    :param vlanid:int or str value of target 802.1q VLAN
    :param vlan_name: str value of the target 802.1q VLAN name. MUST be valid name on target device.
    :return:HTTP Status code of 201 with no values.

 

Creating our Add VLANs function

Now that we understand how the create_dev_vlans function works. We’ll create a new function which will take a full list of VLANs in the desired_vlans_list and check if the it already exists in the dev_vlan_ids variable that we created above. If it already exists; we do nothing. If it doesn’t exist, we will add it.

Just for giggles, I also included a small timer which will allow us to see how long it actually takes for this function to run.

In [18]:
def add_vlans():
    start_time = time.time()
    for vlan in desired_vlan_list['vlans']:
        if vlan['vlanId'] in dev_vlan_ids:
            pass
        else:
            print ('adding vlan ' + str(vlan['vlanId']))
            create_dev_vlan(devid, vlan['vlanId'], vlan['vlanName'], auth=auth.creds, url=auth.url)
            
    print("Operation took --- %s seconds ---" % (time.time() - start_time))
 

Adding the VLANs

Now we simply run the function we defined above to add the VLANs to our target device. You can see from the output below that this took a whopping 0.43 seconds to add the missing three VLANs to the device.

In [19]:
dev_vlan_ids = [ vlan['vlanId'] for vlan in (get_dev_vlans(devid, auth.creds, auth.url))]
add_vlans()
get_dev_vlans(devid, auth.creds, auth.url)
 
adding vlan 2
adding vlan 3
adding vlan 10
Operation took --- 0.43477892875671387 seconds ---
Out[19]:
[{'vlanId': '1', 'vlanName': 'default', 'vlanStatus': '1'},
 {'vlanId': '2', 'vlanName': 'TenantABC', 'vlanStatus': '1'},
 {'vlanId': '3', 'vlanName': 'management', 'vlanStatus': '1'},
 {'vlanId': '5', 'vlanName': 'DoesntBelong', 'vlanStatus': '1'},
 {'vlanId': '10', 'vlanName': 'mgmt', 'vlanStatus': '1'}]
 

Let’s do that again

Now we run the same thing again, but this time all the VLANs already exist so there’s no need to add them. The timer function tells us this took an amazing 3.814e-06 seconds. If memory serves, I think that’s 5 pico seconds.

Let’s run it again a few times to see if that stays the same.

In [20]:
dev_vlan_ids = [ vlan['vlanId'] for vlan in (get_dev_vlans(devid, auth.creds, auth.url))]
add_vlans()
 
Operation took --- 3.814697265625e-06 seconds ---
In [23]:
dev_vlan_ids = [ vlan['vlanId'] for vlan in (get_dev_vlans(devid, auth.creds, auth.url))]
add_vlans()
 
Operation took --- 7.152557373046875e-06 seconds ---
In [24]:
dev_vlan_ids = [ vlan['vlanId'] for vlan in (get_dev_vlans(devid, auth.creds, auth.url))]
add_vlans()
 
Operation took --- 3.814697265625e-06 seconds ---
 

Remove Undesired VLANs from Target Device

Now that we’ve added all the VLANs that SHOULD be there, we need to make sure that we get rid of those “undesirables”. we want the state to be exactly what was defined in the GITHub file, no more, no less, right?

We’ll go back to the pyhpeimc library which has a delete_dev_vlans function pre-built for our usage.

This time we’ll do the exact opposite of above. Instead of adding VLANS which aren’t in the list; we’re going to be removing VLANS which aren’t in the list.

In [25]:
help (delete_dev_vlans)
 
Help on function delete_dev_vlans in module pyhpeimc.plat.vlanm:

delete_dev_vlans(devid, vlanid, auth, url)
    function takes devid and vlanid of specific device and 802.1q VLAN tag and issues a RESTFUL call to remove the
    specified VLAN from the target device.
    :param devid: int or str value of the target device
    :param vlanid:
    :return:HTTP Status code of 204 with no values.

In [26]:
def del_vlans():
    start_time = time.time()
    for vlan in get_dev_vlans(devid, auth.creds, auth.url):
        if vlan['vlanId'] not in desired_vlan_ids:
            print ("Deleting vlan " + vlan['vlanId'])
            delete_dev_vlans(devid, vlan['vlanId'], auth.creds, auth.url)
        else:
            print ('Not touching VLAN ' + str(vlan['vlanId']))
    print("Operation took --- %s seconds ---" % (time.time() - start_time))
In [31]:
del_vlans()
 
Operation took --- 5.9604644775390625e-06 seconds ---
Not touching VLAN 1
Not touching VLAN 2
Not touching VLAN 3
Not touching VLAN 10
Operation took --- 0.1680889129638672 seconds ---
 

And again!

Running this the first time took 0.19 seconds. But, since we’ve not got our target device in the desired state. We should now be able to run the command again and see the time come down considerably as, this time, we’re checking the device and finding out there’s nothing to do.

Let’s take a look:

In [29]:
del_vlans()
 
Not touching VLAN 1
Not touching VLAN 2
Not touching VLAN 3
Not touching VLAN 10
Operation took --- 0.07348895072937012 seconds ---
 

Putting it together

Now that we’ve created both functions, let’s run them both at the same time.

In [32]:
add_vlans()
del_vlans()
 
Operation took --- 5.0067901611328125e-06 seconds ---
Not touching VLAN 1
Not touching VLAN 2
Not touching VLAN 3
Not touching VLAN 10
Operation took --- 0.16545391082763672 seconds ---
 

Embracing the possibilities

So you might be saying “so what?” you just added some vlans to a single switch. With a bit of tweaking, we could easily have the add_vlans() and del_vlans()functions take the IP address of a target device as an input to the function. In this case, we could deploy the VLANS to ALL of the target devices in a specific group, or branch, or campus, or the entire network if we really wanted. That’s the beauty of a little idea.

You can see how the automation of a single small task can quickly save you a lot of time, not to mention the fact that there is no possiblity for human error at the CLI and you will have a predicatable outcome from the centralised YAML file that’s under version control.

Not bad for a network guy, right?

As always, comments or questions are more than welcome. It’s also cool if you just wanted to say “hi”. 

@netmanchris

 

Cleaning up After Ourselves

For those of you following along at home. I have been running this demo a lot lately so I wrote this additional code to get the devices back into the original state. Making it much easier to just run through the whole ipython notebook and perform the same demo in a predicatble manner every time.

I’ve included the code here in case anyone else finds it useful.

In [33]:
create_dev_vlan(devid, '5', 'DoesntBelong', auth.creds, auth.url)
remove_vlans = [ vlan['vlanId'] for vlan in desired_vlan_list['vlans']]
print (remove_vlans)
for i in remove_vlans:
    delete_dev_vlans(devid, i, auth.creds, auth.url)
 
['1', '2', '3', '10']
Unable to delete VLAN.
VLAN does not Exist
Device does not support VLAN function
Vlan deleted
Vlan deleted
Vlan deleted
In [34]:
get_dev_vlans(devid, auth.creds, auth.url)
Out[34]:
[{'vlanId': '1', 'vlanName': 'default', 'vlanStatus': '1'},
 {'vlanId': '5', 'vlanName': 'DoesntBelong', 'vlanStatus': '1'}]
 

Deploying Code to Devices Through your NMS

 
 

note: It’s come to my attention that WordPress is truncating some of my posts so that the right hand side is blocked by the side bar. My apologies for the this. I’ll get it fixed ( or more likely move to GH pages ) as quickly as possible. Thanks for your patience

@netmanchris

 

If you’re luck enough to have an NMS as powerful as HPE IMC then you already have a very capable system which has a ton of APIs that you probably didn’t even know about. IMC isn’t the only NMS which has APIs these days, but it’s the one we’re going to be looking at here.

We’ve spent the last few posts ( herehere, and here running through creating some network configurations through the Jinja2 templating language.

There are at least a couple of immediate benefits to this approach:

  • Consistency in the configuration between devices
  • Accuracy in the commands going into your devices

But the one large draw back is that you’ve still got to cut and paste that configuration into your device somehow, which is not the ideal scenario. We’re trying to get away from touching our devices.

In this post, we’re going to look at taking the rendered configuration and pushing it directly to the desired device through HPE IMC’s RESTful API, refered to as the eAPI in documentation.

Although there used to be a charge for this, HPE recently made some changes and the RESTful API is now included in both the Standard and Enterprise editions of the NMS.

In [2]:
import requests
import yaml
import githubuser
from pyhpeimc.auth import *
from pyhpeimc.plat.device import *
from pyhpeimc.plat.icc import *
from pyhpeimc.plat.vlanm import *
from jinja2 import Environment, FileSystemLoader, Template
 

Loading the templates and values from Git

We’ve gone through this before, so I’m not going to spend much time here going over this. In a nutshell, we’re loading the comware_template and the variables we’d like to use to fill in the template. Again, make sure you’re using the raw URL from Github and not the normal URL or you will end up with the whole HTML structure and not just the content you’re looking for.

In [3]:
comware_template = requests.get('https://raw.githubusercontent.com/netmanchris/Jinja2-Network-Configurations-Scripts/master/comware_vlan.j2').text
gitauth = githubuser.gitcreds() #you didn't think I was going to give you my password did you?
simple = yaml.load(requests.get('https://raw.githubusercontent.com/netmanchris/Jinja2-Network-Configurations-Scripts/master/vlans.yaml', auth=gitauth).text)
cw_template = Template(comware_template)
 

Rendering the template

Here we’re going to take a quick look at the rendered combination of the comware_tempalte and the variables to make sure this is what we want to send during the final push to the device. Automation is great, but it’s going to be a long time before it can replace a human being with knowledge of the environment. Trust… but verify.

In [4]:
my_template = cw_template.render(simple=simple)
print (my_template)
 
#vlan config
vlan 1
    name default
    description default
vlan 2
    name TenantABC
    description TenantABC
vlan 3
    name management
    description management
vlan 10
    name mgmt
    description mgmt

 

Options, Options, Options…

We now have a decision to make. There are a couple of different APIs available to us to push VLANs to the device.

For this example, we’re going to use the executecmd API that allows us to send a series of commands to the device through the HPE IMC REST API.

vlan api

As you can see from the REST documentation, you need to send a JSON object which is a list of the commands that you would type in from the command prompt of the switch.

So there are a couple of things we need to prepare the rendered jinja template into a format that can be sent to the API.

  1. We need to add the command “system-view” to the beginning of the command list.

    system-view on HPE Comware devices is equivalent to the enable + conf t commands using the IOS syntax you’re probably used to

  2. We need to split the giant string that rendering the jinja template gave us into a python list with one command per list item. Thankfully, we can use the python split method to help us through this. We can use the carriage return symbol to identify the end of each line. python identifies the carriage return by the \n characters which is what we’re going to use as the input to the split method.

  3. Once we’ve got those two things done, we simply add the two together and voila!

In [5]:
cmd_list = ['system-view']
cmd_list = cmd_list + my_template.split('\n')
 

Trust but verify

Are you seeing a trend here? If we’re ever going to learn to trust automation, we need to get comfortable that our expectations are met at each step of the journey, so we’re going to take a look at the new cmd_list variable and make sure that

  • it’s a list
  • the first elemend of the list is system-view
  • the rest of the list is one command per element
  • all the commands are in the right order
In [6]:
cmd_list[0:10]
Out[6]:
['system-view',
 '#vlan config',
 'vlan 1',
 '    name default',
 '    description default',
 'vlan 2',
 '    name TenantABC',
 '    description TenantABC',
 'vlan 3',
 '    name management']
 

Sending the commands

So far, other than splitting on the \n, this isn’t much different than what we’ve covered in the other blog posts. Now is where we connect the list of commands we’ve created to the device they are destined for.

The first thing we’re going to do is to create an authentication object that we can use to feed into the requests commands upon sending to the REST API.

In [7]:
auth = IMCAuth("http://", "10.101.0.203", "8080", "admin", "admin")
 
 

Getting the Device ID

The input for the run_dev_cmd is the device ID. This is an internal number that IMC uses to idenitfy that specific device. Thankfully, we’ve also got an RESTful function to get that based on the IP address of the device. To make things a little bit easier on us, we will grab the results of the get_dev_details API and assign the device ID directly to a variable called devid. Once we’ve got the device ID back, this gives us what we need to move on to the next steps.

In [8]:
devid = get_dev_details('10.20.10.10', auth.creds, auth.url)['id']
devid
Out[8]:
'221'
 

Sending the Commands to the target Device

We will now use the run_dev_cmd function from the pyhpeimc library to send the commands directly to the device. You can see that we are using the devidvariable assigned above as the input for the target device. We’re also using the cmd_list variable that containts the list of all the commands that we wish to send to the device.

We’re going to look for the contents of the success response. Which, if we’re lucky, should be true.

In [9]:
run_dev_cmd(devid, cmd_list, auth.creds, auth.url)['success']
Out[9]:
'true'
 

Double Checking the VLANs

Now that we’ve sent the VLANs to the device, the last thing we should be doing is to double check that nothing went wrong in the sending. We’ll use the exact same run_dev_cmd function, but this time, we’ll be sending the display vlan command and looking at the content of the return instead of the success.

In [10]:
cmd_list = ['system-view', 'display vlan']
print (run_dev_cmd(devid, cmd_list, auth.creds, auth.url)['content'])
 
 1(default), 2-3, 5, 10
 

Getting better, right?

So we’ve come a long way in a short time. We’ve

And in this post, we learned how to leverage the first three to deploy configurations directly from code to our devices.

The good part

For those who have done some scripting to device before, you’ll have noticed that using an API provided by an NMS such as HPEIMC makes life much easier. We didn’t have to worry about username and passwords for the individual devces, nor having to worry about deciding what protocol we need to use to connect to the device. The great part about using the NMS as a proxy is that all the credential and protocl negotiations are all handled by the NMS itself, allowing us to get on to the trouble of worrying about what we want to send to our devices and not concerning with how they actually get there.

This is a big step forward, but there are still a couple of small problems that we need to address

Configuration Drift

If you look closely, we’ve actually got an extra VLAN in there. VLAN 5 has been configured on the device, but it’s not in our list of desired_vlans where we have declared exactly which VLANs should be on the target device. This is what is sometimes known as configuration drift. Some people may say

Hey, It’s just an extra VLAN right? That won’t hurt us!

Sorry to respectfully disagree, but this attitude is exactly what causes us issues. This is the death of a thousand cuts. It’s JUST one VLAN, JUST one switch running a differnet version of code, JUST one router that has some unused sub-interfaces on it.

IT’S JUST ONE MORE THING THAT WILL BITE YOU WHEN YOU’RE TROUBLESHOOTING AN ISSUE.

These JUST things are what we sometimes call technical debt. If you can figure out out.

Vendor Syntax

The other problem with this example is that we are bound to a specific vendor’s syntax. If you attmept to run the system-view command on a Juniper/Cisco/Brocade/Extreme/ARISTA device, it’s going to error out. Right? This coule easily be addressed by some conditional logic which figures out which kind of a box it is first and then leverages a specific Jinja template for that vendor, but you can see how this becomes a slippery slope rather quickly.

In the next post, we’re going to look at a way to address both of these issues.

Stay Tuned!

@netmanchris

P.S. As always, comments and questions are more than welcome.

In [ ]:
 

GIT and Jinja – Like Peanut butter and Pickles!

Thanks to @mierdin for point this out. It looks like the wordpress format is causing some strange word-wrap issues. For a better view please click here to see the full post without presentation issues. 

 

Using GITHub to build our Network Configs

As I wrote in this post, one of my goals for this year is to be able to compltely automate the build of my lab environment programatically.

In the last couple of jinja posts, I wrote about the basics of Jinja2 templates and how they can be applied to building network configurations.

In this post, I’m going to take the next step and move those files from my local hard drive out to…

 

duh duh dahhhhhhhhhh

The cloud.

The cloud

 

Before we get started…

We’re going to go over some basics on the tools we’re using to make sure everyone’s on the same page. cool?

What’s GIT?

Git is a widely-used source code management system for software development. It is a distributed revision control system with an emphasis on speed, data integrity, and support for distributed, non-linear workflows. wikipedia

Huh?

GIT is a piece of software that allows you to track changes to files over time.

So what’s GITHub?

“Where software is built Powerful collaboration, code review, and code management for open source and private projects. Public projects are always free. “Github.com

GITHub is like facebook for developers. It’s a place where you can sync your local GIT repository to a central location, and then sync that central location to other local repositories.

Different people can connect to the same repository allowing multiple people to work on the same project.

What’s a repository?

A repository is essentially a collection of files that make up a project. You could think of it like a folder or directory. That analogy is not exact as it’s possible for a repository to have multiple sub-folders or directories, but it’s close enough for our purposes.

Is GIT only for Code?

GIT was definitely designed for software developers to as a versioning control system while developing software, but you can use it for tracking changes to things other than

You could use it for anything text format that you want to track changes to over time. For example

  • grocery lists
  • contact list
  • tracking your weight

There are a lot of interesting uses for GIT, one of those that we’re going to use today is looking at storing our Jinja2 templates on a public GIT repository and loading them directly into our python script as part of the code.

 

Import Required Libraries

Unles you’ve already got them, you’ll need to  pip install jinj2  and  pip install requests these two libraries before loading them into your running environment.

In [1]:
import requests
import yaml
import githubuser
from jinja2 import Environment, FileSystemLoader, Template
 

Loading Templates from GITHub

Like with most things in python, if it’s useful enough, chances are there’s probably someone else who already put a library together for that. In our case, we’re going to use the python request library to handle loading files directly from our Github repository.

 

The first thing we’ll do is load the HPE comware switch template from that we used in this post. If you wanted to take a look at this directly on github, it can be found here. All we have to do is to copy and paste the URL from our browser directly into the first input of the requests.get function.

note: The requests function will return a whole object that has various attributes. the ” .text ” at the end of this tells the function to just give us the contents of the file, not of the other information, like the HTTP status_code.

Simple, right?

In [75]:
comware_template = requests.get('https://github.com/netmanchris/Jinja2-Network-Configurations-Scripts/blob/master/simple_comware.j2').text
 

Looking at the output

So now that we’ve loaded the contents of the simple_comware.j2 template directly from the Github site into the comware_template variable. Let’s take a look to make sure that we have what we need.

In [76]:
print (comware_template)
 
<!DOCTYPE html>
<html lang="en" class="">
  <head prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb# object: http://ogp.me/ns/object# article: http://ogp.me/ns/article# profile: http://ogp.me/ns/profile#">
    <meta charset='utf-8'>
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta http-equiv="Content-Language" content="en">
    <meta name="viewport" content="width=1020">
    
    
    <title>Jinja2-Network-Configurations-Scripts/simple_comware.j2 at master · netmanchris/Jinja2-Network-Configurations-Scripts · GitHub</title>
    <link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="GitHub">
    <link rel="fluid-icon" href="https://github.com/fluidicon.png" title="GitHub">
    <link rel="apple-touch-icon" href="/apple-touch-icon.png">
   
...
 

Hmmmmm. That’s not right?

The requests library is reaching out and grabbing whatever we put into that first variable. If we look at the print contents, we can see the first line is<!DOCTYPE html> . So it looks like we’re grabbing the rendered webpage, not just the contents of the file. Thankfully, looking at the GITHub website, there’s an option to look at any of your files in raw mode. So let’s grab that URL and try this again, ok?

In [77]:
comware_template = requests.get('https://raw.githubusercontent.com/netmanchris/Jinja2-Network-Configurations-Scripts/master/simple_comware.j2').text
In [78]:
print (comware_template)
 
#sysname config
sysname {{ simple['hostname'] }}
#vlan config
{% for vlan in simple['vlans'] -%}
vlan {{ vlan['id'] }}
    name {{ vlan['name'] }}
    description {{ vlan['description'] }}
{% endfor %}#snmp_config
snmp-agent
snmp-agent community read {{ simple['snmp']['read'] }}
snmp-agent community write {{ simple['snmp']['write'] }}
snmp-agent sys-info contact {{ simple['snmp']['syscontact']  }}
snmp-agent sys-info location {{ simple['snmp']['syslocation'] }}
snmp-agent sys-info version all
 

Ahhhh… That’s better.

 

Loading Network Specific Values from GITHub

Now we’re going to load our network specific values which were stored in the YAML file in this post. But this time, we’re going to load them directly from a private github repository.

The free GITHub accounts allow you to have public repositories, which means everyone can see what you’re doing, but if you have a paid version, you can get private repositories for as little as five dollars a month.

The private repositories are secured and can only be accessed by someone with a GIThub username and password who has explicitly been given access to this repository.

I would say that it’s probably a bad idea for us to keep any secure information like usernames, passwords, or SNMP strings in a online repository. But for my purposes, I don’t have anythng of value in this lab environment so I’m not too worried about it.

note: Before you put any sensitive data into an online repository of any kind, be sure to check with your companies data policies to see if you’re breaking any corporate rules.

 

Creating an Auth Object

First, I’m going to create an auth object, which is basically a single object that represents the username and password for my github account. In my case, I’ve got a file on my local hard drive that will automatically create the auth object for me.

In case you’re interested, the file is called githubuser.py and contains the following code. 

 

from requests.auth import HTTPBasicAuth

def gitcreds(): auth = HTTPBasicAuth('netmanchris', 'my_secret_password') return auth

In [79]:
auth = githubuser.gitcreds() #you didn't think I was going to give you my password did you?
 

Loading simple.yaml

We’ll now load the simple.yaml file like we did in this post but instead of opening it from a local file, we’re going to load it directly from the raw version of the file on github. I’d give you the link but it’s in a private repository, so you won’t be able to access it anyways.

Thigs I want to point out

  • yaml.load: takes the response and processes the yaml content directly into a python data structure ( dictionary )
  • .text: takes the “.text” attribute from the requests object which is the content of the page.
  • auth = auth: takes the auth object we created above and passes it as the username and password during the HTTP request.

Make sense?

In [80]:
simple = yaml.load(requests.get('https://raw.githubusercontent.com/netmanchris/PrivateRepo/master/simple_config.yaml', auth=auth).text)
In [81]:
simple
Out[81]:
{'hostname': 'testswitch',
 'ip': '10.101.0.221',
 'snmp': {'read': 'supersecret',
  'syscontact': 'admin.lab.local',
  'syslocation': 'lab',
  'trap': [{'target': '10.101.0.200'},
   {'target': '10.101.0.201'},
   {'target': '10.101.0.202'}],
  'write': 'macdonald'},
 'vlans': [{'description': 'management vlan',
   'id': '10',
   'name': 'management'},
  {'description': 'users vlan', 'id': '15', 'name': 'users'},
  {'description': 'phones vlan', 'id': '16', 'name': 'phones'},
  {'description': 'servers vlan', 'id': '20', 'name': 'servers vlan'}]}
 

Putting it all together

So looking at our list

  • download simple_comware.j2 template from Github public repo: **Check!**
  • download simple.yaml values file from Github private repo: **Check!**
  • rendered templates: **Nope**

So I guess we know what comes next, right?

 

Rendering the final config

We use the Template function to create a jinja2 template object and then we use the simple variable we created during the yaml section as input into the cw_template object.

In [82]:
cw_template = Template(comware_template)
type(cw_template)
Out[82]:
jinja2.environment.Template
In [83]:
print (cw_template.render(simple=simple))
 
#sysname config
sysname testswitch
#vlan config
vlan 10
    name management
    description management vlan
vlan 15
    name users
    description users vlan
vlan 16
    name phones
    description phones vlan
vlan 20
    name servers vlan
    description servers vlan
#snmp_config
snmp-agent
snmp-agent community read supersecret
snmp-agent community write macdonald
snmp-agent sys-info contact admin.lab.local
snmp-agent sys-info location lab
snmp-agent sys-info version all
 

Writing the Config to Disk

So far we’ve only been rendering and printing configurations, but it would be kinda nice to be able to have these on disk so that we can open them in our favorite editor before we cut and paste them into a telnet session to our network device.

The next two commands simply write the rendered template to disk with the filename comware.cfg and then we open and print the file to screen just to make sure it worked.

In [84]:
with open('comware.cfg', "w") as file:
    file.write(cw_template.render(simple=simple))
In [85]:
with open('comware.cfg') as file:
    print (file.read())
 
#sysname config
sysname testswitch
#vlan config
vlan 10
    name management
    description management vlan
vlan 15
    name users
    description users vlan
vlan 16
    name phones
    description phones vlan
vlan 20
    name servers vlan
    description servers vlan
#snmp_config
snmp-agent
snmp-agent community read supersecret
snmp-agent community write macdonald
snmp-agent sys-info contact admin.lab.local
snmp-agent sys-info location lab
snmp-agent sys-info version all
 

What’s next?

So far, we’ve come pretty far. We’ve written a couple of jinja templates, we’ve figure out how to store those files in a centralized control versioning system, but we’re still cut’ing and past’ing those configurations ourselves which is not ideal.

In the next post, we’ll look at using APIs to push the configuraiton directly to a configuraiton management tool.

Questions or comments? Feel free to post below!

@netmanchris