Serial numbers how I love thee…

No one really like serial numbers, but keeping track of them is one of the “brushing your teeth” activities that everyone needs to take care of. It’s like eating your brussel sprouts. Or listening to your mom. You’re just better of if you do it quickly as it just gets more painful over time.

Not only is it just good hygene, but you may be subject to regulations, like eRate in the United States where you have to be able to report on the location of any device by serial number at any point in time.

Trust me, having to play hide-and-go seek with an SSH session is not something you want to do when government auditors are looking for answers.

I’m sure you’ve already guessed what I’m about to say, but I”ll say it anyway…

There’s an API for that!!!

HPE IMC base platform has a great network assets function that automatically gathers all the details of your various devices, assuming of course they supportRFC 4133, otherwise known as the Entity MIB. On the bright side, most vendors have chosen to support this standards based MIB, so chances are you’re in good shape.

And if they don’t support it, they really should. You should ask them. Ok?

So without further ado, let’s get started.


Importing the required libraries

I’m sure you’re getting used to this part, but it’s import to know where to look for these different functions. In this case, we’re going to look at a new library that is specifically designed to deal with network assets, including serial numbers.

In [1]:
from pyhpeimc.auth import *
from pyhpeimc.plat.netassets import *
import csv
In [2]:
auth = IMCAuth("http://", "", "8080", "admin", "admin")
In [3]:
ciscorouter = get_dev_asset_details('', auth.creds, auth.url)

How many assets in a Cisco Router?

As some of you may have heard, HPE IMC is a multi-vendor tool and offers support for many of the common devices you’ll see in your daily travels.

In this example, we’re going to use a Cisco 2811 router to showcase the basic function.

Routers, like chassis switches have multiple components. As any one who’s ever been the victem owner of a Smartnet contract, you’ll know that you have individual components which have serial numbers as well and all of them have to be reported for them to be covered. So let’s see if we managed to grab all of those by first checking out how many individual items we got back in the asset list for this cisco router.

In [4]:

What’s in the box???

Now we know that we’ve got an idea of how many assets are in here, let’s take a look to see exactly what’s in one of the asset records to see if there’s anything useful in here.

In [5]:
{'alias': '',
 'asset': '',
 'assetNumber': '',
 'boardNum': 'FHK1119F1DX',
 'bom': '',
 'buildInfo': '',
 'cleiCode': '',
 'containedIn': '0',
 'desc': '2811 chassis',
 'devId': '15',
 'deviceIp': '',
 'deviceName': 'router.lab.local',
 'firmwareVersion': 'System Bootstrap, Version 12.4(13r)T11, RELEASE SOFTWARE (fc1)',
 'hardVersion': 'V04 ',
 'isFRU': '2',
 'mfgName': 'Cisco',
 'model': 'CISCO2811',
 'name': '2811 chassis',
 'phyClass': '3',
 'phyIndex': '1',
 'physicalFlag': '0',
 'relPos': '-1',
 'remark': '',
 'serialNum': 'FHK1119F1DX',
 'serverDate': '2016-01-26T15:20:40-05:00',
 'softVersion': '15.1(4)M, RELEASE SOFTWARE (fc1)',
 'vendorType': ''}

What can we do with this?

With some basic python string manipulation we could easily print out some of the attributes that we want into what could easily turn into a nicely formated report.

Again realise that the example below is just a subset of what’s available in the JSON above. If you want more, just add it to the list.

In [7]:
for i in ciscorouter:
    print ("Device Name: " + i['deviceName'] + " Device Model: " + i['model'] +
           "\nAsset Name is: " + i['name'] + " Asset Serial Number is: " +
           i['serialNum']+ "\n")
Device Name: router.lab.local Device Model: CISCO2811
Asset Name is: 2811 chassis Asset Serial Number is: FHK1119F1DX

Device Name: router.lab.local Device Model: VIC2-2FXO
Asset Name is: 2nd generation two port FXO voice interface daughtercard on Slot 0 SubSlot 2 Asset Serial Number is: FOC11063NZ4

Device Name: router.lab.local Device Model:
Asset Name is: 40GB IDE Disc Daughter Card on Slot 1 SubSlot 0 Asset Serial Number is: FOC11163P04

Device Name: router.lab.local Device Model:
Asset Name is: AIM Container Slot 0 Asset Serial Number is:

Device Name: router.lab.local Device Model:
Asset Name is: AIM Container Slot 1 Asset Serial Number is:

Device Name: router.lab.local Device Model:
Asset Name is: C2811 Chassis Slot 0 Asset Serial Number is:

Device Name: router.lab.local Device Model:
Asset Name is: C2811 Chassis Slot 1 Asset Serial Number is:


Why not just write that to disk?

Although we could go directly to the formated report without a lot of extra work, we would be losing a lot of data which we may have use for later. Instead why don’t we export all the available data from the JSON above into a CSV file which can be later opened in your favourite spreadsheet viewer and manipulated to your hearst content.

Pretty cool, no?

In [9]:
keys = ciscorouter[0].keys()
with open('ciscorouter.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, keys)

Reading it back

Now we’ll read it back from disk to make sure it worked properly. When working with data like this, I find it useful to think about who’s going to be consuming the data. For example, when looking at this remember this is a CSV file which can be easily opened in python, or something like Microsoft Excel to manipuate further. It’s not realy intended to be read by human beings in this particular format. You’ll need another program to consume and munge the data first to turn it into something human consumable.

In [12]:
with open('ciscorouter.csv') as file:
    print (
"System Bootstrap, Version 12.4(13r)T11, RELEASE SOFTWARE (fc1)",,1,-1,FHK1119F1DX,3,"15.1(4)M, RELEASE SOFTWARE (fc1)",2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,,0,,Cisco,2811 chassis,2811 chassis,V04 ,,,CISCO2811,,FHK1119F1DX,,15
,,14,0,FOC11063NZ4,9,,2016-01-26T15:20:40-05:00,1,,,2,router.lab.local,,13,,Cisco,2nd generation two port FXO voice interface daughtercard,2nd generation two port FXO voice interface daughtercard on Slot 0 SubSlot 2,V01 ,,,VIC2-2FXO,,FOC11063NZ4,,15
,,30,0,FOC11163P04,9,,2016-01-26T15:20:40-05:00,1,,,2,router.lab.local,,29,,Cisco,40GB IDE Disc Daughter Card,40GB IDE Disc Daughter Card on Slot 1 SubSlot 0,,,, ,,FOC11163P04,,15
,,25,6,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,,3,,Cisco,AIM Container Slot 0,AIM Container Slot 0,,,,,,,,15
,,26,7,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,,3,,Cisco,AIM Container Slot 1,AIM Container Slot 1,,,,,,,,15
,,2,0,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,,1,,Cisco,C2811 Chassis Slot,C2811 Chassis Slot 0,,,,,,,,15
,,27,1,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,,1,,Cisco,C2811 Chassis Slot,C2811 Chassis Slot 1,,,,,,,,15


What about all my serial numbers at once?

That’s a great question! I’m glad you asked. One of the most beautiful things about learning to automate things like asset gathering through an API is that it’s often not much more work to do something 1000 times than it is to do it a single time.

This time instead of using the get_dev_asset_details function that we used above which gets us all the assets associated with a single device, let’s grab ALL the devices at once.

In [13]:
all_assets = get_dev_asset_details_all(auth.creds, auth.url)
In [14]:
len (all_assets)

That’s a lot of assets!

Exactly why we automate things. Now let’s write the all_assets list to disk as well.

**note for reasons unknown to me at this time, although the majority of the assets have 27 differnet fields, a few of them actually have 28 different attributes. Something I’ll have to dig into later.

In [15]:
keys = all_assets[0].keys()
with open('all_assets.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, keys)
ValueError                                Traceback (most recent call last)
<ipython-input-15-e4c553049911> in <module>()
 3     dict_writer = csv.DictWriter(file, keys)
 4     dict_writer.writeheader()
----> 5dict_writer.writerows(all_assets)

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ in writerows(self, rowdicts)
 156         rows = []
 157         for rowdict in rowdicts:
--> 158rows.append(self._dict_to_list(rowdict))
 159         return self.writer.writerows(rows)

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ in _dict_to_list(self, rowdict)
 147             if wrong_fields:
 148                 raise ValueError("dict contains fields not in fieldnames: "
--> 149 + ", ".join([repr(x) for x in wrong_fields]))  150         return [rowdict.get(key, self.restval) for key in self.fieldnames]

ValueError: dict contains fields not in fieldnames: 'beginDate'

Well That’s not good….

So it looks like there are a few network assets that have a different number of attributes than the first one in the list. We’ll write some quick code to figure out how big of a problem this is.

In [16]:
print ("The length of the first items keys is " + str(len(keys)))
for i in all_assets:
    if len(i) != len(all_assets[0].keys()):
       print ("The length of index " + str(all_assets.index(i)) + " is " + str(len(i.keys())))
The length of the first items keys is 27
The length of index 39 is 28
The length of index 41 is 28
The length of index 42 is 28
The length of index 474 is 28
The length of index 497 is 28
The length of index 569 is 28
The length of index 570 is 28
The length of index 585 is 28
The length of index 604 is 28
The length of index 605 is 28
The length of index 879 is 28
The length of index 880 is 28
The length of index 881 is 28
The length of index 882 is 28
The length of index 883 is 28
The length of index 884 is 28
The length of index 885 is 28
The length of index 886 is 28

Well that’s not so bad

It looks like the items which don’t have exactly 27 attribues have exactly 28 attributes. So we’ll just pick one of the longer ones to use as the headers for our CSV file and then run the script again.

For this one, I’m going to ask you to trust me that the file is on disk and save us all the trouble of having to print out 1013 seperate assets into this blog post.

In [18]:
keys = all_assets[879].keys()
with open ('all_assets.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, keys)

What’s next?

So now that we’ve got all of our assets into a CSV file which is easily consumable by something like Excel, you can now chose what to do with the data.

For me it’s interesting to see how vendors internally instrument their boxes. Some have serial numbers on power supplies and fans, some don’t. Some use the standard way of doing things. Some don’t.

From an operations perspective, not all gear is created equal and it’s nice to understand what’s supported when trying to make a purchasing choice for something you’re going to have to live with for the next few years.

If you’re looking at your annual SMARTnet upgrade, at least you’ve now got a way to easily audit all of your discovered environment and figure out what line cards need to be tied to a particualr contract.

Or you could just look at another vendor who makes your life easier. Entirely your choice.


I use SNMP SETs and I’m not afraid to admit it.

Do you remember back in CCNA school when we learned all sorts of great things that we very rarely followed. One of the favourites was that we are supposed to put meaningful descriptions on all of our interfaces so we know what the other side is connected to.

How many people actually follow that advice?

Yeah, I never do it either. There’s always just too many things on the list that need to get done and it seems like that extra 5 seconds it would take me to update the description to the interface just doesn’t seem like it’s worth the effort. Of course, then I later check the port and end up knocking out my XYZ services and cause myself an outage.

This is where a little python and a decent NMS can help to solve a problem.

Understanding ifIndex

Before we get into the code. We need to understand a little about ifIndex values and how they relate to the physical interfaces of the devices. If you’re REALLY interested, you can do some reading in RFC 2863.  But in a nutshell, each interface on a device, whether physical or logical has a specific numeric value assigned to it which is the last digit in the interface statistics that can be seen through the SNMP interfaces. This is commonly known as interfaces group stats.

There are a bunch of different tables in the interface group stats that are used to store specific kinds of information or statistics for the interfaces such as

  • ifNumber (. – Which is the total number of Interfaces
  • ifIndex ( . * ) – Which acts as a primary key for all the other interface group stats.
  • ifDescr (.* ) – Which is the name of the interface
  • ifType (.* ) – Which describes the type of the interface
  • ifMTU ( .*) – Which shows the current MTU size configured on the interface
  • ifSpeed (.*) – Which shows the current speed of the interface
  • ifPhysicalAddress (.*) – Which shows the mac address of the interface
  • ifAdminStatus (.*) – Which shows the current admin status of the port
  • ifOperStatus (.*) – Which shows the current operational status

There’s a bunch more which you can see here if you’re interested, but one that I found particularly fun was the ifAlias ( .* ) which actually corresponds to the description command in your friendly neighbourhood network operating system

So when your interface is configured like this

Screen Shot 2015 12 10 at 10 33 12 PM

The ifAlias value for the corresponding ifIndex looks like this

Screen Shot 2015 12 10 at 10 34 30 PM

The Fun Begins

The interesting part about the ifAlias value is that it’s actually a SET-able value through the SNMP interface. That means that if you have a simple piece of python code like the following

It will allow you to run a little command like this >>> set_interface_description(‘’, ‘1’, ‘Changed This’ ) which will result in the following.

Screen Shot 2015 12 10 at 10 52 41 PM

So this is cool, right? We’ve just programatically changed a single interface description on a interface. We could stop here and you would be left with a

“So what? Why would I go to the trouble of doing that. It’s harder than just typing in the description manually!”

But wait! There’s more…

The really cool part about automating something so simple is that now we have a building block that we can do something with. For those of you who might have seen the HPE Intelligent Management Centre, you may already be aware that the topology map does this really cool thing where it actually creates a table of all of the links within a given custom view and automatically creates link-names for them.

Screen Shot 2015 12 10 at 10 58 57 PM

You’ll notice that the auto-generated Link-Name actually tells me who’s connected to both sides of that link. You’ll also notice that I have the left and right nodes ( with the IP address ), as well as the left and right interfaces. And  “yes”, there is an API for this where it’s all represented in a nice little JSON string which can be easily parsed in your favourite IDE.

Time vs ROI

The reason we don’t label interfaces is that most people feel it’s just not worth the effort to keep them up to date. Things change too often and it’s just too easy to say “it’s not that important” and move on before changing that description.  I’m with you. I’m as guilty as everyone else on this. But with the help of a little code, a good NMS, the entire process is now automated.

I can proudly say that, for now at least, my lab is 100% accurate as far as interface descriptions go. And because the whole thing is totally automated, I simply re-run the script overtime I make some changes.

So for those of you who weren’t aware. Yes, there is someone who actually uses SNMP SETs in the world. 🙂

Automating your NMS build – Part 4 Adding Custom Views

This is part four in a series of using python and the RESTful API to automate the configuration of the HP IMC network management station.  Like with all things, it’s easier to learn something when you’re able to find a good reason to use those skills. I decided to extend my python skills by figuring out how to use python to configure my NMS using the RESTful API. 

If you’re interested, check out the other posts in this series

Creating Operators

Adding Devices

Changing Device Categories

Adding Custom Views

In this post, we’re going to use the RESTful API to programatically add a custom view, and then add devices to that custom view.  For those of you who don’t know, a custom view is simply a logical grouping of devices.  In IMC custom views also form the basis of the topology maps. In fact, a custom view and a topology map are essentially the same object internally,  The difference is just whether you chose to look at it as a list of devices in the normal interface or chose to look at it in the typical topology/visio format which we all know and might-not-love. 

Why might we want to add a custom view programatically you ask? While, the answer to that might be simply that we’re lazy and it’s easier and faster.  Or it might be that you don’t want to have to look through all the different devices, or, as in my case, that you simply want an excuse to extend your python skills.  Whatever you’re reasons are, custom views are something that just don’t get used enough in my opinion. 

Custom views are a great way to be able to zoom in on the status of a specific branch, a geographic area, maybe a logical grouping of devices that support a specific application?  It really doesn’t matter and the best part is that a single device can exist in multiple custom views at the same time, so there’s really no limit to how you put these views together. It all depends on what makes sense to you. 


The Code

This code is a little bit more complicated than some of the other examples we’ve looked at. We’re actually going to be using multiple functions together, but the logic should be pretty easy to follow.  I’m sure there are better ways to do this, but this seems to work for me.  I’m sure I’ll be back here in a year going “Why did I write it that way!?!?!?!?!?” but for now, I hope it’s simple enough for someone else to follow and possibly get inspired. 

The main function is really just calling the other functions which we will break down below

Step 1 – Create the Custom View

In this code, we’re going to simply use this small function that I created to gather the name of the custom view ( the view_name variable ) and then use that to create the JSON payload ( the payload variable ).  The other part of the JSON payload is the autoAddDevType variable which is hard-coded to 0.  This could be used to have the system automatically add new devices of a given type, but I”m in interested in doing the automation myself here. The last part of this code will be used as the input for another function, which is the return of the view name.  You can see in the main  function in the the view_name = create_new_view() line. 

Step 2 – Get Custom Views

For the next part, we are going to need to figure out what Id was assigned to this new view, to do this we’re going to have to go through a couple of steps. The first one is to ask the NMS to send us a list of all the known custom views. The following function will request the list, which will be returned as a JSON array, and then convert it over into a python list of dictionaries so we can work with it natively as a python object. You can see this in the main function in the  view_list = get_custom_views() line

Now that we have the view_list which is the list of all the views, we’re going to have to find the ID for the new view that we just created.  We do that by by using the get_view_id() function using the view_name as the input.  Essentially, this will look through each of the views in the view_list that was returned above and let us know when the ‘name’ value is equal to the view_name value that we captured above. Once it’s equal, we then return the ‘symbolId’ which is the internal unique numeric value assigned to this particular custom view. This is the number we’re going to use to identify the view that  that we want to add devices to.  Make sense? Now that we’ve got this number, we’re going to assign it to the object view_id for use later on. 

Note: I actually could have added the devices directly to the view in the original add_custom_view code code above, but then I’d have to write the modify function later if I ever wanted to change or add new devices to the view. I’m trying to follow the DRY ( Don’t Repeat Yourself ) advice here so I just write the modify here and I can then leverage it later without having to re-write the code.  

Step 3 – Generate the Device List

We’re not going to spend too much time on this part as I’m essentially re-using the code from the Changing Device Categories blog.  It’s pretty straight forward. I’m using some user-put to gather a list of devices that we want to add to this specific view and then capture it in the dev_list object.  What’s cool about doing it this way is that the returned list will search through all the IP addresses assigned to your devices, not just the managed address which might come up in the NMS interface where you would normally perform this step. There are ways around that as well, but that’s another blog. 

Step 4 – Add Devices to Existing Custom View

This last step is where things come together. We’re going to run the add_device_to_view(dev_list, view_id) function which is using the dev_list generated in step 3 and the view_id that we captured at the end of step 2 as the inputs.  Essentially, we’re just saying here  “ add all the devices I want to the view I just created “. 

Wrapping it Up

So this is just an example of how you can tie a few pieces of code together to help automate something that might otherwise take you a lot of manual labour. In this case, just a bunch of mouse clicks and depending on the fact that you were able to manually identify all the devices you wanted to add to a specific view.  Personally, I’d rather leave the hard work to the computers and move on to something that requires my brain.

Automating your NMS build – Part 3 Changing Device Categories

In the first couple of posts in this series, we created some operators, then we added some devices.   In this post we’re going to look at something a little more complicated. In this post, we’re going to link a couple of different python functions together to meet the requirement which is to change a device from one category to another.

A bit about Device Categories

For those of you haven’t used HP’s Intelligent Management Centre before, the system automatically categorizes any discovered device, usually based upon SNMP sysobjectid. What this means is that when you run an auto discovery, the majority of your infrastructure will be properly classified right out of the box.

Screen Shot 2015 07 07 at 11 11 51 PM

This works great for devices which are SNMP enabled, as well as some other devices like ESX machines ( Virtual Devices ) which use SOAP as the management protocol, but it fails pretty miserably when dealing with devices which don’t support anything more than PING.  IMC’s default behaviour is to put anything which doesn’t respond to SNMP in the Desktop Category.

IP Phones aren’t smart

I’ve had customers who decided to discover all of their expensive IP phones and suddenly found out that none of them were classified properly. The problem with IP phones is that they are usually pretty stupid devices. Low memory, weak CPUs. Most of the processing/thinking is done by the PBX. I’ve never seen an IP phone that supports SNMP.

In this case we have two choices

  1. Manually change each IP phone from the Desktop category into the Voice category one device at a time.
  2. Use the RESTful API and a bit of code to do it wihile we go have a coffee

Filtering First

So the first thing we need to do is to identify the IP phones from the rest of the devices in the Desktop category. Thankfully, this is where having a well designed network can come in REALLY handy. Most Voice networks are designed so that the IP phones are automatically put into a voice vlan. This means that all phones SHOULD be in the same layer 3 network range.

The first piece of code we need to write is a simple piece of code which will allow the user to identify which of the categories they want to filter by. Although this might sound strange, we actually need to make sure that we only grab the DESKTOP devices in a specific subnet range. Imagine if you accidentally move the router or switch for this subnet into the voice category too. Really sucks leaving when you lose your router, right?

In this piece of code, we’re simply creating a dictionary which creates the link between the categoryId, which is the number IMC internally uses to identify the defined categories and the labels we humans use to identify them.  In a nutshell, this simply prints out the available categories.  Why you ask? Because the human running this script needs to known which categories they want to filter by.

This next performs two different functions

  1. Filters all known devices by the categories listed above
  2. Filters all known devices by an IP range.

Combining the two of them we’re able to easily fine all of the devices that have been classified in the Desktop Category in the L3 subnet of the IP phones and return that as a dictionary which contains, among other things, the device IDs which we’ll need in the next step.

Putting it all together

So the last piece of code is where the magic actually happens.

First we assign the output of the filtering function above into the variable called dev_list.  Essentially, this is a list of devices which meet the search criteria of the filtering function.

In our case, this means the list will consists of all the IP phones in the specific subnet that we filtered.

From there, we use the items in dev_list as input into a for loop which changes them into the new category.  ( see how we use the same print_category function from above? )

Wrapping it up

Hopefully this is a fairly useful example of how putting a few basic python functions together can help to substantially cut down on the amount of time it takes to perform what’s really a simple task. That’s the whole point of automation right?

As I continue learning, I can already see there a bunch of ways to improve this code, but hopefully having a simple working example will help people who are a couple of steps behind me on this journey take another step forward.

If you’re ahead of me on this journey and have suggestions, please feel free to comment!


Automating your NMS build using Python and Restful APIs Part 2 – Adding Devices though Auto-Discovery

This is the second in a series of posts where I’ll be using a RESTful API to automate a bunch of the initial deployment functions within my NMS. There are a bunch of reasons to do this that are right for the business. Being able to push information gathering onto the customer, being able to use lower-skilled ( and hence lower paid!) resources to do higher level tasks. Being able to be more efficient in your delivery, undercut the competitors on price and over deliver on quality. It’s a really good project to sink my teeth and use some of my growing coding skills to make a difference to the business. 

Other posts in this series

Automating your NMS build using Python and Restful APIs Part 1 – Creating Operators


Adding Devices

A Network Management System without devices is a sad, sad thing. It’s just a lonely piece of code with no purpose in life. Empty graphs and arm notifications with nothing to notify. Today we’re going to look at adding devices through launching a basic network range auto discovery. In a follow up post, I’ll be adding device through a CSV import which is my preferred method. 

One thing which I will not be covering in this post to save some time and space is how to authenticate to the NMS system which is covered here

Launching an Autodiscovery

One of the worst things about installing an NMS in a customers environment is the inevitable scope creep that happens. The major reason that this happens is that most customers looking for an NMS are doing so because they really don’t have a clue what’s in their network, and potentially not even how they are configured. They know it’s kinda working most of the time, but not much more than that.

Typically, you have a statement of work to discover N number of devices. The customer says  “I don’t know all the IP addresses, why don’t you just discover this range of IP addresses. So you launch a discovery from your tool and suddenly, you’ve got 2*N devices in your database!  And because they are in your database, you’re now responsible for getting them up and running. Even though your contract is for only N devices, you don’t want to disappoint your customer right? 

There other things that can easily go wrong in an auto discovery as well like

  • Mis-matched SNMP strings: You have the SNMP read, but not the write strings.

This can cause some unpredictable results that are SOOO fun to troubleshoot.  

  • Missing CLI credentials: You have SNMP strings, but not the CLI strings.

There are some things that are only accessible through CLI. It’s great that we’re moving into an era where decent programatic APIs are available devices, but for the mass majority of your infrastructure, there’s a ton of functions which are only available through the good old command line interface.  If you don’t have the right credentials, you’re going to have problems. In my experience, unless they have a centralized authentication system in place, like Cisco ACS, FreeRADIUS or other; you’re probably going to have to troubleshoot half of the device credentials. 

Let’s look at the code. 

First I’ll show the entire function complete, and then we’ll break down the sections individually.  Again, I’ll be skipping the imc_creds() function as it’s covered in the earlier post. 

 import requests, json, sys, time, subprocess, csv, os, ipaddress 

 def plat_auto_discover():

    if auth == None or url == None: # checks to see if the imc credentials are already available
    auto_discover_url = '/imcrs/plat/res/autodiscover/start'
    f_url = url + auto_discover_url
    network_address = input(
    '''What is the the network address of the range you wish to discover?\nPlease input the address in the format "": ''')
    # end_address = input('''What is the last address of the network range you wish to discover?\nIPv4 Address: ''')
        network_address = ipaddress.ip_network(network_address)
    except ValueError:
        print("You have entered an invalid network address. Please try again.")
        print ('\n'*80)
    payload = ''' {
    "mode": "0",
    "ipSection": {
        "begin": "''' + str(network_address[1]) + '''",
        "end": "''' + str(network_address[-2]) + '''"
    "discoverNonSnmpDevice": "true",
    "pingAll": "true"
    r =, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents
    if r.status_code == 200:
        print ("Auto-Discovery Successfully Started")
        print ("An Error has occured")

 Gathering the network range

So the first task here is gathering the actual network range that we want to run the auto discover over. For this example, we’ll keep this very basic and use a simple network. This is going to cause the system to scan the entire 254 hosts of the subnet.  One of the other things to keep in mind here is that I’m going to use the default system templates for SNMP and Telnet which is why I’m not gathering them here. This isn’t magic, right?

So I’m using the input function to gather the IP address network range that I want to discover. I’m also using the python ipaddress standard library to caste the user input as a network_address to ensure that it’s a valid input to the final function. 

network_address = input(
    ”’What is the the network address of the range you wish to discover?\nPlease input the address in the format “″: ”’)
    # end_address = input(”’What is the last address of the network range you wish to discover?\nIPv4 Address: ”’)
        network_address = ipaddress.ip_network(network_address)
    except ValueError:
        print(“You have entered an invalid network address. Please try again.”)
        print (‘\n’*80)

So essentially, this code performs three steps

  1. Gathers the desired network address discovery range in the format
  2. Attempts to use the ipaddress.ip_network method from the python ipaddress library to test if the user input was valid, and stores it in the  variable network_address
  3. If the ipaddress.ip_network method fails, this will raise an error and re-run the function to gather input in the right format.

The other nice thing about storing the user input as a ipaddress.ip_network object means that we can easily gather the start and end of this subnet based on the subnet math that’s included in the library. I’d always prefer  to have someone/something else do the binary math. 🙂

Creating the JSON Array

This particular API uses a JSON array in the HTTP message body to gather all of the information used to launch the auto-discovery.

payload = ”’

    “mode”: “0”,
    “ipSection”: {
        “begin”: “”’ + str(network_address[1]) + ”'”,
        “end”: “”’ + str(network_address[-2]) + ”'”
    “discoverNonSnmpDevice”: “true”,
    “pingAll”: “true”

I’m assuming you’re somewhat comfortable with working with strings in python here. The only thing that’s a little funny is the two str(network_address[1]) and str(network_address[-2]) lines.  This is the magic part of the ipaddress library that I mentioned above. These two lines of code do all the binary math for you. For those who aren’t used to subnet math, the first address in any IP network range is actually the network address, so we don’t want the 1, not 0, will be the first valid IP address in the range.  The last address in the range is actually the broadcast address, which is why we use -1 and not -2.  Pretty obvious when you think about it, right?

The network_address object is actually of the ‘ipaddress.IPv4Network’ class, so I’m using the str method to ensure that it’s a valid string when I’m adding joining it to the other text to create the payload. 

You can see from the following print statement that the first and last address in the range are exactly what we expect them to be. 

>>> print (json.dumps(json.loads(payload),indent=4))
"pingAll": "true",
"mode": "0",
"discoverNonSnmpDevice": "true",
"ipSection": {
"begin": "",
"end": ""


Sending the Request

So the last part of this code is just sending the actual request to the web server and seeing what happens.


r =, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents 
    if r.status_code == 200:
        print ("Auto-Discovery Successfully Started")
        print ("An Error has occured")

Using the requests library POST method, we create the HTTP call and include the PAYLOAD that we created above as the data for the message body of this request. 
I’m also evaluating the return of the request here. If the HTTP response code is 200 OK, then everything is good. If it’s anything else, then there’s an issue. As with almost any code on the planet, we could probably do a lot more error handling here, but more my purposes, this is more than ok.
Do you have better ways of doing anything I’ve got here?  Comments are welcome

Automating your NMS build using Python and Restful APIs Part 1 – Creating Operators

It’s a funny world we live in.  Unless you’re hiding under a rock, there’s been a substantial push in the industry over the last few years to move away from the CLI.  As someone right in the middle of this swirling vortex of inefficiency, I’d like to suggest that it’s not so much the CLI that’s the problem, but the fact that each box is handled on an individual basis and that human beings access the API through a keyboard. Not exactly next-generation technology.


I’ve been spending lot of time learning python and trying to apply it to my daily tasks. I started looking at the HP IMC Network Management station a few months ago. Mainly as a way to start learning about how I can use python to access RESTFul APIs as well as gain some hands on working with JSON and XML. As an observation, it’s interesting to be that I’m using a CLI ( python ) to configure an NMS ( IMC) that I’m using to avoid using the CLI. ( network devices ).   

I’ve got a project I’m working on to try and automate a bunch of the initial deployment functions within my NMS. There are a bunch of reasons to do this that are right for the business. Being able to push information gathering onto the customer, being able to use lower-skilled ( and hence lower paid!) resources to do higher level tasks. Being able to be more efficient in your delivery, undercut the competitors on price and over deliver on quality. It’s a really good project to sink my teeth and use some of my growing coding skills to make a difference to the business. 

This is the first post in which I’ll discuss and document some of the simple functions I’m developing. I make no claims to be a programmer, or even a coder. But I’m hoping someone can find something here usefull, and possibly get inspired to start sharing whatever small project you’re working on as well. 


Without further ado, let’s jump in and look at some code. 

What’s an Operator

Not familiar with HP IMC?  You should be! It’s chock full of goodness and you can get a 60 day free trial here.   In IMC an Operator is someone who has the right to log into the system and perform tasks in the NMS itself.  The reason they use the word operator vs. user is that there’s a full integrated BYOD solution available as an add-on module which treats a user as resource, which of course is not the same thing as an administrator on the system. 

IMC’s got a full RBAC system as well which allows you to assign different privilege levels to your operators, from view only to root-equiv access, as well as splitting up what devices you can perform actions on, as well as segmenting what actions you’re allowed to perform. Pretty powerful stuff once you understand how the pieces go together. 

Adding an Operator in the GUI

 This is a screen capture of the dialog used to add an operator into IMC.  It’s intuitive. You put the username in the username box, you put the password in the password box. Pretty easy right?

If you know what you’re doing and you’re a reasonably good typist, you can add probably add an operator in a minute or less.  

Screen Shot 2015 04 16 at 12 19 17 PM

Where do Operators come from?

Don’t worry. This isn’t a birds and bees conversation.  One of the biggest mistakes that I see when people start into any network management system project, whether that’s Solarwinds, Cisco Prime, What’s up Gold, HP NNMi, or HP IMC, is that they don’t stop to think about what they want/need to do before they start the project.  They typically sit down, start an auto-discovery and then start cleaning up afterwards.  Not exactly the best way to ensure success in your project is it?

When I get involved in a deployment project, I try to make sure I do as much of the information gathering up front. This means I have a bunch of excel spreadsheets that I ask them to fill in before I even arrive onsite. This ensures two things:

  1. I can deliver on what the customer actually wants
  2.  I know when I’m done the project and get to walk away and submit the invoice. 


I won’t make any judgement call on which one of those is more important. 



My Operator Template

My operator template looks like this


The values map to the screen shot above exactly as you would expect them to. 

Full name is the full name. Name is the login name, password is the password etc…  

The authType is a little less intuitive, although it is documented in the API docs. The authType maps to the authentication type above which allows you to choose how this specific operator is going to authenticate, through local auth, LDAP, or RADIUS. 

The operator group, which is “1” in my example, maps to the admin operator group which means that I have root-level access on the NMS and can do anything I want. Which is, of course, how it should be, right?


The Problem

So I’ve got a CSV file and I know it takes about one minute to create an operator because I can type and I know the system. Why am I automating this? Well, there are a couple of reasons for that.

  • Because I can and I want to gain more python experience
  • Because if I have to add ten operators, this just became ten minutes.
  • Because I already have the CSV file from the customer. Why would I type all this stuff again?
  • Because I can reuse this same format at every customer project I get involved in. 
  • Because I can blame any typos on the customer

Given time, I could add to this list, but let’s just get to the code. 

The Code

Authenticating to the Restful API

Although the auth examples in the eAPI documentation use the standard URLIB HTTP library, I’ve found that the requests library is MUCH more user friendly and easier to work with.

So I first create a couple of global variables called URL and AUTH that I will use to store the credentials.  


#url header to preprend on all IMC eAPI calls
url = None

#auth handler for eAPI calls
auth = None 

Now we get to the meat. I think this is pretty obvious, but this function gathers the username and password used to access the eAPI and then tests it out to make sure it’s valid. Once it’s verified as working ( The 200 OK check ). The credentials are then stored in the URL and AUTH global variables for use later on. I’m sure someone could argue that I shouldn’t be using global variables here, but it works for me. :) 
def imc_creds():
    ''' This function prompts user for IMC server information and credentuials and stores
    values in url and auth global variables'''
    global url, auth, r
    imc_protocol = input("What protocol would you like to use to connect to the IMC server: \n Press 1 for HTTP: \n Press 2 for HTTPS:")
    if imc_protocol == "1":
        h_url = 'http://'
        h_url = 'https://'
    imc_server = input("What is the ip address of the IMC server?")
    imc_port = input("What is the port number of the IMC server?")
    imc_user = input("What is the username of the IMC eAPI user?")
    imc_pw = input('''What is the password of the IMC eAPI user?''')
    url = h_url+imc_server+":"+imc_port
    auth = requests.auth.HTTPDigestAuth(imc_user,imc_pw)
    test_url = '/imcrs'
    f_url = url+test_url
        r = requests.get(f_url, auth=auth, headers=headers)
    except requests.exceptions.RequestException as e: #checks for reqeusts exceptions
        print ("Error:\n"+str(e))
        print ("\n\nThe IMC server address is invalid. Please try again\n\n")
    if r.status_code != 200: #checks for valid IMC credentials
        print ("Error: \n You're credentials are invalid. Please try again\n\n")
        print ("You've successfully access the IMC eAPI")
I”m using this function to gather the credentials of the operator accessing the API. By default when you first install HP IMC, these are admin/admin.    You could ask: Why don’t you just hardcode those into the script? Why bother with writing a function for this? 
Answer: Because I want to reuse this as much as possible and there are lots of things that you can do with the eAPI that you would NOT want just anyone doing. Plus, hardcoding the username and password of the NSM system that controls your entire network is just a bad idea in my books. 

Creating the Operators

I used the HP IMC eAPI /plat/operator POST call to as the basis for this call. 

Screen Shot 2015 04 16 at 1 06 21 PM


After doing a bit of testing, I arrived at a JSON array which would allow me to create an operator using the “Try it now” button in the API docs.  ( http://IMC_SERVER:PORTNUMBER/imcrs to access the online docs BTW ).

"password": "access4chris",
"fullName": "Christopher Young",
"defaultAcl": "0",
"operatorGroupId": "1",
"name": "cyoung",
"authType": "0",
"sessionTimeout": "10",
"desc": "admin account"

Using the Try it now button, you can also see the exact URL that is used to call this API. 

The 201 response below means that it was successfully executed. ( you might want to read up on HTTP codes as it’s not quite THAT simple, but for our purposes, it will work ).

Screen Shot 2015 04 16 at 1 10 46 PM

Now that I’ve got a working JSON array and the URL I need, I’ve got all the pieces I need to put this small function together. 

You can see the first thing I do is check to see if the auth and url variables are still set to None. If they are still None I use the IMC_CREDS function from above to gather them and store them. 


I create another variables called headers which stores the headers for the HTTP call. By default, the HP IMC eAPI will respond with XML. After working with XML for a few months, I decided that I prefer JSON. It just seems easier for me to work with.

This piece of code takes the CSV file that we created above and decodes the CSV file into a python dictionary using the column headers as the key and any additional rows as the values. This is really cool in that I can have ten rows, 50 rows, or 100 rows and it doesn’t matter. This script will handle any reasonable number you throw at it. ( I’ve tested up to 20 ).


#headers forcing IMC to respond with JSON content. XML content return is the default

headers = {‘Accept’: ‘application/json’, ‘Content-Type’: ‘application/json’,’Accept-encoding’: ‘application/json’}

def create_operator():
    if auth == None or url == None: #checks to see if the imc credentials are already available
    create_operator_url = ‘/imcrs/plat/operator’
    f_url = url+create_operator_url
    with open (‘imc_operator_list.csv’) as csvfile: #opens imc_operator_list.csv file
        reader = csv.DictReader(csvfile) #decodes file as csv as a python dictionary
        for operator in reader:
            payload = json.dumps(operator, indent=4) #loads each row of the CSV as a JSON string
            r =, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents
            if r.status_code == 409:
                print (“Operator Already Exists”)
            elif r.status_code == 201:
                print (“Operator Successfully Created”)

 Now you run this code and you’ve suddenly got all the operators in the CSV file imported into your system. 

Doing some non-scientific testing, meaning I counted in Mississippi’s, it took me about 3 seconds to create 10 operators using this method.  

Time isn’t Money

Contrary to the old saying, time isn’t actually money. We can always get more money. There’s lots of ways to do that. Time on the other hand can never be regained. It’s a finite resource and I’d like to spend as much of it as I can on things that I enjoy.  Creating Operators in an NMS doesn’t qualify.

Now, I hand off a CSV file to the customer, make them fill out all the usernames and passwords and then just run the script. they have all the responsibility for the content and all I have to do is a visual on the CSV file to make sure that they didn’t screw anything up.


Questions or comments or better ways to do this?  Feel free to post below. I’m always looking to learn.




Sometimes Size Matters: I’m sorry, but you’re just not big enough.

So now that I’ve got your attention…I wanted to put together some thoughts around a design principal of what I call the acceptable unit of loss, or AUL. 

Acceptable Unit of Loss: def. A unit to describe the amount of a specific resource that you’re willing to lose


Sounds pretty simple doesn’t it?  But what does it have to do with data networking? 

White Boxes and Cattle

2015 is the year of the white box. For those of you who have been hiding under a router for the last year, a white box is basically a network infrastructure device, right now limited to switches, that ships with no operating system.

The idea is that you:

  1. Buy some hardware
  2. Buy an operating system license ( or download an open source version) 
  3. Install the operating system on your network device
  4. Use DevOps tools to manage the whole thing

add Ice. Shake and IT operational goodness ensures. 

Where’s the beef?

So where do the cattle come in?  Pets vs. Cattle is something you can research elsewhere for a more thorough dealing, but in a nutshell, it’s the idea that pets are something that you love and care for and let sleep on the bed and give special treat to on Christmas.  Cattle on the other hand are things you give a number, feed from a trough, and kill off without remorse if a small group suddenly becomes ill. You replace them without a second thought. 

Cattle vs. Pets is a way to describe the operational model that’s been applied to the server operations at scale. The metaphor looks a little like this:

The servers are cattle. They get managed by tools like Ansible, Puppet, Chef, Salt Stack, Docker, Rocket, etc… which at a high level are all tools which allow for a new version of the server to be instantiated on a very specific configuration with little to no human intervention. Fully orchestrated,   

Your servers’s start acting up? Kill it. Rebuild it. Put in back in the herd. 

Now one thing that a lot of enterprise engineers seem to be missing is that this operational model is predicated on the fact that you’re application has been built out with a well thought out scale-out architecture that allows the distributed application to continue to operate when the “sick” servers are destroyed and will seamlessly integrate the new servers into the collective without a second thought. Pretty cool, no?


Are your switches Cattle?

So this brings me to the Acceptable Unit of Loss. I’ve had a lot of discussions with enterprise focused engineers who seem to believe that Whitebox and DevOps tools are going to drive down all their infrastructure costs and solve all their management issues.

“It’s broken? Just nuke it and rebuild it!”  “It’s broken? grab another one, they’re cheap!”

For me, the only way that this particular argument that customers give me is if there AUL metric is big enough.

To hopefully make this point I’ll use a picture and a little math:


Consider the following hardware:

  • HP C7000 Blade Server Chassis – 16 Blades per Chassis
  • HP 6125XLG Ethernet Interconnect – 4 x 40Gb Uplinks
  • HP 5930 Top of Rack Switch – 32 40G ports, but from the data sheet “ 40GbE ports may be split into four 10GbE ports each for a total of 96 10GbE ports with 8 40GbE Uplinks per switch.”

So let’s put this together

Screen Shot 2015 03 26 at 10 32 56 PM

So we’ll start with

  • 2 x HP 5930 ToR switches

For the math, I’m going to assume dual 5930’s with dual 6125XLGs in the C7000 chassis, we will assume all links are redundant, making the math a little bit easier. ( We’ll only count this with 1 x 5930, cool? )

  • 32 x 40Gb ports on the HP 5930 – 8  x 40Gb ports saved per uplink ) = 24 x 40Gb ports for connection to those HP 6125XLG interconnects in the C7000 Blade Chassis.
  • 24 x 40Gb ports from the HP 5930 will allow us to connect 6 x 6125XLGs for all four 40Gb uplinks. 

Still with me? 

  • 6 x 6125XLGs means 6 x C7000 which then translates into 6*16 physical servers.
Just so we’re all on the same page, if my math is right; we’ve got 96 physical servers on six blade chassis connected through the interconnects at 320Gb ( 4x40Gb x 2 – remember the redundant links?) to the dual HP 5930 ToR switches which will have (16*40Gb – 8*40Gb from each HP 5930) 640Gb of bandwidth out to the spine.  .  

If we go with a conservative VM to server ratio of 30:1,  that gets us to 2,880 VMs running on our little design. 

How much can you lose?

So now is where you ask the question:  

Can you afford to lose 2,880 VMs? 

According to the cattle & pets analogy, cattle can be replaced with no impact to operations because the herd will move on with out noticing. Ie. the Acceptable Unit of Lose is small enough that you’re still able to get the required value from the infrastructure assets. 

The obvious first objection I’m going to get is

“But wait! There are two REDUNDANT switches right? No problem, right?”

The reality of most of networks today is that they are designed to maximize the network throughput and efficient usage of all available bandwidth. MLAGG, in this case brought to you by HPs IRF, allows you to bind interfaces from two different physical boxes into a single link aggregation pipe. ( Think vPC, VSS, or whatever other MLAGG technology you’re familiar with ). 

So I ask you, what are the chances that you’re running the unit at below 50% of the available bandwidth? 

Yeah… I thought so.

So the reality is that when we lose that single ToR switch, we’re actually going to start dropping packets somewhere as you’ve been running the system at 70-80% utilization maximizing the value of those infrastructure assets. 

So what happens to TCP based application when we start to experience packet loss?  For a full treatment of the subject, feel free to go check out Terry Slattery’s excellent blog on TCP Performance and the Mathis Equation. For those of you who didn’t follow the math, let me sum it up for you.

Really Bad Things.  

On a ten gig link, bad things start to happen at 0.0001% packet loss. 

Are your Switches Cattle or Pets?

So now that we’ve done a bit of math and metaphors, we get to the real question of the day: Are you switches Cattle? Or are they Pets? I would argue that if your measuring your AUL in less that 2,000 servers, then you’re switches are probably Pets. You can’t afford to lose even one without bad things happening to your network, and more importantly the critical business applications that are being accessed by those pesky users. Did I mention they are the only reason the network exists?

Now this doesn’t mean that you can’t afford to lose a device. It’s going to happen. Plan for it. Have spares, Support Contracts whatever. But my point is that you probably won’t be able to go with a disposable infrastructure model like what has been suggested by many of the engineers I’ve talked to in recent months about why they want white boxes in their environments.

Wrap up

So are white boxes a bad thing if I don’t have a ton of servers and a well architected distributed application? Not at all! There are other reasons why white box could be a great choice for comparatively smaller environments. If you’ve got the right human resource pool internally with the right skill set, there are some REALLY interesting things that you can do with a white box switch running an OS like Cumulus linux. For some ideas,  check out this Software Gone Wild podcast with Ivan Pepelnjak and Matthew Stone.

But in general, if your metric for  Acceptable Unit of Loss is not measured in Data Centres, Rows, Pods, or Entire Racks, you’re probably just not big enough. 


Agree? Disagree? Hate the hand drawn diagram? All comments welcome below.