Serial numbers how I love thee…


No one really like serial numbers, but keeping track of them is one of the “brushing your teeth” activities that everyone needs to take care of. It’s like eating your brussel sprouts. Or listening to your mom. You’re just better of if you do it quickly as it just gets more painful over time.

Not only is it just good hygene, but you may be subject to regulations, like eRate in the United States where you have to be able to report on the location of any device by serial number at any point in time.

Trust me, having to play hide-and-go seek with an SSH session is not something you want to do when government auditors are looking for answers.

I’m sure you’ve already guessed what I’m about to say, but I”ll say it anyway…

There’s an API for that!!!

HPE IMC base platform has a great network assets function that automatically gathers all the details of your various devices, assuming of course they supportRFC 4133, otherwise known as the Entity MIB. On the bright side, most vendors have chosen to support this standards based MIB, so chances are you’re in good shape.

And if they don’t support it, they really should. You should ask them. Ok?

So without further ado, let’s get started.

 

Importing the required libraries

I’m sure you’re getting used to this part, but it’s import to know where to look for these different functions. In this case, we’re going to look at a new library that is specifically designed to deal with network assets, including serial numbers.

In [1]:
from pyhpeimc.auth import *
from pyhpeimc.plat.netassets import *
import csv
In [2]:
auth = IMCAuth("http://", "10.101.0.203", "8080", "admin", "admin")
In [3]:
ciscorouter = get_dev_asset_details('10.101.0.1', auth.creds, auth.url)
 

How many assets in a Cisco Router?

As some of you may have heard, HPE IMC is a multi-vendor tool and offers support for many of the common devices you’ll see in your daily travels.

In this example, we’re going to use a Cisco 2811 router to showcase the basic function.

Routers, like chassis switches have multiple components. As any one who’s ever been the victem owner of a Smartnet contract, you’ll know that you have individual components which have serial numbers as well and all of them have to be reported for them to be covered. So let’s see if we managed to grab all of those by first checking out how many individual items we got back in the asset list for this cisco router.

In [4]:
len(ciscorouter)
Out[4]:
7
 

What’s in the box???

Now we know that we’ve got an idea of how many assets are in here, let’s take a look to see exactly what’s in one of the asset records to see if there’s anything useful in here.

In [5]:
ciscorouter[0]
Out[5]:
{'alias': '',
 'asset': 'http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=1',
 'assetNumber': '',
 'boardNum': 'FHK1119F1DX',
 'bom': '',
 'buildInfo': '',
 'cleiCode': '',
 'containedIn': '0',
 'desc': '2811 chassis',
 'devId': '15',
 'deviceIp': '10.101.0.1',
 'deviceName': 'router.lab.local',
 'firmwareVersion': 'System Bootstrap, Version 12.4(13r)T11, RELEASE SOFTWARE (fc1)',
 'hardVersion': 'V04 ',
 'isFRU': '2',
 'mfgName': 'Cisco',
 'model': 'CISCO2811',
 'name': '2811 chassis',
 'phyClass': '3',
 'phyIndex': '1',
 'physicalFlag': '0',
 'relPos': '-1',
 'remark': '',
 'serialNum': 'FHK1119F1DX',
 'serverDate': '2016-01-26T15:20:40-05:00',
 'softVersion': '15.1(4)M, RELEASE SOFTWARE (fc1)',
 'vendorType': '1.3.6.1.4.1.9.12.3.1.3.436'}
 

What can we do with this?

With some basic python string manipulation we could easily print out some of the attributes that we want into what could easily turn into a nicely formated report.

Again realise that the example below is just a subset of what’s available in the JSON above. If you want more, just add it to the list.

In [7]:
for i in ciscorouter:
    print ("Device Name: " + i['deviceName'] + " Device Model: " + i['model'] +
           "\nAsset Name is: " + i['name'] + " Asset Serial Number is: " +
           i['serialNum']+ "\n")
 
Device Name: router.lab.local Device Model: CISCO2811
Asset Name is: 2811 chassis Asset Serial Number is: FHK1119F1DX

Device Name: router.lab.local Device Model: VIC2-2FXO
Asset Name is: 2nd generation two port FXO voice interface daughtercard on Slot 0 SubSlot 2 Asset Serial Number is: FOC11063NZ4

Device Name: router.lab.local Device Model:
Asset Name is: 40GB IDE Disc Daughter Card on Slot 1 SubSlot 0 Asset Serial Number is: FOC11163P04

Device Name: router.lab.local Device Model:
Asset Name is: AIM Container Slot 0 Asset Serial Number is:

Device Name: router.lab.local Device Model:
Asset Name is: AIM Container Slot 1 Asset Serial Number is:

Device Name: router.lab.local Device Model:
Asset Name is: C2811 Chassis Slot 0 Asset Serial Number is:

Device Name: router.lab.local Device Model:
Asset Name is: C2811 Chassis Slot 1 Asset Serial Number is:

 

Why not just write that to disk?

Although we could go directly to the formated report without a lot of extra work, we would be losing a lot of data which we may have use for later. Instead why don’t we export all the available data from the JSON above into a CSV file which can be later opened in your favourite spreadsheet viewer and manipulated to your hearst content.

Pretty cool, no?

In [9]:
keys = ciscorouter[0].keys()
with open('ciscorouter.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, keys)
    dict_writer.writeheader()
    dict_writer.writerows(ciscorouter)
 

Reading it back

Now we’ll read it back from disk to make sure it worked properly. When working with data like this, I find it useful to think about who’s going to be consuming the data. For example, when looking at this remember this is a CSV file which can be easily opened in python, or something like Microsoft Excel to manipuate further. It’s not realy intended to be read by human beings in this particular format. You’ll need another program to consume and munge the data first to turn it into something human consumable.

In [12]:
with open('ciscorouter.csv') as file:
    print (file.read())
 
firmwareVersion,vendorType,phyIndex,relPos,boardNum,phyClass,softVersion,serverDate,isFRU,alias,bom,physicalFlag,deviceName,deviceIp,containedIn,cleiCode,mfgName,desc,name,hardVersion,remark,asset,model,assetNumber,serialNum,buildInfo,devId
"System Bootstrap, Version 12.4(13r)T11, RELEASE SOFTWARE (fc1)",1.3.6.1.4.1.9.12.3.1.3.436,1,-1,FHK1119F1DX,3,"15.1(4)M, RELEASE SOFTWARE (fc1)",2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,10.101.0.1,0,,Cisco,2811 chassis,2811 chassis,V04 ,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=1,CISCO2811,,FHK1119F1DX,,15
,1.3.6.1.4.1.9.12.3.1.9.3.114,14,0,FOC11063NZ4,9,,2016-01-26T15:20:40-05:00,1,,,2,router.lab.local,10.101.0.1,13,,Cisco,2nd generation two port FXO voice interface daughtercard,2nd generation two port FXO voice interface daughtercard on Slot 0 SubSlot 2,V01 ,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=14,VIC2-2FXO,,FOC11063NZ4,,15
,1.3.6.1.4.1.9.12.3.1.9.15.25,30,0,FOC11163P04,9,,2016-01-26T15:20:40-05:00,1,,,2,router.lab.local,10.101.0.1,29,,Cisco,40GB IDE Disc Daughter Card,40GB IDE Disc Daughter Card on Slot 1 SubSlot 0,,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=30, ,,FOC11163P04,,15
,1.3.6.1.4.1.9.12.3.1.5.2,25,6,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,10.101.0.1,3,,Cisco,AIM Container Slot 0,AIM Container Slot 0,,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=25,,,,,15
,1.3.6.1.4.1.9.12.3.1.5.2,26,7,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,10.101.0.1,3,,Cisco,AIM Container Slot 1,AIM Container Slot 1,,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=26,,,,,15
,1.3.6.1.4.1.9.12.3.1.5.1,2,0,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,10.101.0.1,1,,Cisco,C2811 Chassis Slot,C2811 Chassis Slot 0,,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=2,,,,,15
,1.3.6.1.4.1.9.12.3.1.5.1,27,1,,5,,2016-01-26T15:20:40-05:00,2,,,0,router.lab.local,10.101.0.1,1,,Cisco,C2811 Chassis Slot,C2811 Chassis Slot 1,,,http://10.101.0.203:8080/imcrs/netasset/asset/detail?devId=15&phyIndex=27,,,,,15

 

What about all my serial numbers at once?

That’s a great question! I’m glad you asked. One of the most beautiful things about learning to automate things like asset gathering through an API is that it’s often not much more work to do something 1000 times than it is to do it a single time.

This time instead of using the get_dev_asset_details function that we used above which gets us all the assets associated with a single device, let’s grab ALL the devices at once.

In [13]:
all_assets = get_dev_asset_details_all(auth.creds, auth.url)
In [14]:
len (all_assets)
Out[14]:
1013
 

That’s a lot of assets!

Exactly why we automate things. Now let’s write the all_assets list to disk as well.

**note for reasons unknown to me at this time, although the majority of the assets have 27 differnet fields, a few of them actually have 28 different attributes. Something I’ll have to dig into later.

In [15]:
keys = all_assets[0].keys()
with open('all_assets.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, keys)
    dict_writer.writeheader()
    dict_writer.writerows(all_assets)
 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-e4c553049911> in <module>()
 3     dict_writer = csv.DictWriter(file, keys)
 4     dict_writer.writeheader()
----> 5dict_writer.writerows(all_assets)

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/csv.py in writerows(self, rowdicts)
 156         rows = []
 157         for rowdict in rowdicts:
--> 158rows.append(self._dict_to_list(rowdict))
 159         return self.writer.writerows(rows)
 160

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/csv.py in _dict_to_list(self, rowdict)
 147             if wrong_fields:
 148                 raise ValueError("dict contains fields not in fieldnames: "
--> 149 + ", ".join([repr(x) for x in wrong_fields]))  150         return [rowdict.get(key, self.restval) for key in self.fieldnames]
 151

ValueError: dict contains fields not in fieldnames: 'beginDate'
 

Well That’s not good….

So it looks like there are a few network assets that have a different number of attributes than the first one in the list. We’ll write some quick code to figure out how big of a problem this is.

In [16]:
print ("The length of the first items keys is " + str(len(keys)))
for i in all_assets:
    if len(i) != len(all_assets[0].keys()):
       print ("The length of index " + str(all_assets.index(i)) + " is " + str(len(i.keys())))
 
The length of the first items keys is 27
The length of index 39 is 28
The length of index 41 is 28
The length of index 42 is 28
The length of index 474 is 28
The length of index 497 is 28
The length of index 569 is 28
The length of index 570 is 28
The length of index 585 is 28
The length of index 604 is 28
The length of index 605 is 28
The length of index 879 is 28
The length of index 880 is 28
The length of index 881 is 28
The length of index 882 is 28
The length of index 883 is 28
The length of index 884 is 28
The length of index 885 is 28
The length of index 886 is 28
 

Well that’s not so bad

It looks like the items which don’t have exactly 27 attribues have exactly 28 attributes. So we’ll just pick one of the longer ones to use as the headers for our CSV file and then run the script again.

For this one, I’m going to ask you to trust me that the file is on disk and save us all the trouble of having to print out 1013 seperate assets into this blog post.

In [18]:
keys = all_assets[879].keys()
with open ('all_assets.csv', 'w') as file:
    dict_writer = csv.DictWriter(file, keys)
    dict_writer.writeheader()
    dict_writer.writerows(all_assets)
 

What’s next?

So now that we’ve got all of our assets into a CSV file which is easily consumable by something like Excel, you can now chose what to do with the data.

For me it’s interesting to see how vendors internally instrument their boxes. Some have serial numbers on power supplies and fans, some don’t. Some use the standard way of doing things. Some don’t.

From an operations perspective, not all gear is created equal and it’s nice to understand what’s supported when trying to make a purchasing choice for something you’re going to have to live with for the next few years.

If you’re looking at your annual SMARTnet upgrade, at least you’ve now got a way to easily audit all of your discovered environment and figure out what line cards need to be tied to a particualr contract.

Or you could just look at another vendor who makes your life easier. Entirely your choice.

@netmanchris

Advertisements

9 thoughts on “Serial numbers how I love thee…

  1. Hello Chris,

    I stumbled on your posts while trying to get my head around the REST interface of the HP IMC. From your blog posts, you seem to have some deep insight into it’s architecture, and I hope you can help point me to the right direction ..

    I am trying to get some basic ACL operations working on the IMC, via the REST interface. However, the API reference is sparse on many important and required details regarding ACL config..

    For instance,

    * what are all the possible values of aclType ?
    * within an aclRule object:
    1. what is the range of possible protocols? I have been able to figure out that UDP = 17; IP=256 .. but what are the rest?
    2. I have worked out that sourcePortOper / destPortOper should be 1 if a rule is to be applied to specific ports, and -1 if the rule is a non-port specific rule .. but I have also seen a value of 5 used in the API examples .. So, what is the full range of possible values for this attribute?

    And so many other such questions on the fields and what to set them to for desired configs .. I am not particularly asking for specific answers to these questions, but where can I obtain a more complete documentation for the fields than what’s available in the web based API reference accessible within an installation ?

    Thanks in advance ..

    1. Hey Chux,

      Just wanted to add notes to what we talked about offline!

      In a nutshell, the swaggerUI and the online docs are really the best place to get started, but for specific API calls which are more complicated, I find that creating example objects of whatever I want to manipulate in the GUI and then using the GET method in the API is really the best way to reverse engineer and ultimately, understand how the API is working for specific objects.

      This is one of those “integers are better than names” things that computers like. Primary keys as integers are just more efficient for computers, but the kinda suck for developers when we’re trying to understand what that primary key actually represents.

      Hopefully you’ve been able to figure this out! I’d love to see some examples of your code, if shareable, once you’re through with your project!

      @netmanchris

  2. Hi Chris,

    Thanks for your quick response .. and offline message. Your suggestion to work off GUI changes was very useful .. and I have been able to work out that the protocol numbers are standard IANA assigned numbers (http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml) .. I have also been able to address most of my other issues .. Thanks indeed ..

    In terms of my code, I would probably not be able to share it, because it will expose internal details of our network architecture. But the code is not solely focused on the IMC, but integrates into .. and the IMC part of it primarily just does a few basic things:
    — list all the ACLs assigned to a given device
    — from that list, grab 2 of the ACLs
    — for each of the 2 ACLs, add (or remove) some rules depending on command
    — delete the original ACLs
    — write the changed ACLs in place of the deleted ACLs

    best regards

    1. Hey Chux,

      Glad to see you figured this out! The. GUI first trick has been invaluable in reverse engineers some of the more complicated APIs. The docs are useful, but they could always use improvements.

      This is a great pointer.

      I just finished building out an Class for the IP Address management in Resource > Terminal Access > IP Address

      I might have to take a shot at the ACLM now. The API has SOOO many capabilities that it’s hard to decide where to start playing next!

      Glad you worked it out!

      @netmanchris

    1. Thanks for the pointer!

      I actually ended up using the Python ipaddress standard library. I have looked at netaddr before and it looks very useful, but for what I’m doing, the standard library had all the functionality I needed in it.

      I try to stick with the standard libraries unless there’s a specific reason for me to look elsewhere. Example – Requests vs URLIB2.

      Any comments on comparing the netaddr and the standard ipaddress libraries?

  3. Great strategy to use the standard libraries whenever possible .. Keeps your code leaner, and reduces side effects and dependencies that must be maintained .. As to comparisons with netaddr, there is none really .. Only attraction for netaddr is that ipaddress is not available for python 2, which is what I am using still. Besides that, I think ipaddress has all the features one is likely to need for working with IP addresses ..

    1. Hey Pradeep,

      To the best of my knowledge, IMC only uses the Entity MIB which is the standard way to expose asset information. For vendors who have chosen to not follow that standard… no luck. There are other ways to potentially get this content though using the Compliance Center or potentially to use a combination of python, regular expressions and the eAPI to write scripts to manually parse the text data as it comes back into a report which you could merge with the network assets data. I know that’s not what you’re looking for but it’s what I can think of.

      The other thing I would suggest is to request that your vendors support the standard MIB. It’s there for a reason and has existed a long time. 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s