Surfing your NMS with Python

Python is my favourite programming language. But then again, it’s also the only one I know. 🙂

I made a choice to go with python because, honestly, that’s what all the cool kids were doing at the time. But after spending the last year or so learning the basics of the language, I do find that it’s something that I can easily consume, and I’m starting to get better with all the different resources out there. BTW http://www.stackoverflow.com is your friend.  you will learn to love it.

On with the show…

So in this post, I’m going to show how to use python to build a quick script that will allow you to issue the RealTimeLocate API to the HP IMC server. In theory, you can build this against any RESTful API, but I make no promises that it will work without some tinkering.

Planning the project.

I’ve written before how I’m a huge fan of OPML tools like Mind Node Pro.  The first step for me was planning out the pieces I needed to make this:

  • usable in the future
  • actually work in the present
In this case I’m far more concerned about the present as I’m fairly sure that I will look back on this code in a year from now and think some words that I won’t put in print.
Aside: I’ve actually found that using the troubleshooting skills I’ve honed over the years as a network engineer helps me immensely when trying to decompose what pieces will need to go in my code. I actually think that Network Engineers have a lot of skills that are extremely transportable to the programming domain. Especially because we tend to think of the individual components and the system at the same time, not to mention our love of planning out failure domains and forcing our failures into known scenarios as much as possible.

Screen Shot 2014 11 24 at 9 33 50 PM

Auth Handler

Assuming that the RESTful service you’re trying to access will require you to authenticate, you will need an authentication handler to deal with the username/password stuff that a human being is usually required to enter. There are a few different options here. Python actually ships with URLLIB or some variant depending not the version of python you’re working with.  For ease of use reasons, and because of a strong recommendation from one of my coding mentors, I chose to use the REQUESTS library.  This is not shipped by default with the version of python you download over at http://www.python.org but it’s well worth the effort over PIP’ing it into your system.

The beautiful thing about REQUEST’s is that the documentation is pretty good and easily readable.

In looking through the HP IMC eAPI documentation and the Request library – I settled on the DigestAuth

Screen Shot 2014 11 24 at 10 17 07 PM

So here’s how this looks for IMC.

Building the Authentication Info

>>>import requests   #imports the requests library you may need to PIP this in if you don’t have it already

>>> from requests.auth import HTTPDigestAuth    # this imports the HTTPDigestAuth method from the request library.
>>>
>>> imc_user = ”’admin”’   #The username used to auth against the HP IMC Server
>>> imc_pw = ”’admin”’   #The password of the account used to auth against the HP IMC Server.
>>>  

auth = requests.auth.HTTPDigestAuth(imc_user,imc_pw)     #This puts the username and password together and stores them as a variable called auth

We’ve now built the auth handler to use the username “admin” with the password “admin”. For a real environment, you’ll probably want to setup an Operator Group with only access to the eAPI functions and lock this down to a secret username and password. The eAPI is power, make sure you protect it.

Building the URL

So for this to work, I need to assign a value to the host_ip  variable above so that the URL will complete with a valid response. The other thing to watch for are types. Python can be quite forgiving at times, but if you try to add to objects of the wrong type together… it mostly won’t work.  So we need to make sure the host_ip is a string and the easiest way to do that is to put three quotes around the value.

In a “real” program, I would probably use the input function to allow this variable to be input as part of the flow of the program, but we’re not quite there yet.

>>> host_ip = ”’10.101.0.109”’   #variable that you can assign to a host you want to find on the network
>>> h_url = ”’http://”’    #prefix for building URLs use HTTP or HTTPS
>>> imc_server = ”’10.3.10.220:8080”’   #match port number of IMC server default 8080 or 8443
>>> url = h_url+imc_server    #combines the h_url and the IP address of the IMC box as a base URL to use later
>>> find_ip_host_url = (”’/imcrs/res/access/realtimeLocate?type=2&value=”’+host_ip+”’&total=false”’)   # This is the RealTimeLocate API URL with a variable set
>>>

Putting it all together.

This line takes puts the url that we’re going to send to the web server all together. You could ask “Hey man, why didn’t you just drop the whole string in one variable to begin with? “   That’s a great question.  There’s a concept in programming called DRY. (Don’t Repeat Yourself).  The idea is that when you write code, you should never write the same thing twice. Think in a modular fashion which would allow you to reuse pieces of code again and again.

In this example, I can easily write another f_url variable and assign to it another RESTful API that gets me something interesting from the HP IMC server. I don’t need to write the h_url portion or the server IP address portion of the header.  Make sense?

>>> f_url = url + find_ip_host_url
>>>    #  This is a very simple mathematical operation that puts together the url and the f_url which will product the HTTP call. 

Executing the code.

Now the last piece is where we actually execute the code. This will issue a get request, using the requests library.  It will use the f_url as the actual URL it’s going to pass, and it will use the variable auth that we created in the Authentication Info step above to automatically populate the username and password.

The response will get returned in a variable called r.

>>> r = requests.get(f_url, auth=auth)    #  Using the requests library get method, we’re going to pass the f_url as the argument for the URL we’re going to access and pass auth as the auth argument to define how we authenticate Pretty simple actually . 
>>>

The Results

So this is the coolest part. We can now see what’s in r.  Did it work? Did we find out lost scared little host?  Let’s take a look.

>>> r
<Response [200]>

Really? That’s it? .

The answer is “yes”.  That’s what’s been assigned to the variable r.  200 OK may look familiar to you voice engineers who know SIP and it means mostly the same thing here. This is a response code to let you know that your request was successful – But not what we’re looking for. I want that content, right?  If I do a type(r) which will tell me what python knows about what kind of object r is I will get the following.

>>> type(r)

<class ‘requests.models.Response’>

So this tells us that maybe we need to go back to the request documentation and look for info on the responses. Now we know to access the part of the response that I wanted to see, which is the reply to my request on where the host with ip address 10.101.0.111 is actually located on the network.

So let’s try out one of the options and see what we get

>>> r.content
b'<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?><list><realtimeLocation><locateIp>10.101.0.111</locateIp><deviceId>4</deviceId><deviceIp>10.10.3.5</deviceIp><ifDesc>GigabitEthernet1/0/16</ifDesc><ifIndex>16</ifIndex></realtimeLocation></list>’

How cool is that. We put in an IP address and we actually learned four new things about that IP address without touching a single GUI. And the awesome part of this?  This works across any of the devices that HP IMC supports.

Where to from here?

So we’ve just started on our little journey here.  Now that we have some hints to the identity of the network devices and specific interface that is currently harbouring this lost host, we need to use that data as hints to continue filling in the picture.

But that’s in the next blog…

Comments or Questions?  Feel free to post below!

A Network Services Platform

So things are starting to get interesting with the HP IMC eAPI that was recently released. It’s really amazing to see the types of creative projects when technical people are presented with new toys. 🙂

So for those of you who didn’t read my last eAPI blog post, let me catch you up. The eAPI is a RESTFul inteface that allows programmers, or scripters to leverage the various network services that HP IMC presents.

Thanks@ioshintsfor a quick look at SNMP vs. RESTfull interface

Basically it looks a little like this.

note: This is not a full list of the IMC modules or services. Check out the HP website for a complete list.

The RESTfull inteface presents the services in a XML format which is consumable to any programming language that can parse XML. ( I’m not a programmer, but that’s pretty much all of the current ones from what I understand ).

Those services are then applied to specific devices. But what’s COOL about this, is the following.

Say you want to change a VLAN on a bunch of ports. Some of those happen to be HP Comware switches, some of them happen to be HP Procurve Switches, and some of them happen to be Cisco switches. The IMC device adapters at the bottom do all the work for you, providing a device abstraction layer so that you can just say ” add VLAN” rather than having to worry about the syntax of all the individual devices.

So what’s actually available in the HP IMC eAPI? Well you can checkout @neelixx’s blog for the documentation. This is the first release, but I’m told that the eAPI will continue to grow with each future release of the platform AND the modules.

But I think what’s a LOT more interesting is some of the projects that have started to creep up.

For example

1) Wouldn’t it be cool if when you sent someone an outlook invite for a meeting in your office that your network access control system would automatically create guest accounts for the day of the meeting and send them to your guests?

2) Wouldn’t it be cool if when your support desk could simply click on a user in Microsoft Lync and automatically see where they have been logged in the network? Check out what access service is assigned to them. Maybe they are having trouble accessing some resources and you want to make sure they are in the right VLAN.

I’ve also started to see other apps pop up such as an application that searches the entire network for the mac addresses of lost laptops and locates the interface they are plugged into. Pretty handy for a hospital where a lost laptop with patient data is a nightmare. Or something as simple as an app for a a College which allows the teacher to shut down all the interfaces for the switches which are in their classroom, and then to turn them all back on with a click of the button.

No login to the NMS.

No call to the help desk.

Just shutting down the ports when the students aren’t listening, and turning them back on when it’s time to work.

What about you guys? HP has given you some color. What are YOU going to paint?

HP IMC’s New eAPI

Now that I got rum-pooh out of my system…  on to a slightly more technical post.

Not sure if anyone of you caught the recent announcements about the new eAPI from HP’s Intelligent Management center.  In a nutshell, this is a RESTful API which allows programatic access to almost all ( maybe all?) of the IMC functions through an HTTP(s) interface.

Now I’m not a programer at all, but I like to think I have a working knowledge of programing logic. At least enough to give a half-decent programmer enough information to get the job done.

So when I had a co-worker present me with a problem earlier this week, I thought “Hey, I wonder what this new eAPI can really do?”.  I did mention I’m not a programer right?  After this little exercise, I’m thinking I might just have to pick up some scripting skills this year. 🙂

So what’s the value of the eAPI? It functionally allows IMC to act as the progamatic upper layer APIs and abstracts the actual management task from the underlying hardware devices.

In less complicated terms; it means that a program can say  “Hey, IMC change the VLAN on this port” and IMC, assuming IMC actually supports that particular device, it will change the VLAN on that port.

INDEPENDANT OF THE ACTUAL VENDOR

Yup. That’s right.  IMC doesn’t care if that device is a HP 5500EI ( comware ), a HP 3800 ( procurve), or even a Cisco Catalyst 3560.  From the perspective of the developer on the other side. It’s as simple as “Hey, IMC change the VLAN on this port”.

So the actual challenge I was given was the following.

” A customer wants to take a bar-code reader, scan in the MAC address of a device, plug it into the network, push a button and then have IMC automatically put it in the right VLAN”.

Now first I had to break that down into the various components.

1) Bar-code reader scans the MAC address

2) Program has to capture the MAC address for use in the %mac-address% variable in the script.

3) Find the device in the network

Hmmm… this could be more difficult than I thought.

So, I need to mock this up, so I break out to a windows CMD prompt.

Ping a known address ( my Synology NAS — LOVE THIS PRODUCT ).  And then put do a arp -a  to get the following output

10.101.0.51           00-11-32-10-03-8b

Now if I was using the IMC web-interface, I would just use the   Resource-Terminal Access-Real-Time Location feature which will, you guessed it, locate a host in real-time using the mac-address or the IP address.

Hmmm… that kinda sucks for output for the script to leverage.  So I went and looked at the eAPI documentation and came out with this little baby

The eAPI call is the following  ( if you wanted to search for an IP address you would use type=2 instead of 1 )

( Don’t click this, it won’t take you anywhere)

http://10.101.0.201:8080/imcrs/res/access/realtimeLocate?type=1&value=00-11-32-10-03-8b

The return is the following

<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes” ?>

<list>

<realtimeLocation>

<locateIp>00:11:32:10:03:8b</locateIp>

<deviceIp>10.101.0.221</deviceIp>

<ifDesc>GigabitEthernet1/0/21</ifDesc>

</realtimeLocation>

</list>

Yup. Good old XML. Easy to apply transforms or grab variables.  Programmers love this stuff.

4) So now I have the device IP (switch) that this is plugged into, and the ifDesc which is the actual interface it’s located on. So now I have to figure out how to apply the VLAN to this interface. So I break out the trusty eAPI documentation and start looking for the VLAN section.

Hmmm… I have the devIP and the ifDesc.. not the devID and the ifIndex

note to self: Feedback to the developers to have the first command return the devID and the ifIndex variables

So now I have to find the devID and the ifIndex for that devIP and intDesc

5a) Now if I was on the trusty IMC web interface, I would go to the device resource page… hmmm… that doesn’t appear to be there. I guess instead, let’s go to the eAPI documentation and look for something that looks like a dev query.

Yup. It’s actually called devquery. And it looks like I can filter based on the device IP.

Cool. So now I can search for the specific device IP and hope we get the devID variable back that we need for the VLAN call.

http://10.101.0.201:8080/imcrs/plat/res/device?ip=10.101.0.221

The return is the following

  <?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes” ?>

<list>

<device>

<id>15</id>     ———————————————- This is the device ID that we need to reference later

<label>HP_5500EI</label>

<ip>10.101.0.221</ip>

<mask>255.255.255.0</mask>

<status>1</status>

<statusDesc>Normal</statusDesc>

<sysName>HP_E5500EI</sysName>

<contact>HP Montreal</contact>

<location>Marlborough, MA 01752 USA</location>

<sysOid>1.3.6.1.4.1.43.1.16.4.3.36</sysOid>

<sysDescription>HP_5500EI</sysDescription>

<devCategoryImgSrc>Switch</devCategoryImgSrc>

<topoIconName>stack</topoIconName>

<categoryId>1</categoryId>

<symbolId>1022</symbolId>

<symbolName>HP_5500EI</symbolName>

<symbolType>3</symbolType>

<symbolDesc>HP_5500EI</symbolDesc>

<symbolLevel>3</symbolLevel>

<parentId>1003</parentId>

<typeName>3Com S4800G PWR 24-Port</typeName>

<mac>00:1e:c1:dc:fc:01</mac>

<link op=”GET” rel=”self” href=”http://10.101.0.201:8080/imcrs/plat/res/device/15” />

</device>

</list>

There it is.  DevID is  “15”.

5b) So now we need to figure out that ifIndex value associated with <ifDesc>GigabitEthernet1/0/21</ifDesc> that we pulled above.

If I was in the webinterface, I would simply go to the device ( 10.101.0.221 ), click on the interface list, click on interface Gig 1/0/21 and I would pull out the ifIndex from the interface…

But again, those programmers don’t want HTML, they want an easy XML output that they can play with. So let’s find that…

http://10.101.0.201:8080/imcrs/plat/res/device/15/interface?start=1&size=100

This returns a whole bunch of data for all the interfaces on the switch, but I’m sure that any decent programmer can write a regex expression to only return the one who’s ifDesc value is for Gig 1/0/21, right?

<interface>

<ifIndex>21</ifIndex>      ——————————————–In this case, the ifindex value is the same as the port number. That’s not always going to be true. This is the other variable for the set VLAN

<ifType>6</ifType>

<ifTypeDesc>ETHERNETCSMACD</ifTypeDesc>

<ifDescription>GigabitEthernet1/0/21</ifDescription>

<adminStatus>1</adminStatus>

<adminStatusDesc>Up</adminStatusDesc>

<showStatus>2</showStatus>

<statusDesc>Down</statusDesc>

<operationStatus>2</operationStatus>

<operationStatusDesc>Down</operationStatusDesc>

<ifspeed>10000000</ifspeed>

<appointedSpeed>-1</appointedSpeed>

<ifAlias>GigabitEthernet1/0/15 Interface</ifAlias>

<phyAddress>00:1e:c1:dc:fc:4f</phyAddress>

<mtu>1522</mtu>

<lastChange>4 day(s) 21 hour(s) 39 minute(s) 50 second(s) 990 millisecond(s)</lastChange>

<lastChangeTime>42359099</lastChangeTime>

<filterTrapStatus>0</filterTrapStatus>

</interface>

So now we have the DevID     15     and the ifIndex    for the actual interface where that MAC-address is located.

So let’s go back to the set that VLAN

Let’s assume that you wanted to put the device in VLAN 20,   you would run the following

http://10.101.0.201:8080/imcrs/vlan/20?devId=15&ifIndex21

That’s about it for the task. Now any decent programmer is going to have to put in some checking and error handling, for instance, you might want to check whether or not that VLAN actually EXISTS on the switch. ( Can’t put a VLAN on a port if the VLAN doesn’t exist on the switch, right? ) or maybe return an error if the MAC-address is actually seen on two interfaces, but in a nutshell that’s it.

note: I would also suggest that the dev actually bounce the port to make sure that the device hasnt’ gotten locked in with a DHCP address on the wrong subnet.

/plat/res/device/{deviceId}/interface/{ifIndex}/down   to down the interface

so for use that would be  /plat/res/device/15/interface/21/down

and then immediately do a

/plat/res/device/{deviceId}/interface/{ifIndex}/up

or again for us  /plat/res/device/15/21/up

Now whether the actual switch commands are “switchport acces vlan 20 ”  or ” port access vlan 20 ” or some other variation on a theme doesn’t actually matter to your devOps team. They just write the code to follow the steps and IMC and the eAPI will take care of the rest.

Pretty cool stuff. 🙂

@netmanchris