Automating your NMS build using Python and Restful APIs Part 2 – Adding Devices though Auto-Discovery

This is the second in a series of posts where I’ll be using a RESTful API to automate a bunch of the initial deployment functions within my NMS. There are a bunch of reasons to do this that are right for the business. Being able to push information gathering onto the customer, being able to use lower-skilled ( and hence lower paid!) resources to do higher level tasks. Being able to be more efficient in your delivery, undercut the competitors on price and over deliver on quality. It’s a really good project to sink my teeth and use some of my growing coding skills to make a difference to the business.

Adding Devices

A Network Management System without devices is a sad, sad thing. It’s just a lonely piece of code with no purpose in life. Empty graphs and arm notifications with nothing to notify. Today we’re going to look at adding devices through launching a basic network range auto discovery. In a follow up post, I’ll be adding device through a CSV import which is my preferred method.

One thing which I will not be covering in this post to save some time and space is how to authenticate to the NMS system which is covered here.

Launching an Autodiscovery

One of the worst things about installing an NMS in a customers environment is the inevitable scope creep that happens. The major reason that this happens is that most customers looking for an NMS are doing so because they really don’t have a clue what’s in their network, and potentially not even how they are configured. They know it’s kinda working most of the time, but not much more than that.

Typically, you have a statement of work to discover N number of devices. The customer says “I don’t know all the IP addresses, why don’t you just discover this range of IP addresses. So you launch a discovery from your tool and suddenly, you’ve got 2*N devices in your database! And because they are in your database, you’re now responsible for getting them up and running. Even though your contract is for only N devices, you don’t want to disappoint your customer right?

There other things that can easily go wrong in an auto discovery as well like

Mis-matched SNMP strings: You have the SNMP read, but not the write strings.

This can cause some unpredictable results that are SOOO fun to troubleshoot.

Missing CLI credentials: You have SNMP strings, but not the CLI strings.

There are some things that are only accessible through CLI. It’s great that we’re moving into an era where decent programatic APIs are available devices, but for the mass majority of your infrastructure, there’s a ton of functions which are only available through the good old command line interface. If you don’t have the right credentials, you’re going to have problems. In my experience, unless they have a centralized authentication system in place, like Cisco ACS, FreeRADIUS or other; you’re probably going to have to troubleshoot half of the device credentials.

Let’s look at the code.

First I’ll show the entire function complete, and then we’ll break down the sections individually. Again, I’ll be skipping the imc_creds() function as it’s covered in the earlier post.

import requests, json, sys, time, subprocess, csv, os, ipaddress

def plat_auto_discover():

    if auth == None or url == None: # checks to see if the imc credentials are already available
    imc_creds()
    auto_discover_url = '/imcrs/plat/res/autodiscover/start'
    f_url = url + auto_discover_url
    network_address = input(
    '''What is the the network address of the range you wish to discover?\nPlease input the address in the format "192.168.0.0/24": ''')
    # end_address = input('''What is the last address of the network range you wish to discover?\nIPv4 Address: ''')
    try:
        network_address = ipaddress.ip_network(network_address)
    except ValueError:
        print("You have entered an invalid network address. Please try again.")
        time.sleep(2)
        print ('\n'*80)
        plat_auto_discover()
    payload = ''' {
    "mode": "0",
    "ipSection": {
        "begin": "''' + str(network_address[1]) + '''",
        "end": "''' + str(network_address[-2]) + '''"
        },
    "discoverNonSnmpDevice": "true",
    "pingAll": "true"
    }
    '''
    r = requests.post(f_url, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents 
    if r.status_code == 200:
        print ("Auto-Discovery Successfully Started")
    else:
        print ("An Error has occured")

Gathering the network range

So the first task here is gathering the actual network range that we want to run the auto discover over. For this example, we’ll keep this very basic and use a simple 192.168.0.0/24 network. This is going to cause the system to scan the entire 254 hosts of the subnet. One of the other things to keep in mind here is that I’m going to use the default system templates for SNMP and Telnet which is why I’m not gathering them here. This isn’t magic, right?

So I’m using the input function to gather the IP address network range that I want to discover. I’m also using the python ipaddress standard library to caste the user input as a network_address to ensure that it’s a valid input to the final function.

network_address = input(
”’What is the the network address of the range you wish to discover?\nPlease input the address in the format “192.168.0.0/24″: ”’)
# end_address = input(”’What is the last address of the network range you wish to discover?\nIPv4 Address: ”’)
try:
network_address = ipaddress.ip_network(network_address)
except ValueError:
print(“You have entered an invalid network address. Please try again.”)
time.sleep(2)
print (‘\n’*80)
plat_auto_discover()

So essentially, this code performs three steps

Gathers the desired network address discovery range in the 192.168.0.0/24 format
Attempts to use the ipaddress.ip_network method from the python ipaddress library to test if the user input was valid, and stores it in the variable network_address
If the ipaddress.ip_network method fails, this will raise an error and re-run the function to gather input in the right format.

The other nice thing about storing the user input as a ipaddress.ip_network object means that we can easily gather the start and end of this subnet based on the subnet math that’s included in the library. I’d always prefer to have someone/something else do the binary math. 🙂

Creating the JSON Array

This particular API uses a JSON array in the HTTP message body to gather all of the information used to launch the auto-discovery.

payload = ”’

{
“mode”: “0”,
“ipSection”: {
“begin”: “”’ + str(network_address[1]) + ”'”,
“end”: “”’ + str(network_address[-2]) + ”'”
},
“discoverNonSnmpDevice”: “true”,
“pingAll”: “true”
}
”’

I’m assuming you’re somewhat comfortable with working with strings in python here. The only thing that’s a little funny is the two str(network_address[1]) and str(network_address[-2]) lines. This is the magic part of the ipaddress library that I mentioned above. These two lines of code do all the binary math for you. For those who aren’t used to subnet math, the first address in any IP network range is actually the network address, so we don’t want the 1, not 0, will be the first valid IP address in the range. The last address in the range is actually the broadcast address, which is why we use -1 and not -2. Pretty obvious when you think about it, right?

The network_address object is actually of the ‘ipaddress.IPv4Network’ class, so I’m using the str method to ensure that it’s a valid string when I’m adding joining it to the other text to create the payload.

You can see from the following print statement that the first and last address in the range are exactly what we expect them to be.

>>> print (json.dumps(json.loads(payload),indent=4))
{
 "pingAll": "true",
 "mode": "0",
 "discoverNonSnmpDevice": "true",
 "ipSection": {
 "begin": "192.168.0.1",
 "end": "192.168.0.254"
 }
}

Sending the Request

So the last part of this code is just sending the actual request to the web server and seeing what happens.

r = requests.post(f_url, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents 
    if r.status_code == 200:
        print ("Auto-Discovery Successfully Started")
    else:
        print ("An Error has occured")

Using the requests library POST method, we create the HTTP call and include the PAYLOAD that we created above as the data for the message body of this request.

I’m also evaluating the return of the request here. If the HTTP response code is 200 OK, then everything is good. If it’s anything else, then there’s an issue. As with almost any code on the planet, we could probably do a lot more error handling here, but more my purposes, this is more than ok.

Do you have better ways of doing anything I’ve got here? Comments are welcome

@netmanchris

Automating your NMS build using Python and Restful APIs Part 1 – Creating Operators

It’s a funny world we live in. Unless you’re hiding under a rock, there’s been a substantial push in the industry over the last few years to move away from the CLI. As someone right in the middle of this swirling vortex of inefficiency, I’d like to suggest that it’s not so much the CLI that’s the problem, but the fact that each box is handled on an individual basis and that human beings access the API through a keyboard. Not exactly next-generation technology.

I’ve been spending lot of time learning python and trying to apply it to my daily tasks. I started looking at the HP IMC Network Management station a few months ago. Mainly as a way to start learning about how I can use python to access RESTFul APIs as well as gain some hands on working with JSON and XML. As an observation, it’s interesting to be that I’m using a CLI ( python ) to configure an NMS ( IMC) that I’m using to avoid using the CLI. ( network devices ).

I’ve got a project I’m working on to try and automate a bunch of the initial deployment functions within my NMS. There are a bunch of reasons to do this that are right for the business. Being able to push information gathering onto the customer, being able to use lower-skilled ( and hence lower paid!) resources to do higher level tasks. Being able to be more efficient in your delivery, undercut the competitors on price and over deliver on quality. It’s a really good project to sink my teeth and use some of my growing coding skills to make a difference to the business.

This is the first post in which I’ll discuss and document some of the simple functions I’m developing. I make no claims to be a programmer, or even a coder. But I’m hoping someone can find something here usefull, and possibly get inspired to start sharing whatever small project you’re working on as well.

Without further ado, let’s jump in and look at some code.

What’s an Operator

Not familiar with HP IMC? You should be! It’s chock full of goodness and you can get a 60 day free trial here. In IMC an Operator is someone who has the right to log into the system and perform tasks in the NMS itself. The reason they use the word operator vs. user is that there’s a full integrated BYOD solution available as an add-on module which treats a user as resource, which of course is not the same thing as an administrator on the system.

IMC’s got a full RBAC system as well which allows you to assign different privilege levels to your operators, from view only to root-equiv access, as well as splitting up what devices you can perform actions on, as well as segmenting what actions you’re allowed to perform. Pretty powerful stuff once you understand how the pieces go together.

Adding an Operator in the GUI

This is a screen capture of the dialog used to add an operator into IMC. It’s intuitive. You put the username in the username box, you put the password in the password box. Pretty easy right?

If you know what you’re doing and you’re a reasonably good typist, you can add probably add an operator in a minute or less.

Screen Shot 2015 04 16 at 12 19 17 PM

Where do Operators come from?

Don’t worry. This isn’t a birds and bees conversation. One of the biggest mistakes that I see when people start into any network management system project, whether that’s Solarwinds, Cisco Prime, What’s up Gold, HP NNMi, or HP IMC, is that they don’t stop to think about what they want/need to do before they start the project. They typically sit down, start an auto-discovery and then start cleaning up afterwards. Not exactly the best way to ensure success in your project is it?

When I get involved in a deployment project, I try to make sure I do as much of the information gathering up front. This means I have a bunch of excel spreadsheets that I ask them to fill in before I even arrive onsite. This ensures two things:

I can deliver on what the customer actually wants
I know when I’m done the project and get to walk away and submit the invoice.

I won’t make any judgement call on which one of those is more important.

My Operator Template

My operator template looks like this

The values map to the screen shot above exactly as you would expect them to.

Full name is the full name. Name is the login name, password is the password etc…

The authType is a little less intuitive, although it is documented in the API docs. The authType maps to the authentication type above which allows you to choose how this specific operator is going to authenticate, through local auth, LDAP, or RADIUS.

The operator group, which is “1” in my example, maps to the admin operator group which means that I have root-level access on the NMS and can do anything I want. Which is, of course, how it should be, right?

The Problem

So I’ve got a CSV file and I know it takes about one minute to create an operator because I can type and I know the system. Why am I automating this? Well, there are a couple of reasons for that.

Because I can and I want to gain more python experience
Because if I have to add ten operators, this just became ten minutes.
Because I already have the CSV file from the customer. Why would I type all this stuff again?
Because I can reuse this same format at every customer project I get involved in.
Because I can blame any typos on the customer

Given time, I could add to this list, but let’s just get to the code.

The Code

Authenticating to the Restful API

Although the auth examples in the eAPI documentation use the standard URLIB HTTP library, I’ve found that the requests library is MUCH more user friendly and easier to work with.

So I first create a couple of global variables called URL and AUTH that I will use to store the credentials.

#url header to preprend on all IMC eAPI calls
url = None

#auth handler for eAPI calls
auth = None

Now we get to the meat. I think this is pretty obvious, but this function gathers the username and password used to access the eAPI and then tests it out to make sure it’s valid. Once it’s verified as working ( The 200 OK check ). The credentials are then stored in the URL and AUTH global variables for use later on. I’m sure someone could argue that I shouldn’t be using global variables here, but it works for me. :)

def imc_creds():
    ''' This function prompts user for IMC server information and credentuials and stores
    values in url and auth global variables'''
    global url, auth, r
    imc_protocol = input("What protocol would you like to use to connect to the IMC server: \n Press 1 for HTTP: \n Press 2 for HTTPS:")
    if imc_protocol == "1":
        h_url = 'http://'
    else:
        h_url = 'https://'
    imc_server = input("What is the ip address of the IMC server?")
    imc_port = input("What is the port number of the IMC server?")
    imc_user = input("What is the username of the IMC eAPI user?")
    imc_pw = input('''What is the password of the IMC eAPI user?''')
    url = h_url+imc_server+":"+imc_port
    auth = requests.auth.HTTPDigestAuth(imc_user,imc_pw)
    test_url = '/imcrs'
    f_url = url+test_url
    try:
        r = requests.get(f_url, auth=auth, headers=headers)
    except requests.exceptions.RequestException as e: #checks for reqeusts exceptions
        print ("Error:\n"+str(e))
        print ("\n\nThe IMC server address is invalid. Please try again\n\n")
        imc_creds()
    if r.status_code != 200: #checks for valid IMC credentials
        print ("Error: \n You're credentials are invalid. Please try again\n\n")
        imc_creds()
    else:
        print ("You've successfully access the IMC eAPI")

I”m using this function to gather the credentials of the operator accessing the API. By default when you first install HP IMC, these are admin/admin. You could ask: Why don’t you just hardcode those into the script? Why bother with writing a function for this?

Answer: Because I want to reuse this as much as possible and there are lots of things that you can do with the eAPI that you would NOT want just anyone doing. Plus, hardcoding the username and password of the NSM system that controls your entire network is just a bad idea in my books.

Creating the Operators

I used the HP IMC eAPI /plat/operator POST call to as the basis for this call.

Screen Shot 2015 04 16 at 1 06 21 PM

After doing a bit of testing, I arrived at a JSON array which would allow me to create an operator using the “Try it now” button in the API docs. ( http://IMC_SERVER:PORTNUMBER/imcrs to access the online docs BTW ).

    {
 "password": "access4chris",
 "fullName": "Christopher Young",
 "defaultAcl": "0",
 "operatorGroupId": "1",
 "name": "cyoung",
 "authType": "0",
 "sessionTimeout": "10",
 "desc": "admin account"
}

Using the Try it now button, you can also see the exact URL that is used to call this API.

The 201 response below means that it was successfully executed. ( you might want to read up on HTTP codes as it’s not quite THAT simple, but for our purposes, it will work ).

Screen Shot 2015 04 16 at 1 10 46 PM

Now that I’ve got a working JSON array and the URL I need, I’ve got all the pieces I need to put this small function together.

You can see the first thing I do is check to see if the auth and url variables are still set to None. If they are still None I use the IMC_CREDS function from above to gather them and store them.

I create another variables called headers which stores the headers for the HTTP call. By default, the HP IMC eAPI will respond with XML. After working with XML for a few months, I decided that I prefer JSON. It just seems easier for me to work with.

This piece of code takes the CSV file that we created above and decodes the CSV file into a python dictionary using the column headers as the key and any additional rows as the values. This is really cool in that I can have ten rows, 50 rows, or 100 rows and it doesn’t matter. This script will handle any reasonable number you throw at it. ( I’ve tested up to 20 ).

#headers forcing IMC to respond with JSON content. XML content return is the default

headers = {‘Accept’: ‘application/json’, ‘Content-Type’: ‘application/json’,’Accept-encoding’: ‘application/json’}

def create_operator():
if auth == None or url == None: #checks to see if the imc credentials are already available
imc_creds()
create_operator_url = ‘/imcrs/plat/operator’
f_url = url+create_operator_url
with open (‘imc_operator_list.csv’) as csvfile: #opens imc_operator_list.csv file
reader = csv.DictReader(csvfile) #decodes file as csv as a python dictionary
for operator in reader:
payload = json.dumps(operator, indent=4) #loads each row of the CSV as a JSON string
r = requests.post(f_url, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents
if r.status_code == 409:
print (“Operator Already Exists”)
elif r.status_code == 201:
print (“Operator Successfully Created”)

Now you run this code and you’ve suddenly got all the operators in the CSV file imported into your system.

Doing some non-scientific testing, meaning I counted in Mississippi’s, it took me about 3 seconds to create 10 operators using this method.

Time isn’t Money

Contrary to the old saying, time isn’t actually money. We can always get more money. There’s lots of ways to do that. Time on the other hand can never be regained. It’s a finite resource and I’d like to spend as much of it as I can on things that I enjoy. Creating Operators in an NMS doesn’t qualify.

Now, I hand off a CSV file to the customer, make them fill out all the usernames and passwords and then just run the script. they have all the responsibility for the content and all I have to do is a visual on the CSV file to make sure that they didn’t screw anything up.

Questions or comments or better ways to do this? Feel free to post below. I’m always looking to learn.

@netmanchris

Surfing your NMS with Python

Python is my favourite programming language. But then again, it’s also the only one I know. 🙂

I made a choice to go with python because, honestly, that’s what all the cool kids were doing at the time. But after spending the last year or so learning the basics of the language, I do find that it’s something that I can easily consume, and I’m starting to get better with all the different resources out there. BTW http://www.stackoverflow.com is your friend. you will learn to love it.

On with the show…

So in this post, I’m going to show how to use python to build a quick script that will allow you to issue the RealTimeLocate API to the HP IMC server. In theory, you can build this against any RESTful API, but I make no promises that it will work without some tinkering.

Planning the project.

I’ve written before how I’m a huge fan of OPML tools like Mind Node Pro. The first step for me was planning out the pieces I needed to make this:

usable in the future
actually work in the present

In this case I’m far more concerned about the present as I’m fairly sure that I will look back on this code in a year from now and think some words that I won’t put in print.

Aside: I’ve actually found that using the troubleshooting skills I’ve honed over the years as a network engineer helps me immensely when trying to decompose what pieces will need to go in my code. I actually think that Network Engineers have a lot of skills that are extremely transportable to the programming domain. Especially because we tend to think of the individual components and the system at the same time, not to mention our love of planning out failure domains and forcing our failures into known scenarios as much as possible.

Screen Shot 2014 11 24 at 9 33 50 PM

Auth Handler

Assuming that the RESTful service you’re trying to access will require you to authenticate, you will need an authentication handler to deal with the username/password stuff that a human being is usually required to enter. There are a few different options here. Python actually ships with URLLIB or some variant depending not the version of python you’re working with. For ease of use reasons, and because of a strong recommendation from one of my coding mentors, I chose to use the REQUESTS library. This is not shipped by default with the version of python you download over at http://www.python.org but it’s well worth the effort over PIP’ing it into your system.

The beautiful thing about REQUEST’s is that the documentation is pretty good and easily readable.

In looking through the HP IMC eAPI documentation and the Request library – I settled on the DigestAuth

Screen Shot 2014 11 24 at 10 17 07 PM

So here’s how this looks for IMC.

Building the Authentication Info

>>>import requests #imports the requests library you may need to PIP this in if you don’t have it already

>>> from requests.auth import HTTPDigestAuth    # this imports the HTTPDigestAuth method from the request library.
>>>
>>> imc_user = ”’admin”’   #The username used to auth against the HP IMC Server
>>> imc_pw = ”’admin”’   #The password of the account used to auth against the HP IMC Server.
>>>

auth = requests.auth.HTTPDigestAuth(imc_user,imc_pw) #This puts the username and password together and stores them as a variable called auth

We’ve now built the auth handler to use the username “admin” with the password “admin”. For a real environment, you’ll probably want to setup an Operator Group with only access to the eAPI functions and lock this down to a secret username and password. The eAPI is power, make sure you protect it.

Building the URL

So for this to work, I need to assign a value to the host_ip variable above so that the URL will complete with a valid response. The other thing to watch for are types. Python can be quite forgiving at times, but if you try to add to objects of the wrong type together… it mostly won’t work. So we need to make sure the host_ip is a string and the easiest way to do that is to put three quotes around the value.

In a “real” program, I would probably use the input function to allow this variable to be input as part of the flow of the program, but we’re not quite there yet.

>>> host_ip = ”’10.101.0.109”’ #variable that you can assign to a host you want to find on the network
>>> h_url = ”’http://”’ #prefix for building URLs use HTTP or HTTPS
>>> imc_server = ”’10.3.10.220:8080”’ #match port number of IMC server default 8080 or 8443
>>> url = h_url+imc_server #combines the h_url and the IP address of the IMC box as a base URL to use later
>>> find_ip_host_url = (”’/imcrs/res/access/realtimeLocate?type=2&value=”’+host_ip+”’&total=false”’) # This is the RealTimeLocate API URL with a variable set
>>>

Putting it all together.

This line takes puts the url that we’re going to send to the web server all together. You could ask “Hey man, why didn’t you just drop the whole string in one variable to begin with? “ That’s a great question. There’s a concept in programming called DRY. (Don’t Repeat Yourself). The idea is that when you write code, you should never write the same thing twice. Think in a modular fashion which would allow you to reuse pieces of code again and again.

In this example, I can easily write another f_url variable and assign to it another RESTful API that gets me something interesting from the HP IMC server. I don’t need to write the h_url portion or the server IP address portion of the header. Make sense?

>>> f_url = url + find_ip_host_url
>>> # This is a very simple mathematical operation that puts together the url and the f_url which will product the HTTP call.

Executing the code.

Now the last piece is where we actually execute the code. This will issue a get request, using the requests library. It will use the f_url as the actual URL it’s going to pass, and it will use the variable auth that we created in the Authentication Info step above to automatically populate the username and password.

The response will get returned in a variable called r.

>>> r = requests.get(f_url, auth=auth) # Using the requests library get method, we’re going to pass the f_url as the argument for the URL we’re going to access and pass auth as the auth argument to define how we authenticate Pretty simple actually .
>>>

The Results

So this is the coolest part. We can now see what’s in r. Did it work? Did we find out lost scared little host? Let’s take a look.

>>> r
<Response [200]>

Really? That’s it? .

The answer is “yes”. That’s what’s been assigned to the variable r. 200 OK may look familiar to you voice engineers who know SIP and it means mostly the same thing here. This is a response code to let you know that your request was successful – But not what we’re looking for. I want that content, right? If I do a type(r) which will tell me what python knows about what kind of object r is I will get the following.

>>> type(r)

<class ‘requests.models.Response’>

So this tells us that maybe we need to go back to the request documentation and look for info on the responses. Now we know to access the part of the response that I wanted to see, which is the reply to my request on where the host with ip address 10.101.0.111 is actually located on the network.

So let’s try out one of the options and see what we get

>>> r.content
b'<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?><list><realtimeLocation><locateIp>10.101.0.111</locateIp><deviceId>4</deviceId><deviceIp>10.10.3.5</deviceIp><ifDesc>GigabitEthernet1/0/16</ifDesc><ifIndex>16</ifIndex></realtimeLocation></list>’

How cool is that. We put in an IP address and we actually learned four new things about that IP address without touching a single GUI. And the awesome part of this? This works across any of the devices that HP IMC supports.

Where to from here?

So we’ve just started on our little journey here. Now that we have some hints to the identity of the network devices and specific interface that is currently harbouring this lost host, we need to use that data as hints to continue filling in the picture.

But that’s in the next blog…

Comments or Questions? Feel free to post below!

Working with RESTFul APIs

There are a lot of talk about APIs right now. Every vendor has an API, but not all are created equal. What does an API even mean? I’m not going to get too wrapped around definitions. But I’ll provide you a link

A more formal definiition of REST may be found here. For my purposes, I propose the following:

RESTful API

Something that I can work with using the HTTP protocol and probably returns data in XML or JSON.

Some examples

I’m working with HP’s Intelligent Management Center and it’s eAPI, which offers a RESTful interface to the network management system which will return both XML and JSON.

Here’s an example of a call using XML

Screen Shot 2014 11 24 at 8 38 21 PM

Update: For those who noticed – The URL for both are the same. But the content of the HTTP request actually shows a slightly different story

Here’s the Wireshark of the XML request. If you look at the trace below, you can see the Accept: application /xml\r\n which shows that the request is asking for XML.

Screen Shot 2014 11 27 at 8 47 04 PM

Here’s an example of a call using JSON.

Screen Shot 2014 11 24 at 8 38 39 PM

Here’s the JSON request which is essentially the same except the accept: portion in this shows application as JSON.

Screen Shot 2014 11 27 at 8 46 39 PM

As you can see they carry the exact same data, but the way the data is structured is slightly different. From what I can tell, programatically speaking, there’s no difference in that you can definitely work with either one easily.

For right now, I’ve chosen to focus on the just XML under the theory that if I focus on figuring out how to work with just XML for now, I can go back and learn how to work with JSON later after my overall skills as a programer have increased.

It’s a theory.

Note: For those of you who haven’t seen this interface, it’s a standard ( at least for HP ) RS-Docs interface which provides the documentation for the RESTful interface directly on the machine and allows you to test it. This is also available for the HP SDN controller.

For IMC. it can be accessed at http://localhost:8080/imcrs and will require you to authenticate. BTW the port numbers may also change if you installed in something other than the default state.

RealTimeLocation API – Breaking it Down.

For those of you with keen eyes. You can probably guess that this particular API is used to locate a host on the network. Let’s break it down a little so that we can see what the the return is actually telling us here.

<list> # This lets us know this is the start of the list

<realtimeLocation> # This lets us know the data below is about realtimeLocation

<locateIp>10.101.0.111</locateIp> # This is the IP address we wanted to locate

<deviceId>4</deviceId> # This is the device ID of the switch where we found it.

<deviceIp>10.10.3.5</deviceIp> # This is the IP address of the switch where we found it.

<ifDesc>GigabitEthernet1/0/16</ifDesc> # This is the interface description where we found it.

<ifIndex>16</ifIndex> # This is the ifIndex value of the interface where we found it.

</realtimeLocation> # This lets us know that the realtimeLocation data has ended.

</list> # This lets us know that the list has ended.

So a couple of quick notes about this list.

Device ID is an internal numbering scheme that HP IMC uses to keep track of the devices. This has no practical relation to anything outside the IMC system.
ifDesc is the SNMP ifDesc. You may be tempted to look at this and think “ That must be the description on the interface!!!” You would be wrong. The description you configure on the interface when you type in the command “description This_Is_My_Interface” is actually held in the ifAlias ( 1.3.6.1.2.1.31.1.1.1.18 ) . Blame SNMP for this one.
ifIndex is the SNMP ifIndex value. This is, again, an easy way for computers to keep track of the number of the port. Also, important to know that on some vendors devices, these values can change with a reboot. Cisco used to have this issue, but they do allow you to make them persist across reboot

Next Time

So this is a brief introduction into XML and an example of a RESTful API. As you can see, it’s not that intimidating. It’s actually almost readable by a human being.

In the next post. I’m going to look at building some basic python code to use this API directly.

Questions or Comments? Please post below and I”ll be happy to do my best. Again, I’m student in progress here, so please take any answers I give with a grain of salt. If you’re further along on this journey that I am. Please feel free to suggest improvements in the comments as well. I wouldn’t say I’ve got no ego, but I”ll check it at that door if it helps me improve.

HP Discover 2013

It’s that time of the year again. HP Discover 2013 is happening this weekend in Las Vegas and I’m actually sitting in a hotel room wondering just how hot it’s going to get outside while I write this.

I’ve actually been involved in HP Discover, previous known as TechForum, since 2010 in one way or another, but this is the first year where I’m officially involved as part of the HP Networking Technical Marketing organization.

Without letting any secrets out, it’s going to be a good show for us, but I’m really looking forward to checking out the HP Discover Zone, not to mention the various things that are going on at the HP Innovation Theatre.

Some of the official blogs have covered a bit on session highlights as well as the who’s who role call for the Social Media program.

What I love about Discover

Learn about HP

Hewlett Packard is a HUGE company. To put this in context, HP has more employees than the top 26 smallest Countries. Not cities. Countries. As you can imagine, it’s tough to keep up to date on all the things that are happening across the entire company. HP Discover is a place where we all come together and the curious can learn about what the other parts of the companies are doing, not to mention meet some great people.

Learn about HP’s Partners

The fact that the largest IT companies of the world are represented there as well as partner’s is wonderful. We get a up close and personal with VMware, Microsoft, Citrix, Brocade and many others, large and small, who have something to show. Pretty amazing to see all this technology in one place. It’s a phenomenal place to learn about what’s going on in the whole industry for someone who’s willing to brave the show floor and engage with the people who have come out to educate people about their products and services. It truly is a place to Discover. ( <- see what I did there? ).

Talking with Customers

The one session I did want to mention is HOL 2809 , HP Intelligent Management Center custom scripting, I”m co-presenting with Aaron Paxon @neelix and Lindsay Hill @northlandboy. I’m very excited about this presentation, not just because I’m presenting with some great guys, but because it really represents to me the beauty of the SoMe community.

Lindsay is an HP partner, Aaron is an HP customer, and I’m an HP employee.

We’ve spent the last few month getting together the session abstract working over email, twitter, google+ and even the phone! Amazing how technology can bring us all together. I’m hoping that everyone who shows up for our session gets a good show, but also walks away with something they can use.

What I hope to get from the week

My goals for this year are, in no order of importance, the following

Meet as many people as possible
Visit as many people on the HP Discover Zone as possible
Learn more about vCloud and Azure
Blog and Tweet about what’ going on.

As this is the first major event in my new role, it’s going to be great to see the show from an insider’s perspective. I’m not sure how much I’m going to be able to deliver on my goals, but at least it’s going to be a great learning experience and that, after all, is the thing I love the most.

Hope to see you all there,

@netmanchris

Adding Custom Device Fingerprints to the HPN BYOD Solution

So I was working on the new HP BYOD Solution in my lab and I just didn’t have enough wireless devices to really make it interesting. So I decided to look for other devices in my house which I could connect to the HPN BYOD Controlled MSM controller-based wireless networks.

I did find a Nintendo Wii, but we don’t have fingerprints in IMC to properly identify the Nintendo Wii. I guess Nintendo didn’t make the cut. ( They don’t even support WPA2 Enterprise!!! )

Anyways, the great thing about HP’s new BYOD solution, based on IMC and UAM, is the ability for operators to extend the default fingerprints to devices beyond what was shipped with the product. Although the process does require some knowledge of wireshark, it’s nothing that a little google-technician skills can’t get you through. The adding of fingerprints was super easy.

Creating the foundation

So before we actually get to creating the fingerprints, we need to create the custom vendor, endpoint type, and OS type that we’re going to assign to the DHCP and HTTP fingerprints we are going to create. If you’re doing this for a new smart phone, like the blackberry 10, you’ll probably be able to skip this step as RIM is already listed as a vendor. As you can imagine, Nintendo wasn’t

So let’s look at what the process looks like.

Add Vendor

As you can imagine, there’s no default vendor category for Nintendo, so I’m going to go into the Service>>User Access Manager>> Endpoint Identification Management>>Vendor screen and add a new vendor

Add Endpoint Type

IMC ships with a bunch of endpoint types by default to cover all the normal devices you would see in a business environment. I don’t see that many Wii’s in offices these days though, so we’ll have to create this one too.

Add OS type

Again, No love for Nintendo in the OS department. Let’s add that too.

Creating the fingerprints

For those of you who don’t know, IMC uses digital fingerprints to be able to identify devices accessing the network. We use a combination of characteristics that are mostly unique to one specific type of device to be able to make an educated decision on the model, operating system, and type of the endpoint accessing the network. The three types of fingerprints we can use are

DHCP Fingerprint – In this option IMC uses the options requested in the DHCP client option 55 field to identify the device requesting an address. The specific sequence and number of options are considered to be unique to that specific operating system. ie. All Nintendo Wii machines should request the same values in the same order in the option 55 field of the DHCP request packet. This is considered to be the most reliable of the fingerprinting techniques.

HTTP User-Agent – In this option IMC uses the User-agent portion of the HTTP request headers sent to the BYOD web server to be able to identify the device requesting the webpage. As most browsers will identify themselves through the use of HTTP User-Agent, this is a still a good method for making an educated decision.

MAC Address – In this option IMC uses the MAC address, obtained through the RADIUS server, to identify the vendor based on the MAC address OUI. This is considered to be the weakest form of fingerprinting, but necessary as some devices do not use a unique DHCP signature, nor a web browser. An example of this might be an IP Telephone or Printer.

So let’s get started here and setup our first fingerprint.

Capture the DHCP fingerprint

This is where the nerdiness starts. I have a Windows Active Directory Server that is serving up addresses for the network that my Wii connects to. So I just installed wireshark on the domain controller and start capturing packets.

note: I use the filter bootp.option.type == 53 which will get allow me to see just the DHCP traffic. Cuts down on the packets I need to look through.

I turn on my Nintendo Wii and wait a few seconds for it to try and connect to the network

Now that I’ve got the packet, I need to look a little closer for the Option 55 information. INSERT SOME INFO ON OPTION 55 FROM WIKI

You can see in the packet capture above that the option 55 parameters list has a length of 6, and the values are 1,3,6,15,28, and 33.

Creating the DHCP Fingerprint

So now we go back to the IMC console and navigate to Service>>User Access Manager>> Endpoint Identification Management>> DHCP Character Identification Configuration

click the add button and input the values above

Now that we’ve got the DHCP Fingerprint, let’s go after the HTTP Fingerprint.

Capturing the HTTP User-Agent Fingerprint

This time, I”m running a packet trace from wireshark loaded on my IMC machine ( this is handy for a whole bunch of reasons) I use the Internet Channel on my Wii and attempt to login to the IMC server. Now I check Wireshark again, this time using HTTP as the filter. I could also add the filter for the specific host 10.101.0.116, but in this case it’s just as easy to resort by the source server and get to the right packet.

There it is… “Nintendo Wii”.

Now that I’ve got the HTTP User Agent Signature, I can now go back to IMC and add that in as well.

Creating the HTTP User-Agent Fingerprint

Putting it all together

So, we created the DHCP Character Identification as well as the HTTP User Agent Feature Identification. Now we’re going to connect the wii to the BYOD-enabled wireless network and see what the test this out and see what our work has gotten us.

Fingerprinting Successful. As you can see, the Nintendo Wii was identified by the DHCP client identifier and has been successfully registered in the endpoint MAC address management list in UAM.

The one step other step which I did skip here was adding a MAC address finger print to identify devices which would allow you to identify the device by it’s MAC address. To be honest, that doesn’t require a packet trace, so I skipped that step. What fun is something that doesn’t require a packet trace?

@netmanchris

Configuration Management – Configuration Baselines

Many times when I’m speaking with customers, one of the first questions I get asked is

” Ok, I’ve got this NMS, what’s the first thing I should do that’s going to make the biggest difference in my network?”

There are probably a lot of opinions on the answer to this question. For me, the answer is always this:

Start with Configuration Management.

In ITILv3, one of main aspects of the configuration management domain is to track all of the configuration items that relate to an IT service. For more on ITILv3 CI’s check out this video.

For those of you who suffer from insomnia and would like a cure, most of the ITILv3 change management stuff is found in Volume III, Service Transition. In ITILv3, the first thing you need to do is to define your CMS.

Configuration Management System

This is the ITIL term for the software that handles your configs for you.

Again, remember that ITIL is about process. So it’s possible to actually run an ITIL based shop without tools in place. It’s POSSIBLE… but I think this falls in the JBYCDMYS (Just because you can doesn’t mean you should) bucket.

What to look for in your CMS

So for NMS newbie’s who are trying to get into more process driven network operations, your CMS is the software that does basic tasks like

Backup of Configurations

Any NCCM solution should allow you to backup configurations. If you’re lucky you’re NMS may have additional features that allow you to move beyond basic configuration backups. Ideally, your NMS will have features that will enable you to define configuration baselines and snapshots for any given device.

Configuration Baselines : A configuration baseline is the configuration of a service, product or infrastructure that has been formally reviewed and agreed on, that thereafter can be changed only through formal change procedures. Configuration Snapshots: A snapshot of the current state of a configuration item or an environment. It also serves as a fixed historical record.

In plain english terms, a configuration baseline is the place where you absolutely last know that everything was working. A snapshot is an automatic backup that lets you know what the state of the device was at the time of that backup.

We’ll come back to this later on a subsequent blog post, but snapshots are also great to have around for helping to address your compliance initiatives like SOX, PCI, or HIPPA. Having a configuration snapshot from a certain date is an easy way for you to prove to the auditors what the configuration state of a given device was on that date.

Configuration Templates: A complete, or a portion, of a device configuration.

This could be your standard configuration for your access switches, a secure configuration for your routers, or even just a portion of a configuration, such as the config required to change the local admin password on all your switches.

Scheduling Configuration Changes: The ability to schedule changes to your network devices at specific time.

The ability to schedule changes is nice. Assuming your changes have gone through a peer-review process and through your companies Change Approval Board, Why do you need to be up at 3am during your companies change window?

Now there may be cases where you will still need to be onsite to verify that a critical change went through. To perform the change validation tests that I KNOW you all had in your change plan. Right?

But for those cases where you are simply changing a local admin password, or adding an NTP server, or some other low-risk change, you may want to just schedule this for the ‘wee hours of the morning while you are home in your toasty bed.

One last thing…

When making major, or minor changes to your network configurations, it’s a good practice to go back and update your CMS to reflect the new Configuration Baseline for that device. You did actually run through a series of test to make sure you didn’t break something, right?

So although this could be a TFTP server on the network somewhere, hopefully it’s a software that will automate the backup of network device configurations for you. Examples could include HP’s Intelligent Management Center, Solarwinds Orion, Cisco Prime, or perhaps an opensource tool like RANCID.

In this video, I’ll go through the basic CMS functions of HP’s IMC to show how baselining and snapshots can be applied.