Automating your NMS build using Python and Restful APIs Part 1 – Creating Operators

It’s a funny world we live in.  Unless you’re hiding under a rock, there’s been a substantial push in the industry over the last few years to move away from the CLI.  As someone right in the middle of this swirling vortex of inefficiency, I’d like to suggest that it’s not so much the CLI that’s the problem, but the fact that each box is handled on an individual basis and that human beings access the API through a keyboard. Not exactly next-generation technology.

 

I’ve been spending lot of time learning python and trying to apply it to my daily tasks. I started looking at the HP IMC Network Management station a few months ago. Mainly as a way to start learning about how I can use python to access RESTFul APIs as well as gain some hands on working with JSON and XML. As an observation, it’s interesting to be that I’m using a CLI ( python ) to configure an NMS ( IMC) that I’m using to avoid using the CLI. ( network devices ).   

I’ve got a project I’m working on to try and automate a bunch of the initial deployment functions within my NMS. There are a bunch of reasons to do this that are right for the business. Being able to push information gathering onto the customer, being able to use lower-skilled ( and hence lower paid!) resources to do higher level tasks. Being able to be more efficient in your delivery, undercut the competitors on price and over deliver on quality. It’s a really good project to sink my teeth and use some of my growing coding skills to make a difference to the business. 

This is the first post in which I’ll discuss and document some of the simple functions I’m developing. I make no claims to be a programmer, or even a coder. But I’m hoping someone can find something here usefull, and possibly get inspired to start sharing whatever small project you’re working on as well. 

 

Without further ado, let’s jump in and look at some code. 

What’s an Operator

Not familiar with HP IMC?  You should be! It’s chock full of goodness and you can get a 60 day free trial here.   In IMC an Operator is someone who has the right to log into the system and perform tasks in the NMS itself.  The reason they use the word operator vs. user is that there’s a full integrated BYOD solution available as an add-on module which treats a user as resource, which of course is not the same thing as an administrator on the system. 

IMC’s got a full RBAC system as well which allows you to assign different privilege levels to your operators, from view only to root-equiv access, as well as splitting up what devices you can perform actions on, as well as segmenting what actions you’re allowed to perform. Pretty powerful stuff once you understand how the pieces go together. 

Adding an Operator in the GUI

 This is a screen capture of the dialog used to add an operator into IMC.  It’s intuitive. You put the username in the username box, you put the password in the password box. Pretty easy right?

If you know what you’re doing and you’re a reasonably good typist, you can add probably add an operator in a minute or less.  

Screen Shot 2015 04 16 at 12 19 17 PM

Where do Operators come from?

Don’t worry. This isn’t a birds and bees conversation.  One of the biggest mistakes that I see when people start into any network management system project, whether that’s Solarwinds, Cisco Prime, What’s up Gold, HP NNMi, or HP IMC, is that they don’t stop to think about what they want/need to do before they start the project.  They typically sit down, start an auto-discovery and then start cleaning up afterwards.  Not exactly the best way to ensure success in your project is it?

When I get involved in a deployment project, I try to make sure I do as much of the information gathering up front. This means I have a bunch of excel spreadsheets that I ask them to fill in before I even arrive onsite. This ensures two things:

  1. I can deliver on what the customer actually wants
  2.  I know when I’m done the project and get to walk away and submit the invoice. 

 

I won’t make any judgement call on which one of those is more important. 

 

 

My Operator Template

My operator template looks like this

NewImage

The values map to the screen shot above exactly as you would expect them to. 

Full name is the full name. Name is the login name, password is the password etc…  

The authType is a little less intuitive, although it is documented in the API docs. The authType maps to the authentication type above which allows you to choose how this specific operator is going to authenticate, through local auth, LDAP, or RADIUS. 

The operator group, which is “1” in my example, maps to the admin operator group which means that I have root-level access on the NMS and can do anything I want. Which is, of course, how it should be, right?

 

The Problem

So I’ve got a CSV file and I know it takes about one minute to create an operator because I can type and I know the system. Why am I automating this? Well, there are a couple of reasons for that.

  • Because I can and I want to gain more python experience
  • Because if I have to add ten operators, this just became ten minutes.
  • Because I already have the CSV file from the customer. Why would I type all this stuff again?
  • Because I can reuse this same format at every customer project I get involved in. 
  • Because I can blame any typos on the customer

Given time, I could add to this list, but let’s just get to the code. 

The Code

Authenticating to the Restful API

Although the auth examples in the eAPI documentation use the standard URLIB HTTP library, I’ve found that the requests library is MUCH more user friendly and easier to work with.

So I first create a couple of global variables called URL and AUTH that I will use to store the credentials.  

 

#url header to preprend on all IMC eAPI calls
url = None

#auth handler for eAPI calls
auth = None 

Now we get to the meat. I think this is pretty obvious, but this function gathers the username and password used to access the eAPI and then tests it out to make sure it’s valid. Once it’s verified as working ( The 200 OK check ). The credentials are then stored in the URL and AUTH global variables for use later on. I’m sure someone could argue that I shouldn’t be using global variables here, but it works for me. :) 
 
def imc_creds():
    ''' This function prompts user for IMC server information and credentuials and stores
    values in url and auth global variables'''
    global url, auth, r
    imc_protocol = input("What protocol would you like to use to connect to the IMC server: \n Press 1 for HTTP: \n Press 2 for HTTPS:")
    if imc_protocol == "1":
        h_url = 'http://'
    else:
        h_url = 'https://'
    imc_server = input("What is the ip address of the IMC server?")
    imc_port = input("What is the port number of the IMC server?")
    imc_user = input("What is the username of the IMC eAPI user?")
    imc_pw = input('''What is the password of the IMC eAPI user?''')
    url = h_url+imc_server+":"+imc_port
    auth = requests.auth.HTTPDigestAuth(imc_user,imc_pw)
    test_url = '/imcrs'
    f_url = url+test_url
    try:
        r = requests.get(f_url, auth=auth, headers=headers)
    except requests.exceptions.RequestException as e: #checks for reqeusts exceptions
        print ("Error:\n"+str(e))
        print ("\n\nThe IMC server address is invalid. Please try again\n\n")
        imc_creds()
    if r.status_code != 200: #checks for valid IMC credentials
        print ("Error: \n You're credentials are invalid. Please try again\n\n")
        imc_creds()
    else:
        print ("You've successfully access the IMC eAPI")
 
 
I”m using this function to gather the credentials of the operator accessing the API. By default when you first install HP IMC, these are admin/admin.    You could ask: Why don’t you just hardcode those into the script? Why bother with writing a function for this? 
Answer: Because I want to reuse this as much as possible and there are lots of things that you can do with the eAPI that you would NOT want just anyone doing. Plus, hardcoding the username and password of the NSM system that controls your entire network is just a bad idea in my books. 
 

Creating the Operators

I used the HP IMC eAPI /plat/operator POST call to as the basis for this call. 

Screen Shot 2015 04 16 at 1 06 21 PM

 

After doing a bit of testing, I arrived at a JSON array which would allow me to create an operator using the “Try it now” button in the API docs.  ( http://IMC_SERVER:PORTNUMBER/imcrs to access the online docs BTW ).

    {
"password": "access4chris",
"fullName": "Christopher Young",
"defaultAcl": "0",
"operatorGroupId": "1",
"name": "cyoung",
"authType": "0",
"sessionTimeout": "10",
"desc": "admin account"
}

Using the Try it now button, you can also see the exact URL that is used to call this API. 

The 201 response below means that it was successfully executed. ( you might want to read up on HTTP codes as it’s not quite THAT simple, but for our purposes, it will work ).

Screen Shot 2015 04 16 at 1 10 46 PM

Now that I’ve got a working JSON array and the URL I need, I’ve got all the pieces I need to put this small function together. 

You can see the first thing I do is check to see if the auth and url variables are still set to None. If they are still None I use the IMC_CREDS function from above to gather them and store them. 

 

I create another variables called headers which stores the headers for the HTTP call. By default, the HP IMC eAPI will respond with XML. After working with XML for a few months, I decided that I prefer JSON. It just seems easier for me to work with.

This piece of code takes the CSV file that we created above and decodes the CSV file into a python dictionary using the column headers as the key and any additional rows as the values. This is really cool in that I can have ten rows, 50 rows, or 100 rows and it doesn’t matter. This script will handle any reasonable number you throw at it. ( I’ve tested up to 20 ).

 

#headers forcing IMC to respond with JSON content. XML content return is the default

headers = {‘Accept’: ‘application/json’, ‘Content-Type’: ‘application/json’,’Accept-encoding’: ‘application/json’}

def create_operator():
    if auth == None or url == None: #checks to see if the imc credentials are already available
        imc_creds()
    create_operator_url = ‘/imcrs/plat/operator’
    f_url = url+create_operator_url
    with open (‘imc_operator_list.csv’) as csvfile: #opens imc_operator_list.csv file
        reader = csv.DictReader(csvfile) #decodes file as csv as a python dictionary
        for operator in reader:
            payload = json.dumps(operator, indent=4) #loads each row of the CSV as a JSON string
            r = requests.post(f_url, data=payload, auth=auth, headers=headers) #creates the URL using the payload variable as the contents
            if r.status_code == 409:
                print (“Operator Already Exists”)
            elif r.status_code == 201:
                print (“Operator Successfully Created”)

 Now you run this code and you’ve suddenly got all the operators in the CSV file imported into your system. 

Doing some non-scientific testing, meaning I counted in Mississippi’s, it took me about 3 seconds to create 10 operators using this method.  

Time isn’t Money

Contrary to the old saying, time isn’t actually money. We can always get more money. There’s lots of ways to do that. Time on the other hand can never be regained. It’s a finite resource and I’d like to spend as much of it as I can on things that I enjoy.  Creating Operators in an NMS doesn’t qualify.

Now, I hand off a CSV file to the customer, make them fill out all the usernames and passwords and then just run the script. they have all the responsibility for the content and all I have to do is a visual on the CSV file to make sure that they didn’t screw anything up.

 

Questions or comments or better ways to do this?  Feel free to post below. I’m always looking to learn.

 

@netmanchris 

 

Advertisements

Introduction to R and SWIRL

So I’m taking a Cousera course from John Hopkins on Data Science.

The course uses the R programming language which is a derivative of the S programming language that came out of Bell labs in the 70’s. I’m a huge believer in network programability and SDN in general. From a traditional  Network Management point of view, most of the work getting done and discussed today is really around the C in the FCAPS model. There are some people, like Jason Edelman, Matt Oswald, etc… who are using network programability for automating troubleshooting tasks, but most of those are pretty straight forward

  • automate information gathering
  • automate troubleshooting
  • Identify the corrective action

Once you’ve got the corrective action nailed down, you could also automate the fix, but there are a lot of people who are still nervous about having changes happen without a human being involved. 

Automating configuration management and configuration based fault detection and error correction are great things. But there are other parts of the network that can benefit from the application of a programming language to old problems. 

I’m personally interested in the massive amounts of data that the network holds. We’ve got a ton of instrumentation within the network that is just setting there to be accessed, tracked, and mined for useful insights. 

Data Science is all about different methods to scroll through all the data in a scientifically reproducable manner, hopefully gain some insights.

The Tools

Like python, R has an IDE available that will allow you to run R code interactively, or through R files. It can be downloaded at the CRAN site here

There’s also a better IDE available called R Studio that allows some additional functionality which is available here 

SWIRL is a library which allows learners to access some interactive tutorials written in R for R. There’s a GIT repository here which provides a set of tutorials for different courses that allows you to get a feel for the language syntax, creating functions, etc…  

 

R Studio and Swirl

Once you install the SWIRL library, which is really easy using the RStudio Install Packages feature, you load the SWIRL library ( think IMPORT in Python ) using the library(swirl) function. Once you’ve done that, you can either download the course files from the GIT repository, or you can install directly from within R ( uses CURL in the background to download the files directly into your working directory ). 

As you can see in the screen capture, I’ve got a few different course installed, and each of the courses has a bunch of lessons inside them. The screen capture shows the lessons within the R Programming course. What’s also cool about this is, assuming that you’re enrolled in the Coursera R Programming course, you can complete the lesson, input your username and password ( specific to your course, not your cousera password ) and magically, you get extra credit for the course lessons you complete.   

Extra credit is a good thing.

 

Wrap Up

I’ve only been into R for about a week. It’s got some nice features, but to be honest, I don’t have enough coding experience to really give a qualified opinion on the subject. I’ll continue to work with it and see where things go.  There’s still a ton of python that I need to learn, but I’ve already found a native python library called rpy2 that allows me to access native R libraries from within my python code. Best of both words I guess. 🙂

 

Bringing Wireless Back in to the Fold

I’m sitting in the airport in Barcelona just having had an amazing week of conversations ranging from potentially core-belief shattering to crazy ideas for puppet shows. The best part of these events, for those of us who are social, is the ability to interact with people in meatspace that we’ve already “known” for a period of time on twitter. I had the pleasure this week of hanging out with such luminaries of the networking social scene like Tom Hollingsworth (@networkingnerd ), Amy Arnold (@amyengineer), Jon Herbert (@mrtugs ), Ethan Banks @ecbanks and, not to be left out of any conversation, Mr. Greg Ferro.

 

There were a lot of great conversations, and more than a couple of packet pushers shows recorded during the week but the one that’s sticking in my mind right now is a conversation we had around policy and wireless. This has been something on my mind now for awhile and I think I’ve finally thought this through enough to put something down on paper.

Before we get started, I think it’s important that everyone understand that I”m not a wireless engineer, I’m making some assumptions here that I”m hoping that will be corrected in the comments if I’m headed in the wrong direction.

 

Wireless: The original SDN

So in many ways, wireless networking have been snickering at us wired lovers for our relatively recent fascination with SDN. Unsurprisingly, I’ve heard a lot of snark on this particular subject for quite awhile. The two biggest being:

  • Controller Based networking? That’s sooooooo 2006. Right?
  • Overlays?  We’ve been creating our own topologies independent of the physical layout of the network for years!!!!

 

I honestly can’t say I disagree with them in principle, but I love considering the implications of how SDN, combined with the move to 802.11ac is going to really cause our words to crash back together.

 

I’m a consumer of wireless. I love the technology and have great respect for the witchdoctor network engineers who somehow manage to keep it working day-in and day-out. I’m pretty sure where I have blue books on my book shelf, they have a small alter to the wireless gods. I poke fun, but it’s just such a different discipline requiring intense knowledge of the transmission medium that I don’t think a lot of wired engineers can really understand how complicated wireless can be and how much of an art form that creating a good stable wireless design actually is.

On a side note, I heard this week that airplanes actually use sacks of potatoes in their airplanes when performing wireless surveys to simulate the conditions of passengers in the seats. If that doesn’t paint a picture of the differences with wireless, I don’t know what does.

 

The first wireless controller I had a chance to work with was the Trapeze solution back in approx 2006. It was good stuff. It worked. It allowed for centralized monitoring, not to mention centralized application of policy. The APs were 802.11G and it was awesome. I could plug in an AP anywhere in the network and the traffic would magically tunnel back to the controller where I could set my QoS and ACLs to apply the appropriate policies and ensure that users were granted access and priority, or not, to the resources that I wanted. Sounds just like an Overlay doesn’t it?

In campus environments, this was great. An AP consumed a theoretical bandwidth of 54Mbps and we had, typically, dual Gig uplinks. If we do some really basic math here, we see the following equation

Screen Shot 2014 12 05 at 1 08 31 PM

 

Granted, this is a napkin calculation to make a point.  But you can see it would be REALLY hard to oversubscribe the uplinks with this kind of scenario.  There weren’t that many wireless clients at the time. AP density wasn’t that bad. 2.4 Ghz went pretty far and there wasn’t much interference.

Screen Shot 2014 12 05 at 1 09 14 PM

 

Hmmm… things look a little different here.  Now there are definitely networks out there that have gone  to 10Gb connections between their closets in the campus. But there are still substantial amount that are still running dual gig uplinks between their closets and their core switches. I’ve seen various estimates, but consensus seems to suggest that most end-stations connected to the wireless network consume, on average, about 10% of the actual bandwidth. Although I would guess that’s moving up with the rich media (video) getting more and more widely used.

Distributed Wireless

We’ve had the ability to allow the wireless APs to drop the wireless client traffic directly on to the local switch for years. Although vendors have implemented this feature at different times in their product life cycles. I think it’s safe to say this is a me-too feature at this point. I don’t see it implemented that much though because, in my opinion, having a centralized point in the network, aka. the controller, were I can tunnel all of my traffic back to allows me to have a single point to apply policy. Because of the limited bandwidth, we could trade off the potential traffic trombone of wireless going back to the controller to access local resources for the simplicity of centralized policy.

Now that a couple of 802.11ac access points can potentially oversubscribe the uplinks on your switch, I think we’re quickly going to have to rethink that decision. Centralized policy not only won’t be worth the cost of the traffic trombone, but I would argue it’s just not going to be possible because of the bandwidth constraints.

 

I’m sure some people who made the decision to move to 10Gb uplinks will continue to find centralized policy to be the winner of this decision, but for a large section of network designers, this just isn’t going to be practical

Distributed Policy

This is where things start to get really interesting for me. Policy is the new black. Everyone’s talking about it. Promise Theory, Declaritive Intent. Congress, etc… There are a lot of theories and ideas out there right now and it’s a really exciting time to be in networking. I don’t think this is going to be a problem we solve overnight, but I think we’re going to have to start creating the foundation now with more consistent design and configurations allowing us to provide a consistent, semi homogenous foundation when we start to see the policy discussion resulting in real products.

What do I mean by this? Right not there, really two big things that will help to drive this forward.

Globally Significant, but not Unique VLANS

Dot1x, or more accurately, the RADIUS protocol, allows us to send back a tunnel-group ID attribute in the RADIUS response that corresponds to a VLAN ID ( name or dot1q tag are both valid ). We all know the horrors of stretched VLANS, but there’s no reason you can’t refuse the same VLAN number in various places in the network as long as they have a solid L3 boundary in between them and are assigned different L3 address space. This means that we’re going to have to move back towards L3 designs and turn to configuration tools to ensure that VLAN ids and naming conventions are standardized and enforced across the global network policy domain.

Consistent Access Control Lists and QoS Policies

RADIUS can also send back a specific attribute in the RADIUS response that will tell the switch put apply a specific ACL or QoS policy to the authenticated connection for the session time of that connection. Some vendors, but not all, allow for the dynamic instantiation of the ACL/QoS policy, but most still require the ACL or QoS construct to be already present in the network device before the RADIUS policy can consume that object. This means we’re going to be forced to turn to configuration management tools to make sure that these policy enforcement objects are present in all of the network devices across the network, regardless of the medium.

 

The future

I think we’re swiftly arriving at a point where wireless can not be designed in a vacuum as an overlay technology. The business need policy to be consistently applied across the board and bandwidth to be available and efficiently used.  I don’t see any other way for this to be done without ensuring that the we start to ignore the medium type that a client is connecting on.  On the bright side, this should result in more secure, more flexible, and more business policy driven wired connectivity in the coming years. I don’t believe we’ll be thinking about how the client connected anymore. We won’t care.

 

Agree? Disagree? Did I miss something? Feel free to comment below!

@netmanchris

PYSNMP with HP 5500EI Comware Switch

Inspired by @kirkbyers post over here  I wanted to stretch my python skills and see about playing around with the PYSNMP libraries as well as Kurt’s SNMP_HELPER.PY function which is available here.

Clean up the SNMP_HELPER.PY function for Python 3.x

There are some differences in Python 2 vs. Python 3. One of those differences is that the print command now requires you to actually have parans ()   around the content that you wish to print.  This was about the only thing that I had to do to get Kirk’s code working in Python 3.  If you try to run the code in the python IDLE software it will come up with this error right away.  I could also have run the py2to3  scripts, but since this was a small file, it was easy to just search for the 4 or so print statements and edit it manually as I was reading through the code to try and understand what Kirk was doing.

 

Easy Installation

So Kirk takes you through the normal PIP installation. I’m performing this on OS X Mavericks with Python 3. So for those not familiar with the differences yet. Python 2.x is natively installed on OSX. If you do a pip install …  command, this will result in you downloading and making that specific library available to the python 2.x version on your OS.  Since I’m using python 3.x, I instead need to use the pip3 install command which will, instead, make the library you’re downloading available to python 3.x on your system

$pip3 install pysnmp

 

Note: Kirk has a couple of other ways to install the pysnmp library over on his blog, so I won’t repeat them here.

Testing Out SNMP

So it’s a good idea to ensure that SNMP is running and you have the right community strings on the machine you’re going to access. For this, I’m going to use an

SNMP MIB browser that I have installed on my MBA to test this out. You could also use the net-snap utilities as shown on Kirk’s blog if you’d like to do this from the CLI. I highly recommend getting a MIB Browser installed on your system. http://www.ireasoning.com has a nice free one available.

Screen Shot 2014 11 27 at 3 51 04 PM

 

So now that we’ve confirmed this all works. on to the code.

Setting the Stage

So I’m assuming that you’re able to run the SNMP_Helper.py file in IDLE.  If you look at the code, one of the first things it does is import the cmdgen method from the pysnmp library

“from pysnmp.entity.rfc3413.oneliner import cmdgen” 

One of the ways that has really helped me learn is to go through other people’s code and try and understand exactly what they are doing. I don’t think I could have written SNMP_Helper.py on my own yet, but I can understand what it’s doing, and I can DEFINITELY use it. 🙂

Now we set up a few variables, using the exact same names that Kirk used over in his blog here

>>> COMMUNITY_STRING = ‘public’
>>> SNMP_PORT = 161
>>> a_device = (‘10.101.0.221’, COMMUNITY_STRING, SNMP_PORT)

Running the Code

Now we’ll run the exact same SNMP query against the sysDescr OID that Kirk used. And Amazingly enough, get a very similar output.

>> snmp_data = snmp_get_oid(a_device, oid=’.1.3.6.1.2.1.1.1.0′, display_errors=True)
>>> snmp_data
[(MibVariable(ObjectName(1.3.6.1.2.1.1.1.0)), DisplayString(hexValue=’485020436f6d7761726520506c6174666f726d20536f6674776172652c20536f6674776172652056657273696f6e20352e32302e39392052656c6561736520323232315030350d0a48502041353530302d3234472d506f452b204549205377697463682077697468203220496e7465726661636520536c6f74730d0a436f707972696768742028632920323031302d32303134204865776c6574742d5061636b61726420446576656c6f706d656e7420436f6d70616e792c204c2e502e’))]

 

It’s nice to see that we have gotten that same nasty output. SNMP is a standard after all and we should expect to see the same response from Cisco, HP, and other vendors devices when using standard SNMP functions, such as the MIBII sysDescr OIDs.

So now, let’s use Kirk’s cleanup function to be able to see what the data actually looks like. Again, remember Python3 needs those parens for the print statement to work properly.

>>> output = snmp_extract(snmp_data)
>>> print (output)
HP Comware Platform Software, Software Version 5.20.99 Release 2221P05
HP A5500-24G-PoE+ EI Switch with 2 Interface Slots
Copyright (c) 2010-2014 Hewlett-Packard Development Company, L.P.

Just for giggles, I also used this code against my Synology Diskstation

>>> print(output)
Linux DiskStation 2.6.32.12 #4482 Fri Apr 18 02:12:31 CST 2014 armv5tel

Then against my Server Technologies intelligent PDU

>>> print(output)
Sentry Switched CDU

Then against my DIGI console server.

>>> snmp_data = snmp_get_oid(a_device, oid=’.1.3.6.1.2.1.1.1.0′, display_errors=True)
ERROR DETECTED:
error_message No SNMP response received before timeout
error_status 0
error_index 0

The last one was working exactly as expected as I have ACL’s in place to only allow SNMP access from certain devices in my network. 🙂

Observations

It’s nice to see that standards like SNMP and widely available libraries like pysnmp can be used to access the devices regardless of the vendor they come from.

SNMP gets a bad wrap in general as there are new cooler technologies out there like NETCONF, OpenFlow, OVSDB, NetFlow, sFlow, and I’m sure a dozen others that I’m missing that can do a better job of the functions that SNMP was originally designed to go after.

But sometimes, SNMP is what we have, and the reason that it’s still around after all these years is that it’s  “good enough”

 

Questions or comments?  please post below!

@netmanchris

 

 

Working with RESTFul APIs

There are a lot of talk about APIs right now.  Every vendor has an API, but not all are created equal. What does an API even mean?  I’m not going to get too wrapped around definitions. But I’ll provide you a link

A more formal definiition of REST may be found here.  For my purposes, I propose the following:

RESTful API

Something that I can work with using the HTTP protocol and probably returns data in XML or JSON.

 

Some examples

I’m working with HP’s Intelligent Management Center and it’s eAPI, which offers a RESTful interface to the network management system which will return both XML and JSON.

Here’s an example of a call using XML

Screen Shot 2014 11 24 at 8 38 21 PM

Update: For those who noticed – The URL for both are the same. But the content of the HTTP request actually shows a slightly different story

Here’s the Wireshark of the XML request. If you look at the trace below, you can see the Accept: application /xml\r\n which shows that the request is asking for XML.

Screen Shot 2014 11 27 at 8 47 04 PM

Here’s an example of a call using JSON.

Screen Shot 2014 11 24 at 8 38 39 PM

 

Here’s the JSON request which is essentially the same except the accept: portion in this shows application as JSON.

Screen Shot 2014 11 27 at 8 46 39 PM

 

 

 

As you can see they carry the exact same data, but the way the data is structured is slightly different.  From what I can tell, programatically speaking, there’s no difference in that you can definitely work with either one easily.

For right now, I’ve chosen to focus on the just XML under the theory that if I focus on figuring out how to work with just XML for now, I can go back and learn how to work with JSON later after my overall skills as a programer have increased.

 

It’s a theory.

 

Note: For those of you who haven’t seen this interface, it’s a standard ( at least for HP ) RS-Docs interface which provides the documentation for the RESTful interface directly on the machine and allows you to test it. This is also available for the HP SDN controller.  

For IMC. it can be accessed at  http://localhost:8080/imcrs   and will require you to authenticate. BTW the port numbers may also change if you installed in something other than the default state. 

 

RealTimeLocation API – Breaking it Down.

For those of you with keen eyes. You can probably guess that this particular API is used to locate a host on the network.  Let’s break it down a little so that we can see what the the return is actually telling us here.

 

<list> # This lets us know this is the start of the list

<realtimeLocation> # This lets us know the data below is about realtimeLocation

<locateIp>10.101.0.111</locateIp> # This is the IP address we wanted to locate

<deviceId>4</deviceId> # This is the device ID of the switch where we found it.

<deviceIp>10.10.3.5</deviceIp> # This is the IP address of the switch where we found it.

<ifDesc>GigabitEthernet1/0/16</ifDesc> # This is the interface description where we found it.

<ifIndex>16</ifIndex> # This is the ifIndex value of the interface where we found it.

</realtimeLocation> # This lets us know that the realtimeLocation data has ended.

</list> # This lets us know that the list has ended.

 

So a couple of quick notes about this list.

  • Device ID is an internal numbering scheme that HP IMC uses to keep track of the devices. This has no practical relation to anything outside the IMC system.
  • ifDesc  is the SNMP ifDesc.  You may be tempted to look at this and think “ That must be the description on the interface!!!” You would be wrong. The description you configure on the interface when you type in the command “description This_Is_My_Interface” is actually held in the ifAlias ( 1.3.6.1.2.1.31.1.1.1.18 ) . Blame SNMP for this one.
  • ifIndex is the SNMP ifIndex value. This is, again, an easy way for computers to keep track of the number of the port. Also, important to know that on some vendors devices, these values can change with a reboot. Cisco used to have this issue, but they do allow you to make them persist across reboot

 

Next Time

 

So this is a brief introduction into XML and an example of a RESTful API.  As you can see, it’s not that  intimidating. It’s actually almost readable by a human being.

In the next post. I’m going to look at building some basic python code to use this API directly.

 

Questions or Comments?  Please post below and I”ll be happy to do my best. Again, I’m student in progress here, so please take any answers I give with a grain of salt. If you’re further along on this journey that I am. Please feel free to suggest improvements in the comments as well. I wouldn’t say I’ve got no ego, but I”ll check it at that door if it helps me improve.

Rethinking Change Control in a SDN world

I had the opportunity to attend the Open Networking User Group event (ONUG) in New York recently and had a chance to talk through some of my musings around change management in an SDN world with some very smart, knowledgable people from a range of different backgrounds.

Let’s talk a little about change control

In a nutshell, people screw things up when left to their own devices. Individuals will inevitably type a wrong command, misplace a decimal point, not have sufficient information, or just plain not-think-something-through.

People are frail, fragile and error prone. But when people come together in groups, share information, share experience, and double check each other’s work, then the error-rate per change tends to drop significantly and changes start to be implemented in a much higher quality fashion.

Change Policy in Modern Organizations

Most modern organizations have some change management process in place. Whether they have succumbed to a full ITIL based process, gone the DevOps route of continual integration, or fall somewhere in between, people have generally figured out that change management is a good thing.

I’ve seen good change management that promotes healthy growth, and I’ve seen bad change managements that restricts the business into stagnation because nothing is ever allowed to change in the organization. ( There’s another word for something that never changes – dead. 🙂

Change Control in a SDN environment

One of the major issues I see in SDN environments is that many of the changes that we are not only capable, but advocating, are currently heavily restricted through the existing organizations change policies.

To make this example more concrete, let’s talk about an app from Guardicore that uses SDN to detect potential advanced persistent threat attacks in the data center and then uses OpenFlow and the HP VAN SDN Controller to dynamically keep the session alive and re-route ( re-bridge?) the flow directly to a Honeypot which is capable of performing further analysis on that particular session to see if that particular flow is trying to do anything more interesting like trying to execute shell code or some other dubious shenanigans.

Now imagine how the Change Advisory Board is going to react to this request. I imagine it could go something like this.

” What? You want to reconfigure the edge, distribution and core of my data center based on an unknown event at an unknown time because something may or may not be going on?”

How do you think that’s going to go?

ITSM Pre-Approved Changes

There is a concept in ITSM frameworks like ITIL and MOF that allow for a common change to be pre-approved. The change request still has to be fed into the system, but the approvals are automatic and no one has to actively log into a system and click the ” I Approve “

One of the approaches I’ve been advocating is the possibility of repurposing the pre-approved change to allow for dynamic flow modification based on known conditions. This seems to be the simplest way for us to allow the ITSM structures in well-run IT organizations to continue to work without having to scrape the whole change approval process.

This is new ground and I think that this topic requires a lot more discussion that we are currently giving it.

What do you think? Is pre-approved change the way to go? Is there another better way? Is your organization currently using SDN and found a way to rationalize this to the Change Advisory Board?

Please blog it up or post in the comments below.

Solarwinds NPM – Take 2

Ok. So I’m back at it now.

The first step of this mulligan was to remove the activated license from the corrupted windows box that caused me all the trouble in the first place.

While I deploy a brand new Windows 2012 image, I headed over to the solarwinds website and read through this document.  As detailed in the doc, I installed the licensing application. Deactivated the NPM license and everything went as great.

Good news so far. I’m really looking forward to start digging into how NPM manages HP Networking gear.

An Update

So after the fiasco of the last attempted install. I learned a couple of things.

  • The Solarwinds NPM install package from the customer portal does NOT include the embedded Microsoft SQL server.  If you want to run this with SQL express, then you need to install the eval version.  Good thing to know if you are trying to install NPM in a smaller environment.  Keep in mind though, it is STRONGLY recommended – I read it multiple times in the docs – to use an external SQL server when using NPM in production. This makes sense for a “real” network, but for my purposes, I have a small lab so there’s really no need. 
 
  • My Windows image was hosed. screwed. burned out. totally useless.  When I did the install on a brand new Windows 2012 server, it went totally smooth. I pre-installed the IIS server, as mentioned in the docs, and everything else went off without a hitch, so much so that the only reason I’m mentioning it is the fact that I had so much trouble the first time.   The blame for that one goes on a bad windows build.

 

First thoughts

Initial Discovery

It’s been a couple of years since I was at the helm of an NPM  box, but to be honest, it feels pretty comfortable. Having a lot of sticktime on some other products, I had a bit of trouble with getting the desired results from the discovery process ( IP ranges vs. Subnets didn’t do exactly what I wanted – I kept getting more ranges that I wanted to. ) but after a few tries, I managed to get the initial discovery up and running without any trouble.

The Good:

In general. The discovery process went smooth. Interestingly, NPM asked me for windows, vmware, telnet/ssh, and SNMP credentials. The nice thing, which kind of surprised me, was that NPM was now able to discover my VMware ESXi and vCenter servers. This is a good thing as I’m a big fan of providing a consolidated view of the entire network, whether that’s physical or virtual, wired or wireless. I’ll check later into what Virtualization support is actually offered in NPM, but for now, I’m happy to see that I can at least identify the resources on my network. 

 

NewImage

 The not so good:

There were a couple of mis–labeled devices. Specifically, the HP 5500EI and the HP 5120EI which are a couple of boxes that have been in the market now for a few years. As you can see from the images below, both of these devices are HP devices. The description ( which is pulled directly from the device through the sysdesc OID  ( .1.3.6.1.2.1.1.1.0  for anyone who’s counting ) does show that this is an HP device.

 

NewImage

 

 

 

 

 

 

 

 

 

 

 

 

 

On the bright side, the error has been submitted to the NPM unknown device thread here so hopefully this will be addressed in a future update. 

Topology Maps

In previous versions of Solarwinds, one of the things that did bother me was having to jump back and forth between the web interface and the windows console depending on the task that I needed to accomplish. I know Solarwinds has done a lot of work to move all the administrative functions into the web interface, but it doesn’t look like Network Atlas has made the cut yet. 

This is first glance, so it’s possible I just haven’t clicked on the right button yet. One of the most powerful pieces of a good NMS is an accurate topology map. Now that I’ve got the network discovered and up and running, creating some network maps are going to be my next task. 

 

NewImage

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Closing

In general, Solarwinds feels familiar. It’s not too far removed from the versions I was more familiar with so I’m hoping that digging in is going to go more smoothly. I’m also VERY happy that I’m over my initial install issues. That was a painful experience and it’s nice to be able to say I just had a corrupted windows build.  The new install went perfectly.  I’ve been spending some time upgrading my lab to ESX 5.5 this week, as well as playing with the HP SDN Controller as well, so I might take a break from Solarwinds for a bit, but expect more info in the future as I start to spend some more time with NPM.

 

@netmanchris