Working with RESTFul APIs

There are a lot of talk about APIs right now.  Every vendor has an API, but not all are created equal. What does an API even mean?  I’m not going to get too wrapped around definitions. But I’ll provide you a link

A more formal definiition of REST may be found here.  For my purposes, I propose the following:

RESTful API

Something that I can work with using the HTTP protocol and probably returns data in XML or JSON.

 

Some examples

I’m working with HP’s Intelligent Management Center and it’s eAPI, which offers a RESTful interface to the network management system which will return both XML and JSON.

Here’s an example of a call using XML

Screen Shot 2014 11 24 at 8 38 21 PM

Update: For those who noticed – The URL for both are the same. But the content of the HTTP request actually shows a slightly different story

Here’s the Wireshark of the XML request. If you look at the trace below, you can see the Accept: application /xml\r\n which shows that the request is asking for XML.

Screen Shot 2014 11 27 at 8 47 04 PM

Here’s an example of a call using JSON.

Screen Shot 2014 11 24 at 8 38 39 PM

 

Here’s the JSON request which is essentially the same except the accept: portion in this shows application as JSON.

Screen Shot 2014 11 27 at 8 46 39 PM

 

 

 

As you can see they carry the exact same data, but the way the data is structured is slightly different.  From what I can tell, programatically speaking, there’s no difference in that you can definitely work with either one easily.

For right now, I’ve chosen to focus on the just XML under the theory that if I focus on figuring out how to work with just XML for now, I can go back and learn how to work with JSON later after my overall skills as a programer have increased.

 

It’s a theory.

 

Note: For those of you who haven’t seen this interface, it’s a standard ( at least for HP ) RS-Docs interface which provides the documentation for the RESTful interface directly on the machine and allows you to test it. This is also available for the HP SDN controller.  

For IMC. it can be accessed at  http://localhost:8080/imcrs   and will require you to authenticate. BTW the port numbers may also change if you installed in something other than the default state. 

 

RealTimeLocation API – Breaking it Down.

For those of you with keen eyes. You can probably guess that this particular API is used to locate a host on the network.  Let’s break it down a little so that we can see what the the return is actually telling us here.

 

<list> # This lets us know this is the start of the list

<realtimeLocation> # This lets us know the data below is about realtimeLocation

<locateIp>10.101.0.111</locateIp> # This is the IP address we wanted to locate

<deviceId>4</deviceId> # This is the device ID of the switch where we found it.

<deviceIp>10.10.3.5</deviceIp> # This is the IP address of the switch where we found it.

<ifDesc>GigabitEthernet1/0/16</ifDesc> # This is the interface description where we found it.

<ifIndex>16</ifIndex> # This is the ifIndex value of the interface where we found it.

</realtimeLocation> # This lets us know that the realtimeLocation data has ended.

</list> # This lets us know that the list has ended.

 

So a couple of quick notes about this list.

  • Device ID is an internal numbering scheme that HP IMC uses to keep track of the devices. This has no practical relation to anything outside the IMC system.
  • ifDesc  is the SNMP ifDesc.  You may be tempted to look at this and think “ That must be the description on the interface!!!” You would be wrong. The description you configure on the interface when you type in the command “description This_Is_My_Interface” is actually held in the ifAlias ( 1.3.6.1.2.1.31.1.1.1.18 ) . Blame SNMP for this one.
  • ifIndex is the SNMP ifIndex value. This is, again, an easy way for computers to keep track of the number of the port. Also, important to know that on some vendors devices, these values can change with a reboot. Cisco used to have this issue, but they do allow you to make them persist across reboot

 

Next Time

 

So this is a brief introduction into XML and an example of a RESTful API.  As you can see, it’s not that  intimidating. It’s actually almost readable by a human being.

In the next post. I’m going to look at building some basic python code to use this API directly.

 

Questions or Comments?  Please post below and I”ll be happy to do my best. Again, I’m student in progress here, so please take any answers I give with a grain of salt. If you’re further along on this journey that I am. Please feel free to suggest improvements in the comments as well. I wouldn’t say I’ve got no ego, but I”ll check it at that door if it helps me improve.

Network Developer: A network engineers Journey into Python

Like most other people in the networking industry, I’ve been struggling with answering the question as to whether or not Network Engineers need to become programmers. It’s not an easy question to answer and after a few years down this SDN journey, I’m still no closer to figuring out whether or not network engineers need to fall into one of the following categories

Become Full-Time Software Developers

DaveTucker

For those of you who don’t know @dave_tucker, he was a talented networking engineer who choose to make the jump to becoming a full time programmer. Working on creating consumption libraries using python for the HP VAN SDN Controller, contributing to the OpenDayLight controller, and now joined up with @networkstatic, another great example. and @MadhuVenugopal   to form SocketPlane focused on the networking stack in Docker. 

Gain some level of proficiency in a modern programming language

One of the people that i think has started to lead in this category is @jedelman8. Jason is a CCIE who glimpsed what the future may hold for some in our profession and has done a great job sharing what he’s been learning on his journey on his blog at http://www.jedelman.com/.  Definitely check it out if you haven’t already. 

This is also where I’ve chosen to be for now. The more I code, I think it’s possible that I could go full programmer, but I also love networking too much. I guess the future will tell with that one. 

For this category, this will mean putting in extra time on nights and weekends to focus on learning the craft.  As someone once told me, it takes about 10 years to become a really good network engineer, no one can be expected to become a good programmer in a year, especially not with a full time day-job. 

On the bright side there are a lot of resources out there like

Coursera.org – Just search for the keyword “python” and there are several good courses that can help you gain the basics of this language.

CodeAcademy.com – CodeAcademy has a focused python track that will allow you to get some guided hands on labs as long as you have an internet connection.

 pynet.twb-tech.com – @kirkbyers has put together an email led python course specifically for network engineers over at   He’s also got some great blogs  that discuss how to use python for different functions that are specifically related to network engineers day-to-day jobs. Having something relevant always helps to make you’re live easier. 

Gain the ability to think programmatically and articulate this in terms software developers understand

I don’t have any really good examples of this particular category.  For some reason, that has so far eluded me, there just isn’t many network engineers in this category. If you know of any great examples, please comment below and I’ll be happy to update the post!

This is where I was a coupe of years ago. I knew logic. I could follow simplistic code if it was already written, and I could do a good enough job communicating to my programming friends enough to ensure that the bottle of tequila I was bribing them with would most likely result in something like what I had in my head. 

 

Stay right where they are today. 

The star fish is one of the few creatures in the history of evolution that went “ Hmmm. I think I’m good! “   This isn’t a judgement, but you need to decide where you want to be and if Star Fish is it… you might find your future career prospects limited. 

starfish

 

 

Journey Ahead

 

As I get back into actually posting, I’m planning on sharing some of the simplistic code that I’ve been able to cobble together. I make no claims as to how good this code is, but I hope that it will inspire some one else reading this to take some classes, find a project, and then write and share some small script or program that makes their life just a little bit easier. Guys like Jason have done this for me. I recently hit a place where I finally have enough skills to be able to accomplish some of the the goals I had in mind. My code is crap, but it’s so simplistic that it’s easy to understand exactly what I’m doing.  And that’s where I think the value comes from sharing right now.

 

Comments or thoughts? Please feel free to comment below!

 

 

A different kind of SDN? Programatic NMS with Management Layer Abstractions

I was at an internal event this week where we were going over some of the amazing things that HP is working on, some which will come to market soon, and some which may never make it out of the labs. 

To answer the obvious question: No, I’m not going to talk about that.  

 

But during one of the sessions, an interesting conversation took place which I wanted to put some additional thoughts down on paper, or e-paper as it were. 

As is the case so much in the industry right now, the conversation came down to a discussion around this question

What is SDN?

The speakers statement, which I’ll look at below, was that programatic NMS is NOT SDN.  Now he is a very smart guy and I do have to say that his presentation was very good, but I had to object to that particular point.  I’m still not sure if I’m right or wrong, but I think it’s worth at least examining the point. 

So first I’ll refer to Martin Casado’s slideshare presentation which you should check out as I’m going to take parts of it out of it’s context in an attempt to make my point.  ( As they say, when you take the text out of the context, all you’re left with is a con ).

For those of you who didn’t check it out, the presentation focuses on the idea that the true benefit of SDN is the power of abstractions. The idea that abstractions allow for much greater flexibility and evolution because you don’t have to worry about the fundamental underlying complexity.

Imagine if all the software in the world had to be written in assembler. I’ll wait while your body works out the sudden urge to scream and jump out a window.

Screen Shot 2012 08 23 at 3 08 07 PM

 

So one of the fundamental tenants of SDN is that

Abstractions are the way forward.

I agree with this 100%. Abstractions take away the underlying complexity and allows us to concentrate on the task at hand. 

So the slideshare preso above deals primarily with the idea of control plane abstraction in the sense that OpenFlow’s purpose is to allow for instructions sets abstractions to be sent to OpenFlow enabled switches which will allow them to bypass all the current control plane complexities that we all know and love in the current network paradigm.

 

I’m 100% in agreement with that.

But what is of particular interest to me is the idea of management plane abstractions. Now for those who don’t know already, I’m an HPN Solutions Architect and an avid evangelist of HPN’s NMS platform, HP IMC.  ( Full Disclosure ).  So I will be attempting to make this argument in context of what IMC is providing now and where I see it going. 

DISCLOSURE: I AM NOT PLM AND I AM NOT REPRESENTING HP IN ANY CAPACITY, OFFICIAL OR OTHERWISE, THIS IS STRICTLY THE THOUGHTS BOUNCING AROUND IN MY HEAD ).

 

If one of the core ideas behind SDN is the idea that abstractions NEED to be created in the networking world to allow us to evolve past the current “protocol-per-problem” paradigm that we are all living in,

It seems to me that allowing management plane abstractions should also have value and should qualify as a type of software defined networking. 

I fully admit that this will not be as revolutionary as the idea of a complete control-plane abstraction like what OpenFlow is promising in the next 2-10 years. ( I’m not holding my breath ). But I do strongly believe that there is some pretty amazing benefits that can be gained today IF the management abstractions are in place.

As well, I also believe that this could easily be the current missing-link that will allow us to bridge the gap between where we are and where we want to be.

 

Example

So let’s look at the following simple example so I can try and make my case.

Imagine, if you will, you are running a dual-vendor strategy. I know you all are, Gartner said it’s a good thing, right?  But now you have discovered that MOST of the current networking hardware manufacturers are offering NMS platforms that limited to managing their own devices. 

So let’s imagine that you are running Cisco and HP equipment in your network and you want to deploy a new VLAN across your data centre. 

Assuming that your NMS’s are single-vendor NMSs ( like Cisco Prime LMS ). You’re stuck  You MIGHT be able to login in to your Cisco Prime LMS interface and deploy VLANs, or you might be able to rely on VTP or GVRP/MVRP to take care of this for you. 

Now if you were using HP IMC, we have already abstracted the management layer to allow you to just say ” DEPLOY VLAN 15″ and the programatic NMS actually takes care of the rest. 

Check out this video for a demo.

So how does this work? 

ABSTRACTIONS

Let’s take the a step further and look at this in the context of SDN. Imagine the following scenario if you will.

A user plugs in to the switch. What happens

1) The dot1x supplicant kicks in and says ” Hey NAC ( sorry BYOD ) server; Is this guy allowed?”

2) RADIUS server says ” Yup, but he’s a sales user, so he needs to go in VLAN 15 

So far, this is pretty standard, right? RADIUS server responds back with the tunnel-group attribute or some other proprietary VSA and the switch puts the user in the right VLAN, right?

 

Wrong.

 

The NAC software has no knowledge of whether or not that particular VLAN is actually present on switch where the user is plugged in.

 

So imagine if there was a programatic interface, like the eAPI in HP’s IMC that would allow the conversation to go like this. 

1) User plugs in to a switch, dot1X process takes place, switch contacts NAC software through RADIUS protocol and the RADIUS server says ” This user should be in VLAN 15 “

2) The NAC software checks with the NMS to see whether or not the VLAN which we are trying to deploy is actually present on the switch in question. 

3) The Programatic NMS would then check the VLAN database to see if that specific VLAN is present on the switch in question. In this case, the the NMS will respond with a no.

4) So now the NAC program knows that the user authentication is going to fail because the underlying infrastructure does not have the necessary services in place ( the VLAN ) to be able to complete the transaction, so it then asks the programatic NMS to deploy the VLAN in question to that switch. 

 5) The programatic NMS would then connect to the switch, and add the VLAN. The NMS would check the bridge MIB, or whatever other MIB might be present to verify that the transaction was completed successfully.

6) After some arbitrary time value, the NAC program would then ask the NMS if the VLAN is present on the switch.

7) The NMS would then respond with a resounding ” Yes! It’s there!” 

 8) The NAC program would then complete the RADIUS request, place the user in the right VLAN and everything is great, right?

 

The best part of this, no human intervention, the program is able to request a service from the network, the network is able to respond that it’s not present and reconfigure itself to meet the needs of the application. 

 

So where’s the abstractions?

Look at the flow above. The NAC program asked ” Is this VLAN present” it didn’t say “When I run the command show VTP Database, after you parse through it, is VLAN 15 present?”  which of course would not work on a juniper or hp switch. 

The NAC program uses the self-describing RESTful API calls like

http://127.0.0.1:8080/GetVLAN device id15

And the NMS responds with an XML output which does not contain that VLAN ID.

Then the NAC program sends an HTTP post to the NMS and says 

http://127.0.0.1:8080/addVLAN:15?devID15

 

note: I don’t have the eAPI SDK in front of me, so I’m making up those calls, I’ll try and fix them later with the real syntax when I have a chance. 

And then the really cool part happens.

If it’s a Cisco IOS based Switch, it uses the Cisco IOS switch device adapters which define the specific commands or SNMP OIDS to set the VLAN on that particular platform.

If it’s an HP Comware switch, it uses the HP Comware switch device adapters which define the specific commands or SNMP OIDS to set the VLAN on that particular platform.

If it’s an HP Procurve switch, it uses the HP Procurve switch device adapters which define the specific commands or SNMP OIDS to set the VLAN on that particular platform.

 

ABSTRACTIONS

No matter what kind of switch that user is connected to, HP’s Intelligent Management Center will hide all of the underlying complexity from the NAC program and respond to the same specific RESTFull API call. 

 

So instead of all the device specific syntax, it’s almost like a simple conversation.

NAC Program to HP IMC ” Is VLAN 15 present? “

HP IMC to NAC Program  ” Nope”

NAC Program to IMC ” Can you please add it for me?”

HP IMC to switch ” Add VLAN 15 to your configuration “

HP IMC to switch ” You done yet? “

NAC Program to HP IMC ” You done yet? “

HP IMC to NAC Program ” I’m done”

 

Now are their other failure modes which should probably be checked? Absolutely, off the top of my head, you would also have to check the dot1q trunks between the device and the L3 interface for that particular VLAN to ensure that you have end2end L2 connectivity and you’re not going to create an isolated orphan VLAN which dumps you user into no connectivity, but that’s just adding more logic to the program.

 

So back to the original question: Does a programatic NMS qualify as SDN or not?

Let me through out the following description. 

A paradigm where a application can request a specific service from the network and the network is able to dynamically reconfigure itself to respond to the applications specific requirements. 

 

That kinds sound like software defined networking to me.

What do you think? Is it the law-of-physic changing paradigm that much of the media would have you believe is coming? Nope. But does it have value? I’ll let you decide.