I had the opportunity to attend the Open Networking User Group event (ONUG) in New York recently and had a chance to talk through some of my musings around change management in an SDN world with some very smart, knowledgable people from a range of different backgrounds.
Let’s talk a little about change control
In a nutshell, people screw things up when left to their own devices. Individuals will inevitably type a wrong command, misplace a decimal point, not have sufficient information, or just plain not-think-something-through.
People are frail, fragile and error prone. But when people come together in groups, share information, share experience, and double check each other’s work, then the error-rate per change tends to drop significantly and changes start to be implemented in a much higher quality fashion.
Change Policy in Modern Organizations
Most modern organizations have some change management process in place. Whether they have succumbed to a full ITIL based process, gone the DevOps route of continual integration, or fall somewhere in between, people have generally figured out that change management is a good thing.
I’ve seen good change management that promotes healthy growth, and I’ve seen bad change managements that restricts the business into stagnation because nothing is ever allowed to change in the organization. ( There’s another word for something that never changes – dead. :)
Change Control in a SDN environment
One of the major issues I see in SDN environments is that many of the changes that we are not only capable, but advocating, are currently heavily restricted through the existing organizations change policies.
To make this example more concrete, let’s talk about an app from Guardicore that uses SDN to detect potential advanced persistent threat attacks in the data center and then uses OpenFlow and the HP VAN SDN Controller to dynamically keep the session alive and re-route ( re-bridge?) the flow directly to a Honeypot which is capable of performing further analysis on that particular session to see if that particular flow is trying to do anything more interesting like trying to execute shell code or some other dubious shenanigans.
Now imagine how the Change Advisory Board is going to react to this request. I imagine it could go something like this.
” What? You want to reconfigure the edge, distribution and core of my data center based on an unknown event at an unknown time because something may or may not be going on?”
How do you think that’s going to go?
ITSM Pre-Approved Changes
There is a concept in ITSM frameworks like ITIL and MOF that allow for a common change to be pre-approved. The change request still has to be fed into the system, but the approvals are automatic and no one has to actively log into a system and click the ” I Approve “
One of the approaches I’ve been advocating is the possibility of repurposing the pre-approved change to allow for dynamic flow modification based on known conditions. This seems to be the simplest way for us to allow the ITSM structures in well-run IT organizations to continue to work without having to scrape the whole change approval process.
This is new ground and I think that this topic requires a lot more discussion that we are currently giving it.
What do you think? Is pre-approved change the way to go? Is there another better way? Is your organization currently using SDN and found a way to rationalize this to the Change Advisory Board?
Please blog it up or post in the comments below.