5 Rules for growing your SE

After reading Greg Ferro’s ( @etherealmind ) hilarious blog on fashion tips, I thought I would add my own experience and put together 5 rules for Account Executives ( aka. Sales Guys ) to follow when helping their SE’s ( aka. the brains ) to do put their best foot forward. Part of your job as our handler is to make sure that you mitigate the little eccentricities that get in the way of the dashing brilliance that we unleash in each and every meeting you bring us into.

So without further delay…

Be kind.

We SE’s have lived a hard live. Although we were blessed with an natural intelligence and an uncanny wit, our ability to understand networking mostly comes from our childhood experience playing with our imaginary friends.

We don’t dress like this on purpose. We really don’t get that certain colours don’t go together, or that comic book camera ties are never, ever, EVER appropriate. (unless your son or daughter buys you one and INSISTS you were it the whole day!) If you make fun of us, we won’t like you. If we don’t like you. We won’t work for you. Keep that in mind.

We are the brains, and if you don’t treat us nicely, there may suddenly be a lot of dead air after you introduce us at the beginning of the meeting. Or even worse, you might find that the only service you get is SAaaS. Smart Ass as a Service.

Advice your SE on body appropriate attire.

SE’s are a special breed, and we don’t always get that certain clothes don’t belong on certain people. Depending on how bad your SE suffers from lack of fashion, you may be able to give him some small tips, or you may have to take him shopping. Once he understands that his lack of style is working against him, he will let you dress him in a too-too if he thinks this will help make his point heard!

The best example of this happened a few years back. We had an manufacturer come in to present to my SE team on their product. This guy was brilliant. He VERY clearly knew the product inside and out. He was articulate and passionate, but unfortunately, the 42″ muffin-top that was dropping over his 32″ pants had caused sensory overload, preventing us from even hearing the words coming out of his mouth.

My Point? It doesn’t matter how smart your SE is, if his fashion style prevents us from hearing what his message.

Body Language and Hand Gestures:

We’ve already established SE’s are not social animals. Many of us spent large parts of our adolecense connecting to BBS’s on 2400 baud modems. I even had friends who could whistle modem tones. ( We thought he was soooooo cooolllll!!!!!!). This means that we probably didn’t interact with many people, and definitely not enough girls. So we never learned some basic social rules. This includes appropriate hand gestures and body language. Your job is to help guide us in the right direction, let us know what works and what doesn’t. ( please refer to rule number 1! We don’t know any better!)

Funny enough, the best example that comes to mind was the same guy with the muffin-top. Same meeting in fact. So you can imagine how bad this was looking already, but then add a set of hand gestures that looked like a combination of KRS-1, Marcel Marceau, and a Shaolin master. By itself, this was bad enough, but the fact that he stopped every couple of minutes or so to wipe his sweaty palms on his man-boobs.

He looked like a mime stopping his act to play with his nipples.

Do you think that anyone could have REALLY heard what this guy was talking about? We were a room full of SE’s and we could barely stop ourselves from laughing out loud, let alone pay attention to the words coming out of his mouth!

Point: If your SE suffers from mimism, tie his hands together. If that doesn’t work. Video him and make him watch it on mute. He might even tie his own hands together at this point.

Warning: He may also try to cut his own hands off. A twenty-four hour watch may be required after the initial viewing.

You are not a bus driver

As a general rule, we SE’s don’t like buses. I’m sure you’ve heard the example ” But what if he gets run over by a bus?”. These saying comes from somewhere.

We don’t like buses. we like them less when they run over us. and even less when our friend, you the AE, are at the wheel.

If you are planning something, let us know in advance. And please don’t ask us questions in front of a customer if you don’t already know the answer. We will most likely answer that question honestly. and you may not like the answer or the delivery, or both.

Lunch is Good

When we go for lunch, you pay. Always.

If you should ever doubt, please refer to the first rule.

Proprietary? So what? Don’t take away my right to choose!

I was reading @networkingnerd blog and he made a comment that started me thinking.

First the quote

Proprietary is very cut and dried. You are either entirely open and run the same implementation with everyone, like OSPF or IS-IS, or you are closed and only play well with your own kind, like EIGRP or IRF. I’ve always been of the mind that proprietary implementations of things aren’t necessarily a bad thing.

Full disclosure: I work For HP Networking. Normally I don’t bring that up because this isn’t a work blog and my tweets are my own, but I think it’s important for this particular blog post since some will think that obviously skews my position. It doesn’t.

Ok so I agree 100% with Tom’s point here. What I don’t agree with was the examples he used for proprietary protocols.

Before I go further, let me point out where Tom and I agree again. Proprietary is neither good or bad. It simply is. If it meets my business requirements and solves a problem I’m having in a cost justifiable way, GREAT!

But what any good engineer knows is that any technology decision is done with trade offs in mind. If I do X then Y will suffer a little, but that’s ok because Y is of lesser importance to our business requirements that the technology design is supposed to address.

What I see too often is that people make emotional, short term decisions without considering the consequences. This always leads them down a bad place eventually.

An example: EIGRP.

I’m not going to debate the merits of this protocol or try to Cisco-bash this in any way. It does have its challenges, but if it didn’t work, people would not still be using it more than 10 years later, right? But it does have its consequences.

I hope you REALLY liked Pix firewalls. I hope you are learning to love those ASAs. Because you couldn’t put in a checkpoint if you wanted to! Ok… I understand no one REALLY wants to put in a Checkpoint. But the point is that you can’t even make the choice because of the proprietary protocol. Right?

You can’t choose a sonicwall, a Fortinet, or even a Palo Alto. Because they don’t support EIGRP either. Now I’m not going to start questioning whether or not the ASAs are a good product. They are “good enough” for a lot of people. But wouldn’t it be nice to choose what was best for you? Of course there is route redistribution, etc… But I have to believe that it’s got to be just painful because I meet people everyday who make EIGRP support as a reason for not been able to make a move off their current network supplier. There’s of course no way this could be a rationalization to support an emotional decision, right?

Now let’s look at IRF.

First: IRF is not really a protocol. It doesn’t run on IP. It’s actually control plane extension between two hardware compatible switches. Is it proprietary? Sure! 100%

You will never get an HP 5500EI to form an IRF stack with an Cisco 3750 or a Juniper EX4200. Not ever going to happen. IRF is HP’s secret sauce.

So where is the difference?

The differences is the extent of the consequences. HP’s IRF is just a control plane technology. Very similar in principal to Cisco’s VSS on the 6500 platform, or Juniper’s Virtual Chassis on the EX line of switches. All of those technologies are very limited in how far they can extend in their current implementations.

This means that if I make a choice to move towards IRF, I am at most going to be implementing it on a couple of chassis switches, or up to 9 stackables ( typically in the same closet). Now this is where the difference starts to happen….

HP has made a choice to support open standards. All of the protocols running on top of IRF are based on the standards. Instead of HSRP, there is VRRP that will run together with Juniper, Cisco, or any other vendors standard implementation of VRRP. OSPF/ISIS/BGP? Exactly the same thing. STP/RSTP/MSTP? Same thing!

Has there been a limitation of choice?

For sure!

If you WANT IRF, you need to buy the second switch from HP as well. But that’s because it’s a good technology that you really found value in, not because we’ve created a artificial limitation which prevents you from going anywhere else. and for everything else? You can choose what load balancer you want without worrying about WCCP support, or which firewall you want without worrying about EIGRP support, or what ever server platform you want without worrying about VNTag support. Everything else just works on top of it, and it plays nice with the other kids.

Perhaps a small distinction, but I think it’s an important one.

I grew up in place where independence and freedom where strongly ingrained in the culture. I would even dare to use the word “cowboy” to describe it. And anything that outs an unreasonable amount of restriction on my choices is something which I’ve been taught to resist from birth.

so Tom; If you would have compared IRF to VSS, Stackwise, or Virtual Chassis. I would have been just fine. I hope you’ll forgive my semantics debate, but I just felt the need to get this out.

@netmanchris

Through the eyes of a child

Wrote this last summer and apparently didn’t publish. Still amazes me.

Listening to the packet pushers podcast there was a listeners question on studying and learning. Coincedently, I had just had one of the most amazing experiences of my life. I had just watched my 6 year old son ride his bike alone for the first time.

We’ve been working on it all summer, and he was more scared than anything else. It was late September and he was discouraged and didn’t want to practice anymore, and I literally had to force him back on that bike. But I knew that this day was the day.

And it was.

The look of absolute wonder on his face at his new, seemingly superhuman, ability to ride his bike by himself was awe inspiring. And it got me thinking just how lucky we are in this industry.

We have the privilege every day to learn. what a great job we have.

It’s not my fault! Someone moved my line!

Open Letter to a Server Guy

Intro

Ok first, I’d like to refer everyone to a great post on Greg Ferro’s blog over at http://www.etherrealmind.com. Server people and Network people don’t mix; Oil and Water, Cats and Dogs, Cisco and anybody else… But just like the real world, sometimes we end up in situations where we have no choice but to work together, and for that to happen, we have to understand each other and find some common ground.

This whole series springs from a conversation I had with a server administrator. Honestly, I was complaining about the lack of visibility into the server environment that he was giving me. His response was “Don’t worry about it. It’s in the server”. This post is my response to him, and all the server and network people in the world who are still living in this us-and-them world. Converged Infrastructure is not just about blending the boundaries between your server, network, and storage infrastructure. It’s also about tearing down the walls between the application, server, network, and storage groups that have grown over the years.

In this post, I hope to share my view of the where we came from as network people, give my view of where we are today, and hopefully help to create a common starting point to engage the server folk from for the rest of this discussion.

My view of the world: From a network persons perspective, they are responsible for the point in the environment where a host ( an operating system that provides or consumes services) enters the network, be it virtual or physical, to the point where another host receives the request.

I have compliance, governance, and security initiatives that I need to address, not to mention the application performance management that we’re all been asked to ensure.

Then

In the beginning… ok, so maybe not the beginning, but at least since the early 90’s, the a network persons domain of responsibility looked like a little like this

Figure 1 – Traditional Demarcation Point

Network Folk stayed on our side of the red line and server folk dealt with the other stuff. Ahhh… the easy days when virtualization was the NT hardware abstraction layer and the words application and server meant, for all intent and purpose, the same thing. Responsibility ended at the end of that beautiful crystal RJ-45 connector. ( and we crimped those ourselves darnnit!!!!)

This gave all of us a very simple view of the world. As network folk, we had our routers, cores, distributions, and access switches, and anything beyond that was labeled with a sign “Here there be servers!!!” VTP was a good thing and spanning-tree was something we all wanted in our network! DNS and DHCP were things those server guys “took care of” ( <- does anyone know how to insert the sarcasm font?)

All jokes aside; the world was simpler in that we ALL understood where our responsibly started and stopped and it was easy for us to understand our place in the world. The side affect is that it also enabled us to fall into an “us and them” paradigm which resulted in rampant finger pointing and as we all know it was always “the networks fault”.

Now

At some point in the last 5 years, a few things fundamentally broke this model.

Multi-Tier Apps: Applications went from living on a single server to a multi-tear model where the presentation, logic, and data tiers may all reside on different physical servers. In fact, each of the individual tiers may also exist on multiple physical servers.
Service Oriented Architectures: Web 2.0 Apps have become common further complicating the environment by having multiple multi-tier applications cross-leveraging information.
Blade Chassis: HP introduced the Blade Chassis to the server market, while fundamentally keeping the traditional network demarcation point in place. This resulted in a lot of “network traffic” flowing across the backplane of the switch without the knowledge, guidance, or protection of the network team.
Server virtualization: vSwitches have broken the traditional demarcation point and extended it inside, between, and through the traditional network.

These three shifts have shattered the traditional network demarcation point to the point where it’s impossible to tell where your responsibility as a network person ended.

Was it here?

Figure 2 -Simple Demarcation Point

Or here?

Figure 3 – Blade Chassis Demarcation Point

Or here?

Figure 4 -vSwitch in a Blade Chassis Demarcation Point.

To paraphrase Dr. Seuss, “I.T.’s not easy I’m sure you will find, for a mind-maker-upper to make up his mind”.

So, what’s my point?

Once upon a time, life was simple. The cable was where my world ended. I’m still responsible for managing, monitoring, securing, and troubleshooting the network. It’s just that I don’t know where the network ends anymore.

Server Guy: I understand that your world changed too. To run the same applications, you need multiple servers with multiple cores, leveraging multiple data sources spread out throughout multiple data centers. You have virtual servers running on physical servers and you’re still responsible for managing, monitoring, securing, and troubleshooting the applications. Most of all, to make all this stuff work, we need each other.

I get it, and I’m here to help.

I hope this helps you server folk to understand why we network folk are asking the question we are asking. It’s not our fault; it’s just that someone keeps moving our line.

@netmanchris

Bringing the network out of the darkness

Once upon a time, I had to interview with the new HR director after the company I was working for was acquired. The new management had a much better, process driven approach to HR and employee development which included a clear job description of what each and every employee did. You can imagine how this conversation went…

me I’m the network administrator,

her Ok, but what do you do?

me make sure that the dynamic routing protocols are working. Ensure our qos policies are working properly,etc.. I’m also currently working on an active directory migration from our current nt4 environment,

her …… I don’t understand a thing you just said.

me hmmm… Let me put it like this. If I do my job well, then no one knows I exist. If I don’t do my job, no one else can do there’s.

her wow. So you’re pretty important then?

me definitely. Can I have a raise?

You can imagine how the rest of that conversation went. But hey, to paraphrase Dinero in A Bronx Tale ” What are you gonna do? He took a shot! Nice try, son”

So it got me thinking? What can we, as a profession, do to raise the profile of the networking engineer. We are happy to be in back rooms, hiding in the cold noisy data centers but hiding is not going to get us any more budget in the next round of funding.

This is a major issue as, for many organizations, the network is an afterthought at best. How many of us can relate to the story of a server guy coming to us with a request for a few dozen 10Gb ports to plug in their brand new servers that were purchased using grant money in an EDU? Except the money was spent on servers and there’s nothing left over for the network portion and there are no 10Gb ports available in the network.

Anyone have any ideas or creative ways that you’ve been able to highlite the importance of the network to the business?

Looking forward to reading your ideas!

@netmanchris

Code is like my wife.

So it’s obvious I’m a management guy. And I don’t mean I manage people, I manage my network, and I help advise my customers on managing their networks. One of the things that never ceases to amaze me is how little people in our industry actually know about change management. In fact I find it down right ironic that an industry who’s only constant is change hasn’t really embraced change management at all. In fact, I think it’s safe to say we run screaming from change management as quickly as possible. The only thing we avoid more than change management is it’s dreaded enforcer

THE PROJECT MANAGER

Personally, I think that a little more time spent in ITSM ( IT service management ) school could really do a lot to change the stereotype of our profession as networking focused IT workers. The most common ITSM framework today is, of course, ITIL. The Information Technology Infrastructure Library is a CBOK ( common body of knowledge ) that focuses on how to run an IT organization. It’s most lofty goal is to give us a common vocabulary to be able to effectively communicate across the IT silos. It’s as simple as that. The focus of ITIL is really just to standardize and codify the best practices and wisdom that other IT professionals have gathered over the years and allow us to leverage their prior experience.

def: WISDOM – The experience we get from making bad decisions.

So what does ITIL have to do with code and my wife? We’re getting there, but first we need a little more ITIL knowledge.

ITIL is comprised of 5 core volumes, ( Strategy, Design, Transition, Operations, and CSI for those of you who are counting ). There are a couple of volumes, transition and operations, that deal specifically with change management. There’s a LOT of good content in there, but I’m going to focus on just one piece

def: Configuration Item: A CI is an asset, service component or other item that is, or will be, under the control of Configuration Management. CI’s may vary widely in complexity, size and type, ranging from an entire service or system including all hardware, software, documentation and support staff to a single software module or a minor hardware component. Configuration items may be grouped and managed together, e.g. a set of components may be grouped into a release. CI’s should be selected using established selection criteria, grouped, classified and identified in such a way that they are manageable and traceable through the service lifecycle.

One of the goals of Configuration Management is really to offer stability in the IT environment. One of the ways to get more stability is to ensure consistency of operations, and one of the way to ensure consistency of operations is to ensure that you have common components. This is way telco’s often standardize on a single box, even though in many cases it may be vastly overkill for the application. From an operational cost point of view, the CAPEX cost is offset by the stability and consistency gained by having a single common CI.

So let me bring this back to my wife… One of the MOST common ~~mistakes~~ inefficiencies that I see in many customers networks is the following.

Joe Admin receives a brand new Cisco 3750/Juniper EX4200/HP 5500EI switch for the new wiring closet. It arrived on time, and he got a good price on it. In fact he got a great price because it’s the exact same model that he’s been buying now for a couple of years. So Joe grabs his trusty serial-to-usb adapter, grabs his MacBook Pro and console cable and 10 minutes later, the box is assigned an IP address to VLAN interface 1 and he’s setup telnet and a local user account for remote administration purposes here and he’s good to go!

Now there are a bunch of things that he’s just done that are less than ideal (see if you can spot the all! ) but I’m just going to focus on one today; Joe didn’t check the version of code on this switch. And this brings me back to my point.

Code is like my wife

Let me try to explain that one: Every version of code has things that we love, things that we don’t. It has features that we may or may not use, and always, always, always bugs. Of course vendors do their best to ensure that code is as bug free as possible before it goes out the door, but there is always something unintended or unexpected that comes up.

I’ve been married for a few years now, and like any married person I’ve found that my wife has certain… special charms that make me love her even more. I’m not allowed to call them bugs. But there are certain things in peoples personalities that are just like bugs in code. You can’t predict them, they really don’t seem to make any sense, and they usually pop up at the LEAST desirable moment.

So how is code like my wife? Both have bugs of course! ( I mean undocumented “features”!!!) But… we’ve been together for a few years now and I’ve got a lot of the bugs figured out. I understand that when I do X she will exhibit a certain symptom that may not make sense to me, but I can tell you that it’s repeatable on a somewhat consistent basis. ( Yes, I’m male. So that does make me a slow learner ).

So let’s apply that back to Joe. Joe just grabbed a brand new switch out of the box, with a brand new software load and a brand-new set of undocumented “feature” that he has yet to discover. Now there are certain people in the world who enjoy the excitement of new experiences. The thrill of discovery. The rush of jumping headfirst off a cliff with no clue whether or not there are rocks in that water. Me? Not so much.

I love my wife, but I have to admit that the practical engineer side of me really loves the stability as well. I know a lot of her bugs. I know what sets her off. I know how to avoid them, and when they happen, I know how to diffuse them quickly. I also know that if I have to troubleshoot for a new bug, I have a wide range of experience with this particular version to draw on, which should speed up the whole MTTR (mean time to recovery) process.

Note: I usually use MTTI ( Mean Time to Innocense) , but in my experience so far with my wife has taught me that it’s VERY rarely not me who’s at fault.

So What Should Joe have done here?

1) Pick a version of code. Any version of code. but only pick one. Get to know it inside and out. Read the configuration guide, the release notes and make sure he’s happy with the features and documented bugs.

2) Install THIS specific version of code on every single compatible hardware platform in the entire network.

3) When Joe receives a new switch with a new version of code. Read the release notes. Check for the new features. Are they required? Is there a solid business reason for upgrading? Did the code fix any specific bugs which are impacting the availability of the network or the profitability of the business? If the answer is No to all of these questions.

DOWNGRADE THE SWITCH!

Yup. You heard me, I’m recommending that you actually downgrade your switches to the chosen version of code. Your knee-jerk reaction would be to upgrade everything else in the network to the new version, right? Wrong.

Wrong. Wrong. Wrong. Wrong. Wrong.

In a industry where there is so much change on a daily basis, we often don’t take the time to actually sit down and think about WHY we are changing thing in the environment. What’s the net benefit for this change? Are we changingg just for changes sake? Or is there a compelling reason for us to invest the time and energy into learning an entire new set of bugs? To link this back to the ITIL CI, if you have a single hardware platform with a single version of code on it, you have a single CI. The fewer CI’s you have to manage, the fewer CIs you have to manage. And I think we can all agree that eliminating complexity and reducing the number of managed elements we have to think about in a day is a good thing.

@netmanchris

I’ve got control issues. And I’m ok with that.

As the title says; I’ve got control issues. I want things to work the way they should. When things fail, I want them to fail exactly like I believe they should. I want to know that if my performance down a certain path starts to drop that my network will dynamically choose a better path. Oh, and I want to know in advance what path that will be.

I want to know that if a power supply in one of my core or distribution switches starts to go a little wonky. I want to know BEFORE it causes an outage, not after.

How many network administrators have any proactive monitoring in place. Far too few if my customers are to be taken as a representative sample. It strikes me as a little funny that we have such great tools at our disposal and we just don’t bother to take advantage of them.

From free tools such as OPENNMS and Spiceworks, to older traditional tools such as SolarWinds and Ciscoworks LMS, to newer tools based on modern software architectures like HP’s Intelligent management center. There are a lot of low cost options nowadays for network admins to be able to get a little insight into what’s REALLY happening in their environments.

What about you? Where are you in your network management journey? What tools are you using and what value you do you find in them?