It Generalist or Network Specialist?

Very shortly after posting this blog, I was pleasantly surprised by a comment from Ethan Banks. Apparently my posting and his free time aligned. 🙂   One line of his comment really got thinking about where I want to take my career in the future. You can read the whole comment, but the part that got thinking was Ethan’s comment “I’m looking into becoming more of an IT generalist in the long run.”.


IT Generalist vs. Network Specialist

When I went back and read my post, I can definitely see where the IT Generalist thought came up. There’s a lot of different skills that I”m trying to develop this year, and to be honest, there’s probably a lot more that I’m going to have to gain before I can start developing in the areas that I really want to go after.  For an example, I just spent a few hours reading about GIT.

GIT was not on the list, but I’ve just spent a few very precious hours of my time because it’s almost unthinkable now to be doing any software development without using GIT to share your work. It’s the standard and I just don’t have a job enough working knowledge of repos and forks, and merge’s etc…  just not something I’ve had to pick up in my career yet. So although it’s not on my list, it’s a pre-requisite for really being able to use the skills I want to develop.

So, back to the question; Am I trying to become a IT generalist or a network specialist? I’m not quite sure I have an answer to that yet. But i think that we need to first ask the question: “what is the network?” before I can figure out an answer.

I think this was an easy question a few years ago, but it’s getting much much harder to figure out where my areas of responsibility as a networking professional now ends.


How many Tiers?

I was having a conversation with @networkstatic a couple weeks ago and we were laughing the whole idea of a two tier network. Now, I work in marketing and I understand that the point of the phrase “two tier network” is really designed to communicate that there’s less hardware involved, therefore less cost, less latency, less complexity, etc… But once the marketing is down and the POs have been cut, it becomes time to operate the network and then the number tiers suddenly is a whole different number.

Think about a typical connection from a user to an application. Let’s assume that we’ve got typical architecture where there’s a user on a tablet trying to access an application in a data centre somewhere. I’m sure someone is going to disagree and we could easily expand the DMZ into a few more tiers, but for discussion purposes, just work with me here.


Network Tiers

Now by my count, this “two tier” network is actually going through thirteen different tiers in the network where policy can be applied which would potentially alter how the traffic would flow through the network, which of course impact the quality of the specific application that the user is trying to access.

Arguably, the last OVS/Linux Bridge layer would not be there for a lot of typical networks, but with technologies like Docker and Rocket, I think we’re going to see that become more commonplace over the next year. This also doesn’t even factor in the whole overlay/underlay and VTEP mess that can also throw another wrench into the mix. But I think you get my point.

Closing the loop

So bringing this back to the opening question; If a networking professional has responsibility to understand the end-to-end path and where issues may arise for a connection between a consumer and a service, it would seem that we need to develop skills across the entire stack.  There are a lot of other places where a problem can arise; database seek times, noisy neighbour issues in a storage array. Badly coded applications, etc…  I would not argue that a network engineer be knowledgable on the ins and outs of ALL of things that can go wrong, but I do believe that we should be making a fighting attempt at understanding the parts that affect our craft.

In the long run, I think we’re going to continue to see networking divide into sub-professions that specialize into specific architectural blocks of the network. Although there may be a lot of common knowledge, I would also argue there’s also a lot of specialized knowledge that can only be gained through experience dedicated to one of these architectural blocks over a period of time.

Network Disciplines List


Then again, I might be over thinking the whole thing. It’s also possible that the network knowledge will start to become considered generalist knowledge.

What do you think? Post your comments below.



Plans for 2015: Where to from here?

I know I’m a little bit late for New Years resolutions, but it’s been a tough decision making process. There is so much going on right now in the networking industry and, to be honest, I’m not sure that networking is going to be a skill that will demand the premium that it’s been able to for the last 10-15 years.  Don’t get me wrong, I’m not saying that networking is dead. In fact, just the opposite, networking is going to flourish. There is going to be so much networking that needs to be done that the only way we will be able to deal with it is to dump all of our collective knowledge into code and start to automate what would have previously been the domain of the bit-plumbers that we are. 


What skills to pick up in 2015:

So the question: What skills am I looking at picking up in 2015?  I am a huge believer the infrastructure-as-code movement. Looking at what leaders like Matt Oswald, Jason Edelman, Brent Salisbury, Dave Tucker, Colin McNamara, Jeremy Schulman, etc… are taking us, it’s obvious that coding skills are becoming a mandatory skill for anyone in the networking field who wants to become, or remain, at the top of the field.  That’s not to say that core networking skills are not going to be important, but I’m definitely branching out this year in trying to gain some another language, as well as improve my chops with what I already know.

Increase Python Skills

As anyone who’s been here for the last year knows, I’ve been playing around with python a lot. I’m hoping that 2015 will allow me to continue to increase my python skills, specifically as focused around networking, and I’m hoping that I will have enough time to go from just learning to actually contributing back to some code to the community. I’m signed up for Kirk Bayles Python for Networking Engineers course starting in January, as well as going through a few different books. Bets of all, my 9 year old son has also shown some interest in learning to code, so this might actually become a father son project.

I’m also hoping to get more involved with things like Ansible, Schprokits, as well as possibly releasing some of my own all projects.  Crossing my fingers on the stretch goals. 🙂

Gain Data Analysis Skills

Cousera is awesome. If you haven’t checked it out, you need to. You would have to be living under a rock buried in a lead can stored in a faraday cage at the bottom of the ocean to not have heard about SDN. I believe that there’s an ENORMOUS opportunity within the networking space for applying data analysis techniques to the massive amounts of information that flows across our networks every day. There’s a Cousera Data Science Specialization that I’m signed up for that I”m hoping will start me down the path of being able to execute on some the ideas that I’ve had bouncing around in my skull for more than half a decade. I’m sure I will be blogging on the course, but you might have to wait for some of the ideas.  


Docker, Rocket, NSX, ESX, KVM, OVS. They are all going to get a little love this year from this guy. I’m not sure how much I’m going to be able to consume, but I believe these are all technologies that are going to be relevant in the coming years. I believe that Containers are going to get a lot of love in the industry and companies like are going to be something I”m watching closely. 

Networking Networking Networking

This is my core knowledge set and, I believe, what will continue to be the foundation of my value for the foreseeable future. I hit my CCIE Emeritus this year and also had a chance to attend a Narbik bootcamp. It was an incredibly humbling experience and reminded me of how much there is still to learn in this space that I love. If you get a chance to attend a Micronics CCIE bootcamp, I couldn’t recommend it highly enough. There are very few people who understand and can TEACH this information at the level Narbik can. I’m actually planning on finding time to resit the bootcamp this year just soak up more of the goodness. 


Plans Plans Plans

2014 was a bit of a mess for me. But I think I still did fairly well in executing on gaining some of the programming skills that I wanted. 2015 is going to be a crazy time for the whole industry. I’m not sure which of these four areas is going consume the most of my time. The way our industry has been going, it’s entirely possible that I will fall in love with something else entirely. 🙂  

If at the end of 2015 I have managed to move forward in these four areas by at least a few steps, I think I will consider the year a success. 


What about you?




Bringing Wireless Back in to the Fold

I’m sitting in the airport in Barcelona just having had an amazing week of conversations ranging from potentially core-belief shattering to crazy ideas for puppet shows. The best part of these events, for those of us who are social, is the ability to interact with people in meatspace that we’ve already “known” for a period of time on twitter. I had the pleasure this week of hanging out with such luminaries of the networking social scene like Tom Hollingsworth (@networkingnerd ), Amy Arnold (@amyengineer), Jon Herbert (@mrtugs ), Ethan Banks @ecbanks and, not to be left out of any conversation, Mr. Greg Ferro.


There were a lot of great conversations, and more than a couple of packet pushers shows recorded during the week but the one that’s sticking in my mind right now is a conversation we had around policy and wireless. This has been something on my mind now for awhile and I think I’ve finally thought this through enough to put something down on paper.

Before we get started, I think it’s important that everyone understand that I”m not a wireless engineer, I’m making some assumptions here that I”m hoping that will be corrected in the comments if I’m headed in the wrong direction.


Wireless: The original SDN

So in many ways, wireless networking have been snickering at us wired lovers for our relatively recent fascination with SDN. Unsurprisingly, I’ve heard a lot of snark on this particular subject for quite awhile. The two biggest being:

  • Controller Based networking? That’s sooooooo 2006. Right?
  • Overlays?  We’ve been creating our own topologies independent of the physical layout of the network for years!!!!


I honestly can’t say I disagree with them in principle, but I love considering the implications of how SDN, combined with the move to 802.11ac is going to really cause our words to crash back together.


I’m a consumer of wireless. I love the technology and have great respect for the witchdoctor network engineers who somehow manage to keep it working day-in and day-out. I’m pretty sure where I have blue books on my book shelf, they have a small alter to the wireless gods. I poke fun, but it’s just such a different discipline requiring intense knowledge of the transmission medium that I don’t think a lot of wired engineers can really understand how complicated wireless can be and how much of an art form that creating a good stable wireless design actually is.

On a side note, I heard this week that airplanes actually use sacks of potatoes in their airplanes when performing wireless surveys to simulate the conditions of passengers in the seats. If that doesn’t paint a picture of the differences with wireless, I don’t know what does.


The first wireless controller I had a chance to work with was the Trapeze solution back in approx 2006. It was good stuff. It worked. It allowed for centralized monitoring, not to mention centralized application of policy. The APs were 802.11G and it was awesome. I could plug in an AP anywhere in the network and the traffic would magically tunnel back to the controller where I could set my QoS and ACLs to apply the appropriate policies and ensure that users were granted access and priority, or not, to the resources that I wanted. Sounds just like an Overlay doesn’t it?

In campus environments, this was great. An AP consumed a theoretical bandwidth of 54Mbps and we had, typically, dual Gig uplinks. If we do some really basic math here, we see the following equation

Screen Shot 2014 12 05 at 1 08 31 PM


Granted, this is a napkin calculation to make a point.  But you can see it would be REALLY hard to oversubscribe the uplinks with this kind of scenario.  There weren’t that many wireless clients at the time. AP density wasn’t that bad. 2.4 Ghz went pretty far and there wasn’t much interference.

Screen Shot 2014 12 05 at 1 09 14 PM


Hmmm… things look a little different here.  Now there are definitely networks out there that have gone  to 10Gb connections between their closets in the campus. But there are still substantial amount that are still running dual gig uplinks between their closets and their core switches. I’ve seen various estimates, but consensus seems to suggest that most end-stations connected to the wireless network consume, on average, about 10% of the actual bandwidth. Although I would guess that’s moving up with the rich media (video) getting more and more widely used.

Distributed Wireless

We’ve had the ability to allow the wireless APs to drop the wireless client traffic directly on to the local switch for years. Although vendors have implemented this feature at different times in their product life cycles. I think it’s safe to say this is a me-too feature at this point. I don’t see it implemented that much though because, in my opinion, having a centralized point in the network, aka. the controller, were I can tunnel all of my traffic back to allows me to have a single point to apply policy. Because of the limited bandwidth, we could trade off the potential traffic trombone of wireless going back to the controller to access local resources for the simplicity of centralized policy.

Now that a couple of 802.11ac access points can potentially oversubscribe the uplinks on your switch, I think we’re quickly going to have to rethink that decision. Centralized policy not only won’t be worth the cost of the traffic trombone, but I would argue it’s just not going to be possible because of the bandwidth constraints.


I’m sure some people who made the decision to move to 10Gb uplinks will continue to find centralized policy to be the winner of this decision, but for a large section of network designers, this just isn’t going to be practical

Distributed Policy

This is where things start to get really interesting for me. Policy is the new black. Everyone’s talking about it. Promise Theory, Declaritive Intent. Congress, etc… There are a lot of theories and ideas out there right now and it’s a really exciting time to be in networking. I don’t think this is going to be a problem we solve overnight, but I think we’re going to have to start creating the foundation now with more consistent design and configurations allowing us to provide a consistent, semi homogenous foundation when we start to see the policy discussion resulting in real products.

What do I mean by this? Right not there, really two big things that will help to drive this forward.

Globally Significant, but not Unique VLANS

Dot1x, or more accurately, the RADIUS protocol, allows us to send back a tunnel-group ID attribute in the RADIUS response that corresponds to a VLAN ID ( name or dot1q tag are both valid ). We all know the horrors of stretched VLANS, but there’s no reason you can’t refuse the same VLAN number in various places in the network as long as they have a solid L3 boundary in between them and are assigned different L3 address space. This means that we’re going to have to move back towards L3 designs and turn to configuration tools to ensure that VLAN ids and naming conventions are standardized and enforced across the global network policy domain.

Consistent Access Control Lists and QoS Policies

RADIUS can also send back a specific attribute in the RADIUS response that will tell the switch put apply a specific ACL or QoS policy to the authenticated connection for the session time of that connection. Some vendors, but not all, allow for the dynamic instantiation of the ACL/QoS policy, but most still require the ACL or QoS construct to be already present in the network device before the RADIUS policy can consume that object. This means we’re going to be forced to turn to configuration management tools to make sure that these policy enforcement objects are present in all of the network devices across the network, regardless of the medium.


The future

I think we’re swiftly arriving at a point where wireless can not be designed in a vacuum as an overlay technology. The business need policy to be consistently applied across the board and bandwidth to be available and efficiently used.  I don’t see any other way for this to be done without ensuring that the we start to ignore the medium type that a client is connecting on.  On the bright side, this should result in more secure, more flexible, and more business policy driven wired connectivity in the coming years. I don’t believe we’ll be thinking about how the client connected anymore. We won’t care.


Agree? Disagree? Did I miss something? Feel free to comment below!


Things I learned as a voice engineer

 I have something to admit. I’m a recovering Voice Engineer. It’s been almost 10 years since my last installation. I occasionally experiment in the privacy of my home. But I’ve mostly broken the habit. It’s actually been so long that most of the people I work with have no idea that my CCIE is actually in Voice. I was actually one of the first 50 or so in the world which means I’ve been out of that world longer than most people have been in it. I try to stay current, but to be honest, my passion has moved to other things. 


For someone reason, Voice Engineers are viewed as the bottom feeders of the networking world. But I never seem to be surprised that some of the best networking people I know seem to have a solid voice background. Just to call out a few of the twitter personalities 


@networkingnerd – Yup. Tom used to be a voice engineer.

@amyengineer – Uh huh. Her too. She slips occasionally, but she’s getting better.

@treylayton  – CTO over at that little VCE company?  Before vBlocks and Netapp. He was one too.

and then tonight I just found out

@colinmcnamara has a great voice background from the early days as well. Yup. The DevOps/OpenStack/automation guy? He’s one too.


I’m not talking about the kind of voice engineers that we see today. One who reads one of the many Cisco Press purple books which are available. ( in my day they were teal darn it! ) Logs into a website which automatically puts together a complete Cisco Communication Manager Express ( #hursttoevenwritethat ) configuration based on your input into a webpage. People who think that SIP has always been here and that H.323 is a “legacy protocol”. 

I spent some time reminiscing about AS5300s, MCS 3800s, MGCP inconsistencies, importing hundreds of MAC addresses with a Bar code reader and various other shenanigans that we had to do at that point of the industry.   To be honest, it feels so good to know that we will never. ever. ever. have to do any of those things again.

But there were also some really great lessons I learned from that point in my career.


Be nice to everyone

I’m lucky that I’m Canadian and had my parents and country to thank for teaching me good manners. Unfortunately, I’ve seen this go very very wrong when I was paired with a sales person who didn’t have the same upbringing I did.

Situation:  We walk into a potential customer. He’s immediately rude to the receptionist and tells ( not asks ) here to let the CIO know we’re here and to go get him a coffee. Not even a please. I don’t think he called her honey or sweetie, but it was definitely implied. I was speechless and young enough to not step up and defend her. 

So the sale goes on. We get the buy in of the CIO, the CFO, the accountants ( they eventually became known as the IT department ), and basically nailed the sale. We had everyone’s buy in and the sale guy was looking forward to a big commission check.  Then we were marched back to the front of the office and sat down in front of the receptionist that he had offended from the moment he walked in the office.  She was shown the receptionist station phone and asked what she thought.  Anyone want to guess at how this turned out?

This just reinforced what mom and dad taught me. Be nice to everyone. It’s important. You never know when a lack of manners is going to come back to haunt you.


It’s all about getting from point A to point B. The rest is just details. 


For anyone who lived through the early days of Voice, you will all remember the media frenzy of the time telling everyone that the network team was going to absorb the comms team and that all the legacy voice people were going to lose their jobs. In fact, Cisco at the time was using the headcount reduction as one of the selling features.  This setup for a huge war between the Voice and Networking teams. Never a good thing. The legacy comms teams were afraid of this new technology. They didn’t understand what IP, TCP, CME, CCM, SRST, H.323, SIP, etc… meant. And they didn’t even know where to start.

There was one particular account where I was dropped into and all of the information sat in the head of the legacy comms guy who was running a multi-site Nortel network with some Option 11s and various norstars ( nortel key switch ) in a fairly complicated voice network. I was the 3rd or 4th person to get dropped in and I was warned about him. He was feeling threatened and had been intentionally difficult.  And rightly so. He was told that we were coming in to replace him and he had no path forward. 

I remember sitting down with Dave and I spent about 2 hours with him explaining routing in voice engineer terms. Routing in terms of dial plans, etc…  At the end of it he just looked at me and  shook his head and laughed.  Then he looked up and said “ That’s it?”

Apparently, the guys before me had never taken the time to explain any of what we were doing in terms that he could understand. Turns out the guy is a routing genius and understand the route summarization, advanced routing concepts, ACLs, and much much more. He was actually running a complicated voice network spanning 6 separate LATAs and doing Tail-End-Hop-Off ( TEHO ) while dealing with non-contiguous NSX codes.  

The network ops teams came to me a couple of days later and wanted to know what kind of magic I had pulled with Dave because they had never seen him more cooperative or helpful. Plus, he suddenly was a lot more interested in what they were doing and seemed to have gained 10 years of network knowledge overnight.

This project taught me that when you have a good conceptual model and you really understand it. The rest is just implementation details.  


Just because it’s a standard, doesn’t mean it’s a standard.

SOOOOOO many examples of this, but I think the first time I learned this was when SIP first came out. Wow. What a mess. Now we see non-standard implementations of protocols all the time. It’s sad that we’ve gotten to the point where we don’t even have outrage over it anymore. This brings me to the next lesson.


Trust but verify.

I worked a customer where we had ordered 2 PRIs ( ISDN Primary Rate interfaces ) for Voice service. We were clear that we HAD to have CallerID as part of the bundle. We received the sheet from the vendor ( Telco who doesn’t exist anymore ) who was actually subleasing the lines from another Telco.  It had all the regular line information on there. Protocol DMS 100, etc… So we spun up the line and we didn’t have CallerID.   Long Story short, the customer didn’t pay their bill for almost 2 years because the Telco could never get CallerID to work and every time we started getting somewhere, the account team changed and we had to start over from zero. 

One night during an outage window, just for giggles, we changed the protocol on our end from DMS 100 to NI1 or NI2. For those of you who don’t know, they are all pretty similar as ISDN protocols and the one thing which I remember is that the place where the incoming CallerID was stored was different among the three implementations.  The customer didn’t say a word to the vendor until for another few months. 🙂  


The application is what’s important. 

The most important thing I learned as a voice engineer is that the application is what’s important. Or more accurately, the value that the business derives from the application. In those days, if you said Callmanager had a minimum of features, it would have been an understatement.  I remember Lucent ( now Avaya ) competing against us with the “ We have 250+ features! You might need them! You’ll want to make sure that you have them!”.  The response was usually “Name 10 of those features”.   I’ve never had a customer get past 6 or 7.  Funny enough, I know see network vendors doing the same thing. 

All that really matters is If the product you’re pitching has the right features and you can align the to what the business is doing or wants to be doing.



Funny enough. I find myself having a lot of the same conversations know as our industry does another paradigm shift into the world of SDN, APIs, and Orchestration. Hopefully, I learned the lessons well enough the first time to not repeat them this time around. 


To all my other crusty old voice guys. Thanks for all the lessons you taught me. 

How to be a Network Jedi

So it’s almost 2012 and as I get a little older each year, I’m trying to become more comfortable with my role as a mentor and influencer of the next generation. When I look at my kids at Christmas and watch all the little behaviors and endearing little qualities that come from me, it’s really amazing how much influence that we can have simply by the example we set in the world.

In light of that, I’m going to start the New Years resolution a little bit early this year, and just throw out the goals that I’m going to try to live professionally this year, and hopefully rolemodel out to the next generation.  To be honest, it’s not really a New Years resolution since these are pretty much the goals I tried to live by last year too. But here they are. Hope you enjoy!


Funny enough: This was 99% written before @cloudtoad posted this great post on

Note: So, I blatantly stole these images from a Youtube talk by Artur Bergmen of You should check it out because it’s funny, and it’s wisdom.


  • 1) Learn the basics

I’ve had the honor to learn from many great masters in my years in the martial arts, and one of the lessons that really stuck with me is that the difference between a student and a master is the quality of the basic movements.

As I go into 2012 again, I’m going to really re-focus on learning the basics as I go for another round of Cisco Trivial pursuit. ( CCIE renewal ). Understanding the basics of technology will help you in so many ways and solve so many of the other mysteries for you.

2) Come ready to learn

We have been blessed with a career where we are rewarded for our insatiable curiosity.Respect how amazing a gift that is and embrace the learning opportunities which are placed before us everyday.

How does this work? Why did that break? What happens when I touch this button?Hmmm. I wonder why it did that?

I’m going to continue on in 2012 asking these questions, preferably in the safety of my own lab. 🙂

Yes honey… I promise I’ll get that fixed!


  • 3) RTFM – Don’t ask stupid questions.

There’s nothing more annoying to me when someone asks me a question that can be found in the first 5 hits of a simple google search ( ). A stupid question is one that you obviously have spent no time trying to figure out. One that it takes me one google search to find, or one which you should have known in the first place.

I’m not saying don’t ask questions, because those often lead to design breakthroughs or really great learning experiences because of the nuance brought on by that particular angle of a question. But also, don’t be afraid to sit down and go through the manuals to see if it’s actually documented somewhere. Don’t be afraid to read through some blog posts, and also, embrace social media. Twitter has an enormous amount of shared wisdom and experience.

If you are going to ask a stupid question; come prepared. Don’t waste someone’s time.

Warning: Ask stupid questions on twitter with caution. Someone might call you stupid.

  • 4) Figure it out

If Rule #3 fails you and you can’t find the answer that you’re looking for, then you need to go back to rule #1. If you know your basics, there should be almost no problem that you can’t break into its composite parts and figure out what’s happening.


Networks aren’t magic. Packets don’t teleport from point A to point B, and there are certainly no gnomes, trolls, magics elfs, or unicorn tears. ( sorry guys! ). If you don’t understand how they work from the basics, you stand no chance of understanding how they work when things get complicated.

VxLAN/NVGRE, LISP, and other overlay technologies are just that; Overlays. They don’t fix the underlying issues, and they can actually abstract the problem. As Artur says in the video, if you don’t understand the stack, you can spend a lot of resources trying to fix the problem at the wrong layer. This is the biggest issue I have with the overlays is that it will abstract the system and potentially allow the various data center factions to continue not-talking to each other. This is not a good thing.

  • 5) Learn about the other parts of the system

I believe that the network folk are actually the most well rounded out of the IT disciplines. (personal bias notwithstanding )

You will often find that network professionals were once programmers, or server/os guys, or even applications guys. ( Or in the case of Voice Network guys, they still are! ).

I actually believe that this is a result of the fact that we are always the ones who get blamed and we need to understand the system to prove our innocence. ( MTTI: Mean Time to Innocence )

As blah-blah cloud becomes more and more of a reality, and technologies like Openflow start to abstract the reality of the underlying system, it is becoming more and more important for SOMEONE to have an omniscient understanding of the datacenter.

That someone could be you. 🙂


If anyone has any additional goals for the year, please feel free to post below. I’d love to hear.


Through the eyes of a child

Wrote this last summer and apparently didn’t publish. Still amazes me.

Listening to the packet pushers podcast there was a listeners question on studying and learning. Coincedently, I had just had one of the most amazing experiences of my life. I had just watched my 6 year old son ride his bike alone for the first time.

We’ve been working on it all summer, and he was more scared than anything else. It was late September and he was discouraged and didn’t want to practice anymore, and I literally had to force him back on that bike. But I knew that this day was the day.

And it was.

The look of absolute wonder on his face at his new, seemingly superhuman, ability to ride his bike by himself was awe inspiring. And it got me thinking just how lucky we are in this industry.

We have the privilege every day to learn. what a great job we have.