Playing with Arista eAPI – Dynamic VLAN Assignment-ish

A note

Before we get started, let me first say that just because we can, doesn’t mean we should. I got a request to show an example of how dynamic VLAN assingment based on mac-address could work and this is the results of that request. Before anyone takes this code and runs with it, let me first say that you really need to think about whether or not this is a good idea for you and your organization.

If you want dynamic VLAN assignment, there are other ways to accomplish this and I would encourage you to look into them before going down this path.

That being said – It was a fun little project and I wanted to share it with you all.

The Project

For this project, the goal is to write a simple script to help us automatically assign the right VLAN to a host based on the MAC address of that host.

Now we all know there are different ways to do this, like pure mac-authentication using a RADIUS server, or even using a mac-based VLAN configuration, but that wouldn’t be as fun as trying to get this work with Arista’s eAPI now would it?

The Plan

So our basic plan is the following

Query the Arista eAPI to request all of the mac addresses.
Look through the eAPI response for the target mac address while ignoring the uplink ports that connects the switches
Once the interface where the host is connected has been identified, send another Arista eAPI call to assign the appropriate VLAN to the target interface.

*Note: This is a simple project and should NOT be implemented into a production environment without REALLY thinking about the various failure modes.

First we’ll do the imports. For this project, we’ll import the standard json library as well as the Arista pyeapi library.

note: This assumes you’ve actually pip install pyeapi’d this library into your environment. You did do that, right?

In [30]:

import json

import pyeapi

Create the Switches Variable

Now that we’ve imported the libraries we’ll need, let’s start setting up the variables

First we’ll create a variable called switches that will be a python list of strings which in IPv4 address format. One for each switch which we want to search through.

In [23]:

switches = ['10.101.0.249', '10.101.0.250']

Create the Uplinks Variable

As with most things with automation, you have two choices, standardize or write a LOT of code to keep up with the variable conditions. For my purposes, I want to keep this straightforward so I will be deciding that uplinks, or connections/trunks, between the two switches will be using interface Ethernet1 only.

This may sound a bit strange, but remember that in normal operations, an ARP will show up on the link that interconnects two switches as it travels through the network. I’m assuming you don’t want to be changing the VLAN on the interfaces that connect your switches, right?

We will now create a variable called uplinks that will be a python string object with the name of the ethernet interface as it would be displayed in the eAPI response. Basically we’ll be saying if the name is what’s contained in the uplink variable, then don’t do anything.

In [24]:

uplink = 'Ethernet1'

Create the target variable

Our last variable will be to set up the actual target host that we want to find and assign to a specific VLAN. You will need to make sure that the format of the mac-address must be consistent with what’s in the Arista eAPI response.

As we all know, mac-address can be displayed in various different formats including

xx:xx:xx:xx:xx:xx
xxxx.xxxx.xxxx
xxxx:xxxx:xxxx
xxxxxxxxxxxx
note: If we were doing this right, we would probably want to include some code that would normalize the mac-addresses into a consistent format regardless of the input format. But that’s code for another day. *

The other thing we’ll need to do is to include the dot1q vlan tag.

We will now create a variable called target that will be a python dict object with two key-value pairs. One for the mac-address, and one for the dot1q VLAN ID. This should be very obvious

In [25]:

target = {"mac" : '00:0d:6f:40:1b:e0', "vlan" : "3"}

Gathering the MAC addresses

Now that we’ve staged all variables, we can get down to writing some code.

The first function we’re going to create a python function to gather the mac-addresses from our list of switches and return a list of the mac-addresses.

In [26]:

def gather_macs(devices):

"""

    Function which takes a list of devices as input and runs the show mac address-table command against them

    using the Arista pyeapi library

    :param devices:

    :return: list of mac-address output

"""

    found_macs = []

    for switch in devices:

        node = pyeapi.connect(host=switch, username='admin', password='password')

        output = node.send('show mac address-table')

        output['switch'] = switch

        found_macs.append(output)

    print (len(found_macs))

    return found_macs

Now we can run this and assign the output to a python variable.

In [34]:

mac_list = gather_macs(switches)

mac_list

Out[34]:

[{'id': None,

  'jsonrpc': '2.0',

  'result': [{'multicastTable': {'tableEntries': []},

    'unicastTable': {'tableEntries': [{'entryType': 'dynamic',

       'interface': 'Ethernet48',

       'lastMove': 1521485142.203959,

       'macAddress': '00:0d:6f:40:1b:e0',

       'moves': 1,

       'vlanId': 1},

      {'entryType': 'dynamic',

       'interface': 'Ethernet17',

       'lastMove': 1516790007.97397,

       'macAddress': '00:1c:73:8c:92:75',

       'moves': 1,

       'vlanId': 1}]}}],

  'switch': '10.101.0.249'},

 {'id': None,

  'jsonrpc': '2.0',

  'result': [{'multicastTable': {'tableEntries': []},

    'unicastTable': {'tableEntries': [{'entryType': 'dynamic',

       'interface': 'Ethernet1',

       'lastMove': 1521485429.933993,

       'macAddress': '00:0d:6f:40:1b:e0',

       'moves': 1,

       'vlanId': 1},

      {'entryType': 'dynamic',

       'interface': 'Ethernet1',

       'lastMove': 1521421510.48407,

       'macAddress': '00:1c:73:8c:92:75',

       'moves': 1,

       'vlanId': 1}]}}],

  'switch': '10.101.0.250'}]

Now that we’ve got the list of mac-address from the switches that we defined in the switches variable. To do that, we need to write some more python code which will look through the list we created in the last step and look through that mac-list to see if the mac-address is present

In [21]:

def find_mac(target, list_of_macs):

    mac_location = []

    for switch in list_of_macs:

        for entry in switch['result'][0]['unicastTable']['tableEntries']:

            if entry['macAddress'] == target['mac']:

                print ("I found the mac_address " + target['mac'] + " on " + switch['switch'] + " on interface " + entry['interface'])

                mac_location.append({'mac': target['mac'], 'switch': switch['switch'], 'interface' : entry['interface']})

    return mac_location

Now that we’ve created the function, we can test this out using the following bit of code. Remember we created the variable target up above? That’s what we’re using here as the input.

In [28]:

host = find_mac(target, mac_list)

host

I found the mac_address 00:0d:6f:40:1b:e0 on 10.101.0.249 on interface Ethernet48

I found the mac_address 00:0d:6f:40:1b:e0 on 10.101.0.250 on interface Ethernet1

Out[28]:

[{'interface': 'Ethernet48',

  'mac': '00:0d:6f:40:1b:e0',

  'switch': '10.101.0.249'},

 {'interface': 'Ethernet1',

  'mac': '00:0d:6f:40:1b:e0',

  'switch': '10.101.0.250'}]

As you can see, we found two occurrences of the mac-address, the bottom one of two items in this list has the interface of Ethernet1 which means this is the uplink port between the two switches, remember?

So now we have to write some code to exclude the uplink interfaces which you can see below in the if entry[‘interface’] == uplink
phrase below.

If it’s not in the uplink list, then we need to use issue a command to the specific Arista switch to the specific interface where we found that mac-address.

In [35]:

def set_vlan(mac_location, target):

    for entry in mac_location:

        if entry['interface'] == uplink:

            print ("Skip the uplink")

            continue

        else:

            node = pyeapi.connect(host=entry['switch'], username='admin', password='password')

            data = {

  "jsonrpc": "2.0",

  "method": "runCmds",

  "params": {

    "format": "json",

    "timestamps": False,

    "autoComplete": False,

    "expandAliases": False,

    "cmds": [

        "cmd": "enable",

        "input": "my_enable_passw0rd"

},

      "configure",

      "interface " + entry['interface'],

      "switchport access vlan " + target['vlan']

],

    "version": 1

},

  "id": "EapiExplorer-1"

            print (entry['switch'])

            print (data)

            output = node.send(json.dumps(data))

            print (output)

Now we use the output of the find_mac() location that we got above combined with the VLAN information that we stored in the target variable.

In [36]:

output = set_vlan(host, target)

print (output)

10.101.0.249

{'method': 'runCmds', 'params': {'expandAliases': False, 'autoComplete': False, 'timestamps': False, 'version': 1, 'format': 'json', 'cmds': [{'cmd': 'enable', 'input': 'my_enable_passw0rd'}, 'configure', 'interface Ethernet48', 'switchport access vlan 3']}, 'jsonrpc': '2.0', 'id': 'EapiExplorer-1'}

{'result': [{}, {}, {}, {}], 'jsonrpc': '2.0', 'id': 'EapiExplorer-1'}

Skip the uplink

None

Now for the last part, we will write a quick bit of code to check the vlan of the interface to make sure that it’s actually been set in the right VLAN.

In [56]:

def check_int_vlan(switch, interface):

    node = pyeapi.connect(host=switch, username='admin', password='password')

    output = node.send('show mac address-table')

    return output

vlans = check_int_vlan('10.101.0.249', 'Ethernet48')['result'][0]['unicastTable']

for i in vlans['tableEntries']:

    print ("Mac address " + i['macAddress'] + " is located on interface " + i['interface'] + " in VLAN " + str(i['vlanId']))

Mac address 00:1c:73:8c:92:75 is located on interface Ethernet17 in VLAN 1

Mac address 00:0d:6f:40:1b:e0 is located on interface Ethernet48 in VLAN 3

Closing Thoughts¶

Hopefully you follow the logical process of how you would go about fulfilling the requirements. One thing I do want to leave you thinking about though is whether the goal made sense in the first place? As well, what additional assumptions and operational process needs to be in place to make this a reality. Questions like: Is the mac-address right right identifier? Where did you get the mac-address? Is this the physical mac-address or a virtual mac-address?

Feel free to post in the comments below if you feel inspired.

@netmanchris

Dipping my toes in the IoT pool: Microsoft DevKit IoT Board

In my personal life, I’ve jumped in the SmartHome head first, but I’ve been really reluctant to blur the lines into my professional life. Recently, I saw something that changed all that. The Microsoft IoT DevKit board.

NewImage

What is it?

This is Microsoft hardware product that allows developers to jump into the IoT pool. Specifically, the Microsoft Azure IoT hub pool. This is a very capable board with multiple sensors.

What sensors you ask?

Humidity:
Air Pressure
Temperature
Magnetometer
Motion
Microphone

Basically, it’s a really good sensor that can grab the majority of the physical measurements that we want to look at in the IoT world.

What do I do with it?

Right now, this is really just a tech toy for me. I have no specific projects that I’m trying to achieve. Rather this is a device that I’m using to try and really understand HOW the IoT ecosystem works and ensure my employability in years to come. So I don’t have any specific goals, but that’s really ok because Microsoft has been wonderful enough to supply us with the Microsoft IoT DevKit page over at Github which has a few different projects that allow me to grow my skills in several IoT use cases until my own specific use-case pops into my head.

What Projects are available?

The great thing about the DevKit project page is that there are a bunch of projects that are intended to take us from zero to hero in the IoT developer world. Or at least from noob to semi-competent.

From basic projects like Connect to Azure IoT hub and Remote Monitoring to more advanced projects like DevKit translator which leverages Microsoft’s Bing to create a speech translator or Shake,Shake for a tweet which integrates motion sensors with twitter.

There are ten projects in all and I wouldn’t be surprised if more projects are released in the future.

What Next?

This is an interesting project for me. It’s not python, it’s not really networking. And I’ve never used any Microsoft Azure services before. So this is going to be an interesting ride.

My plan is to start with the first tutorial, and see if I can bang my head against it until it works. 🙂 Look for that blog soon!

Interested in joining me on the journey? I picked up a board for $39 from DFRobot and it arrived in just a couple of days. If anyone wants to jump in and compare notes, please feel free to reach out. I’m more than happy to share what I’ve learned.

@netmanchris

Hey Alexa, Turn my lab on!

TL/DR Put together a custom Alexa Skill so I can turn switches and routers off in my lab as shown in the video here. Feels pretty great.

As most of my twitter followers have noticed, I’ve been doing a lot of Home Automation, mostly with Apple #homeKit. But I also picked up an Amazon Dot because… well why not?

One of the great things abut the digital voice assistance from Amazon, is that they have created an extensible framework that enables those with a little bit of coding skills to add to Mrs. A’s already already impressive impressive array of abilities.

The Amazon Alexa developer page is pretty impressive. There’s a ton of information and tutorials there, as well as an SDK and code examples in Node.js. I’m almost exclusively a python coder at this point, so I decided to look for something a little more familiar and came upon this.

Flask-Ask

Flask-Ask is a Flask extension that makes building Alexa skills for the Amazon Echo easier and much more fun.

Essentially, John Wheeler took the flask WSGI ( web) framework and made it super easy to be able to create Amazon Alexa skills using this familiar library. I’ve used Flask in the past for a few projects, so this was a no-brainer for me.

John also put together a set of tutorials here which can be used to jumpstart the Alexa skills development process. There’s also a flask-ask Quickstart on the amazing developers blog which pointed me towards ngrok which came in really handy!

*Ngrok allows you to create secure tunnels to a local host. You run ngrok with the port number you want to expose and it automatically exposes the host as a resource on the grok website. It’s really really really cool.

The Project

Like many of us, I have a physical lab in my house from my CCIE studies. As well, specializing in network management over the years requires access to physical gear in a lot of instances. Powering on that gear full time is out of the question because of the cost and power drain. As I’m sure you can imagine, going back and forth to turn things on and off gets old real quick.

To address that problem, I picked up a couple of intelligent PDU’s on eBay. There are many “smart” PDUs out there and I happen to have a set of Server Technologies that allows me to control each socket on a 16 port power bar. Pretty cool, right? No more walking to the garage, which is a good thing when you’re trying to focus on a problem.

So things are heading in the right direction; I can pop over to the local web interface of my PDU and turn my devices on and off. That’s nice…. But all the home automation stuff I’ve been doing lead me to wonder…

Why can’t I just ask for the device to be turned on?

I can ask Siri or Alexa to turn on the lights or adjust the temperature of my house. I can ask the about the weather or to check my calendar. There’s no reason why I shouldn’t be able to do the same with my lab gear.

So I decided to make that a reality.

What’s not covered in this blog

The one step which is not covered in this blog is writing the pyservertech library which I built on top of the pysnmp library. Essentially I walked the MIBs until I found how to gather the info I needed and figured out which specific MIB I needed to set to turn an individual power socket on or off. I might do a blog on that specific piece too, but for now, I’m trying to focus on the Alexa piece.

If there’s interest, please let me know in comments or on twitter and I’ll prioritize the SNMP set blog. 🙂

Building the Alexa Skill

Alexa skills are a combination of three components

Ask-Flask – This is the actual code and includes the templates file shown below
Intent_Schema – Kinda obvious, but this includes the various intents that you’re going to use in your skill
Sample Utterances – Are a collection of the various verbal phrases and how they are connected to the intents.

I’ll do my best to connect these in the code below, but I’d really recommend going through a couple of the tutorials above and play around with the examples to built some intuition on how these components connect.

The code below let’s a user do the following using Amazon Alexa’s voice assistant.

Ask Alexa to open the Lab skill ( Lab is what I called it )
Alexa asks the user “Welcome to the lab. I’m going to ask you which plug you want me to turn on. Ready?”
User responds with “Yes”or “Sure”
Alexa asks the user “Please tell me which power socket you would like to turn on?”
User responds with a number which is the power socket they would like to turn on
Alexa decodes the response and returns the number in a JSON array to the local Flask server
Python code takes the number from the JSON array and uses that as input into the power_on() function.
Power_On() function sends an SNMP SET command to the appropriate input.
Device powers on.Alexa says “I’ve turned the power socket on.”
I don’t walk to the garage.

Now that we understand how the code is supposed to work, let’s take a look at the individual pieces and how they fit together.

Alexa Skill

This is the python code that you’ll run on your local machine. This contains only a portion of the logic of the “program” as Amazon is really doing the majority of the lifting on their side as far as the speech recognition and returning the appropriate data in a JSON array.

Templates file

This file contains the various phrases that Alexa is going to speak on behalf of your application. You can see we’ve only got a few different.

Intent_Schema

This file gets loaded on the Amazon website. Using the developer interface, you load the JSON which defines the Intent Schema directly into the intent schema location on the Interaction Model page.

NewImage

Sample Utterences File

Just like the Intent Schema, the Sample Utterances is also loaded directly into the Amazon developer portal into the Interaction model for this specific skill

NewImage

What’s next

This is just the start of this skill. All it does right now is turn things on, which is cool, but I want more. Just off the top of my head here are some of the things I’d like to do

Turn individual devices on or off
Turn individual devices on or off by name “Alexa ask the lab to turn on the HPE 2920 switch!”
Turn groups of devices on or off “Alexa ask the lab to turn on the Juniper branch!”
Request data from the PDUs “Alexa ask the lab How many devices are currently turned on?” Or “ask the lab how much power is currently being used”

As you can imagine, this would require a lot more code and logic to accomplish all these goals. Definitely something I’m going to pursue, but I’m hoping that the simple example above helps to inspire someone else in their journey down this path.

Questions? Comments? You know what to do…

@netmanchris

Home Automation Setup with Apple #HomeKit

As many of you know, I’ve been diving into the home automation pond for awhile now. I’ve been asked to blog out my current home setup and this is an attempt to do that. There’s MUCH too much to be put into a single post, which is why I started a new blog for this subject over at www.homekitgeek.com as well as doing some video reviews of different HomeKit accessories. Work in process, but I’ll do the short version here.

NewImage

Apple HomeKit

NewImage

I chose to use the Apple homeKit framework as the base for my home automation journey for a few reason. The biggest one is that I already owned a couple of Apple TV (gen4) devices which fit the home automation hub role. These devices are a homeKit hub and are the always-on/always-present devices that are used to perform orchestration/automation actions when I’m home or away. These also tie directly into Apple’s iCloud which allows me remote access to my homeKit gear without having to VPN into my home network.

Home Setup Room-By-Room

There’s a lot to talk about here, but I thought I would just do a quick description of what’s going on room-by-room with any specific features or automations that I’ve got setup.

Front Entrance (outside)

DoorBell

I’ve got a Ring Pro Doorbell which is NOT HomeKit compatible yet. They’ve been promising for a couple of years and have assured me, both publicly and in DMs, that they are committed to updating the Ring Pro for HomeKit support. Small issues with the wiring of my house, but I was able to eliminate the doorbell chime itself and get the Ring Pro up and running. The Ring Chime Pro was a nice addition as well which I’ve got setup in the living room.

NewImage

Lights

For the lights, I’ve got the front entrance lights tied to a Luton Caseta light switch in the house which is setup with an automation to turn the lights on at Sundown and turn them off at Sunrise. This is nice in that the system automatically checks the internet to see exactly what time those will be everyday. Living in a Northern country, Sunrise/Sunset can vary pretty wildly throughout the year. This automation lets me get the benefits of a timer system but without the hassle of changing the timer every month as the times change.

The one thing I haven’t automation yet is the flood motion light. I would really like to get a ring floodlight cam but I’m not investing any more money into Ring until they deliver on their promise around the doorbell.

Front Entrance (inside)

Lock

I decided to not replace the whole deadbolt system and just use the August lock which fits nicely on top of most existing locks. I particularly like the August iPhone App feature which automatically unlocks your front door as you pull in the driveway. Really nice to not have to mess around with keys. 🙂

NewImage

Sensor

In addition to the lock, I also installed an Elgato Eve Door & Window sensor on the front door. Just because it’s locked, doesn’t mean it’s closed. 🙂 Yes, I found that out the hardware.

NewImage

Lights

In the mud-room, I’ve got a light which is connected to the same Lutron Caseta which controls the outdoor lights. Once you’re in the house, I’ve got a Philips Hue motion sensor which turns on a Philipps Hue GU10 light bulb at the bottom of the stairs. Unfortunately, the Philippe Hue motion sensors are not exposed directly to HomeKit, so I was forced to setup an on/off automation pair for a Lutron Caseta switch which controls 6 GU10 lights in the ceiling.

Living Room

Sensors

I’ve got a Elgato Eve Degree for tracking general room stuff ( temperature, air pressure, humidity ).

Cameras

I’ve got a Dlink Omna 180HD camera which, at the time of this writing, is the only camera on the market with official HomeKit support. I was unsure about it when I first installed it, but it’s started to grow on me. The device is billed as a 180 degree camera which gives me full view of my kitchen/dining room/living room areas ( open concept ) and also provides night vision. The one feature that seems to get used the most though is the 2-way audio which lets me talk with someone in the living room without having to be physically present.

Parent note: Also handy to check in on the kids discretely to see what’s happening when you hear things starting to go sideways.

NewImage

Lights

The living room definitely has the most complicated light setup in the whole house. I’ve got GU10 ( recessed lights.) through the whole upstairs which can be quite expensive with the Philipps Hue systems. So the living room is the only place I used the GU10 in mass. Typically, I would use a Lutron Caseta switch to power a bunch of GU10 bulbs ( anything over 2 bulbs and the Lutron is cheaper!!!), but for the entertainment area, I really wanted to have the ability to set different scenes. Philipps Hue has a bunch of different models available and for the living room, I decided to use

Philipps Hue GU10 lights ( both coloured and white )
Philipps Hue Bloom ( accent light )
Philipps Hue Light Strip

The combination of these products let’s me go from regular family to chill movie watching with the touch of a button. I’ve also got a coupe of IFTT automations setup to help manage the kids schedule.

At 8:10pm, the coloured GU10 lights go purple which is a visual signal for them to stop what they’re doing and get ready for bed. If they are quick, they can come back and finish whatever they are doing. At 8:30, every light in the living room flashes to let them know it’s time to go to bed. To be honest, the flashing light is pretty rough, but it is definitely impossible to ignore, even for a kid who’s deep into a Minecraft session.

Other

The other thing I added to the Living room is a Logitech Pop button with the HomeKit Bridge. Voice control with Siri is great and all, but it’s also nice to be able to just tap a button to set the scene. You can set three different scenes with the button which I’ve chosen as follow

Single Press – Turns on the Living Room lights bright scene. This is “normal” mode
Double Press – Turns on Movie time Scene. Dims lights and turns on the Philipps Hue lights for accent lighting.
Long Press – Turns on Good night Scene which turns off all the lights, locks the doors, etc…

Dining Room

Lights

Dining Room is pretty simple. I have 6 lights controlled by a Lutron Caseta light switch. I also got a couple of Ikea Accent Light Boxes which are hooked up to an iDevice plug. This gives me a candle-light vibe without the fire.

BackDoor

The dining room opens directly onto a small deck. This is an entry point with I wanted to add a bit of security to. I used the Elgato Eve Door & Window sensors

To let me know when the door opens and closes. This is really nice when you’re leaving the house for a quick “Hey, Did I close the back door?” check. It doesn’t tell me if I locked it or not. Still haven’t found a solution to that problem.

Kitchen

Lights

Kitchen lights are also pretty simple. Overhead is 6 GU10 bulbs controller by a single Lutron Caseta switch. I’ve got under cabinet lighting as well that I’m planning on adding another Lutron switch to control. Idea here is to setup a motion sensor to turn on the under cabinet lighting at night. No need to blind anyone, right?

Sensors

I have an iHome iS550 5-1 sensor in the kitchen. No particular reason to be honest. I originally bought this for the master bedroom but it didn’t work out at all. Constantly lost wifi signal and the fact that it’s a wall-powered device makes it difficult to hide.

Hallway to Bedrooms

Lights

In the hallway to bedrooms I have a couple of Philipe Hue GU10 bulbs with a motion sensor setup. This automatically turns on the lights as you walk down the hallways. I have young kids ( 6 -10 -11) who need to get up at night, so I also setup a rule that turns the lights in to a “nightlight” mode if motion is detected after 9pm. Keeps the blinding of the children down and makes sure that they can still hit the target in the ‘wee hours of the morning.

Kids Bedrooms

Kid1

Lutron Caseta Switch for the ceiling lights. I also setup a Philipps Hue bloom with the sunrise scene turned on to help him get out of bed in the morning. His room is in the basement, so I also got a fibaro window & door sensor as his room is in the basement.

Kid2

I’ve got a Philipps Hue color light bulb paired with the Philipps Hue Light switch. He’s also got a colorful “moonlight” which is connected to an iDevice plug. This is connected to a time-of-day based automation which turns it on at bedtime and off at midnight ( after he’s fallen asleep). The Philipps Hue color bulb is also setup for the sunrise scene.

Kid3

Of the three kids room, this was the hardest. We had legacy fluorescent light that simply didn’t work with the Lutron Caseta light switch. I ended up replacing the light fixture completely as I REALLY like the Lutron switches. I also have a Philipps Bloom setup with, you guessed it, the sunrise scene.

Master BedRoom

Lights

No ceiling lights in the master bedroom, so I use a couple of different lamps. One lamp I’ve used a Philipps Color bulb, and then other large lamp I use a Lutron Caseta wall plug. The large lamp uses small candle bulbs ( not sure the exact model ) and it was far easier to just grab the Lurtron Caseta wall plug. The Caseta wall plug also acts as a repeater extending the range and reliability of the Lutron automation. ( Luton uses a protocol called ClearConnect, not wi-fi or bluetooth). There’s also a Philipps Go light in the master bedroom as well. The Philipps Go is great as it’s got an internal battery. This came in handy recently when we got hit by a power outage. 🙂

Sensors

I originally used the iHome iS550 sensor in the master bedroom, but it just didn’t work out. Iswapped it for the Elgato Eve Room Sensor which has been working great. The Elgato Eve Room sensor is a bluetooth device and is power by batteries, so no issues with disconnecting wifi networks or losing the device completely during a power outage. The Eve room sensor gives me temperature and humidity. I then use this measurement as a trigger to either turn on the humidifier if the air is too dry.

The room also measures VoC ( volatile organic compound ) or air quality. As allergy season is about to hit, I’m going to get an indoor room air-purifier and use the VoC measurement to turn the purifier on or off.

Humidifier

The humidifier is a generic Honeywell humidifier. I got lucky in this is an analog switch based unit. This means that if it’s in the ON position and you plug it in, it just starts working. I use an iDevices wall plug (with Nightlight!) to control whether or not the power is turned on to the humidifier. Basically, if it gets to dry, as measured by the Elgato Eve Room sensor, a trigger is sent to the iDevices wall plug to turn on and start pumping a little moisture into the air.

Air Purifier

I don’t have the air purifier setup yet, but planning on this as my next purchase. Same basic principal as the Humidifier, but replace with the purifier. If the air quality is poor, as measured by the Eve Elgato room sensor, then the power to the plug turns on and the air starts getting clearer.

Home Office

Lights

The home office lights are using the Lutron Caseta switch. I also included an Elgato Eve Motion Sensor to automatically turn on the lights when I walk in the office. The Eve is Bluetooth based and doesn’t have the response that the Philipps Hue motion sensor does, but the 2-3 second delay doesn’t bother me at all when I know I’m going to be in the office for awhile. The Eve motion sensor also turns on some desk accent lighting which has been plugged into an Eve Elgato Energy wall plug. In a nutshell, the lights turns on when I walk in the office and the lights turn off when I’m no longer there ( after a 15 minute period of no movement detected ).

Sensors

In addition to the Eve Elgato motion sensor mentioned above, My home office is in the basement, so I also included a couple of Eve Elgato Door & Window sensors to make sure the house is secured. Living in a northern country, leaving the window open can have consequences.

Laundry Hallway

Lights

This was my latest addition. I noticed that when I walked downstairs with the laundry basket in my hands, I had to put it down to turn on the light. I know. The horror!!!! I installed 2 Philipps Hue GU10 bulbs with a Philipps Hue Motion sensor to automatically turn on the lights as I carry my basket. Kid1’s room is also at the end of this hallway, so I also setup the nightlight ( 5% light.) to trigger if it’s after 9pm. No need to blind the kid, right?

BackYard

Plug

I have a single iDevices outdoor plug which I use to plug in my electric lawn mower. Although this is a dual-plug unit, both plugs are controller simultaneously meaning that they are both ON or OFF. The plug is set up to be off during the week and power on at 9pm on Friday night. Just enough time to get a full charge to mow the lawn on the weekend.

As the season gets colder, I’m planning on using this to power the Christmas lights as well. Plan is to setup a time-based trigger to turn on the lights at Sundown to save a little electricity.

Garage

Lights

I have a single Philipps A19 bulb with a Philipps Hue motion detector installed to run on as motion is detected in the garage. I also had a spare Koogeek Smartsocket that I plugged a regular A19 light bulb into. I’ve setup an ON/OFF trigger pair to turn the Koogeek light to match the Philipps hue light.

The one other thing to note in the garage is a time-of-day automation I setup to prevent me from banging my head until the wee hours of the morning. At 11pm, the lights go into nightlight mode reminding me to go to bed and work on it tomorrow.

Boiler Room

This is really just a small closet which contains the hot water tank and my furnace. I installed a Fibaro flood sensor in here to detect any leaks before they become a big problem. The hot water tank is a little bit older. Installing the leak sensor just gives me a little peace of mind. This sensor also includes a temperature sensor which is super important for a water tank in a northern country. If it starts to get cold, bad things can happen. Not a bill I’m interested in seeing.

Downstairs Windows

There are a few different windows in my basement that I installed the Elgato Eve Door & Window sensors on. Peace of mind to make sure that we’re all locked up when we leave at night. Already mentioned, but also important when you live in a Northern country. Bad things can happen when the inside of your house goes below freezing temperatures.

What’s next?

As you can tell, I’ve really developed a passion for home automation. I currently only have two projects I’m planning, but as new HomeKit enabled devices come out, I expect this list to grow.

HomeBridge Nest Thermostat integration

The Nest thermostat is not currently Apple homeKit compatible. The Homebridge project exists to let non-Homekit devices participate in a HomeKit ecosystem. I recently purchased a RaspberryPi3 for this purpose. My goal here is basic thermostat integration into the rest of my smart home.

Soma Smart Shades

Soma has recently announced a HomeKit bridge for pre-order. I’m pretty wary about ordering any products without the HomeKit certification. Too many companies which have announced HomeKit support and never delivered. Soma is an after-market device which connects to your existing shades and allows you to control them as part of your smart home. Combine this with the different temp sensors and we can do some interesting things such as

Shut Blinds at 12pm every day
Shut blinds when outdoor temperature exceeds desired indoor temperature
Open blinds at 4pm
Shut blinds for Movie Time scene

Hopefully this was interesting. If you have any questions or want more information, feel free to post in the comments below or to reach out on twitter

@netmanchris

The First IoT Culling: My Devices are Dying.

Cull: to reduce the population of (a wild animal) by selective slaughter

As an early adopter of technology, I sometimes feel like I get to live in the future. Or as William Gibson said “The future is already here, it’s just not evenly distributed”. There are a lot of benefits to be gained from this, but there are also risks. One of the biggest risks is

How long is the product you choose going to be around?

I was an early adopter in the first wave of IoT devices, from wearables to home convenience devices, I dipped my toes in the pool early. Most of these platforms were Kickstarter projects and I’ve been generally happy with most of them, at least the ones that were actually delivered. ( That’s a story for another time…).

But in the last six months, the market seems to have decided that there are just too many of these small companies.

The Death Bells are Ringing

In the last year, I’ve noticed that there’s starting to be a trend. Many of the early platforms that I invested in seem to be disappearing. Some have been bought off and killed. Remember the Pebble watches which was acquired by Fitbit? I’ve got an original and a Pebble time that are now little more than short-term battery traditional watches.

NewImage Some are just dying on the vine.

The latest victim? The Sense sleep monitor system by Hello. This was a Kickstarter project that really helped to define a new category. When the project was launched in 2013, there was nothing else like it in the market, at least nothing that I’m aware of. Like most Kickstarter projects, they shipped later than their Aug 2014 estimate, but when it arrived it was definitely worth the wait.

This device had multiple sensors including light, sound, humidity, VoC ( air quality ), and temperature. It also had remote bluetooth motion sensors that attached to your pillows to track body movement while you sleep. The basic idea is that you sleep 1/3 of your life. Shouldn’t we make sure we are doing it right? The combination of the sensor data combined with sleep research will help users understand why they feel good or bad in the morning. How to create the optimal conditions in your bedroom etc… Obviously I’m not a sleep expert, but I can say that sense has improved the quality of my sleep since I started using it.

Just last week, we all received some sad news that Hello, the company behind the Sense product is shutting down.

NewImage

What’s happening?

Although there are some people who believe that Sense was just a faulty product, I think there is something deeper going on here. This shows a fundamental flaw with the business models that some of the early IoT and wearables players came to market with. There is a very simple business principle that somehow they seemed to completely miss.

If you want to survive as a business, you’re incoming cash must be more than you’re outgoing case.

Pebble and Sense, and several other wearable and IoT products in the market right now were built on a single-purchase model. You buy the product and you get unlimited right to use that product. This is the consumers preferred model. When I spend my money on something, I want to own it. I want to be able to use it, and I don’t want to have to pay for it again and again. At least this is the simple version of the thought process.

Spending some time watching various devices become little more than expensive bricks has made me re-examine that thought process though.

Why I’m looking for Subscription models now

Yup. That’s right. You heard me.

I want to find companies that are actively looking to provide value funded through a subscription model of some kind. Companies like Nest and Ring who are providing cloud storage for security cameras are a great example of this in action.

Looking at the failing companies; the one thing in common that I’m starting to see in common with these different devices is that they tend to make a whole bunch of money up front. ( $10M+ for the initial Pebble Kickstarter project. One of the largest ever on that platform! ). But they tend to be niche products that have a limited target audience, and when that target market has been saturated…. No more money comes in and they’re left having to continue to pay for the “cloud” infrastructure required to keep their products going.

Looking at Sense and Pebble, both of these platforms sold with a hardware model. They have a product offering where your devices connect to cloud-based infrastructure, whether that’s AWS based or other is irrelevant. What most consumers don’t realize is that cloud-based infrastructure has a reoccurring monthly cost to it. This also doesn’t include the cost of ongoing platform development, whether that’s adding new features, creating a better user-experience, or just upgrading to stay current with the newest versions of Apple iOS or Android that are shipping on current devices.

This is fine as long as you continue to sell new hardware product, but as the number of new users start to trend down and your costs stay the same… we start to see what’s happening in the market right now.

Are Subscription models that only way?

Absolutely not. There are other companies, like Dlink, iHome or iDevices that have a fairly broad portfolio of products and are continuously creating new products. These helps to ensure they have a healthy income stream as individual product segments become saturated. They can afford to continue to fund app development and the infrastructure required to host them as they are spreading that cost over many devices.

More Deaths in the future

There have been some notable passings, such as Pebble and Sense, but I don’t think they are going to be the last by any stretch of the imagination. 2017 and 2018 are going to be a hard year on early adopters as we start to look at the mirrors, watches, and gadgets blink eternally as they have no home in the cloud to call back to. Hoping that many of the new IoT players start to realize that having a good technology idea isn’t enough if you want to survive. Strange that I’m now looking at business models in a consumer product purchasing decision. I guess this just goes to show how educated the consumer is truly becoming.

As I invest in my SmartHome products, I look for companies who are established with multiple streams of revenue. Companies like Lutron or Philipps. In some cases, like the Soma Smart Blinds, I really don’t have another option. I’ll probably buy them, but I’m not expecting to these to last the long term. I wish Soma the best of luck, but I don’t see a subscription model and it’s not like shades are something you replace every year.

Bottom line is enjoy your first generation wearables now. They might not be around for that much longer.

@netmanchris

Using JSONSchema to Validate input

There are a lot of REST APIs out there. Quite a few of them use JSON as the data structure which allows us to get data in and out of these devices. There are a lot of network focused blogs that detail how to send and receive data in and out of these devices, but I wasn’t able to find anything that specifically talked about validating the input and output of the data to make sure we’re sending and receiving the expected information.

Testing is a crucial, and IMO too often overlooked, part of the Infrastructure as Code movement. Hopefully this post will help others start to think more about validating input and output of these APIs, or at the very least, spend just a little more time thinking about testing your API interactions before you decide to automate the massive explosion of your infrastructure with a poorly tested script. 🙂

What is JSONSchema

I’m assuming that you already know what JSON is, so let’s skip directly to talking about JsonSchema. This is a pythonlibrary which allows you to take your input/output and verify it against a known schema which defined the data types you’re expecting to see.

For example, consider this snippet of a schema which defines what a valid VLAN object looks like

"vlan_id":
{
    "description": "The unique ID of the VLAN. IEEE 802.1Q VLAN identifier (VID)", 
    "minimum": 1, 
    "maximum": 4094, 
    "type": "integer", 
    "sql.not_null": true
}

You can see that this is a small set of rules that defines what is a valid entry for the vlan_id property of a VLAN. As a network professional, it’s obvious to us that a valid VLAN ID must be between 1 and 4094. We know this because we deal with it every day. But our software doesn’t know this. We have to teach it what a valid vlan_id property looks like and that’s what the schema does.

our software doesn’t know this. We have to teach it

Why do we care?

Testing is SUPER important. By being able to test the input/output of what you’re feeding into your automation/orchestration framework, it can help you to avoid, at worst, a total meltdown, or, at best, a couple of hours trying to figure out why your code doesn’t work.

Using JSONSchema

So the two things you’re going to need to use JSONSchema are

The JSON Schema for a specific API endpoint
The input/output that you want to validate.

In this case, we’ll use a VLAN object that is coming out of an ArubaOS-Switch REST API.

You did know the ArubaOS-Switches have a REST API, right?

Step1 – Loading the VLAN object

We’re going to gather the output from the VLANS API. Instead of writing custom code, we’ll just use the pyarubaoss library. I’ll leave you to check out the GitHub repo and just paste the contents of the output of a single VLAN JSON output here. I’m also going to create a second VLAN with a VLAN_ID of 5000. Just to show how this works. 5000 of course, is not valid and we’d like to prove that. Right?

Step 2 – Loading the JSON Schema Definition

Now we have the output, we want to make sure that the output here complies with the JSON Schema definition that we’ve been provided.

Loading the JSON schema

Here’s a sub-set of the JSON schema that defines what a valid VLAN looks like

Step 3 – Importing the JSON Schema Library and Validating

Now we’re going to load the JSON Schema library into our python session and use it to validate the VLAN object using the Schema we defined above.

First we’ll look at the vlan_good object and run it through the validate function

As you can see, there’s nothing to see here. Basically this means that the vlan_good object is conforming properly to the provided JSON Schema. The VLAN ID is valid as it’s a integer value between 1 and 4094

Now let’s take a look at the vlan_bad object and run it through the same validate function

We can see that the validate function now raises an exception because and let’s us know very specifically that the VLAN ID 5000 is not valid

jsonschema.exceptions.ValidationError: 5000is greater than the maximum of 4094

Pretty cool right? We can still definitely shoot ourselves in the foot, but at least we know the input/output data that we’re using for our API is valid. To me this is important for two reasons

I can validate that the data I’m trying to send to a given API conforms to what that API is expecting
I can validate that the vendor didn’t suddenly change their API which is going to break my code

Wrap Up

There are a lot of networking folk who have started to take on the new set of skills required to automate their infrastructure. One of the most crucial parts of this is testing and validating the code to ensure that you’re not just blowing up your network more efficiently. JSON Schema is one of the tools in your tool box that can help you do that.

Comments, questions? Let me know

@netmanchris

Amazon S3 Outage: Another Opinion Piece

So Amazon S3 had some “issues” last week and it’s taken me a few days to put my thoughts together around this. Hopefully I’ve made the tail-end of the still interested-enough-to-find-this-blog-valuable period.

Trying to make the best of a bad situation, the good news, in my opinion, is that this shows that infrastructure people still have a place in the automated cloudy world of the future. At least that’s something right?

What happened:

You can read the detailed explanation on Amazon’s summary here.

In a nutshell

there was a small problem
they tried to fix it
things went bad for a relatively short time
They fixed it

What happened during:

The internet lost it’s minds. Or more accurately, some parts of the internet went down. Some of them extremely ironic

UNADJUSTEDNONRAW thumb bbfd

Initial thoughts

The reaction to this event is amusing and it drives home the point that infrastructure engineers are as critical as ever, if not even more important considering the complete lack of architecture that seems to have gone into the majority of these “applications”.

First let’s talk about availability: Looking at the Amazon AWS S3 SLA, available here, it looks like they did fall below there 99.9% SLA for availability. If we do a quick look at https://uptime.is/ we can see that for the monthly period, they were aiming for no more than 43m 49.7s of outage. Seems like they did about 6-8 hours of an outage so clearly they failed. Looking at the S3 SLA page, looks like customers might be eligible for 25% service credits. I’ll let you guys work that out with AWS.

Don’t “JUST CLICK NEXT”

One of the first things that struck me as funny here was the fact that this was the US-EAST-1 Region which was affected. US-EAST is the default region for most of the AWS services. You have to intentionally select another region if you want your service to be hosted somewhere else. But because it’s easier to just cllck next, it seems that the majority of people just clicked past that part and didn’t think about where they were actually hosting there services or the implications of hosting everything in the same region and probably the same availability zone. For more on this topic, take a look here.

There’s been a lot of criticism of the infrastructure people when anyone with a credit card can go to amazon sign up for a AWS account and start consuming their infrastructure. This has been thrown around like this is actually a good thing, right?

Well this is exactly what happens when “anyone” does that. You end up with all your eggs in one basket. (m/n in round numbers)

“Design your infrastructure for the four S’s. Stability Scalability, Security, and Stupidity” — Jeff Kabel

Again, this is not an issue with AWS, or any Cloud Providers offerings. This is an issue with people who think that infrastructure and architecture don’t matter and it can just be “automated” away. Automation is important, but it’s there so that your infrastructure people can free up some time from mind numbing tasks to help you properly architect the infra components your applications rely upon.

Why o Why o Why

Why anyone would architect their revenue generating system on an infrastructure that was only guaranteed to 99.9% is beyond me. The right answer, at least from an infrastructure engineers point of view is obvious, right?

You would use redundant architecture to raise the overall resilience of the application. Relying on the fact that it’s highly unlikely that you’re going to lose the different redundant pieces at the same time. Put simply, what are the chances that two different systems, both guaranteed to 99.9% SLA are going to go down at the exact same time?

Well doing some really basic probability calculations, and assuming the outages are independent events, we multiple the non-SLA’d time happening ( 0.001% ) in system 1 times the same metric in system 2 and we get.

0.001 * 0.001 = 0.000001 probability of both systems going down at the same time.

Or another way of saying that is 0.999999% of uptime. Pretty great right?

Note: I’m not an availability calculation expert, so if I’ve messed up a basic assumption here, someone please feel free to correct me. Always looking to learn!

So application people made the mistake of just signing over responsibility to “the cloud” for their application uptime, most of whom probably didn’t even read the SLA for the S3 service or sit down to think.

Really? We had people armed with an IDE and a credit card move our apps to “the cloud” and wonder why things failed.

What could they have done?

There’s a million ways to answer this I’m sure, but let’s just look at what was available within the AWS list of service offerings.

Cloudfront is AWS’s content delivery system. Extremely easy to use. Easy to setup and takes care of automatically moving your content to multiple AWS Regions and Availability Zones.

Route 53 is AWS’s DNS service that will allow you to perform health checks and only direct DNS queries to resources which are “healthy” or actively available.

There are probably a lot of other options as well, both within AWS and without, but my point is that the applications that went down most likely didn’t bother. Or they were denied the budget to properly architect resiliency into their system.

On the bright side, the latter just had a budget opening event.

Look who did it right

Unsurprisingly, there were companies who weathered the S3 storm like nothing happened. In fact, I was able to sit and binge watch Netflix well the rest of the internet was melting down. Yes, it looks like it cost 25% more, but then again, I had no problems with season 4 of Big Bang Theory at all last week, so I’m a happy customer.

Companies still like happy customers, don’t they?

@TheSteve0 @bryanrbeal @randybias the cost model estimated about 25% more cost for active-active. Insurance policy…

— adrian cockcroft (@adrianco) September 20, 2015

The Cloud is still a good thing

I’m hoping that no one reads this as a anti-cloud post. There’s enough anti-cloud rhetoric happening right now, which I suppose is inevitable considering last weeks highly visible outage, and I don’t want to add to that.

What I do want is for people who read this to spend a little bit of time thinking about their applications and the infrastructure that supports them. This type of thing happens in enterprise environments every day. Systems die. Hardware fails. Get over the it and design your architecture to take into consideration these failures as a foregone conclusion. It IS going to happen, it’s just a matter of when. So shouldn’t we design up front around that?

Alternately, we could also chose to take the risk for those services that don’t generate revenue for the business. If it’s not making you money, maybe you don’t want to pay for it to be resilient. That’s ok too. Just make an informed decision.

For the record, I’m a network engineer well versed in the arcane discipline of plumbing packets. Cloud and Application architectures are pretty far away from the land of BGP peering and routing tables where I spend my days. But for the low low price of $15 and a bit of time on Udemy, I was able to dig into AWS and build some skills that let me look at last weeks outage with a much more informed perspective. To all my infrastructure engineer peeps I highly encourage you to take the time, learn a bit, and get involved in these conversations at your companies. Hoping we can all raise the bar collectively together.

Comments, questions?

@netmanchris