Intro to Jinja2

 
 

What is Jinja2

Jinja2 is a templating language that was originally used as part of the Flask python web framework. From the Jinja2 website

Jinja2 is a full featured template engine for Python. It has full unicode support, an optional integrated sandboxed execution environment, widely used and BSD licensed

It was originally developed to help automatically generate HTML dynamically as part of the flask framework, more on that in another post, but it can also easily be used to help us generate our configuration files for our infrastructure devices.

This is going to be a very simple introduction to a few of the basic concepts of that jinja uses which, hopefully, will help to understand how Jinja can be used as a first step down the road of gaining automation skills.

We’ll take a look at a developing some intuition on how Jinja2 can be used to create basic network infrastructure device configurations. This is definitly not the modern method of interfacing directly into the control/data/management plane of devices using APIs, but it’s definitely a step in the right direction on understanding how a bit of code can help make your life better.

Prereqs

I’m assuming you’ve already got python installed on your system. You’re also going to need to run the pip install jinja2 command from a terminal window to get the latest version of jinja2 which should work just fine here.

 

Learning by Example

In this section we’ll start with a small example on how to create a few VLANs using the typical syntax from a modern networking OS. In this case, I used the HPE Comware syntax, but it would be easy enough to create this using a Cisco or Juniper configuration and you’re encouraged to try to get this working with your own network vendor.

 

Import required libraries

First We’ll import the required modules from the Jinja2 library. This is pretty much stolen directly from the jinja2 docs.

In [1]:
from jinja2 import Environment, FileSystemLoader, Template
 

Creating the VLANS

For this example, we’re going to create a python list of dicts which contains six different VLANS as listed in the tabel below.

A python dictionary is just a key-value pair, where the value for a specific key in the dictionary can be accessed using the key name.

  • Name: Name of the VLAN
  • Description: Descrition of the VLAN
  • VLAN ID: Dot1q VLAN ID.
Name Description VLAN ID
Management Management VLAN 10
Users Users VLAN 15
Phones Phones VLAN 16
Servers Servers VLAN 20
Mobility Mobility VLAN 30
Guest Guest VLAN 40
In [5]:
vlans_list = [{'name': 'management', 'description': 'management vlan', 'id': '10'},
         {'name': 'users', 'description': 'users vlan', 'id': '15'},
         {'name': 'phones', 'description': 'phones vlan', 'id': '16'},
         {'name': 'servers', 'description': 'servers vlan', 'id': '20'},
         {'name': 'mobility', 'description': 'mobility vlan', 'id': '30'},
         {'name': 'guest', 'description': 'guest vlan', 'id': '40'},
         {'name': 'rob', 'description': 'guest vlan', 'id': '45'}
         ]
 

VLAN Jinja2 Template

In this step, we’re going to create a variable called text_file which will contain the content of a jinja2 template. This is a basic python string object which means, at this point, it’s just a bunch of text.

In normal circunstances, we would actually be reading this template from a file located on the hard drive, but for our purposes today, we’ll just put the templatein by hand.

What makes Jinja2 powerful is the control structures that allow it to perform programatic operations. In this example, we’re creating a For loop.

Following the code we will each vlan in the vlans object we created above and then render the template using the ‘id’ key for the first variable, the ‘name’ key for the second variable, and the ‘description’ key for the last variable.

Hopfully, this makes sense, but if not, just hold on and it should become clear before the end.

In [6]:
text_file = ('''
#vlan config
{% for vlan in vlans -%}
vlan {{ vlan['id'] }}
 name {{ vlan['name'] }}
 description {{ vlan['description'] }}
{% endfor %}''')
 

If I was to write the same as a traditional python iterator it would look something like this. You can see how they are related I hope?

In [7]:
for vlan in vlans_list:
    print ('''vlan ''' +vlan['id']+
           '''\n name '''+vlan['name']+
           '''\n description '''+vlan['description'])
 
vlan 10
    name management
    description management vlan
vlan 15
    name users
    description users vlan
vlan 16
    name phones
    description phones vlan
vlan 20
    name servers
    description servers vlan
vlan 30
    name mobility
    description mobility vlan
vlan 40
    name guest
    description guest vlan
vlan 45
    name rob
    description guest vlan
 

That’s a lot of work typing isn’t it?

You could ask

That’s more typing than I would do by hand? Why would I use this?

Great question. The point of automating anything is to cut down on the repetitive effort it takes to accomplish a given goal. In this case, we can simply count the number of key strokes it would take to create a single new VLAN on a switch.

In [6]:
count_chars = "vlan', 'name', 'description"
keystrokes = len(count_chars)
print (keystrokes)
 
27
 

Now let’s pretend we had to type that 10 times

In [7]:
keystrokes * 10
Out[7]:
270
 

Or maybe we had to create 100 VLANs.

In [10]:
keystrokes * 100
Out[10]:
2700

Ir maybe we had to create the full 4094 VLANs available

In [11]:
keystrokes * 4094
Out[11]:
110538
 

Not sure about you, but if I don’t have to type 110,000 keystrokes, my fingers will love me at the end of the day. Not to mention the fact that it’s also repeated perfectly every single time, not a single typo in there.

 

Create the Template Object

Now that we’ve created the text_file string object, we need to transform it into a jinj2 template which will allow us to then render it. We will create a new object called vlan_template and assign an instance of the Template class using the text_file contents as the input.

In [12]:
vlan_template = Template(text_file)
 

Make the Magic Happen

We will now use the render method on the vlan_template that we created above. We have a single argument to pass into the function. In this case we are passing the vlans_list list of dictionaries we create above in to the function as the vlans variable.

In [13]:
vlan_template.render(vlans=vlans_list)
Out[13]:
'\n#vlan config\nvlan 10\n    name management\n    description management vlan\nvlan 15\n    name users\n    description users vlan\nvlan 16\n    name phones\n    description phones vlan\nvlan 20\n    name servers\n    description servers vlan\nvlan 30\n    name mobility\n    description mobility vlan\nvlan 40\n    name guest\n    description guest vlan\n'
 

Hmmm What happened there?

That doesn’t look like a configuration file does it? The output of this file is actually a python string object. In python, we need someway to represent a carriage return (enter-key) and the \n just happens to have that honour.

Instead of running the template rendering directly, we can instead capture the output into a string object which we will then pass to the print command.

In [14]:
rendered_template = vlan_template.render(vlans=vlans_list)
In [15]:
print (rendered_template)
 
#vlan config
vlan 10
    name management
    description management vlan
vlan 15
    name users
    description users vlan
vlan 16
    name phones
    description phones vlan
vlan 20
    name servers
    description servers vlan
vlan 30
    name mobility
    description mobility vlan
vlan 40
    name guest
    description guest vlan

 

Clear?

Hopefully, this has shown you a bit of how a basic jinja control structure, like a For loop, can be used to cut down on a lot of key strokes, increase the accuracy of the configurations and help to streamline the operations.

In the next post, I’ll look at loading YAML files directly into python and using their contents as input into some more advanced jinja2 templates.

 
Advertisements

FCAPS – A Quick Introduction

It occurs to me that I’ve been writing the last few posts about network management tasks based on an ITSM model and I didn’t even introduce what is probably the more arguably more useful model for breaking down and understanding network management tasks; the FCAPS model.
FCAPS has it’s roots in the ISO, similar to another model we all know and love; the OSI model. Everyone remember that one? Please Don’t Take Sales’ Peoples Advice?  You may have learned another acronym for it, but this is the probably the most basic conceptual model that every networking person uses to understand the world we live in.

For those of you who are looking for some extra credit reading, or need a cure for insomnia, you can find the actual FCAPS standards in the ITU-T M.3400 recommendations. For the rest, I’m hoping to give a brief overview to help you understand the different aspects of the disciplines of network management.

F is for Fault

This involves the detection, isolation, and correction of a fault condition. Or in plain english, this lets you know when things are broken.

Fault Management could involve things like syslog, SNMP traps been escalated to Alarms. Root-Cause-Analysis and Alarm suppression or some AI which tries to seperate the signal from the noise during event storms.  Alarm notification policies ( sending out an e-mail once you get an alarm ).

Traditionally this was implemented in a lot of NMSs as Green-is-good management. Basically, if everything is green. Things are ok. If they are yellow or red, you’ve probably got along night ahead of you.

In recent years, Fault Management has started to include application performance management as well. In modern networks, it’s not enough to know that an application is “up”. Now we must also make sure that the level of service, or SLA, that is been delivered to the end-user is adequate to meet their needs.

Note: Whether an activity falls into one category of FCAPS or another might depend on your perspective. If you are measuring bandwidth on a particular port, you may be in the “P”, but if you are measuring the bandwidth and raising an alarm if you cross a certain threshold, you’re now in the “F”.

This may seem confusing at first, but remember that FCAPS is just a conceptual model.  This is similar to the 7 Layer OSI model. Ask any good network person what layer MPLS falls at and they will either answer ” It depends” or potentially ” 2.5 “.

C is for Configuration

This involves the configuration of the software and hardware in the network. This includes the versions of software, the actual configurations, change management, etc…

This is probably the easiest to understand. If you’re upgrading code on a switch or router, if you’re logging into a router to make a configuration change, or if you’re just plugging a network cable in to a PC, you’re in the “C”s.

Accounting

This involves the identification of cost to the service provider and payment due for the customer. Ie: Billing.

Personally, I find this definition a little restrictive and prefer to apply the definition that I heard in a presentation.  I wish I could remember the name of the gentleman to give him credit. He started out in a thick southern drawl

The thing to remember about a’counting, is that the rest of the world just calls it counting.

I know. Barely funny, right?

But it does allow us to use this to include things like

  • netflow for counting the different protocols running across a certain WAN link.
  • SNMP polling of T1/PRI interfaces for ensuring that you’re Erlang calculations are accurate and you don’t need to raise or lower the number of trunks on your voice gateways.
  • RADIUS to track how long a user was logged into a specific port on the network or how much bandwidth he actually used.

You get the picture. Basically, accounting is just counting things which might be interesting to you.

Although this is not the strict definition from the ITU M.3400, this amended version makes it easier for me to apply this because I don’t have very many customer (read: any) who actually do charge-backs for their services.

Obviously, in a XaaS service, this domain is probably going to get a lot of attention in the coming years.

P is for Performance

This involves evaluating and reporting on the effectiveness of the network, and individual network devices.

Way back when I did my CCNA, one of the things I remember reading about was how you should be checking your routers and switches often to see if their CPU or memory was running high. I’ve never actually met anyone who logged into a device to check on a daily basis, but the advice was actually really good.

With a good NMS, you can

  • use SNMP polling for the CPU and Memory to track their trending over time.
  • use ICMP to track availability of the devices ( assuming it responds!)
  • use ICMP to track the latency of the device to test the quality of the link.

As I mentioned in the Fault section, performance often blurs with fault in that good performance management habits can alert you to  faults in the network. In fact, good performance management can even allow you to proactively avoid faults by identifying a potential performance block in the network, and addressing the issue before it turns into a fault.

Probably the most important thing to know about performance management is that it helps you make better decisions.

Most good network engineers can instinctively know where the bottlenecks are in their networks and can usually correctly identify what needs to be upgraded to get the most benefit.

Most great network engineers can use the pretty graphs from a good performance management tool to get the money from their CFO for those upgrades.

In my home network, I actually track the response time of all my links, as well as additional services, such as the one below which allows me to keep my wife happy.

Facebook Response Time Performance Tracking

note: probably the most recognizable performance management tool would be MRTG/PRTG. I can’t even imagine how many network upgrades were justfied by the pretty graphs that came out of these tools.

Security

Security is… well security. These are the network management activites that involve securing the network and the data running over it.

In a lot of ways, I strongly believe that security should be addressed in every waking (and sleeping!) moment that you’re thinking about your networks. Security should become so second nature to us that it should be almost impossible to perform any of the other tasks without security entering the conversation.

What do I mean?

Fault – CIA – Confidentiality, Availability, and Integrity. Hard to be secure when it’s not available and the Fault domain helps us keep it that way!

Configuration – Auditing – Good configuration management practices can involve automated IT Control objective verification tools, otherwise known as “scripts” which will allow us to have the NMS ensure all the configurations are what they should be and no unneeded services are on our routers and switches.

Performance – You can’t get performance data without SNMP, and if you’re using SNMP, PLEASE USE SNMPv3 if possible!  It can be encrypted with integrity. Also, lock down your management interfaces with ACLs on your devices.
FCAPS

It’s just a model

Please don’t take it too seriously. It’s not a binary model. Feel free to apply some fuzzy logic here and be confident that it’s 46% Fault Management and 54% Performance Management.

The important thing here is that it helps us understand the network management world we live in. It gives us a conceptual model to be able to understand the different activities involved in network management. As an added bonus, it also gives us a handy tool to evaluate different NMS software packages.

Think about the tools you’re using. Are you using a point solution, like Solarwinds Orion NPM which focuses on Performance monitoring, or an Open Source tool like RANCID which focuses on Configuration?

Or are you looking at a SPOG solution like HP’s IMC which provides full FCAPS (and more!) in the base package?

What tools are you using? Are they full FCAPS?Or are they more focused on one particular area?