Documentation of NUTS

Introduction

The NetTowel Network Unit Testing System or “nuts” in short is the testing component of the NetTowel Project, which is developed at the Institute of Networked Solutions in Rapperswil, Switzerland. It draws on the concept of unit tests, known from the domain of programming, and applies it to the domain of networking.

One major difference between unit tests in programming and network tests is the definition of what a test actually is. In programming, unit tests normally focus on testing edge cases, since the amount of non-edge cases is not definable. In the network testing domain, tests are less about edge cases, but more about testing existing network states with pre-defined test cases. Such a single test case might be “can host A reach neighbors X, Y, Z?” on many different devices. This is what nuts tries to achieve: Apply test cases based on your pre-defined network topology to your actual network and have the tests confirm the correct state.

How nuts works

In order to run nuts, two components are required:

  1. Inventory of the network. Currently, these are YAML-files in the form of a nornir inventory. They contain all details of your network, such as hosts, login-information and other configuration.
  2. Test bundles in the form of YAML-files that specify the actual tests. A test bundle is a series of tests that are logically related to each other. Each test bundle is structured in a similar way, but has specific fields tailored to the test bundle. Some field values in a test bundle are directly related to your inventory.

Head over to the Usage section to see how those two components are structured and how you get nuts up and running.

The project relies on the pytest framework to setup and execute the tests. Nuts itself is written as a custom pytest plugin. In the background, nornir executes specific network tasks for the actual tests.

Pytest reads in the test bundle (step 2 above) and transforms it into test runs. In the background, nornir uses the network information provided in the inventory (step 1 above), queries the network based on the specific test bundle and passes the results of those queries to pytest. Pytest then evaluates if the expectations defined in the test bundle match those results.

Nuts will take the YAML-test bundles and generate pytest test classes from them. These test classes will run the respective Nornir tasks and get the results from them in form of a Nornir 'MultiResult' or 'AggregatedResult' respectively. NUTS will provide these test results to the pytest test functions in form of 'NutsResult' instances.

Enhance nuts

Nuts is written in python and designed as a pytest plugin. It provides some base tests described in the section about all test bundles, but since it’s a plugin, you can write your own, self-written test classes for your custom tests. A dev documentation on how to write your own test classes is planned for a future release.

We do not support self-written tests at the moment, since planned refactorings might introduce breaking changes. Still, you can already write your own test classes! The nuts/base_tests folder in the code repository conatins sample implementations and the development section has a short introduction.

Indices and tables