First Steps with NUTS

This tutorial guides you through a minimal setup of the NetTowel Unit Testing System (NUTS, or nuts). A showcase is included in nuts’ code that allows you to learn the basic mechanics of nuts without the need for an actual network – find those details at the end of this tutorial.

Two major components are needed for nuts:

  1. A network inventory

  2. Test bundles

Here’s an overview of how to organize your files so that NUTS can find everything it needs. The root folder also contains the virtual environment with the nuts installation. This example uses the Nornir SimpleInventory, but all inventory plugins are supported.

.                       # NUTS root
├── inventory           # your inventory
│   └── hosts.yaml      # information on hosts
├── nr-config.yaml      # nornir configuration
└── tests               # your test bundles
    └── test-definition-ping.yaml

NOTE: If you already have an existing nornir configuration including an inventory you do not need to copy it into your NUTS test setup. Instead, you can directly refer to it when executing the tests.

1. Network Inventory

You must provide information on your network configuration so that nuts can interact with it. Currently, the implemented test cases in NUTS use nornir and a config file nr-config.yaml (default) which provides the inventory.

A sample hosts.yaml (Nornir SampleInventory) might look like this:

R1:
  hostname: 10.0.0.42
  platform: ios
  username: Cisco
  password: cisco
  interfaces:
    - name: GigabitEthernet2
      ipv4_address: 172.16.0.42
    - name: Loopback0
      ipv4_address: 172.16.255.1
R2:
  hostname: 10.20.0.24
  platform: ios
  username: Cisco
  password: cisco
  interfaces:
    - name: GigabitEthernet2
      ipv4_address: 172.16.0.24
    - name: Loopback0
      ipv4_address: 172.16.255.2

We provide the most basic example here with sample IP addresses and credentials. Please see nornir’s documentation on how to structure the inventory according to your needs.

A sample nr-config.yaml might look like this:

inventory:
  plugin: SimpleInventory
  options:
    host_file: "inventory/hosts.yaml"
runner:
  plugin: threaded
  options:
    num_workers: 100

If you set up the above folders and files, you are ready to write test bundles.

2. Test Bundle

A test bundle is a collection of tests that are logically related to each other, for example, tests that all revolve around “information on BGP neighbors”. The test bundle describes which test definition should be collected and executed and provides data for those tests. The bundles are written as individual entries in a YAML file.

Currently, only YAML files are supported as test bundle format, but other data sources could be integrated with later versions of nuts.

Structure of a Test Bundle

Each test bundle contains the following structure:

- test_module: <module that contains the test class> # optional
  test_class: <name of the test class>
  label: <label to uniquely identify the test> # optional
  test_execution: <additional data used to execute the test> # optional
  test_data: <data used to generate the test instances>

test_module: Optional. The full path of the Python module that contains the test class to be used. This value is optional if the test class is registered in index.py of the pytest-nuts plugin. Note that it can be relevant in which directory pytest is started if local test modules are used. Using test_modules allows you to write your own test classes. Note: We currently do not support self-written test modules, since upcoming refactorings might introduce breaking changes.

test_class: Required. The name of the Python class which contains the tests that should be executed. Note that currently every test in this class is executed.

label: Optional. An additional identifier that can be used to distinguish between multiple occurrences of the same test class in a test bundle.

test_execution: Optional. Nuts uses nornir tasks to automatically interact with the network. This field contains additional information that is directly passed to the nornir task in the background. Therefore the key-value pairs must be consistent with the key-value pairs of the specific nornir task. As an example, the test definition TestNapalmPing calls a nornir task to execute napalm’s ping-command. This allows the additional count parameter in test execution, since it is in turn pre-defined by napalm. Please see the chapter on test bundles for more detailed explanations.

test_data: Required. Data that is used to parametrize the tests - basically what information each test instance needs. The structure of this section is specific to every test bundle, detailed in the chapter on test bundles.

Since each test bundle looks a little different, please see the chapter on test bundles to see how each one is structured.

Sample Test-Bundle: Ping

As an example, we now want to test if R1 can ping R2. Here’s our sample test bundle:

- test_class: TestNapalmPing
  test_execution:
    count: 5
  test_data:
    - host: R1
      destination: 172.16.0.24
      expected: SUCCESS
      max_drop: 1

Notes:

  • test_execution: By using the pre-defined key-value pair count: 5, we indicate that the ping should be executed 5 times.

  • test_data.expected: SUCCESS. The ping should be successful. The pre-defined values are either SUCCESS, FAIL, or FLAPPING.

  • test_data.max_drop: 1. Maximum one ping attempt is allowed to fail to still count as SUCCESS ping.

We save this file as test-definition-ping.yaml into the tests folder.

3. Running the Tests

If everything is set up as shown above, run the test from the root folder:

$ pytest tests/test-definition-ping.yaml

Pytest’s output should then inform you if the test succeeded or not.

Custom nornir configuration

If you already have a nornir configuration and inventory for your network you can reuse it by passing the parameter --nornir-config to the pytest command:

$ pytest tests/test-definition-ping.yaml --nornir-config path/to/nr-config.yaml

4. Sample Test-Bundle Without a Network

The sample test bundle above requires a network inventory and a running network in the background. In case you want to learn how nuts works but do not have a network at hand, nuts comes with an offline showcase to display its functionality. Use it as follows:

  1. Clone the nuts repository and change into the cloned folder.

  2. Create a virtual environment (venv) in it and activate it.

  3. Install nuts in the venv.

  4. Run the showcase test bundle.

$ git clone https://github.com/INSRapperswil/nuts && cd nuts
$ python -m venv .venv && source .venv/bin/activate
$ pip install .
$ pytest tests/showcase_test/test-expanse.yaml

How it works: Each test module implements a context class to provide module-specific functionality to its tests. This context class is a NutsContext or a subclass of it. This guarantees a consistent interface across all tests for test setup and execution.

The predefined test classes which depend on a network all use nornir in order to communicate with the network devices. Those test classes all derive from a more specific NornirNutsContext, which provides a nornir instance and nornir-specific helpers.

In order for the offline showcase to work, the test class derives from NutsContext and implements its own context class. See the code in nuts/tests/showcase/showcase_expanse.py to see the structure of this offline context class.