How To Write Your Own Test¶
Nuts is written as a Pytest plugin and allow you to write your own tests, apart from the existing ones in nuts/base_tests
. The short introduction here shows you the required ingredients to cook up your own tests.
Three classes are needed for a test - they define the context in which a test is run, process raw data and then run the tests:
- Context class: Holds all necessary information that is needed for a specific test. Yields the raw, unprocessed result. Base class:
NutsContext
- Extractor class: Processes the raw overall results from a network query (very often a nornir task), so that it can be passed on to the test class. Base class:
AbstractResultExtractor
- Test class: Contains the actual test methods. They test data from the YAML file against the retrieved and processed data from the network.
A Simple Test¶
The project features a showcase that displays the most basic test possible, one that also works without a network to gather data. You find the module that contains the code in tests/showcase
and the test bundle (the YAML file) in tests/showcase_test
.
Writing the Test itself¶
The showcase test implements the three above-mentioned classes and the necessary methods:
ExpanseContext
derives fromNutsContext
.class ExpanseExtractor
derives fromAbstractResultExtractor
.class TestExpanseCrew
is the actual test class.
The context to be used by nuts is defined by setting CONTEXT = ExpanseContext
.
Here’s a code snippet from class TestExpanseCrew
to illustrate how the test class is written:
class TestExpanseCrew:
# ...
@pytest.mark.nuts("name, role")
def test_role(self, single_result: NutsResult, name: str, role: str) -> None:
assert single_result.result[name]["role"] == role
@pytest.mark.nuts("name, role")
is a custom pytest marker: The arguments name, role
refer to the fields as they are defined in the YAML file under test_data
. They are then passed on as a pytest fixture to the test itself.
Writing the YAML-File¶
The sample YAML file that uses the test has the following features:
test_module
directly refers to your module and its location.test_class
is the name of the test class you have defined in step 3 in the previous section.test_data
contains the data against which the tests should be run. Be aware that the fields undertest_data
and those used in the test class itself must match, i.e. the arguments in the custom pytest marker.
Fields in test_data
are optional except for the key into the results (ship
in our case here, usually host
). If you leave out other fields than the key, the test that requires that particular entry will be skipped. The following snipped comments out origin
, which therefore skips test_origin()
for Bobbie Draper.
- ship: rocinante
name: "bobbie draper"
role: marine
# origin: mars
A Test Using Nornir¶
To illustrate how this test works, the BGP test module (nuts/base_tests/napalm_bgp_neighbors.py
) shall serve as an example. The test module implements these three classes:
BgpNeighborsContext
derives fromNornirNutsContext
.BgpNeighborsExtractor
derives fromAbstractHostResultExtractor
.class TestNapalmBgpNeighborsCount
is the actual test class.
The context to be used is defined by setting CONTEXT = BgpNeighborsContext
. This time, nuts uses nornir in the background to connect to a network. Therefore, the new class does not directly derive from NutsContext
, but rather from NornirNutsContext
. This requires the setup of a nornir inventory too - please see First Steps with NUTS for more information.
There is an important distinction on what kind of extractor you want to derive from. Basically, there are two broad type of tests in nuts: (1) test properties of a host and (2) test connection properties between two hosts. Examples of (1) are nearly all test bundles in nuts including the BGP test. Examples of (2) are the ping and iperf tests. These two types determine from which extractor you want to derive:
- Test host properties: derive from
AbstractHostResultExtractor
. The required field in alltest_data
entries ishost
.- Test connection properties: derive from
AbstractHostDestResultExtractor
. The required fields in alltest_data
entries arehost
anddestination
.
Both extractors wrap the processed results in a new NutsResult
: This must be done so that when one nornir task fails (or something else breaks during data gathering), only that task is turned into a failed pytest test, and not the whole test run. The NutsResult
is then passed on to the test class.
The test class is written very similarly to the simple case above: Set the pytest custom marker with the required arguments, use them as fixture and write the test.
Class Diagram of How Tests are Embedded into NUTS¶
The following diagram provides a detailed overview of the class structure of NUTS relative to test class implementation.