The test suite is intended to run a binary and compare its standard output,
standard error and return value against provided values. This is implemented
using Python's unittest
module and thus all test files are Python files.
The simplest test has the following structure:
# -*- coding: utf-8 -*- import system_tests class GoodTestName(metaclass=system_tests.CaseMeta): filename = "$data_path/test_file" commands = ["$exiv2 $filename", "$exiv2 $filename" + '_2'] stdout = [""] * 2 stderr = ["""$exiv2_exception_msg $filename: $kerFailedToReadImageData """] * 2 retval = [1] * 2
The test suite will run the provided commands in commands
and compare them to
the output in stdout
and stderr
and it will compare the return values.
The strings after a $
are variables either defined in this test's class or are
taken from the suite's configuration file (see doc.md
for a complete
explanation).
When creating new tests, follow roughly these steps:
Choose an appropriate subdirectory where the test belongs. If none fits
create a new one and put an empty __init__.py
file there.
Create a new file with a name matching test_*.py
. Copy the class definition
from the above example and choose an appropriate class name.
Run the test suite via python3 runner.py
and ensure that your test case is
actually run! Either run the suite with the -v
option which will output all
test cases that were run or simply add an error and check if errors occur.