The test suite is intended to run a binary and compare its standard output,
standard error and return value against provided values. This is implemented
unittest module and thus all test files are Python files.
The simplest test has the following structure:
# -*- coding: utf-8 -*- import system_tests class GoodTestName(metaclass=system_tests.CaseMeta): filename = "$data_path/test_file" commands = ["$exiv2 $filename", "$exiv2 $filename" + '_2'] stdout = [""] * 2 stderr = ["""$exiv2_exception_msg $filename: $kerFailedToReadImageData """] * 2 retval =  * 2
The test suite will run the provided commands in
commands and compare them to
the output in
stderr and it will compare the return values.
The strings after a
$ are variables either defined in this test's class or are
taken from the suite's configuration file (see
doc.md for a complete
When creating new tests, follow roughly these steps:
Choose an appropriate subdirectory where the test belongs. If none fits
create a new one and put an empty
__init__.py file there.
Create a new file with a name matching
test_*.py. Copy the class definition
from the above example and choose an appropriate class name.
Run the test suite via
python3 runner.py and ensure that your test case is
actually run! Either run the suite with the
-v option which will output all
test cases that were run or simply add an error and check if errors occur.