<page xmlns="http://projectmallard.org/1.0/" xmlns:its="http://www.w3.org/2005/11/its" type="topic" id="unit-testing" xml:lang="de">

  <info>

    <link type="guide" xref="index#general-guidelines"/>

    <credit type="author copyright">

      <name>Philip Withnall</name>

      <email its:translate="no">philip.withnall@collabora.co.uk</email>

      <years>2015</years>

    </credit>

    <include xmlns="http://www.w3.org/2001/XInclude" href="cc-by-sa-3-0.xml"/>

    <desc>Designing software to be tested and writing unit tests for it</desc>

    <mal:credit xmlns:mal="http://projectmallard.org/1.0/" type="translator copyright">

      <mal:name>Mario Blättermann</mal:name>

      <mal:email>mario.blaettermann@gmail.com</mal:email>

      <mal:years>2016</mal:years>

    </mal:credit>

    <mal:credit xmlns:mal="http://projectmallard.org/1.0/" type="translator copyright">

      <mal:name>Christian Kirbach</mal:name>

      <mal:email>christian.kirbach@gmail.com</mal:email>

      <mal:years>2016</mal:years>

    </mal:credit>

  </info>

  <title>Unit Testing</title>

  <synopsis>

    <title>Zusammenfassung</title>

      Unit testing should be the primary method of testing the bulk of code

      written, because a unit test can be written once and run many times —

      manual tests have to be planned once and then manually run each time.

      Development of unit tests starts with the architecture and API design of

      the code to be tested: code should be designed to be easily testable, or

      will potentially be very difficult to test.

    <list>

      <item>

        Write unit tests to be as small as possible, but no smaller.

        (<link xref="#writing-unit-tests"/>)

      
</item>

      <item>

        Use code coverage tools to write tests to get high code coverage.

        (<link xref="#writing-unit-tests"/>)

      
</item>

      <item>

        Run all unit tests under Valgrind to check for leaks and other problems.

        (<link xref="#leak-checking"/>)

      
</item>

      <item>

        Use appropriate tools to automatically generate unit tests where

        possible. (<link xref="#test-generation"/>)

      
</item>

      <item>

        Design code to be testable from the beginning.

        (<link xref="#writing-testable-code"/>)

      
</item>

    </list>

  </synopsis>

  <section id="writing-unit-tests">

    <title>Writing Unit Tests</title>

      Unit tests should be written in conjunction with looking at

      <link xref="tooling#gcov-and-lcov">code coverage information gained from

      running the tests</link>. This typically means writing an initial set of

      unit tests, running them to get coverage data, then reworking and

      expanding them to increase the code coverage levels. Coverage should be

      increased first by ensuring all functions are covered (at least in part),

      and then by ensuring all lines of code are covered. By covering functions

      first, API problems which will prevent effective testing can be found

      quickly. These typically manifest as internal functions which cannot

      easily be called from unit tests. Overall, coverage levels of over 90%

      should be aimed for; don’t just test cases covered by project

      requirements, test everything.

      Like <link xref="version-control">git commits</link>, each unit test

      should be ‘as small as possible, but no smaller’, testing a single

      specific API or behavior. Each test case must be able to be run

      individually, without depending on state from other test cases. This is

      important to allow debugging of a single failing test, without having to

      step through all the other test code as well. It also means that a single

      test failure can easily be traced back to a specific API, rather than a

      generic ‘unit tests failed somewhere’ message.

      GLib has support for unit testing with its

      <link href="https://developer.gnome.org/glib/stable/glib-Testing.html">GTest

      framework</link>, allowing tests to be arranged in groups and hierarchies.

      This means that groups of related tests can be run together for enhanced

      debugging too, by running the test binary with the <cmd>-p</cmd> argument:

      <cmd>./test-suite-name -p /path/to/test/group</cmd>.

  </section>

  <section id="installed-tests">

    <title>Installierte Tests</title>

      All unit tests should be installed system-wide, following the

      <link href="https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests">installed-tests

      standard</link>.

      By installing the unit tests, continuous integration (CI) is made easier,

      since tests for one project can be re-run after changes to other projects

      in the CI environment, thus testing the interfaces between modules. That

      is useful for a highly-coupled set of projects like GNOME.

      To add support for installed-tests, add the following to

      <file>configure.ac</file>:

    # Installed tests

AC_ARG_ENABLE([modular_tests],

              AS_HELP_STRING([--disable-modular-tests],

                             [Disable build of test programs (default: no)]),,

              [enable_modular_tests=yes])

AC_ARG_ENABLE([installed_tests],

              AS_HELP_STRING([--enable-installed-tests],

                             [Install test programs (default: no)]),,

              [enable_installed_tests=no])

AM_CONDITIONAL([BUILD_MODULAR_TESTS],

               [test "$enable_modular_tests" = "yes" ||

                test "$enable_installed_tests" = "yes"])

AM_CONDITIONAL([BUILDOPT_INSTALL_TESTS],[test "$enable_installed_tests" = "yes"])

      Then in <file>tests/Makefile.am</file>:

    insttestdir = $(libexecdir)/installed-tests/[project]

all_test_programs = \

	test-program1 \

	test-program2 \

	test-program3 \

	$(NULL)

if BUILD_MODULAR_TESTS

TESTS = $(all_test_programs)

noinst_PROGRAMS = $(TESTS)

endif

if BUILDOPT_INSTALL_TESTS

insttest_PROGRAMS = $(all_test_programs)

testmetadir = $(datadir)/installed-tests/[project]

testmeta_DATA = $(all_test_programs:=.test)

testdatadir = $(insttestdir)

testdata_DATA = $(test_files)

testdata_SCRIPTS = $(test_script_files)

endif

EXTRA_DIST = $(test_files)

%.test: % Makefile

	$(AM_V_GEN) (echo '[Test]' > $@.tmp; \

	echo 'Type=session' >> $@.tmp; \

	echo 'Exec=$(insttestdir)/$<' >> $@.tmp; \

	mv $@.tmp $@)

  </section>

  <section id="leak-checking">

    <title>Leak Checking</title>

      Once unit tests with high code coverage have been written, they can be run

      under various dynamic analysis tools, such as

      <link xref="tooling#valgrind">Valgrind</link> to check for leaks,

      threading errors, allocation problems, etc. across the entire code base.

      The higher the code coverage of the unit tests, the more confidence the

      Valgrind results can be treated with. See <link xref="tooling"/> for more

      information, including build system integration instructions.

      Critically, this means that unit tests should not leak memory or other

      resources themselves, and similarly should not have any threading

      problems. Any such problems would effectively be false positives in the

      analysis of the actual project code. (False positives which need to be

      fixed by fixing the unit tests.)

  </section>

  <section id="test-generation">

    <title>Test Generation</title>

      Certain types of code are quite repetitive, and require a lot of unit

      tests to gain good coverage; but are appropriate for

      <link href="http://en.wikipedia.org/wiki/Test_data_generation">test data

      generation</link>, where a tool is used to automatically generate test

      vectors for the code. This can drastically reduce the time needed for

      writing unit tests, for code in these specific domains.

    <section id="json">

      <title>JSON</title>

        One example of a domain amenable to test data generation is parsing,

        where the data to be parsed is required to follow a strict schema — this

        is the case for XML and JSON documents. For JSON, a tool such as

        <link href="http://people.collabora.com/~pwith/walbottle/">Walbottle</link>

        can be used to generate test vectors for all types of valid and invalid

        input according to the schema.

        Every type of JSON document should have a

        <link href="http://json-schema.org/">JSON Schema</link> defined for it,

        which can then be passed to Walbottle to generate test vectors:

json-schema-generate --valid-only schema.json

json-schema-generate --invalid-only schema.json

        These test vectors can then be passed to the code under test in its unit

        tests. The JSON instances generated by <cmd>--valid-only</cmd> should be

        accepted; those from <cmd>--invalid-only</cmd> should be rejected.

    </section>

  </section>

  <section id="writing-testable-code">

    <title>Writing Testable Code</title>

      Code should be written with testability in mind from the design stage, as

      it affects API design and architecture in fundamental ways. A few key

      principles:

    <list>

      <item>

        Do not use global state. Singleton objects are usually a bad idea as

        they can’t be instantiated separately or controlled in the unit tests.

      
</item>

      <item>

        Separate out use of external state, such as databases, networking, or

        the file system. The unit tests can then replace the accesses to

        external state with mocked objects. A common approach to this is to use

        dependency injection to pass a file system wrapper object to the code

        under test. For example, a class should not load a global database (from

        a fixed location in the file system) because the unit tests would then

        potentially overwrite the running system’s copy of the database, and

        could never be executed in parallel. They should be passed an object

        which provides an interface to the database: in a production system,

        this would be a thin wrapper around the database API; for testing, it

        would be a mock object which checks the requests given to it and returns

        hard-coded responses for various tests.

      
</item>

      <item>

        Expose utility functions where they might be generally useful.

      
</item>

      <item>

        Split projects up into collections of small, private libraries which are

        then linked together with a minimal amount of glue code into the overall

        executable. Each can be tested separately.

      
</item>

    </list>

  </section>

  <section id="external-links">

    <title>Externe Links</title>

      The topic of software testability is covered in the following articles:

    <list>

      <item>
<link href="http://msdn.microsoft.com/en-us/magazine/dd263069.aspx">Design for testability</link>
</item>

      <item>
<link href="http://en.wikipedia.org/wiki/Software_testability">Software testability</link>
</item>

      <item>
<link href="http://en.wikipedia.org/wiki/Dependency_injection">Dependency injection</link>
</item>

      <item>
<link href="http://c2.com/cgi/wiki?SoftwareDesignForTesting">Software design for testing</link>
</item>

    </list>

  </section>

</page>