|
Packit |
6c4009 |
README for libm-test math test suite
|
|
Packit |
6c4009 |
====================================
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The libm-test math test suite tests a number of function points of
|
|
Packit |
6c4009 |
math functions in the GNU C library. The following sections contain a
|
|
Packit |
6c4009 |
brief overview. Please note that the test drivers and the Perl script
|
|
Packit |
6c4009 |
"gen-libm-test.pl" have some options. A full list of options is
|
|
Packit |
6c4009 |
available with --help (for the test drivers) and -h for
|
|
Packit |
6c4009 |
"gen-libm-test.pl".
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
What is tested?
|
|
Packit |
6c4009 |
===============
|
|
Packit |
6c4009 |
The tests just evaluate the functions at specified points and compare
|
|
Packit |
6c4009 |
the results with precomputed values and the requirements of the ISO
|
|
Packit |
6c4009 |
C99 standard.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Besides testing the special values mandated by IEEE 754 (infinities,
|
|
Packit |
6c4009 |
NaNs and minus zero), some more or less random values are tested.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Files that are part of libm-test
|
|
Packit |
6c4009 |
================================
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The main files are "libm-test-<func>.inc". They are independent of
|
|
Packit |
6c4009 |
the target platform and the specific real floating type and format and
|
|
Packit |
6c4009 |
contain placeholder test "templates" for math functions defined in
|
|
Packit |
6c4009 |
libm. These files, along with generated files named
|
|
Packit |
6c4009 |
"auto-libm-test-out-<func>", are preprocessed by the Perl script
|
|
Packit |
6c4009 |
"gen-libm-test.pl" to expand the templates and produce a set of test
|
|
Packit |
6c4009 |
cases for each math function that are specific to the target platform
|
|
Packit |
6c4009 |
but still independent of the real floating type. The results of the
|
|
Packit |
6c4009 |
processing are "libm-test-<func>.c" and a file "libm-test-ulps.h" with
|
|
Packit |
6c4009 |
platform specific deltas by which the actual math function results may
|
|
Packit |
6c4009 |
deviate from the expected results and still be considered correct.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The test drivers "test-double-<func>.c", "test-float-<func>.c", and
|
|
Packit |
6c4009 |
"test-ldouble-<func>.c", generated by the Makefile, test the normal
|
|
Packit |
6c4009 |
double, float and long double implementation of libm. The test
|
|
Packit |
6c4009 |
drivers with an 'i' in their name ("test-idouble-<func>.c",
|
|
Packit |
6c4009 |
"test-ifloat-<func>.c", and "test-ildoubl-<func>.c") test the
|
|
Packit |
6c4009 |
corresponding inline functions (where available - otherwise they also
|
|
Packit |
6c4009 |
test the real functions in libm). Each driver selects the desired
|
|
Packit |
6c4009 |
real floating type to exercise the math functions to test with (float,
|
|
Packit |
6c4009 |
double, or long double) by defining a small set of macros just before
|
|
Packit |
6c4009 |
including the generic "libm-test.c" file. Each driver also either
|
|
Packit |
6c4009 |
defines or undefines the __NO_MATH_INLINES macro just before including
|
|
Packit |
6c4009 |
"libm-test-<func>.c" to select either the real or inline functions,
|
|
Packit |
6c4009 |
respectively. Each driver is compiled into a single executable test
|
|
Packit |
6c4009 |
program with the corresponding name.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
As mentioned above, the "gen-libm-test.pl" script looks for a file
|
|
Packit |
6c4009 |
named "libm-test-ulps" in the platform specific sysdep directory (or
|
|
Packit |
6c4009 |
its fpu or nofpu subdirectory) and for each variant (real floating
|
|
Packit |
6c4009 |
type and rounding mode) of every tested function reads from it the
|
|
Packit |
6c4009 |
maximum difference expressed as Units of Least Precision (ULP) the
|
|
Packit |
6c4009 |
actual result of the function may deviate from the expected result
|
|
Packit |
6c4009 |
before it's considered incorrect.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The "auto-libm-test-out-<func>" files contain sets of test cases to
|
|
Packit |
6c4009 |
exercise, the conditions under which to exercise each, and the
|
|
Packit |
6c4009 |
expected results. The files are generated by the
|
|
Packit |
6c4009 |
"gen-auto-libm-tests" program from the "auto-libm-test-in" file. See
|
|
Packit |
6c4009 |
the comments in gen-auto-libm-tests.c for details about the content
|
|
Packit |
6c4009 |
and format of the -in and -out files.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
How can I generate "libm-test-ulps"?
|
|
Packit |
6c4009 |
====================================
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
To automatically generate a new "libm-test-ulps" run "make regen-ulps".
|
|
Packit |
6c4009 |
This generates the file "math/NewUlps" in the build directory. The file
|
|
Packit |
6c4009 |
contains the sorted results of all the tests. You can use the "NewUlps"
|
|
Packit |
6c4009 |
file as the machine's updated "libm-test-ulps" file. Copy "NewUlps" to
|
|
Packit |
6c4009 |
"libm-test-ulps" in the appropriate machine sysdep directory. Verify
|
|
Packit |
6c4009 |
the changes, post your patch, and check it in after review.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
To manually generate a new "libm-test-ulps" file, first remove "ULPs"
|
|
Packit |
6c4009 |
file in the current directory, then you can execute for example:
|
|
Packit |
6c4009 |
./testrun.sh math/test-double -u --ignore-max-ulp=yes
|
|
Packit |
6c4009 |
This generates a file "ULPs" with all double ULPs in it, ignoring any
|
|
Packit |
6c4009 |
previously calculated ULPs, and running with the newly built dynamic
|
|
Packit |
6c4009 |
loader and math library (assumes you didn't install your build). Now
|
|
Packit |
6c4009 |
generate the ULPs for all other formats, the tests will be appending the
|
|
Packit |
6c4009 |
data to the "ULPs" file. As final step run "gen-libm-test.pl" with the
|
|
Packit |
6c4009 |
file as input and ask to generate a pretty printed output in the file
|
|
Packit |
6c4009 |
"NewUlps":
|
|
Packit |
6c4009 |
gen-libm-test.pl -u ULPs -n NewUlps
|
|
Packit |
6c4009 |
Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep
|
|
Packit |
6c4009 |
directory.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Note that the test drivers have an option "-u" to output an unsorted
|
|
Packit |
6c4009 |
list of all epsilons that the functions have. The output can be read
|
|
Packit |
6c4009 |
in directly but it's better to pretty print it first.
|
|
Packit |
6c4009 |
"gen-libm-test.pl" has an option to generate a pretty-printed and
|
|
Packit |
6c4009 |
sorted new ULPs file from the output of the test drivers.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Contents of libm-test-ulps
|
|
Packit |
6c4009 |
==========================
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Since libm-test-ulps can be generated automatically, just a few notes.
|
|
Packit |
6c4009 |
The file contains lines for maximal errors of single functions, like:
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Function "yn":
|
|
Packit |
6c4009 |
idouble: 6
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The keywords are float, ifloat, double, idouble, ldouble and ildouble
|
|
Packit |
6c4009 |
(the prefix i stands for inline).
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Adding tests to libm-test-<func>.inc
|
|
Packit |
6c4009 |
====================================
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The tests are evaluated by a set of special test macros. The macros
|
|
Packit |
6c4009 |
start with "TEST_" followed by a specification the input values, an
|
|
Packit |
6c4009 |
underscore and a specification of the output values. As an example,
|
|
Packit |
6c4009 |
the test macro for a function with input of type FLOAT (FLOAT is
|
|
Packit |
6c4009 |
either float, double, long double) and output of type FLOAT is
|
|
Packit |
6c4009 |
"TEST_f_f". The macro's parameter are the name of the function, the
|
|
Packit |
6c4009 |
input parameter, output parameter and optionally one exception
|
|
Packit |
6c4009 |
parameter.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
The accepted parameter types are:
|
|
Packit |
6c4009 |
- "f" for FLOAT
|
|
Packit |
6c4009 |
- "j" for long double.
|
|
Packit |
6c4009 |
- "a" for ARG_FLOAT, the argument type for narrowing functions.
|
|
Packit |
6c4009 |
- "b" for boolean - just tests if the output parameter evaluates to 0
|
|
Packit |
6c4009 |
or 1 (only for output).
|
|
Packit |
6c4009 |
- "c" for complex. This parameter needs two values, first the real,
|
|
Packit |
6c4009 |
then the imaginary part.
|
|
Packit |
6c4009 |
- "i" for int.
|
|
Packit |
6c4009 |
- "l" for long int.
|
|
Packit |
6c4009 |
- "L" for long long int.
|
|
Packit |
6c4009 |
- "u" for unsigned int.
|
|
Packit |
6c4009 |
- "M" for intmax_t.
|
|
Packit |
6c4009 |
- "U" for uintmax_t.
|
|
Packit |
6c4009 |
- "p" for an argument (described in the previous character) passed
|
|
Packit |
6c4009 |
through a pointer rather than directly.
|
|
Packit |
6c4009 |
- "F" for the address of a FLOAT (only as input parameter)
|
|
Packit |
6c4009 |
- "I" for the address of an int (only as input parameter)
|
|
Packit |
6c4009 |
- "1" for an additional output (either output through a pointer passed
|
|
Packit |
6c4009 |
as an argument, or to a global variable such as signgam).
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
How to read the test output
|
|
Packit |
6c4009 |
===========================
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
Running each test on its own at the default level of verbosity will
|
|
Packit |
6c4009 |
print on stdout a line describing the implementation of math functions
|
|
Packit |
6c4009 |
exercised by the test (float, double, or long double), along with
|
|
Packit |
6c4009 |
whether the inline set has been selected, regardless of whether or
|
|
Packit |
6c4009 |
not any inline functions actually exist. This is then followed by
|
|
Packit |
6c4009 |
the details of test failures (if any). The output concludes by
|
|
Packit |
6c4009 |
a summary listing the number of test cases exercised and the number
|
|
Packit |
6c4009 |
of test failures uncovered.
|
|
Packit |
6c4009 |
|
|
Packit |
6c4009 |
For each test failure (and for each test case at higher levels of
|
|
Packit |
6c4009 |
verbosity), the output contains the name of the function under test
|
|
Packit |
6c4009 |
and its arguments or conditions that triggered the failure. Note
|
|
Packit |
6c4009 |
that the name of the function in the output need not correspond
|
|
Packit |
6c4009 |
exactly to the name of the math function actually invoked. For example,
|
|
Packit |
6c4009 |
the output will refer to the "acos" function even if the actual function
|
|
Packit |
6c4009 |
under test is acosf (for the float version) or acosl (for the long
|
|
Packit |
6c4009 |
double version). Also note that the function arguments may be shown
|
|
Packit |
6c4009 |
in either the decimal or the hexadecimal floating point format which
|
|
Packit |
6c4009 |
may or may not correspond to the format used in the auto-libm-test-in
|
|
Packit |
6c4009 |
file. Besides the name of the function, for each test failure the
|
|
Packit |
6c4009 |
output contains the actual and expected results and the difference
|
|
Packit |
6c4009 |
between the two, printed in both the decimal and hexadecimal
|
|
Packit |
6c4009 |
floating point format, and the ULP and maximum ULP for the test
|
|
Packit |
6c4009 |
case.
|