Blame docs/limitations.md

rpm-build a7f80b
rpm-build a7f80b
# Known limitations
rpm-build a7f80b
rpm-build a7f80b
Catch has some known limitations, that we are not planning to change. Some of these are caused by our desire to support C++98 compilers, some of these are caused by our desire to keep Catch crossplatform, some exist because their priority is seen as low compared to the development effort they would need and some other yet are compiler/runtime bugs.
rpm-build a7f80b
rpm-build a7f80b
## Implementation limits
rpm-build a7f80b
### Sections nested in loops
rpm-build a7f80b
rpm-build a7f80b
If you are using `SECTION`s inside loops, you have to create them with different name per loop's iteration. The recommended way to do so is to incorporate the loop's counter into section's name, like so
rpm-build a7f80b
```cpp
rpm-build a7f80b
TEST_CASE( "Looped section" ) {
rpm-build a7f80b
    for (char i = '0'; i < '5'; ++i) {
rpm-build a7f80b
        SECTION(std::string("Looped section ") + i) {
rpm-build a7f80b
            SUCCEED( "Everything is OK" );
rpm-build a7f80b
        }
rpm-build a7f80b
    }
rpm-build a7f80b
}
rpm-build a7f80b
```
rpm-build a7f80b
rpm-build a7f80b
## Features
rpm-build a7f80b
This section outlines some missing features, what is their status and their possible workarounds.
rpm-build a7f80b
rpm-build a7f80b
### Thread safe assertions
rpm-build a7f80b
Because threading support in standard C++98 is limited (well, non-existent), assertion macros in Catch are not thread safe. This does not mean that you cannot use threads inside Catch's test, but that only single thread can interact with Catch's assertions and other macros.
rpm-build a7f80b
rpm-build a7f80b
This means that this is ok
rpm-build a7f80b
```cpp
rpm-build a7f80b
    std::vector<std::thread> threads;
rpm-build a7f80b
    std::atomic<int> cnt{ 0 };
rpm-build a7f80b
    for (int i = 0; i < 4; ++i) {
rpm-build a7f80b
        threads.emplace_back([&]() {
rpm-build a7f80b
            ++cnt; ++cnt; ++cnt; ++cnt;
rpm-build a7f80b
        });
rpm-build a7f80b
    }
rpm-build a7f80b
    for (auto& t : threads) { t.join(); }
rpm-build a7f80b
    REQUIRE(cnt == 16);
rpm-build a7f80b
```
rpm-build a7f80b
because only one thread passes the `REQUIRE` macro and this is not
rpm-build a7f80b
```cpp
rpm-build a7f80b
    std::vector<std::thread> threads;
rpm-build a7f80b
    std::atomic<int> cnt{ 0 };
rpm-build a7f80b
    for (int i = 0; i < 4; ++i) {
rpm-build a7f80b
        threads.emplace_back([&]() {
rpm-build a7f80b
            ++cnt; ++cnt; ++cnt; ++cnt;
rpm-build a7f80b
            CHECK(cnt == 16);
rpm-build a7f80b
        });
rpm-build a7f80b
    }
rpm-build a7f80b
    for (auto& t : threads) { t.join(); }
rpm-build a7f80b
    REQUIRE(cnt == 16);
rpm-build a7f80b
```
rpm-build a7f80b
rpm-build a7f80b
rpm-build a7f80b
_This limitation is highly unlikely to be lifted before Catch 2 is released._
rpm-build a7f80b
rpm-build a7f80b
### Process isolation in a test
rpm-build a7f80b
Catch does not support running tests in isolated (forked) processes. While this might in the future, the fact that Windows does not support forking and only allows full-on process creation and the desire to keep code as similar as possible across platforms, mean that this is likely to take significant development time, that is not currently available.
rpm-build a7f80b
rpm-build a7f80b
### Running multiple tests in parallel
rpm-build a7f80b
Catch's test execution is strictly serial. If you find yourself with a test suite that takes too long to run and you want to make it parallel, there are 2 feasible solutions
rpm-build a7f80b
 * You can split your tests into multiple binaries and then run these binaries in parallel.
rpm-build a7f80b
 * You can have Catch list contained test cases and then run the same test binary multiple times in parallel, passing each instance list of test cases it should run.
rpm-build a7f80b
rpm-build a7f80b
Both of these solutions have their problems, but should let you wring parallelism out of your test suite.
rpm-build a7f80b
rpm-build a7f80b
## 3rd party bugs
rpm-build a7f80b
This section outlines known bugs in 3rd party components (this means compilers, standard libraries, standard runtimes).
rpm-build a7f80b
rpm-build a7f80b
### Visual Studio 2017 -- raw string literal in assert fails to compile
rpm-build a7f80b
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation error when preprocessor attempts to stringize a raw string literal (`#` preprocessor is applied to it). This snippet is sufficient to trigger the compilation error:
rpm-build a7f80b
```cpp
rpm-build a7f80b
#define CATCH_CONFIG_MAIN
rpm-build a7f80b
#include "catch.hpp"
rpm-build a7f80b
rpm-build a7f80b
TEST_CASE("test") {
rpm-build a7f80b
    CHECK(std::string(R"("\)") == "\"\\");
rpm-build a7f80b
}
rpm-build a7f80b
```
rpm-build a7f80b
rpm-build a7f80b
Catch provides a workaround, it is possible to disable stringification of original expressions by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`:
rpm-build a7f80b
```cpp
rpm-build a7f80b
#define CATCH_CONFIG_FAST_COMPILE
rpm-build a7f80b
#define CATCH_CONFIG_DISABLE_STRINGIFICATION
rpm-build a7f80b
#include "catch.hpp"
rpm-build a7f80b
rpm-build a7f80b
TEST_CASE("test") {
rpm-build a7f80b
    CHECK(std::string(R"("\)") == "\"\\");
rpm-build a7f80b
}
rpm-build a7f80b
```
rpm-build a7f80b
rpm-build a7f80b
_Do note that this changes the output somewhat_
rpm-build a7f80b
```
rpm-build a7f80b
catchwork\test1.cpp(6):
rpm-build a7f80b
PASSED:
rpm-build a7f80b
  CHECK( Disabled by CATCH_CONFIG_DISABLE_STRINGIFICATION )
rpm-build a7f80b
with expansion:
rpm-build a7f80b
  ""\" == ""\"
rpm-build a7f80b
```
rpm-build a7f80b
rpm-build a7f80b
### Visual Studio 2015 -- Alignment compilation error (C2718)
rpm-build a7f80b
rpm-build a7f80b
VS 2015 has a known bug, where `declval<T>` can cause compilation error
rpm-build a7f80b
if `T` has alignment requirements that it cannot meet.
rpm-build a7f80b
rpm-build a7f80b
rpm-build a7f80b
A workaround is to explicitly specialize `Catch::is_range` for given
rpm-build a7f80b
type (this avoids code path that uses `declval<T>` in a SFINAE context).
rpm-build a7f80b
rpm-build a7f80b
rpm-build a7f80b
### Visual Studio 2015 -- Wrong line number reported in debug mode
rpm-build a7f80b
VS 2015 has a known bug where `__LINE__` macro can be improperly expanded under certain circumstances, while compiling multi-file project in Debug mode.
rpm-build a7f80b
rpm-build a7f80b
A workaround is to compile the binary in Release mode.
rpm-build a7f80b
rpm-build a7f80b
### Clang/G++ -- skipping leaf sections after an exception
rpm-build a7f80b
Some versions of `libc++` and `libstdc++` (or their runtimes) have a bug with `std::uncaught_exception()` getting stuck returning `true` after rethrow, even if there are no active exceptions. One such case is this snippet, which skipped the sections "a" and "b", when compiled against `libcxxrt` from master
rpm-build a7f80b
```cpp
rpm-build a7f80b
#define CATCH_CONFIG_MAIN
rpm-build a7f80b
#include <catch.hpp>
rpm-build a7f80b
rpm-build a7f80b
TEST_CASE("a") {
rpm-build a7f80b
    CHECK_THROWS(throw 3);
rpm-build a7f80b
}
rpm-build a7f80b
rpm-build a7f80b
TEST_CASE("b") {
rpm-build a7f80b
    int i = 0;
rpm-build a7f80b
    SECTION("a") { i = 1; }
rpm-build a7f80b
    SECTION("b") { i = 2; }
rpm-build a7f80b
    CHECK(i > 0);
rpm-build a7f80b
}
rpm-build a7f80b
```
rpm-build a7f80b
rpm-build a7f80b
If you are seeing a problem like this, i.e. a weird test paths that trigger only under Clang with `libc++`, or only under very specific version of `libstdc++`, it is very likely you are seeing this. The only known workaround is to use a fixed version of your standard library.
rpm-build a7f80b
rpm-build a7f80b
### Clang/G++ -- `Matches` string matcher always returns false
rpm-build a7f80b
This is a bug in `libstdc++-4.8`, where all matching methods from `<regex>` return false. Since `Matches` uses `<regex>` internally, if the underlying implementation does not work, it doesn't work either.
rpm-build a7f80b
rpm-build a7f80b
Workaround: Use newer version of `libstdc++`.