Tests

[Leon0402]

I think it would be nice if the project would allow one to choose between how it is now (without any framework) or use directly some framework.

I would suggest supporting Catch2 and GTest. Optionally also GMock and trompeloeil for mocking.
These have also special functions to use them in cmake, which is one of the reasons I would like them to see in such a template.

Another consideration would be the folder structure for tests. I've never seen a "source" or "src" folder in tests. What I often do is to add folders for the test types. Common is "unit_tests" and "integration_tests", but I guess that is far from a standard as well.
What would be considerable is an include folder I could imagine. I often use this for some "resuable" stuff like mock definitions. Again not sure how much standard that is.

[friendlyanon]

I prefer not to use or suggest using Google products as they are always a liability. I have a fork of utest.h, which also supports handling shared libraries on Windows. I have not seen any other library implement something like this, so if anything, the library in use for a cmake-init template would be that fork.

There is a source directory in test, because it's good to keep things separate and not spilled into a single directory without structure. For example, you can see here that the source code for the tests and the assets they use at runtime are in separate directories. The working directory for the test is also set to that dir, so it's trivial to pick an asset by filename alone.

[friendlyanon]

GTest is missing the mechanism to trivially specify dependencies to add to the PATH on Windows as well. If I included GTest, then my response to people using dependencies on Windows would have to be "figure it out lol". Right now, the windows_set_path provides such a functionality and for a testing library, my utest.h fork does too as part of the test discovery procedure.

If Catch2's CMake build scripts were in an impeccable state and their test discovery provided the above mentioned feature, then I would have absolutely no qualms w.r.t. including it.

[Leon0402]

Hmm one option I could think of is copying dlls, so something like:

    add_custom_command(TARGET MyTest POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_RUNTIME_DLLS:MyTest> $<TARGET_FILE_DIR:MyTest>
        COMMAND_EXPAND_LISTS
    )

I use this for my main target anyway, so I can just execute it on Windows, so this works out of the box for tests I believe.

For tests in general one can use set_tests_environment(TestName PROPERTIES ENVIRONMENT "PATH=SetPathHere") with some generator expressions for building the path. I'm not quite sure if this integrates well with GTest or Catch2, would have to test that.

Based on a quick look catch2 allows to pass down some options, so these might can be used. Apart from that it also states "By default, the CTest test name is the
same as the Catch name; see also TEST_PREFIX and TEST_SUFFIX.".

But as I said that would need some testing.

[friendlyanon]

I linked the commit where this mechanism got implemented for utest.h, so yo don't have to wonder how that is done. Copying is not something I would recommend, since DLLs could depend on some other files relative to them.

[Leon0402]

Ah okay, I think you could use https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#genex:SHELL_PATH to further simplify generating the path for windows.

But I actually meant implementing this outside of catch / gtest, if gtest / catch2 doesn't want to support it directly. Is there already an issue, where it's stated that it's not implemented or won't be implemented?

Isn't copying pretty much the only way in the end? Not for tests, but for applications. At least this is how every application does it on Windows (no windows expert here, I use linux and I'm happy with my rpath :-))

[friendlyanon]

That genex is not necessary for it. Win32 API functions understand forward slashes just fine, only the NtKernel API doesn't and practically every user facing application uses Win32 API.

I have not opened those issues. I simply have not afforded myself the time to do that.

No, copying is not the only way and it's not always going to work. Windows lacks any mechanism similar to RPath, so stuffing DLL paths into PATH prior to calling the executable is the safest option according to my research.

[friendlyanon]

Hey, it wasn't all that recent of a change, but with the addition of package manager integration, Catch2 is now provided by default for testing when a package manager is chosen.

That ought to resolve this topic.