Meson tests fail but pass when ran locally

[ErichZimmer]

Hi all,

I had an issue that has been plaguing me for some time now. When I run unit test executables through Meson or Ninja, I keep getting timeouts on unit tests that read image files using std::ifstream. These image files are copied from the source directory to the tests directory during the build process and were verified post-build via simple directory trees. However, I can't seem to find a reason why they are not able to be opened with std::ifstream and later cause the test to timeout. When the executable is ran without the use of Meson or Ninja, all assertians pass which makes me think that this may be some resource or path related issue.

Tested OSs:
Windows 11
Ubuntu 22.04.3

Example test

// catch
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_all.hpp>

// std
#include <fstream>
#include <iostream>
#include <sstream>
#include <string>

TEST_CASE("image_test - scale_test")
{
    std::ifstream is("some_file.tif", std::ios::binary);
    REQUIRE(is.is_open()); // <-- here is my issue, it always returns false under Meson, but true when ran manually
}

Example build script (for subdir tests)

test_source_files = [
  'my_test_file.cpp'
]

foreach test_file: test_source_files
  source_name = test_file.split('.')[0]
  
  exec = executable(
    source_name,
    test_file,
    include_directories: my_lib_include,
    dependencies: my_lib_deps + [catch2_dep],
    cpp_args: ['-D_USE_MATH_DEFINES'],
    link_with: my_lib
  )
  
  test(
    source_name,
    exec
  )
  
endforeach

If necessary, I can make a working example sometime in the future.

Additional Context
This issue may be related to this stack overflow issue covering issues caused by shared resources (in my case, image files).

I tried the following:

• Manually verifying all files are in tests directory and are correctly named
• Setting number of processes to 1
• Disabling timeout
• Setting workdir to source/build directory in test()
• Setting is_parrelel: false in test()

If this issue is a shared file related issue, then why? std::ifstream supports concurrency, if I remember correctly.

For additional context, the library I am working on is a port from vcpkg to meson and is located here.

[ErichZimmer]

As I suspected, it was an absolute path issue. What is happening is that the relative path is set to the build directory and not the directory containing the executables. For instance. the working directory set by meson on my Windows 11 environment is current working dir: "C:\\Users\\Erich\\meson_test\\builddir" while the test executable and shared test resources is in test dir: "C:\\Users\\Erich\\meson_test\\builddir\\tests". That is why all tests using shared resources fail when using meson or ninja, but pass when ran by executing the executable.

[eli-schwartz]

I tried the following:

* Setting `workdir` to source/build directory in test()

Is this included in your current build files? I don't see it.

For what it's worth, the tests do pass for me, with the following patch:

diff --git a/tests/meson.build b/tests/meson.build
index 7be87be..b9f99fc 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -33,7 +33,8 @@ foreach test_file: test_source_files
   
   test(
     source_name,
-    exec
+    exec,
+    workdir: meson.current_build_dir(),
   )
   
 endforeach
@@ -58,7 +59,8 @@ if found_benchmark
   
     test(
       source_name,
-      exec
+      exec,
+      workdir: meson.current_build_dir(),
     )
   
   endforeach

[ErichZimmer]

I tested it last month using a separate debug environment, so it was not pushed onto my meson port branch. However, I must have done some sort of user error if it works on your end but not on mine. I'll retry that solution again later tonight when I have time.

[ErichZimmer]

I tested you solution and all tests pass. However, I am still running into another issue where the tests timeout after successful completion. It appears that these timeouts only happen when a test uses std::ifstream and std::stringstream as all other tests complete successfully. Do you know what could be causing this issue too?

[ErichZimmer]

Well, the hanging part is not a Meson problem as it pertains to either Catch2 or std::ifstream. In other words, all questions I have are solved. Thanks for your help!

[eli-schwartz]

Okay, good to know. :)