Where should we put files on disk necessary for test and run commands?

[lihaoyi]

• Classpath resources aren't enough all the time because they don't allow filesystem operations (e.g. os.list, os.walk, etc.)
• Reading from the resource dir directly isn't enough all the time because it gets bundled in the jar, which you might not want.

Existing Solutions:

• Maven's best practice seems to just put it in the resources folder e.g. https://stackoverflow.com/questions/2948861/how-to-deal-with-the-test-data-in-junit. The standard directory layout doesn't contain any non-resource test data files https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
• Gradle's current advise (as far as I could find at least) also seems to be src/test/resources https://stackoverflow.com/questions/27046642/access-file-in-junit-test-in-gradle-environment
• Bazel has data dependencies and runfiles which get copied into the test/run sandbox before execution. Maven and Gradle don't have any standard for this AFAICT

Follow up of #3347

[lefou]

Just quoting myself here, to have the idea recorded:

#3347 (comment)

I could imaging a dedicated testSandboxResources targets, which automatically get copied into the sandbox when preparing the test run. This can be helpful if the app under test needs some files.

In contrast, to the other resources, which always specify resources meant to be found on the classpath, this new target explicitly is about files in the FS at runtime, but not on the classpath.

#3347 (comment)

The reason why I proposed a new testSandboxResouces is the fact that test resources are already bundles up in the main jar and therefore using them directly from the filesystem blurs their correct purpose. This leads to projects having lots of test reosurces in the resouces directory, which will effectively also end up in the jar, but were never supposed to end up there. I think the fact that so many projects use the test resources folder is just convenience. Maven, who invented the src/test/resources folder is simply too hard to customized to use a better location for such resources, to begin with.

In contrast, we could easily make a distinction here, make it easy to do the right thing, and avoid encouraging users to do the not-so-correct thing. resources is supposed to be packaged up in the jar, so the correct usage of these files is via the classpath. (In Maven slang, we're speaking of something like a src/run/resources.) A new type of resources (testSandboxResources or some other name) will never end up in the package. Looks like a better place for resources a test case shall look up at run time from the filesystem. Even if we don't copy them into the sandbox but make the path accessible via a Java sysprop, it looks to me as the cleaner approach.