-
Notifications
You must be signed in to change notification settings - Fork 210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test suite doesn't offer a way to override the default image #959
Comments
For reference, here's the downstream patch from RHEL:
It introduces the |
I see two approaches here:
For the v0.1 release I'd try the short solution because I'd like to get the release out soon. For the short term solution, a way could be making the suite more configurable using env vars or a dedicated TOML parser could be used to read a configuration. The first option sounds better to me as the second one will increase the complexity even more and most likely even add an additional dependency that most likely won't be widely available. |
Umm... I don't think this related to the test framework or changing it to something else. The issue here is that the test suite has it's own logic to determine what the default distro, image and release are, that has diverged from the actual logic inside the One way to solve it is to add a private command to I think I had mentioned this approach before in past conversations, but good to finally put it down in writing. |
I don't like the idea of adding a command into the main binary that is meant only for testing. Instead, let's create a separate, test-only binary that uses the same code as |
Sure, sounds like a better idea, indeed. :) |
#840 has already introduced a command that's of no use to end users and should stay hidden. Let's stick to that approach, and not create a separate binary. There's no real philosophical difference between a command for testing, and a command for generating shell completions. These are different from |
As much as I like the separation of concerns the preliminary work I did (https://github.com/HarryMichal/toolbox/tree/test/system/better-generalization-and-configurability) has showed that the introduced binary would further unnecessarily complicate our setup. It's definitely more convenient to keep everything in a single binary. And the final binary size should not be of concern really, too. Though maybe hiding these commands a bit more would be welcome. Instead of |
Yes, namespacing them with __ or some other prefix sounds like a good idea. Whatever we do, we should retain consistency across all such hidden commands. |
Commit 20f4f68 added support for configuration files. If keys in the configuration file were set the system test suite would not honor them. This puts additional strain on distributors overriding the default settings. This solves the problem by creating a new hidden command in Toolbx that provides interface for getting some information Toolbx uses under-the-hood (e.g., the default container name, image). This commit does not change the way images are cached in the system test suite. In case a different image is needed, the tests still need to be patched. Fixes containers#959
Commit 20f4f68 added support for configuration files. If keys in the configuration file were set the system test suite would not honor them. This puts additional strain on distributors overriding the default settings. This solves the problem by creating a new hidden command in Toolbx that provides interface for getting some information Toolbx uses under-the-hood (e.g., the default container name, image). This commit does not change the way images are cached in the system test suite. In case a different image is needed, the tests still need to be patched. Fixes containers#959
Commit 20f4f68 added support for configuration files. If keys in the configuration file were set the system test suite would not honor them. This puts additional strain on distributors overriding the default settings. This solves the problem by creating a new hidden command in Toolbx that provides interface for getting some information Toolbx uses under-the-hood (e.g., the default container name, image). This commit does not change the way images are cached in the system test suite. In case a different image is needed, the tests still need to be patched. Fixes containers#959 containers#1050
Commit 20f4f68 added support for configuration files. If keys in the configuration file were set the system test suite would not honor them. This puts additional strain on distributors overriding the default settings. This solves the problem by creating a new hidden command in Toolbx that provides interface for getting some information Toolbx uses under-the-hood (e.g., the default container name, image). This commit does not change the way images are cached in the system test suite. In case a different image is needed, the tests still need to be patched. Fixes containers#959 containers#1050
Commit 20f4f68 added support for configuration files. If keys in the configuration file were set the system test suite would not honor them. This puts additional strain on distributors overriding the default settings. This solves the problem by creating a new hidden command in Toolbx that provides interface for getting some information Toolbx uses under-the-hood (e.g., the default container name, image). This commit does not change the way images are cached in the system test suite. In case an additional image is needed, the tests still need to be patched. Fixes containers#959 containers#1050
We currently offer a configuration file to override the default image that's used for things like the
create
command. Some downstream distributors use this to significantly change the default image to the point that the test suite can't be run unless it uses the same default image that's specified in the configuration file.As an aside, I am worried that the test suite seems to be growing it's own logic to resolve container and image names parallel to the actual Toolbox code itself. We should try to fix this, because the logic isn't trivial.
When running the test suite on RHEL, @olivergs was using a downstream patch to introduce some environment variables to override the defaults, but I don't see how it can possibly work with the state of the test suite in Toolbox 0.0.99.3.
I think that we need a proper upstream solution for this.
The text was updated successfully, but these errors were encountered: