Add build scripts to run tests under a privileged container #5246
Assignees
Labels
Comments
|
I have most of the basic infrastructure/scripts ready. Hoping to do a PR soon. |
This was referenced Dec 10, 2018
zuercher
pushed a commit
that referenced
this issue
Dec 13, 2018
We want to run some integration tests which require various privileges. I have added some scripts which wrap a bazel test target in another script which sets up such a remote environment. The general approach is similar to bazel-test-gdb, except no intermediate script is generated. The script works fairly well. See the documentation portion of the PR for some examples. There are a few gotchas with it, however, that we should probably resolve over time: It won't work with tests that depend on anything outside the test itself -- e.g. input files. If compiling locally, there may be some challenges invoking other binaries in the target container, since we end up overwriting some runtime libraries that the container's gdb may need. There is no strategy just yet to run a privileged test in gdb... I suspect that sudo bazel-test-gdb will have to be sufficient. Example: tools/bazel-test-docker.sh //test/integration:integration_test --jobs=4 -c dbg Risk Level: Low. This is not used by anything, and it will only ever be used explicitly, so it should be tested substantially then. Testing: Tested locally with various integration tests, as well as in circle with my integration test which requires privileges: https://circleci.com/gh/klarose/envoy-circle/11. This commit was based off my hacked up fork of envoy, which only ran the one test: https://github.com/klarose/envoy-circle Docs Changes: I added a description of the script to bazel/README.md. Release Notes: Only affects tooling, so I don't think one is required. Fixes Issue #5246 Signed-off-by: Kyle Larose <kyle@agilicus.com>
|
This issue is fixed with #5628. I'll make use of them in my integration test, and put together the framework to run them in circle later. |
fredlas
pushed a commit
to fredlas/envoy
that referenced
this issue
Mar 5, 2019
) We want to run some integration tests which require various privileges. I have added some scripts which wrap a bazel test target in another script which sets up such a remote environment. The general approach is similar to bazel-test-gdb, except no intermediate script is generated. The script works fairly well. See the documentation portion of the PR for some examples. There are a few gotchas with it, however, that we should probably resolve over time: It won't work with tests that depend on anything outside the test itself -- e.g. input files. If compiling locally, there may be some challenges invoking other binaries in the target container, since we end up overwriting some runtime libraries that the container's gdb may need. There is no strategy just yet to run a privileged test in gdb... I suspect that sudo bazel-test-gdb will have to be sufficient. Example: tools/bazel-test-docker.sh //test/integration:integration_test --jobs=4 -c dbg Risk Level: Low. This is not used by anything, and it will only ever be used explicitly, so it should be tested substantially then. Testing: Tested locally with various integration tests, as well as in circle with my integration test which requires privileges: https://circleci.com/gh/klarose/envoy-circle/11. This commit was based off my hacked up fork of envoy, which only ran the one test: https://github.com/klarose/envoy-circle Docs Changes: I added a description of the script to bazel/README.md. Release Notes: Only affects tooling, so I don't think one is required. Fixes Issue envoyproxy#5246 Signed-off-by: Kyle Larose <kyle@agilicus.com> Signed-off-by: Fred Douglas <fredlas@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Title: Add build scripts to run tests under a privileged container
Description:
We want to be able to run some of the integration tests with CAP_NET_ADMIN. The only way to do that right now in circleci is to either use a "machine" executor running docker locally, or use a remote docker. A major challenge with those approaches is that they only have a 2CPU 8GB machine. Building all tests and running them there is slow. We don't want to add yet another slow process to the pipeline.
So, what do we want?
I have proven that you can run a privileged docker container from a normal executor using remote docker here: https://github.com/klarose/test-circle-permissions/blob/master/.circleci/config.yml.
I think a reasonable approach to solving the problem is to put together some scripts which allow us to copy over the necessary binaries/etc to a remote docker container, then start it with the privileges we want. I have prototyped this locally, and it worked quite nicely.
The text was updated successfully, but these errors were encountered: