Cache bincapz-samples repository to speed up subsequent tests #448
Conversation
Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
Makefile
Outdated
|
|
||
| update-samples: out/samples-$(SAMPLES_HASH) | ||
| rm -rf samples; cp -a out/samples-$(SAMPLES_HASH) samples/ | ||
| cp -a test_data/. samples/ |
There was a problem hiding this comment.
Can we leave the test_data where it is? It seems weird to pollute the samples data this way. I'm OK if we leave this for a future PR though as it would involve changing tests around.
There was a problem hiding this comment.
I've thought about that and it's definitely messy as-is. I'll keep thinking on it.
Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
egibs
force-pushed
the
cache-samples
branch
from
5a0f132 to
74ebc63
Compare
| @@ -50,11 +50,11 @@ jobs: | |||
|
|
|||
| - name: Prepare samples | |||
| run: | | |||
| cp -a ${{ github.workspace }}/test_data/. ${{ github.workspace }}/samples/ | |||
There was a problem hiding this comment.
Can we run the makefile rule instead of duplicating this logic?
There was a problem hiding this comment.
I had some hang up about that mentally but it may just be because I cloned the other repository separately and wasn't sure if our anonymous clone would work.
The make targets should work in CI as well.
There was a problem hiding this comment.
Updated in 4ad123f (#448).
Makefile
Outdated
| mkdir -p out | ||
| git clone https://github.com/chainguard-dev/bincapz-samples.git out/samples-$(SAMPLES_HASH).tmp | ||
| git -C out/samples-$(SAMPLES_HASH).tmp checkout $(SAMPLES_HASH) | ||
| for file in caddy.xz chezmoi.xz minio_x86_64.xz mongosh.xz neuvector_agent_aarch64.xz opa.xz ; do \ |
There was a problem hiding this comment.
Could this work to avoid hardcoding filenames?
find <path> -name "*.xz" -exec xz -d {}
There was a problem hiding this comment.
Implemented a variant of this in 4ad123f (#448).
egibs
force-pushed
the
cache-samples
branch
from
ff2bc32 to
04a6d33
Compare
Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
egibs
force-pushed
the
cache-samples
branch
from
04a6d33 to
4ad123f
Compare
| @@ -113,16 +126,7 @@ update-third-party: | |||
| .PHONY: refresh-sample-testdata out/bincapz | |||
| refresh-sample-testdata: clone-samples out/bincapz | |||
| cp ./test_data/refresh-testdata.sh samples/ | |||
There was a problem hiding this comment.
note: in a future PR, we can switch to running this script from the test_data directory.
|
merging so I can work on bincapz without wifi :) |
…uard-dev#448) * Cache bincapz-samples repository to speed up subsequent tests Signed-off-by: egibs <20933572+egibs@users.noreply.github.com> * Update -> prepare Signed-off-by: egibs <20933572+egibs@users.noreply.github.com> * Address PR comments; rework out/samples target Signed-off-by: egibs <20933572+egibs@users.noreply.github.com> * Use Make targets in CI; simplify + clean up targets Signed-off-by: egibs <20933572+egibs@users.noreply.github.com> --------- Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
Closes: #445
Repeatedly cloning the
bincapz-samplesdirectory (which is >150MB) wastes a lot of time. This PR attempts to cache the output inout/so that subsequent test runs do not need to re-clone the entire repository.First run:
Subsequent runs: