-
-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Runtime of oci_image is proportional to the total size of of all tars plus the base image #365
Comments
After reading through the source code, I think I understand the reason this is slower than I expected. Building an
I wonder if there may be room to speed up the For |
#324 is gonna solve this. I can't continue my work on that right now, packed with other stuff. |
Blocked by bazelbuild/bazel#20891 |
Working on it at bazelbuild/bazel#21263 |
@thesayyn Thank you for the great work here! I was trying these changes out from the tip of the I did run into one issue I wanted to bring up - I understand these changes aren't officially released yet so apologies if this is a known issue. When trying to include tar layers that are generated from a repository rule, the generated
I was able to fix this with a patch to include the layers as inputs to the action, even when symlinking: diff --git a/oci/private/image.bzl b/oci/private/image.bzl
index 93a6668..bca45a6 100644
--- a/oci/private/image.bzl
+++ b/oci/private/image.bzl
@@ -175,8 +175,7 @@ def _oci_image_impl(ctx):
# If tree artifact symlinks are supported just add tars into runfiles.
if use_symlinks:
transitive_inputs = transitive_inputs + ctx.files.tars
- else:
- inputs = inputs + ctx.files.tars
+ inputs = inputs + ctx.files.tars
# add layers
for (i, layer) in enumerate(ctx.files.tars): Is this an appropriate fix or am I doing something wrong on my end? |
It's very possible that we broke something, could you please file an issue with a reproducer? i'd like to take a look |
Filed #566, thanks! |
Principled fix is here: #567 |
The time required to build
oci_image
seems to be proportional to the total size of the tars plus the size of the base image. This was surprising to me. I expectedoci_image
to take time proportional to the size of only the modified or added layers. After reading through the source code and running some tests, it seems like the reason for this may have something to do with crane mutate.Are these runtime characteristics an intended or necessary property of
rules_oci
or the OCI spec? Or are they due to implementation details ofrules_oci
and its dependencies likecrane
?How to reproduce the issue
I tested this behavior by comparing the build time of four scenarios:
For both scenarios, I built the target, then modified
app.py
and measured the time required to build again.BUILD
The text was updated successfully, but these errors were encountered: