-
Notifications
You must be signed in to change notification settings - Fork 528
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crane export: Support reading tarball from a stream #1274
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1274 +/- ##
==========================================
+ Coverage 74.01% 74.07% +0.05%
==========================================
Files 112 112
Lines 8386 8386
==========================================
+ Hits 6207 6212 +5
+ Misses 1575 1570 -5
Partials 604 604
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this actually work? We need to read through the tarball multiple times (or buffer it) to effectively export the filesystem.
This doesn't work correctly. Why the |
One reason is we can't assume the ordering of files in the archive. We go through a few times to a.) find the We can't guarantee the ordering of these files in the archive, so we don't (currently) attempt to do this in a single read. Since the (c) depends on (b), we'd have to rely on archives being well-ordered to be able to read them in one go, and I don't think tools currently enforce that ordering. Aside from that, we also peek at a layer blob to tell if it's compressed. There might be other places I'm forgetting about where we assume a tar file is reopenable. It might be possible to refactor all this so we can read a tar archive without repeatedly restarting the read, but we don't today. Doing so might break reading some images that we support today. |
A working concept that reads the image from "test.tar" if first argument is specified as "-". The goal is to read from a stdin, but that part (commented) fails with Error: reading tarball from stdin: archive/tar: invalid tar header
./crane export - ubuntu.tar < test.tar Error: reading tarball from stdin: archive/tar: invalid tar header
d7ef2db
to
82ff3e6
Compare
I chose to write This is probably doable for this PR, but the change surface might get too vast to handle. Rebased as some test was failing for unclear reason. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not blocking, but any idea why this doesn't work?
$ ./crane pull ubuntu - | ./crane export - flat.tar
Error: reading tarball from stdin: file manifest.json not found in tar
...but this works:
./crane pull ubuntu ubuntu.tar
./crane export - - < ubuntu.tar | tar -Oxf - etc/passwd
I was hoping to be able to demonstrate this needless monstrosity:
$ ./crane pull ubuntu - | ./crane export - - | tar -Oxf - etc/passwd
Error: reading tarball from stdin: file manifest.json not found in tar
tar: etc/passwd: Not found in archive
tar: Error exit delayed from previous errors.
Co-authored-by: Jason Hall <jasonhall@redhat.com>
Because |
@jonjohnsonjr Any remaining concerns? If not, feel free to merge. |
Implementation for #1273.
The goal is to read from stdin if the first argument is "-".
I was not able to trace what is wrong with reading fromos.Stdin
, and my Go skills are not that good to understand it from the code.