add support for tarerofs for TurboOCI #320
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
hsiangkao
reviewed
Mar 26, 2024
| #include <photon/fs/fiemap.h> | ||
| #include "../lsmt/file.h" | ||
| #include "../lsmt/index.h" | ||
|
|
There was a problem hiding this comment.
it might be better to add
#define TAREROFS_BLOCK_SIZE 4096
#define __stringify(s) #s
src/overlaybd/tar/tarerofs.cpp
Outdated
| struct stat st; | ||
| char buf[128*1024]; | ||
| char base_path[64] = "/tmp/tarerofs_base_XXXXXX"; | ||
| char command_line[256] = "mkfs.erofs --tar=0,upper.map,1073741824 --aufs"; |
There was a problem hiding this comment.
char command_line[256] = "mkfs.erofs --tar=0,upper.map,1073741824 -b" __stringify(TAREROFS_BLOCK_SIZE) " --aufs";
src/overlaybd/tar/tarerofs.cpp
Outdated
| } | ||
| while (fscanf(fp, "%" PRIx64" %x %" PRIx64 "\n", &blkaddr, &nblocks, &toff) >= 3) { | ||
| LSMT::RemoteMapping lba; | ||
| lba.offset = blkaddr * 4096; |
There was a problem hiding this comment.
4096 -> TAREROFS_BLOCK_SIZE
src/overlaybd/tar/tarerofs.cpp
Outdated
| while (fscanf(fp, "%" PRIx64" %x %" PRIx64 "\n", &blkaddr, &nblocks, &toff) >= 3) { | ||
| LSMT::RemoteMapping lba; | ||
| lba.offset = blkaddr * 4096; | ||
| lba.count = nblocks * 4096; |
There was a problem hiding this comment.
4096 -> TAREROFS_BLOCK_SIZE
src/overlaybd/tar/tarerofs.cpp
Outdated
| char buf[128*1024]; | ||
| char base_path[64] = "/tmp/tarerofs_base_XXXXXX"; | ||
| char command_line[256] = "mkfs.erofs --tar=0,upper.map,1073741824 --aufs"; | ||
| char command_line2[] = " upper.erofs"; |
There was a problem hiding this comment.
const char command_line2[] = " upper.erofs";
src/overlaybd/tar/tarerofs.cpp
Outdated
| } | ||
|
|
||
| while ((read = file->read(buf, sizeof(buf))) > 0) { | ||
| if (fwrite(buf, read, 1, fp) != 1) { |
There was a problem hiding this comment.
better to fix indentation.
src/overlaybd/tar/tarerofs.cpp
Outdated
| std::strcat(command_line, " --base "); | ||
| std::strcat(command_line, base_path); | ||
|
|
||
| for (int i = 0; i < 800; i++) { |
There was a problem hiding this comment.
800 here is quite hard-coded. But I'm not sure how we could detect the real written data in the first segment(extent) since we don't need the following LSMT::IFileRW::RemoteData ranges.
salvete2022
force-pushed
the
dev_erofs
branch
2 times, most recently
from
b3e2cfc to
4289d9b
Compare
Currently erofs-utils is integrated into overlaybd by using `mkfs.erofs` executable file. It's not optimial since raw data needs to be dumped first and output data needs to be write into overlaybd then. Later API integration may be a better form but it needs more work. Signed-off-by: Hongzhen Luo <hongzhen.lhz@linux.alibaba.com>
hsiangkao
approved these changes
Mar 26, 2024
|
thanks @hsiangkao |
liulanzheng
approved these changes
Mar 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Currently erofs-utils is integrated into overlaybd by using
mkfs.erofsexecutable file. It's not optimial since raw data needs to be dumped first and output data needs to be write into overlaybd then.Later API integration may be a better form but it needs more work.
The corresponding accelerated-container-image PR:
containerd/accelerated-container-image#272
The corresponding erofs-utils for test can be found at:
git clone https://gitee.com/anolis/erofs-utils.git -b overlaybd-dev
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #
Please check the following list: