Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add erofs-util API support for overlaybd #332

Merged
merged 1 commit into from
Jun 27, 2024

Conversation

salvete
Copy link
Contributor

@salvete salvete commented Jun 24, 2024

What this PR does / why we need it:

Previously, erofs-utils was integrated into overlaybd by using the mkfs.erofs executable file. It's not optimial since raw data needs to be dumped first and output data needs to be write into overlaybd then.

This adds EROFS API support for overlaybd.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Please check the following list:

  • Does the affected code have corresponding tests, e.g. unit test, E2E test?
  • Does this change require a documentation update?
  • Does this introduce breaking changes that would require an announcement or bumping the major version?
  • Do all new files have an appropriate license header?

@hsiangkao
Copy link
Contributor


erofs_inode_manager_init();

root = erofs_mkfs_alloc_root(sbi);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's rename this as erofs_rebuild_make_root(sbi);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


#define EROFS_UNIMPLEMENTED 1

struct erofs_sector {
Copy link
Contributor

@hsiangkao hsiangkao Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could just use a std::map<addr, struct liberofs_inmem_sector> and stl::set for dirty bitmap?

struct liberofs_inmem_sector {
     char data[SECTOR_SIZE];
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

public:
LibErofs(photon::fs::IFile *target);
~LibErofs();
int extract_tar(photon::fs::IFile *source, uint64_t blksize, bool meta_only, bool first_layer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uint64_t blksize should be moved to the constructor.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@salvete salvete force-pushed the erofs-io-incremental branch from afeb875 to 528d634 Compare June 25, 2024 09:22
Previously, erofs-utils was integrated into overlaybd by
using the `mkfs.erofs` executable file. It's not optimial
since raw data needs to be dumped first and output data
needs to be write into overlaybd then.

This commit introduces the I/O API in tarerofs, that the data
stream of the:
- output image
- input tar file
can be controlled by the I/O manager.

This also adds block read/write cache support for IFile to
improve the efficiency of making file system images.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
@salvete salvete force-pushed the erofs-io-incremental branch from 528d634 to 652bc76 Compare June 26, 2024 07:02
Copy link
Contributor

@hsiangkao hsiangkao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL thanks @BigVan

@BigVan
Copy link
Member

BigVan commented Jun 27, 2024

LGTM

@BigVan BigVan merged commit 6c1bf35 into containerd:main Jun 27, 2024
2 checks passed
@BigVan
Copy link
Member

BigVan commented Jun 27, 2024

will build a new release after improving CI test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants