Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lz4 writer does not close properly #94

Closed
rongcuid opened this issue Mar 3, 2023 · 8 comments · Fixed by #95
Closed

lz4 writer does not close properly #94

rongcuid opened this issue Mar 3, 2023 · 8 comments · Fixed by #95

Comments

@rongcuid
Copy link

rongcuid commented Mar 3, 2023

When I use FrameEncoder, the last part of the file is not actually written properly:

let output = Box::new(lz4_flex::frame::FrameEncoder::new(File::create(output)?));
// many write calls
output.flush()?;

Using lz4cat on resulting file gives Error 68: unfinished stream.

I need this boxing because I am generalizing multiple output backends with Box<dyn Write>. I am not sure if this is causing the issue here, but flate2 with its gzip decoder closes files properly.

@rongcuid
Copy link
Author

rongcuid commented Mar 3, 2023

As a note, since I am using Box<dyn Writer>, I cannot actually call .finish(). In my opinion, this should really be called on drop().

edit:

zstd provides a type called AutoFinishEncoder. Maybe you can add something like this?

PSeitz added a commit that referenced this issue Mar 4, 2023
PSeitz added a commit that referenced this issue Mar 4, 2023
@PSeitz PSeitz closed this as completed in #95 Mar 4, 2023
PSeitz added a commit that referenced this issue Mar 4, 2023
@PSeitz
Copy link
Owner

PSeitz commented Mar 4, 2023

Thanks for the suggestion, I added auto_finish for FrameEncoder on the main branch. Can you test if that works for you?

@PSeitz PSeitz reopened this Mar 4, 2023
@rongcuid
Copy link
Author

rongcuid commented Mar 6, 2023

I think it works.

@PSeitz
Copy link
Owner

PSeitz commented Mar 12, 2023

Okay cool, I'll try to make a release next week

@rongcuid
Copy link
Author

I'd just like to check on the status, thanks

@PSeitz
Copy link
Owner

PSeitz commented Jun 2, 2023

I'm preparing a release. I did a lot of changes before auto_finish and want to make sure everything works properly (fuzzing, benchmarks etc.).

@PSeitz
Copy link
Owner

PSeitz commented Jun 3, 2023

Related issue ouch-org/ouch#163 (comment)

@PSeitz
Copy link
Owner

PSeitz commented Jun 18, 2023

Released with 0.11

@PSeitz PSeitz closed this as completed Jun 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants