Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail fast if cgo is disabled and compression is ZSTD #1284

Merged
merged 2 commits into from
Apr 8, 2020

Conversation

jarifibrahim
Copy link
Contributor

@jarifibrahim jarifibrahim commented Mar 31, 2020

As seen in issue hypermodeinc/dgraph#4995, if badger is built without CGO and compression is set to ZSTD, badger would crash when level 0 is compacted. This could take from a few seconds to a few minutes for the crash to show up.

With this PR, badger.Open() call will return an error if compression is set to ZSTD and badger was built without CGO.


This change is Reviewable

Copy link
Contributor

@mangalaman93 mangalaman93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good, just a minor comment. Though, I think it would be better to throw a compile time error. We can use build flags (linux, cgo) and not define the compression functions for the case when !linux or !cgo.

Reviewable status: 0 of 3 files reviewed, 1 unresolved discussion (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)


y/y.go, line 40 at r1 (raw file):

	ErrEOF = errors.New("End of mapped region")

	ErrZstdCgo = errors.New("zstd compression requires building badger with cgo enabled")

comment missing, go vet will complain.

Copy link
Contributor

@mangalaman93 mangalaman93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 0 of 3 files reviewed, 1 unresolved discussion (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)

Copy link
Contributor Author

@jarifibrahim jarifibrahim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a chat with @mangalaman93 and we agreed that the !linux the approach wouldn't work.

Reviewable status: 0 of 3 files reviewed, 1 unresolved discussion (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 3 files at r1, 1 of 1 files at r2.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @ashish-goswami and @jarifibrahim)

@jarifibrahim jarifibrahim merged commit 2e708d9 into master Apr 8, 2020
@jarifibrahim jarifibrahim deleted the ibrahim/failfast-zstd branch April 8, 2020 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants