"leveled.compression_method = lz4" for "riak_3.0.3-OTP22.3_amd64.deb" doesn't work

[ghost]

Install from the deb: 'https://files.tiot.jp/riak/kv/3.0/3.0.3/ubuntu/bionic64/riak_3.0.3-OTP22.3_amd64.deb'

When I set "storage_backend = leveled" and "leveled.compression_method = lz4" in riak.conf Riak fails to put an object.

You can detect it with the "riak admin test" command that fails with timeout. The "native" compression method works well.

In logs I see complains to lz4:

[error] <0.15776.3> gen_server <0.15776.3> terminated with reason: {'module could not be loaded',[{lz4,pack,[...
2021-03-02 11:43:50.254 [error] <0.15776.3>@lz4:pack CRASH REPORT Process <0.15776.3> with 5 neighbours crashed with reason: call to undefined function lz4:pack(<<...>>)

Best regards.

[martinsumner]

This fails in a build from source as well. The dependency is being fetched, so perhaps there's something happening at compilation. lz4 is tested within the leveled tests prior to release (but not in the riak tests). Im 99% certain this has been tested in production with 2.9, so it is likely an issue that got introduced as we updated to rebar3/relx.

I will investigate this now.

[martinsumner]

The problem can be resolved by adding lz4 to the list of applications in the relx stanza of the riak rebar.config (if installing from source).

I need to check the rebar3/relx documentation - I'm not sure if it needs to be added here because of a mistake within the leveled rebar.config, or for some other reason.

I will try and get a new release out urgently, as although native can be used as a workaround, this is not helpful for those upgrading from 2.9 who have already used lz4.

[ P.S. The lz4 option was included as it was assumed that it would be faster, and in small scale tests it was. However on large scale tests we kept observing better performance overall with native (presumably as more stuff fitted in the page cache)].

[martinsumner]

Issue was that lz4 dep was not listed as an application in the app.src file of leveled. I don't know why dialyzer or xref didn't see this as a problem, but ultimately this is a basic mistake by me.

Need to see if there is an easy way to stop this recurring (other than adding in a riak_test).

[martinsumner]

Done a manual check to confirm this is the only such issue. There is an exception wrt protobuffs, which is used in the riak_pb repo, but only at build/compile time - so isn't referred to in app.src doesn't get copied through to the release. This is correct.

[martinsumner]

Resolved in 3.0.4 - https://github.com/basho/riak/blob/develop-3.0/RELEASE-NOTES.md#riak-kv-304-release-notes