SHAKE-128 and SHAKE-256 implementations have no default digest length

[David-Klemenc]

Describe the bug
Since OpenSSL 3.4 the SHAKE-128 and SHAKE-256 implementations have no default digest length. link

To Reproduce

:crypto.hash_init(:shake128) |> :crypto.hash_update("test") |> :crypto.hash_final()
** (ErlangError) Erlang error: {:error, {~c"hash.c", 218}, ~c"Low-level call EVP_DigestFinal failed"}
    crypto.erl:1119: :crypto.hash_final(#Reference<0.945702049.3715760132.202667>)
    iex:1: (file)

this works:

:crypto.hash_xof(:shake128, "test", 16 * 8)
<<211, 176, 170, 156, 216, 183, 37, 86, 34, 206, 188, 99, 30, 134, 125, 64>>

check which of the "supported" hashes work:

:crypto.supports(:hashs) 
|> Enum.map(fn hash -> 
	try do 
		:crypto.hash_init(hash) 
		|> :crypto.hash_update("test") 
		|> :crypto.hash_final()

		"#{hash} works"
	rescue 
		_ -> "#{hash} does not work"
	end
end)

Expected behavior
Docs should be updated

Affected versions
Updating Openssl creates this problem

openssl -v
OpenSSL 3.4.0 22 Oct 2024 (Library: OpenSSL 3.4.0 22 Oct 2024)

[sverker]

OpenSSL 3.3 docs says

For backwards compatibility reasons the default xoflen length for SHAKE-128 is 16 (bytes) which results in a security strength of only 64 bits. To ensure the maximum security strength of 128 bits, the xoflen should be set to at least 32.

and OpenSSL 3.4 docs says

Since OpenSSL 3.4 the SHAKE-128 and SHAKE-256 implementations have no default digest length.

Not sure what to do here...

• Just document "sorry if you use OpenSSL 3.4, shake128 and shake256 will not work"?
• Make shake128 and shake256 work as before by setting the old default lengths towards OpenSSL 3.4 (16 and 32 bytes)?
• Change shake128 and shake256 to use the more secure digest lengths (32 and 64 bytes)?
• Introduce more secure variants shake128_256 and shake256_512 (32 and 64 bytes)?
• Expose crypto:hash_final_xof(State, Length) where you can specify the digest length you want?

[David-Klemenc]

This crypto:hash_final_xof(State, Length) would be cool, in the in the meantime making them work as before sounds the best to me.

Just my 2 cents 😄

[sverker]

PR #9002 makes shake128 and shake256 work as before on OpenSSL 3.4.

[sverker]

Maybe a crypto:hash_init({shake128, Length}) would be an even better API. Then you select your preferred digest with hash_init and can then forget about it and call hash_update and hash_final like all other digests.