fix: add a limit of number of bytes while scale decoding a slice #3733
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
While scale decoding we first read the length of bytes to decode and then we decode that many bytes. Someone could ask us to decode some malicious bytes such that the read length is unreasonably big. In such case, we would have to create a byte slice as big as the length. So, we would either panic because of `makeslice: len out of range` or because the asked length would be more than the memory we have available in our machine. Both of these are the situations we don't want. Hence, I have put a limit on decodable bytes (maxDecodableBytes), 16MB. I have kept it 16MB because for the usecase of polkadot specs, we would never need to decode a bigger set of bytes.
kishansagathiya
requested review from
edwardmack,
timwu20,
EclesioMeloJunior,
jimjbrettj,
dimartiro and
axaysagathiya
as code owners
jimjbrettj
approved these changes
Feb 2, 2024
EclesioMeloJunior
requested changes
Feb 5, 2024
…/gossamer into kishan/fix/panic-unmarshalling
EclesioMeloJunior
approved these changes
Feb 12, 2024
dimartiro
approved these changes
Feb 12, 2024
github-actions bot
pushed a commit
that referenced
this pull request
Mar 1, 2024
# [0.9.0](v0.8.0...v0.9.0) (2024-3-1) ### Bug Fixes * add a limit of number of bytes while scale decoding a slice ([#3733](#3733)) ([5edbf89](5edbf89)) * **docs:** Fixing link to polkadot runtime fundamentals to the right one ([#3763](#3763)) ([a785d32](a785d32)) * don't panic if we fail to convert hex to bytes ([#3734](#3734)) ([12234de](12234de)) * **dot/sync:** execute p2p handshake when there is no target ([#3695](#3695)) ([a9db0ec](a9db0ec)) * fix index out of range undeterministic error in rpc test ([#3718](#3718)) ([d099384](d099384)) * fix non deterministic panic during TestStableNetworkRPC integration test ([#3756](#3756)) ([ee3d243](ee3d243)) * **lib/trie:** use `MustBeHashed` for V1 trie nodes with larger storage values ([#3739](#3739)) ([f5e48a9](f5e48a9)) * **mocks:** Set fixed version for uber mockgen in CI ([#3656](#3656)) ([ea9877e](ea9877e)) * **runtime/storage:** support nested storage transactions ([#3670](#3670)) ([3e99f6d](3e99f6d)) * segfault on node restart ([#3736](#3736)) ([d1ca7aa](d1ca7aa)) * **state-version:** should be uint8 instead of uint32 ([#3779](#3779)) ([c8fdb14](c8fdb14)) * update paseo chain spec ([#3770](#3770)) ([6a54f28](6a54f28)) * use last finalized block on startup ([#3737](#3737)) ([c262642](c262642)) ### Features * **config:** dynamically set version based on environment ([#3693](#3693)) ([5c534c9](5c534c9)) * **staging:** Expose RPC on Westend Staging Node ([#3687](#3687)) ([c374eaa](c374eaa)) * **tests/scripts:** create script to retrieve trie state via rpc ([#3714](#3714)) ([5ccea40](5ccea40))
|
🎉 This PR is included in version 0.9.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
timwu20
pushed a commit
that referenced
this pull request
Apr 18, 2024
While scale decoding we first read the length of bytes to decode and then we decode that many bytes. Someone could ask us to decode some malicious bytes such that the read length is unreasonably big. In such case, we would have to create a byte slice as big as the length. The length in byte slice is an encoded as `Compact<u32>`. Current we are just reading length as uint and not checking if it goes beyond the bounds of uint32. So, we would either panic because of `makeslice: len out of range` or because the asked length would be more than the memory we have available in our machine. We are going to put a check to makes sure that this length is less than max of uint32.
timwu20
pushed a commit
that referenced
this pull request
Apr 19, 2024
While scale decoding we first read the length of bytes to decode and then we decode that many bytes. Someone could ask us to decode some malicious bytes such that the read length is unreasonably big. In such case, we would have to create a byte slice as big as the length. The length in byte slice is an encoded as `Compact<u32>`. Current we are just reading length as uint and not checking if it goes beyond the bounds of uint32. So, we would either panic because of `makeslice: len out of range` or because the asked length would be more than the memory we have available in our machine. We are going to put a check to makes sure that this length is less than max of uint32.
timwu20
pushed a commit
that referenced
this pull request
Apr 19, 2024
# [0.9.0](v0.8.0...v0.9.0) (2024-3-1) ### Bug Fixes * add a limit of number of bytes while scale decoding a slice ([#3733](#3733)) ([5edbf89](5edbf89)) * **docs:** Fixing link to polkadot runtime fundamentals to the right one ([#3763](#3763)) ([a785d32](a785d32)) * don't panic if we fail to convert hex to bytes ([#3734](#3734)) ([12234de](12234de)) * **dot/sync:** execute p2p handshake when there is no target ([#3695](#3695)) ([a9db0ec](a9db0ec)) * fix index out of range undeterministic error in rpc test ([#3718](#3718)) ([d099384](d099384)) * fix non deterministic panic during TestStableNetworkRPC integration test ([#3756](#3756)) ([ee3d243](ee3d243)) * **lib/trie:** use `MustBeHashed` for V1 trie nodes with larger storage values ([#3739](#3739)) ([f5e48a9](f5e48a9)) * **mocks:** Set fixed version for uber mockgen in CI ([#3656](#3656)) ([ea9877e](ea9877e)) * **runtime/storage:** support nested storage transactions ([#3670](#3670)) ([3e99f6d](3e99f6d)) * segfault on node restart ([#3736](#3736)) ([d1ca7aa](d1ca7aa)) * **state-version:** should be uint8 instead of uint32 ([#3779](#3779)) ([c8fdb14](c8fdb14)) * update paseo chain spec ([#3770](#3770)) ([6a54f28](6a54f28)) * use last finalized block on startup ([#3737](#3737)) ([c262642](c262642)) ### Features * **config:** dynamically set version based on environment ([#3693](#3693)) ([5c534c9](5c534c9)) * **staging:** Expose RPC on Westend Staging Node ([#3687](#3687)) ([c374eaa](c374eaa)) * **tests/scripts:** create script to retrieve trie state via rpc ([#3714](#3714)) ([5ccea40](5ccea40))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While scale decoding we first read the length of bytes to decode and then we decode that many bytes.
Someone could ask us to decode some malicious bytes such that the read length is unreasonably big. In such case, we would have to create a byte slice as big as the length. So, we would either panic because of
makeslice: len out of rangeor because the asked length would be more than the memory we have available in our machine.Both of these are the situations we don't want. Hence, I have put a limit on decodable bytes (maxDecodableBytes), 16MB. I have kept it 16MB because for the usecase of polkadot specs, we would never need to decode a bigger set of bytes.
Changes
Tests
go test -tags integration github.com/ChainSafe/gossamerIssues
Fixes #3476
Primary Reviewer
@timwu20 @EclesioMeloJunior