[NRBF] Fix bugs discovered by the fuzzer #107368
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… when the referenced record is missing or it points to a record of different type
… it's being thrown by BinaryReader (or sth else that we use)
dotnet-issue-labeler
bot
added
the
needs-area-label
adamsitnik
added
binaryformatter-migration
|
Tagging subscribers to 'binaryformatter-migration': @adamsitnik, @bartonjs, @jeffhandley, @terrajobst |
| { | ||
| return Decode(reader, options ?? new(), out recordMap); | ||
| } | ||
| catch (FormatException) // can be thrown by various BinaryReader methods |
There was a problem hiding this comment.
Pass to inner exception?
There was a problem hiding this comment.
I don't do that on purpose, to avoid leaking any information from the inner exception that could be attacker controlled (like some weird string that could somehow affect the log file).
There was a problem hiding this comment.
|
@adamsitnik somehow cannot run the fuzzer locally with this fix, had cherry-picked all commits still see the |
|
Anyway there is one thing that is not fixed, getting InvalidCastExeption when there is no valid header, on: I think decoder should throw when there is no header |
|
In your PR, you are referencing the NuGet package, not the local project:
What I did to get this working (it's ugly, there must be a better way):
<ItemGroup>
<Reference Include="System.Formats.Nrbf">
<HintPath>..\..\..\..\artifacts\bin\System.Formats.Nrbf\Debug\net8.0\System.Formats.Nrbf.dll</HintPath>
</Reference>
</ItemGroup>
.\dotnet.cmd build .\src\libraries\System.Formats.Nrbf\System.Formats.Nrbf.sln
D:\projects\runtime\dotnet.cmd publish -o publish; publish/DotnetFuzzing.exe prepare-onefuzz deployment
.\deployment\NrbfDecoderFuzzer\local-run.bat |
It will (I have added the fix for that with a test in my first commit), the More info: the The fuzzer was smart enough to generate a value that made it pass (64 IIRC), I have simply added a check that rejects values out of the enum range: |
…en parsing the decimal fails
Local testing unblocked with this, thanks! |
buyaa-n
approved these changes
Sep 5, 2024
|
|
adamsitnik
force-pushed
the
nrbfFuzzer
branch
from
13422c2
to
acfd78e
Compare
|
…ad (so far an ArgumentException was thrown)
|
@MihuBot fuzz NrbfDecoder |
|
/ba-g the one unknown CI failure is unrelated and the failure log has no any info for filing an issue |
This was referenced Sep 11, 2024
adamsitnik
added a commit
to adamsitnik/runtime
that referenced
this pull request
Sep 13, 2024
* bug #1: don't allow for values out of the SerializationRecordType enum range * bug #2: throw SerializationException rather than KeyNotFoundException when the referenced record is missing or it points to a record of different type * bug #3: throw SerializationException rather than FormatException when it's being thrown by BinaryReader (or sth else that we use) * bug #4: document the fact that IOException can be thrown * bug #5: throw SerializationException rather than OverflowException when parsing the decimal fails * bug #6: 0 and 17 are illegal values for PrimitiveType enum * bug #7: throw SerializationException when a surrogate character is read (so far an ArgumentException was thrown) # Conflicts: # src/libraries/System.Formats.Nrbf/src/System/Formats/Nrbf/NrbfDecoder.cs
4 tasks
jtschuster
pushed a commit
to jtschuster/runtime
that referenced
this pull request
Sep 17, 2024
* bug #1: don't allow for values out of the SerializationRecordType enum range * bug dotnet#2: throw SerializationException rather than KeyNotFoundException when the referenced record is missing or it points to a record of different type * bug dotnet#3: throw SerializationException rather than FormatException when it's being thrown by BinaryReader (or sth else that we use) * bug dotnet#4: document the fact that IOException can be thrown * bug dotnet#5: throw SerializationException rather than OverflowException when parsing the decimal fails * bug dotnet#6: 0 and 17 are illegal values for PrimitiveType enum * bug dotnet#7: throw SerializationException when a surrogate character is read (so far an ArgumentException was thrown)
carlossanlop
pushed a commit
that referenced
this pull request
Sep 17, 2024
* [NRBF] Don't use Unsafe.As when decoding DateTime(s) (#105749) * Add NrbfDecoder Fuzzer (#107385) * [NRBF] Fix bugs discovered by the fuzzer (#107368) * bug #1: don't allow for values out of the SerializationRecordType enum range * bug #2: throw SerializationException rather than KeyNotFoundException when the referenced record is missing or it points to a record of different type * bug #3: throw SerializationException rather than FormatException when it's being thrown by BinaryReader (or sth else that we use) * bug #4: document the fact that IOException can be thrown * bug #5: throw SerializationException rather than OverflowException when parsing the decimal fails * bug #6: 0 and 17 are illegal values for PrimitiveType enum * bug #7: throw SerializationException when a surrogate character is read (so far an ArgumentException was thrown) # Conflicts: # src/libraries/System.Formats.Nrbf/src/System/Formats/Nrbf/NrbfDecoder.cs * [NRBF] throw SerializationException when a surrogate character is read (#107532) (so far an ArgumentException was thrown) * [NRBF] Fuzzing non-seekable stream input (#107605) * [NRBF] More bug fixes (#107682) - Don't use `Debug.Fail` not followed by an exception (it may cause problems for apps deployed in Debug) - avoid Int32 overflow - throw for unexpected enum values just in case parsing has not rejected them - validate the number of chars read by BinaryReader.ReadChars - pass serialization record id to ex message - return false rather than throw EndOfStreamException when provided Stream has not enough data - don't restore the position in finally - limit max SZ and MD array length to Array.MaxLength, stop using LinkedList<T> as List<T> will be able to hold all elements now - remove internal enum values that were always illegal, but needed to be handled everywhere - Fix DebuggerDisplay * [NRBF] Comments and bug fixes from internal code review (#107735) * copy comments and asserts from Levis internal code review * apply Levis suggestion: don't store Array.MaxLength as a const, as it may change in the future * add missing and fix some of the existing comments * first bug fix: SerializationRecord.TypeNameMatches should throw ArgumentNullException for null Type argument * second bug fix: SerializationRecord.TypeNameMatches should know the difference between SZArray and single-dimension, non-zero offset arrays (example: int[] and int[*]) * third bug fix: don't cast bytes to booleans * fourth bug fix: don't cast bytes to DateTimes * add one test case that I've forgot in previous PR # Conflicts: # src/libraries/System.Formats.Nrbf/src/System/Formats/Nrbf/SerializationRecord.cs * [NRBF] Address issues discovered by Threat Model (#106629) * introduce ArrayRecord.FlattenedLength * do not include invalid Type or Assembly names in the exception messages, as it's most likely corrupted/tampered/malicious data and could be used as a vector of attack. * It is possible to have binary array records have an element type of array without being marked as jagged --------- Co-authored-by: Buyaa Namnan <bunamnan@microsoft.com>
sirntar
pushed a commit
to sirntar/runtime
that referenced
this pull request
Sep 30, 2024
* bug #1: don't allow for values out of the SerializationRecordType enum range * bug #2: throw SerializationException rather than KeyNotFoundException when the referenced record is missing or it points to a record of different type * bug #3: throw SerializationException rather than FormatException when it's being thrown by BinaryReader (or sth else that we use) * bug dotnet#4: document the fact that IOException can be thrown * bug dotnet#5: throw SerializationException rather than OverflowException when parsing the decimal fails * bug dotnet#6: 0 and 17 are illegal values for PrimitiveType enum * bug dotnet#7: throw SerializationException when a surrogate character is read (so far an ArgumentException was thrown)
This was referenced Sep 6, 2024
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SerializationRecordTypeenum range (I've checked the parsing of all other enums and they are already very defensive, it seems to be the only enum where we were not defensive enough)SerializationExceptionrather thanKeyNotFoundExceptionwhen the referenced record is missing or it points to a record of different type (SerializationExceptionis what we promise in the docs)SerializationExceptionrather thanFormatExceptionwhen it's being thrown byBinaryReader(or sth else that we use)IOExceptioncan be thrownSerializationExceptionrather thanOverflowExceptionwhen parsing the decimal fails