Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AV in GcInfoDecoder #103894

Closed
jkotas opened this issue Jun 24, 2024 · 6 comments · Fixed by #103938
Closed

AV in GcInfoDecoder #103894

jkotas opened this issue Jun 24, 2024 · 6 comments · Fixed by #103938
Assignees
Labels
area-VM-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
Milestone

Comments

@jkotas
Copy link
Member

jkotas commented Jun 24, 2024

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=717653
Build error leg or test failing: System.Text.RegularExpressions.Tests.WorkItemExecution
Pull request: #103852

Crash dump: runfo get-helix-payload -j 8c930c16-d447-4906-ad98-4d7818d90641 -w System.Text.RegularExpressions.Tests -o c:\helix_payload\System.Text.RegularExpressions.Tests

Crash at:

0:010> k
 # Child-SP          RetAddr               Call Site
00 (Inline Function) --------`--------     coreclr!BitStreamReader::SetCurrentPos+0x20 [D:\a\_work\1\s\src\coreclr\inc\gcinfodecoder.h @ 350] 
01 (Inline Function) --------`--------     coreclr!BitStreamReader::Skip+0x3d [D:\a\_work\1\s\src\coreclr\inc\gcinfodecoder.h @ 358] 
02 000000c9`9d2e5370 00007ffc`e85649c6     coreclr!GcInfoDecoder::EnumerateLiveSlots+0x211 [D:\a\_work\1\s\src\coreclr\vm\gcinfodecoder.cpp @ 911] 
03 000000c9`9d2e57e0 00007ffc`e849b9f0     coreclr!EECodeManager::EnumGcRefs+0x1f6 [D:\a\_work\1\s\src\coreclr\vm\eetwain.cpp @ 1568] 
04 000000c9`9d2e5980 00007ffc`e855a7db     coreclr!GcStackCrawlCallBack+0x170 [D:\a\_work\1\s\src\coreclr\vm\gcenv.ee.common.cpp @ 361] 
05 (Inline Function) --------`--------     coreclr!Thread::MakeStackwalkerCallback+0x50 [D:\a\_work\1\s\src\coreclr\vm\stackwalk.cpp @ 825] 
06 000000c9`9d2e6b70 00007ffc`e8559783     coreclr!Thread::StackWalkFramesEx+0xef [D:\a\_work\1\s\src\coreclr\vm\stackwalk.cpp @ 905] 
07 000000c9`9d2e6f50 00007ffc`e84e6c9b     coreclr!Thread::StackWalkFrames+0xbf [D:\a\_work\1\s\src\coreclr\vm\stackwalk.cpp @ 986] 
08 000000c9`9d2e8060 00007ffc`e84e6a7c     coreclr!ScanStackRoots+0x73 [D:\a\_work\1\s\src\coreclr\vm\gcenv.ee.cpp @ 207] 
09 000000c9`9d2e80d0 00007ffc`e84ed52c     coreclr!GCToEEInterface::GcScanRoots+0xdc [D:\a\_work\1\s\src\coreclr\vm\gcenv.ee.cpp @ 306] 
0a (Inline Function) --------`--------     coreclr!GCScan::GcScanRoots+0x19 [D:\a\_work\1\s\src\coreclr\gc\gcscan.cpp @ 152] 
0b 000000c9`9d2e8110 00007ffc`e84e8228     coreclr!WKS::gc_heap::mark_phase+0x214 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 29546] 
0c 000000c9`9d2e81c0 00007ffc`e84e71cc     coreclr!WKS::gc_heap::gc1+0xbc [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 22275] 
0d (Inline Function) --------`--------     coreclr!GCToOSInterface::GetLowPrecisionTimeStamp+0x5 [D:\a\_work\1\s\src\coreclr\gc\windows\gcenv.windows.cpp @ 1100] 
0e 000000c9`9d2e8220 00007ffc`e84e62d2     coreclr!WKS::gc_heap::garbage_collect+0x1a0 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 24355] 
0f 000000c9`9d2e8270 00007ffc`e84e6166     coreclr!WKS::GCHeap::GarbageCollectGeneration+0x13e [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 50570] 
10 000000c9`9d2e82d0 00007ffc`e84e491c     coreclr!WKS::gc_heap::trigger_gc_for_alloc+0x26 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 18829] 
11 000000c9`9d2e8300 00007ffc`e84e47b1     coreclr!WKS::gc_heap::try_allocate_more_space+0x144 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 18956] 
12 000000c9`9d2e8360 00007ffc`e84eb268     coreclr!WKS::gc_heap::allocate_more_space+0x31 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 19456] 
13 (Inline Function) --------`--------     coreclr!WKS::gc_heap::allocate+0x62 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 19487] 
14 000000c9`9d2e8390 00007ffc`e856834e     coreclr!WKS::GCHeap::Alloc+0x88 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 49491] 
15 (Inline Function) --------`--------     coreclr!Alloc+0xa7 [D:\a\_work\1\s\src\coreclr\vm\gchelpers.cpp @ 227] 
16 (Inline Function) --------`--------     coreclr!AllocateSzArray+0x24f [D:\a\_work\1\s\src\coreclr\vm\gchelpers.cpp @ 489] 
17 000000c9`9d2e83d0 00007ffc`8e1841c4     coreclr!JIT_NewArr1+0x37e [D:\a\_work\1\s\src\coreclr\vm\jithelpers.cpp @ 2027] 

We are trying to read first byte in a page:

coreclr!BitStreamReader::SetCurrentPos+0x20 [inlined in coreclr!GcInfoDecoder::EnumerateLiveSlots+0x211]:
00007ffc`e85655a1 498b3a          mov     rdi,qword ptr [r10] ds:00007ffc`8ed80000=????????????????

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
@jkotas jkotas added area-VM-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' labels Jun 24, 2024
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Jun 24, 2024
Copy link
Contributor

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

@jkotas
Copy link
Member Author

jkotas commented Jun 24, 2024

@VSadov Could you please take a look? It looks like an off-by-one buffer overrun in the GC decoder.

@VSadov
Copy link
Member

VSadov commented Jun 24, 2024

Will take a look.

@VSadov
Copy link
Member

VSadov commented Jun 25, 2024

Ugh, we assume it is always safe to prefetch one word in the bitstream when decoding. In rare edge cases it might not be safe to prefetch. It is not just a problem for Skip, in theory it can affect Read as well.

As I remember prefetching helps noticeably when decoding, so I'd like to keep that.

@VSadov
Copy link
Member

VSadov commented Jun 25, 2024

I wonder if I could just add one bit terminator to GC info...

@VSadov
Copy link
Member

VSadov commented Jun 25, 2024

on Release NativeAOT System.Collections.Concurrent.Tests.exe adding one bit padding to GC info changed size

from: 20,788,224 bytes
to: 20,790,784 bytes

that is increase by 0.01%

@mangod9 mangod9 removed the untriaged New issue has not been triaged by the area owner label Jun 26, 2024
@mangod9 mangod9 added this to the 9.0.0 milestone Jun 26, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Jul 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-VM-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants