Segmentation fault in System.Text.RegularExpressions.Tests #93206
Comments
akoeplinger
added
area-System.Text.RegularExpressions
blocking-clean-ci
ghost
added
the
|
Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions Issue DetailsBuild InformationBuild: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=431489 Error MessageFill the error message using step by step known issues guidance. {
"ErrorMessage": "",
"ErrorPattern": "Segmentation fault.*System.Text.RegularExpressions.Tests",
"BuildRetry": false,
"ExcludeConsoleLog": false
}
|
|
Looks similar to #85046 |
|
Some detail from the logs; no dump: |
|
Both this and #85046 occurred on OSX. |
steveharter
removed
the
untriaged
|
Without a dump it'll be impossible to make progress on this. It's also very unlikely to be in regex itself, and much more likely to be an issue either in the span-related functionality regex sits on top of, or in codegen / the runtime. |
|
I was able to capture a core dump on my local Mac using the Helix artifacts, but it is 8GB so uploading will take a while :) |
|
Here's the coredump compressed with 7z: https://microsofteur-my.sharepoint.com/:u:/g/personal/alkpli_microsoft_com/Ed36-eUF0PZOm-1hEL6QVwMBNRoTbiyBIgX5sh9dY6WR6Q?e=fGpWUP This was from the artifacts from Helix job 5472e3d3-beb1-49db-af99-5d3100d2a736.
Let me know if you need anything else. |
Cool. Is there a crashlog (.crash) file that you can run lldb crashlog against? This looks suspect however: |
|
There's no .crash but a .ips which is basically the same but json encoded: dotnet-2023-10-09-182901.ips.zip (also added the rendered report) Btw. the binary was built from commit d3a782e Interestingly the .ips points to Thread 18 (== Thread 19 in lldb since that uses 1-based indexing) as the thread that had the SIGSEGV, which points to Here's the disassembly of the function https://gist.github.com/akoeplinger/621f3de8abf8dfd01f62c941d5d552fe I wasn't able to get |
|
Poking at the function a bit: |
|
@akoeplinger should this be in the codegen area? |
|
Happened to have a look at this and it does appear to only be failing on Mac. It's a bummer we aren't getting dumps there yet @hoyosjs @carlossanlop - this type of issue would really benefit from crash symbolization. I thought with the latest changes that should be working on Macs? Agree with @danmoseley that this looks more like codegen issue. |
ericstj
added
area-CodeGen-coreclr
|
I'm actively investigating a product issue where dumps are not getting collected. |
|
@kunalspathak, it seems loop alignment related. PTAL. It is blocking clean ci. |
|
are we still seeing this issue? I don't think so. |
|
@kunalspathak that's from yesterday, the dump sadly didn't get egressed. The method that failed was |
|
@jakobbotsch @amanasifkhalid - can one of you please take a look as you recently touched the loops/block layout code. This seems to be accessing a null BasicBlock and we get seg fault. |
|
@amanasifkhalid, PTAL. |
|
Unless I'm missing something, the seg fault doesn't seem to be happening in the JIT. |
|
The two recent appearances are from preliminary runs in PRs that had issues. So I would probably hold off looking at the crash dumps (if any). |
|
Since this hasn't hit recently, I'm going to unmark |
amanasifkhalid
removed
the
blocking-clean-ci
|
Hard to say, I think codegen is a good a guess as any. Are all the crashes on osx-x64? |
|
It's worth noting that the most recent failure (#100658) was from an intermediate commit that hit other issues in CI, so maybe that was a false positive?
Not all of them. Some of them hit on Linux arm64. |
|
@amanasifkhalid I apologize for deleting one of your comments. Reminder that internal helix logs should not be shared in GitHub comments. |
|
@riarenas no worries, sorry about that. |
|
This hasn't hit on a "functional" CI run in quite a while. Are we ok with closing this? |
|
The most recent hit was on a draft PR with other failures. Since we haven't had a failure block CI recently, I think we ought to close this to avoid instilling a false sense of confidence on affected PRs. |
Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=431489
Build error leg or test failing: Libraries Test Run release coreclr osx x64 Release
Pull request: N/A
Error Message
Fill the error message using step by step known issues guidance.
{ "ErrorMessage": "", "ErrorPattern": "Segmentation fault.*System.Text.RegularExpressions.Tests", "BuildRetry": false, "ExcludeConsoleLog": false }Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=431489
Error message validated:
Segmentation fault.*System.Text.RegularExpressions.TestsResult validation: ✅ Known issue matched with the provided build.
Validation performed at: 10/9/2023 11:24:45 AM UTC
Report
Summary
The text was updated successfully, but these errors were encountered: