Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm debugger test timing out #83847

Closed
agocke opened this issue Mar 23, 2023 · 14 comments
Closed

Wasm debugger test timing out #83847

agocke opened this issue Mar 23, 2023 · 14 comments
Assignees
Labels
arch-wasm WebAssembly architecture area-Infrastructure-mono blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab
Milestone

Comments

@agocke
Copy link
Member

agocke commented Mar 23, 2023

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=214012
Build error leg or test failing: chrome-DebuggerTests.SteppingTests.WorkItemExecution
Pull request: #83787

Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "DXVAVDA fatal error: could not LoadLibrary: mf.dll: The specified module could not be found",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Report

Build Definition Test Pull Request
228097 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84335
228059 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84271
228055 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84273
228042 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #74623
228035 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84329
227981 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84148
227824 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84170
227763 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution
227758 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution
227643 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #84148
227254 dotnet/runtime chrome-DebuggerTests.DelegateTests.WorkItemExecution #84289
227157 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution
226099 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #84242
226564 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #84148
226490 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #83461
226452 dotnet/runtime DebuggerTests.EvaluateOnCallFrameTests.DebuggerTests.EvaluateOnCallFrameTests.EvaluateTypeInstanceMembers #82981
226249 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #82005
226127 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84247
225939 dotnet/runtime chrome-DebuggerTests.PointerTests.WorkItemExecution #83714
225875 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests2.WorkItemExecution
225567 dotnet/runtime DebuggerTests.EvaluateOnCallFrameTests.DebuggerTests.EvaluateOnCallFrameTests.EvaluateTypeInstanceMembers
225411 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
224800 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84191
224455 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #84148
223906 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #83716
223892 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution
222367 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
222333 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #80297
221876 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #84097
221800 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution #84085
221596 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests2.WorkItemExecution #84083
221504 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #84080
221205 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution #84004
221207 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84017
221056 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests2.WorkItemExecution #84068
221041 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #84067
220983 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #83776
220770 dotnet/runtime DebuggerTests.EvaluateOnCallFrameTests.DebuggerTests.EvaluateOnCallFrameTests.EvaluateTypeInstanceMembers #84058
220319 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #84027
220279 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution
220262 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #83838
219923 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests2.WorkItemExecution #84004
219916 dotnet/runtime chrome-DebuggerTests.SetVariableValueTests.WorkItemExecution #83657
219516 dotnet/runtime chrome-DebuggerTests.PointerTests.WorkItemExecution #83776
219412 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #84011
219349 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution #82981
218775 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution
218041 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
217847 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
216609 dotnet/runtime chrome-DebuggerTests.DelegateTests.WorkItemExecution #83896
217523 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #82981
217087 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #83630
216733 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #83882
216694 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution #83838
216631 dotnet/runtime chrome-DebuggerTests.GetPropertiesTests.WorkItemExecution
216554 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #83894
216472 dotnet/runtime chrome-DebuggerTests.ArrayTests.WorkItemExecution #80297
216419 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #83882
216403 dotnet/runtime chrome-DebuggerTests.CallFunctionOnTests.WorkItemExecution #83624
216232 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests2.WorkItemExecution #83882
216166 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #83670
215642 dotnet/runtime chrome-DebuggerTests.MiscTests.WorkItemExecution #83856
215329 dotnet/runtime chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution #83624
214831 dotnet/runtime chrome-DebuggerTests.BreakpointTests.WorkItemExecution #82981
214012 dotnet/runtime chrome-DebuggerTests.SteppingTests.WorkItemExecution #83787

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 26 65
@agocke agocke added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab labels Mar 23, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Mar 23, 2023
@ghost
Copy link

ghost commented Mar 23, 2023

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=214012
Build error leg or test failing: chrome-DebuggerTests.SteppingTests.WorkItemExecution
Pull request: #83787

Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false,
  "ErrorPattern": "Test timed out(.|\n|\r)*wasm",
  "ExcludeConsoleLog": true
}
Author: agocke
Assignees: -
Labels:

area-Infrastructure-libraries, blocking-clean-ci, Known Build Error

Milestone: -

@agocke
Copy link
Member Author

agocke commented Mar 23, 2023

@jkoritzinsky @hoyosjs Could you check the regex I used here and see if it matches like I think it does? I'm not sure if multi-line works or whether it works like I think.

@agocke
Copy link
Member Author

agocke commented Mar 23, 2023

Yeah, I don't think this worked. @markwilkie any chance you could provide insight to what I'm doing wrong?

@hoyosjs
Copy link
Member

hoyosjs commented Mar 23, 2023

I am looking at it, but I am not sure what you are trying to match it against.

@agocke
Copy link
Member Author

agocke commented Mar 23, 2023

Ah, I'm trying to match this console log: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-83787-merge-8c3242e441a340b79e/chrome-DebuggerTests.SteppingTests/1/console.cbbe6fd0.log?helixlogtype=result

In particular, the test output "Test timed out", then having it walk all the way down to the stack trace, where the string "wasm" appears.

Basically, I want to catch timeouts where wasm is involved.

@carlossanlop
Copy link
Member

carlossanlop commented Mar 23, 2023

"Test timed out(.|\n|\r)*wasm"

This would match:

  • "Test timed out"
  • Followed by zero or more:
    • Any character, or
    • a "\n" character
    • a "\r" character
  • Followed by "wasm"

I am unsure if these regex searches work with multiple lines.

Is there a reason why you don't want to regex-match the error lines that look like this?:

browser-stderr [0322/182943.716:ERROR:dxva_video_decode_accelerator_win.cc(1426)] DXVAVDA fatal error: could not LoadLibrary: mf.dll: The specified module could not be found. (0x7E)

@ghost
Copy link

ghost commented Mar 23, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=214012
Build error leg or test failing: chrome-DebuggerTests.SteppingTests.WorkItemExecution
Pull request: #83787

Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false,
  "ErrorPattern": "Test timed out(.|\n|\r)*wasm",
  "ExcludeConsoleLog": true
}

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
Author: agocke
Assignees: -
Labels:

arch-wasm, blocking-clean-ci, untriaged, area-Infrastructure-mono, Known Build Error

Milestone: -

@agocke
Copy link
Member Author

agocke commented Mar 23, 2023

Is there a reason why you don't want to regex-match the error lines that look like this?:

I assumed the test could fail for more reasons than this and we would still want to catch it, but tbh that's probably better to start with.

@carlossanlop
Copy link
Member

@agocke confirmed, the search is not multiline: https://github.com/dotnet/arcade/blob/main/Documentation/Projects/Build%20Analysis/KnownIssues.md#regex-matching

Can only match one line, no backtracking, and is case insensitive.

I think if we have many different types of timeouts, it's ok to have multiple issues open if we are able to match unique lines from the error output. Even if they have the same root cause in the end, at least we can get them marked as Known Build Error.

@hoyosjs
Copy link
Member

hoyosjs commented Mar 23, 2023

@agocke you set ExcludeConsoleLog=true

@agocke
Copy link
Member Author

agocke commented Mar 23, 2023

Oops. Fixed :)

@hoyosjs
Copy link
Member

hoyosjs commented Mar 23, 2023

Hmm, it's definitely sad if multiline is not allowed though... And that new test string seems a bit too wide. That sounds like a red herring for a debugger hang.

@hoyosjs
Copy link
Member

hoyosjs commented Mar 23, 2023

I am not surprised if that's always the case even if it's a good run.

@lewing lewing added this to the 8.0.0 milestone Mar 24, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Mar 24, 2023
@lewing
Copy link
Member

lewing commented Apr 6, 2023

@thaystg committed a fix and it seems to be working

@lewing lewing closed this as completed Apr 6, 2023
@ghost ghost locked as resolved and limited conversation to collaborators May 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture area-Infrastructure-mono blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab
Projects
None yet
Development

No branches or pull requests

5 participants