You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I start running a job and i geting this error many times and the job is cancelled
System.UnauthorizedAccessException: Access to the path '/proc/554/oom_score_adj' is denied.
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Starting process:
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] File name: '/usr/bin/bash'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Arguments: '--noprofile --norc -e -o pipefail /home/runner/_work/_temp/0e164abb-bd20-4274-91b5-72f26c8fae7f.sh'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Working directory: '/home/runner/_work/devops'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Require exit code zero: 'False'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Encoding web name: ; code page: ''
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Force kill process on cancellation: 'False'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Redirected STDIN: 'False'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Persist current code page: 'False'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Keep redirected STDIN open: 'False'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] High priority process: 'False'
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] Failed to update oom_score_adj for PID: 554.
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] System.UnauthorizedAccessException: Access to the path '/proc/554/oom_score_adj' is denied.
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] ---> System.IO.IOException: Permission denied
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] --- End of inner exception stack trace ---
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
test-t82bf-runner-tzkbq runner [WORKER 2024-07-10 13:30:38Z INFO JobServerQueue] Try to append 1 batches web console lines for record 'd8be3eeb-808b-5cb1-9c71-884a43769e10', success rate: 1/1.
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at System.IO.Strategies.BufferedFileStreamStrategy.FlushWrite()
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at System.IO.Strategies.BufferedFileStreamStrategy.Dispose(Boolean disposing)
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at System.IO.StreamWriter.CloseStreamFromDispose(Boolean disposing)
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at System.IO.StreamWriter.Dispose(Boolean disposing)
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at System.IO.File.WriteAllText(String path, String contents)
test-t82bf-runner-vcgwt runner [WORKER 2024-07-10 13:30:38Z INFO ProcessInvokerWrapper] at GitHub.Runner.Sdk.ProcessInvoker.WriteProcessOomScoreAdj(Int32 processId, Int32 oomScoreAdj)
test-t82bf-runner-bzrls runner [RUNNER 2024-07-10 13:29:47Z WARN GitHubActionsService] GET request to https://pipelinesghubeus26.actions.githubusercontent.com/0TagnS0qaEOWaiMRmM6TEfnPxHtFutTo92gSnylMXf5cdGZrsW/_apis/distributedtask/pools/10/messages?sessionId=7259e804-7eac-4621-ba0e-6bcd61f8a395&lastMessageId=1&status=Busy&runnerVersion=2.317.0&os=Linux&architecture=X64&disableUpdate=true has been cancelled.
test-t82bf-runner-bzrls runner [RUNNER 2024-07-10 13:29:47Z INFO MessageListener] Get next message has been cancelled.
test-t82bf-runner-bzrls runner [RUNNER 2024-07-10 13:29:47Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
test-t82bf-runner-bzrls runner [RUNNER 2024-07-10 13:29:47Z INFO JobDispatcher] Ensure WorkerDispatcher for job 1cfa91de-17dd-561a-f553-3e6471df7df2 run to finish, cancel any running job.
test-t82bf-runner-bzrls runner [RUNNER 2024-07-10 13:29:47Z INFO JobDispatcher] Send job cancellation message to worker for job 1cfa91de-17dd-561a-f553-3e6471df7df2.
test-t82bf-runner-bzrls runner [RUNNER 2024-07-10 13:29:47Z INFO ProcessChannel] Sending message of length 0, with hash 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessChannel] Receiving message of length 0, with hash 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO Worker] Cancellation/Shutdown message received.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO HostContext] Runner will be shutdown for UserCancelled
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO JobServerQueue] Try to append 1 batches web console lines for record '1cfa91de-17dd-561a-f553-3e6471df7df2', success rate: 1/1.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO StepsRunner] Cancel current running step.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Sending SIGINT to process 175.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Successfully send SIGINT to process 175.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Waiting for process exit or 7.5 seconds after SIGINT signal fired.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Process exit successfully.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Process cancelled successfully through Ctrl+C/SIGINT.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Process Cancellation finished.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO ProcessInvokerWrapper] Finished process 175 with exit code 130, and elapsed time 00:00:48.9883086.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO CreateStepSummaryCommand] Step Summary file (/home/runner/_work/_temp/_runner_file_commands/step_summary_6931528a-638b-4f31-8479-e73982adfb14) is empty; skipping attachment upload
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] Caught cancellation exception from step: System.OperationCanceledException: The operation was canceled.
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at System.Threading.CancellationToken.ThrowOperationCanceledException()
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at GitHub.Runner.Sdk.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Channel`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at GitHub.Runner.Common.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Channel`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at GitHub.Runner.Worker.Handlers.DefaultStepHost.ExecuteAsync(IExecutionContext context, String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, String standardInInput, CancellationToken cancellationToken)
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at GitHub.Runner.Worker.Handlers.ScriptHandler.RunAsync(ActionRunStage stage)
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at GitHub.Runner.Worker.ActionRunner.RunAsync()
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z ERR StepsRunner] at GitHub.Runner.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
test-t82bf-runner-bzrls runner [WORKER 2024-07-10 13:29:47Z INFO StepsRunner] Step result: Canceled```
{
"$id": "1",
"innerException": null,
"message": "The user 'System:PublicAccess;aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource.",
"typeName": "Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server",
"typeKey": "UnauthorizedRequestException",
"errorCode": 0,
"eventId": 3000
}
i am not sure why this happens , working on eks + dind
**To Reproduce**
Steps to reproduce the behavior:
NA
## Runner Version and Platform
2.317.0
OS of the machine running the runner? OSX/Windows/Linux/...
## What's not working?
job is not finishing
## Job Log Output
If applicable, include the relevant part of the job / step log output here. All sensitive information should already be masked out, but please double-check before pasting here.
The text was updated successfully, but these errors were encountered:
So i did full checkup .... i can explain the issues but i cant explain the error . long story short the runner is burst the CPU, MEM to a point of OOM and if you use k8s you will star evection process
i think the runner needs to have some mem + cpu limits ( k8s "resources" will not work ) not sure why i see this CPU and MEM issues
I had the same issue. I also saw some other issues in the same direction here, for conservation and also if someone else has this issue. I had to increase the shm-size of the docker container.
container:
image: your/image:latestoptions: --shm-size=4gb # otherwise we get a bus error (core dumped), do not ask me how long it took to debug this :-)
Describe the bug
I start running a job and i geting this error many times and the job is cancelled
System.UnauthorizedAccessException: Access to the path '/proc/554/oom_score_adj' is denied.
{
"$id": "1",
"innerException": null,
"message": "The user 'System:PublicAccess;aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource.",
"typeName": "Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server",
"typeKey": "UnauthorizedRequestException",
"errorCode": 0,
"eventId": 3000
}
The text was updated successfully, but these errors were encountered: