Skip to content
This repository has been archived by the owner on Apr 22, 2023. It is now read-only.

Deadlock in V8 when CPU profiling is active #25295

Closed
dmelikyan opened this issue May 13, 2015 · 3 comments
Closed

Deadlock in V8 when CPU profiling is active #25295

dmelikyan opened this issue May 13, 2015 · 3 comments

Comments

@dmelikyan
Copy link

In current version of V8 3.28.73 a deadlock causes the process to hang and consume 100% CPU when CPU profiling is active and the application is serving requests. The reason is this line of code https://github.com/joyent/node/blob/v0.12.2/deps/v8/src/isolate.cc#L50, which leads to pthread_mutex_lock deadlock. It is removed in version 3.29.79.

To reproduce start the CPU profiling using V8 API (e.g. with v8-profiler) and run some load agains the server. My sample app would also do some requests to DBs to simulate real world apps. When it hangs, you can use gdb to verify it hangs because of pthread_mutex_lock.

Looks like the patching would be dangerous since the V8 commit fixing the issue has multiple changes and was actually nothing to do with this issue. So the solution is to upgrade V8.

Are there any plans to upgrade V8 in the 0.12 at some point? The issue makes it impossible to profile applications under load test and in production.

@misterdjules
Copy link

It's the same issue as #14576. There is no plan to upgrade V8 in v0.12, as it would be a breaking change. We would need to find a proper way to fix this issue for V8 included in v0.12 and float a patch.

@misterdjules
Copy link

Closing as a duplicate of #14576, thank you very much for filing an issue with a clear and detailed description, it is very much appreciated 👍

@misterdjules
Copy link

@mhdawson Does anyone on your team familiar with V8 internals has the time to investigate coming up with a floating patch for this?

dmelikyan added a commit to dmelikyan/node that referenced this issue May 14, 2015
A deadlock happens when sampler initiated by SIGPROF tries to lock
the thread and the thread is already locked by the same thread. As
a result, other thread involved in sampling process hangs. The
patch adds a check for thread lock before continuing sampler
operation.

The fix has been tested on a sample app under load with and without
profiling turned on.

Fixes issue nodejs#14576 and specifically the duplicate issue nodejs#25295
dmelikyan added a commit to dmelikyan/node that referenced this issue Jun 8, 2015
A deadlock happens when sampler initiated by SIGPROF tries to lock
the thread and the thread is already locked by the same thread. As
a result, other thread involved in sampling process hangs. The
patch adds a check for thread lock before continuing sampler
operation.

The fix has been tested on a sample app under load with and without
profiling turned on.

Fixes issue nodejs#14576 and specifically the duplicate issue nodejs#25295
dmelikyan added a commit to dmelikyan/node that referenced this issue Jun 10, 2015
A deadlock happens when sampler initiated by SIGPROF tries to lock
the thread and the thread is already locked by the same thread. As
a result, other thread involved in sampling process hangs. The
patch adds a check for thread lock before continuing sampler
operation.

The fix has been tested on a sample app under load with and without
profiling turned on.

Fixes issue nodejs#14576 and specifically the duplicate issue nodejs#25295
misterdjules pushed a commit that referenced this issue Jun 10, 2015
A deadlock happens when sampler initiated by SIGPROF tries to lock
the thread and the thread is already locked by the same thread. As
a result, other thread involved in sampling process hangs. The
patch adds a check for thread lock before continuing sampler
operation.

The fix has been tested on a sample app under load with and without
profiling turned on.

Fixes issue #14576 and specifically the duplicate issue #25295

Reviewed-By: Julien Gilli <julien.gilli@joyent.com>
PR-URL: #25309
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants