[SPARK-4342] [Core]connection ack timeout improvement, replace Timer with ScheudledExecutor... #3207
Conversation
haitaoyao
commented
Nov 11, 2014
haitaoyao
commented
- replace java.util.Timer with ScheduledExectorService
- avoid reference to the whole message, use message id only.
…tor to avoid long timeout cause reference message body to OOME
|
Can one of the admins verify this patch? |
haitaoyao
changed the title
| @@ -77,7 +77,8 @@ private[nio] class ConnectionManager( | |||
| } | |||
|
|
|||
| private val selector = SelectorProvider.provider.openSelector() | |||
| private val ackTimeoutMonitor = new Timer("AckTimeoutMonitor", true) | |||
| private val ackTimeoutMonitor = Executors.newScheduledThreadPool(2, | |||
There was a problem hiding this comment.
Is there any reason for the magic number 2?
There was a problem hiding this comment.
to avoid some task block the whole scheduler. ScheduledExecutorService executes the task in its own thread pool, instead of fork a new thread for the task execution.
There was a problem hiding this comment.
I just checked the codes:
public ScheduledThreadPoolExecutor(int corePoolSize,
ThreadFactory threadFactory) {
super(corePoolSize, Integer.MAX_VALUE, 0, TimeUnit.NANOSECONDS,
new DelayedWorkQueue(), threadFactory);
}2 is the min thread number, not the max thread number.
There was a problem hiding this comment.
corePoolSize the number of threads to keep in the pool, even if they are idle
Here I think 1 is enough.
|
This looks like a duplicate of another JIRA, which was resolved by #3259, so do you mind closing this? Thanks! (Sorry that I missed this PR; I guess it got buried in a backlog of other stuff) |
