Allow some tasks waiting for node per stage before blocking scheduling #11740
Conversation
losipiuk
force-pushed
the
lo/stage-scheduler-non-block
branch
from
b84e9f8 to
a165d03
Compare
losipiuk
changed the title
losipiuk
force-pushed
the
lo/stage-scheduler-non-block
branch
from
a165d03 to
48659f5
Compare
core/trino-main/src/main/java/io/trino/execution/scheduler/FaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/execution/scheduler/FaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
losipiuk
force-pushed
the
lo/stage-scheduler-non-block
branch
from
48659f5 to
18741f2
Compare
| @@ -153,6 +153,7 @@ | |||
| public static final String QUERY_RETRY_ATTEMPTS = "query_retry_attempts"; | |||
| public static final String TASK_RETRY_ATTEMPTS_OVERALL = "task_retry_attempts_overall"; | |||
| public static final String TASK_RETRY_ATTEMPTS_PER_TASK = "task_retry_attempts_per_task"; | |||
| public static final String MAX_TASKS_WAITING_FOR_NODE_PER_STAGE = "max_tasks_waiting_for_node_per_stage"; | |||
There was a problem hiding this comment.
nit: maybe prefix it with FAULT_TOLERANT_EXECUTIO_ (similar to other Tardigrade related configs)?
There was a problem hiding this comment.
it gets very long then :) Also session properties above are not prefixed - and also tardigrade specific .
There was a problem hiding this comment.
That's a good point. It looks like we are somehow inconsistent. Some of the properties are prefixed when some of them are not. I wonder what should be the guideline. What do you think?
There was a problem hiding this comment.
If property is only applicable to fault-tolerant execution I am option for prefixing. THough it would be nice to have something shorter than fault_tolerant_execution_ :) No idea for alternative though.
On the other hand at some point we may migrate to just having a single execution mode - then prefixing is not needed. Then dropping prefix already is more future proof :)
|
CI: #11876 |
losipiuk
force-pushed
the
lo/stage-scheduler-non-block
branch
from
18741f2 to
ab53e91
Compare
losipiuk
force-pushed
the
lo/stage-scheduler-non-block
branch
from
ab53e91 to
9bbf603
Compare
|
CI: #11893 |
Description
Allow some tasks waiting for node per stage before blocking scheduling
improvement
core query engine
Related issues, pull requests, and links
Fixes #10808
Documentation
( ) No documentation is needed.
(x) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
(x) No release notes entries required.
( ) Release notes entries required with the following suggested text: