-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
core: implement system batch scheduler
This PR implements a new "System Batch" scheduler type. Jobs can make use of this new scheduler by setting their type to 'sysbatch'. Like the name implies, sysbatch can be thought of as a hybrid between system and batch jobs - it is for running short lived jobs intended to run on every compatible node in the cluster. As with batch jobs, sysbatch jobs can also be periodic and/or parameterized dispatch jobs. A sysbatch job is considered complete when it has been run on all compatible nodes until reaching a terminal state (success or failed on retries). Feasibility and preemption are governed the same as with system jobs. In this PR, the update stanza is not yet supported. The update stanza is sill limited in functionality for the underlying system scheduler, and is not useful yet for sysbatch jobs. Further work in #4740 will improve support for the update stanza and deployments. Closes #2527
- Loading branch information
Showing
36 changed files
with
2,757 additions
and
603 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
job "sysbatchjob" { | ||
datacenters = ["dc1"] | ||
|
||
type = "sysbatch" | ||
|
||
constraint { | ||
attribute = "${attr.kernel.name}" | ||
value = "linux" | ||
} | ||
|
||
parameterized { | ||
payload = "forbidden" | ||
meta_required = ["KEY"] | ||
} | ||
|
||
group "sysbatch_job_group" { | ||
count = 1 | ||
|
||
task "sysbatch_task" { | ||
driver = "docker" | ||
|
||
config { | ||
image = "bash:5" | ||
|
||
command = "bash" | ||
args = ["-c", "ping -c 10 example.com"] | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
job "sysbatchjob" { | ||
datacenters = ["dc1"] | ||
|
||
type = "sysbatch" | ||
|
||
constraint { | ||
attribute = "${attr.kernel.name}" | ||
value = "linux" | ||
} | ||
|
||
group "sysbatch_job_group" { | ||
count = 1 | ||
|
||
task "sysbatch_task" { | ||
driver = "docker" | ||
|
||
config { | ||
image = "bash:5" | ||
|
||
command = "bash" | ||
args = ["-c", "ping -c 10 example.com"] | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
job "sysbatchjob" { | ||
datacenters = ["dc1"] | ||
|
||
type = "sysbatch" | ||
|
||
constraint { | ||
attribute = "${attr.kernel.name}" | ||
value = "linux" | ||
} | ||
|
||
group "sysbatch_job_group" { | ||
count = 1 | ||
|
||
task "sysbatch_task" { | ||
driver = "docker" | ||
|
||
config { | ||
image = "bash:5" | ||
|
||
command = "bash" | ||
args = ["-c", "ping -c 100000 example.com"] | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
job "sysbatchjob" { | ||
datacenters = ["dc1"] | ||
|
||
type = "sysbatch" | ||
|
||
constraint { | ||
attribute = "${attr.kernel.name}" | ||
value = "linux" | ||
} | ||
|
||
periodic { | ||
cron = "*/15 * * * * *" | ||
prohibit_overlap = true | ||
} | ||
|
||
group "sysbatch_job_group" { | ||
count = 1 | ||
|
||
task "sysbatch_task" { | ||
driver = "docker" | ||
|
||
config { | ||
image = "bash:5" | ||
|
||
command = "bash" | ||
args = ["-c", "ping -c 10 example.com"] | ||
} | ||
} | ||
} | ||
} |
Oops, something went wrong.