Skip to content

Commit

Permalink
Given the lessons from jenkinsci#109 (INFRA-2283), trying to throttle…
Browse files Browse the repository at this point in the history
… parallelism using the trick from jenkinsci/workflow-basic-steps-plugin#98.
  • Loading branch information
jglick committed Sep 26, 2019
1 parent 6e1a791 commit b099e6b
Showing 1 changed file with 19 additions and 9 deletions.
28 changes: 19 additions & 9 deletions Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,28 @@ stage('prep') {
}
}

// TODO would much rather parallelize *all* PCT tests, but (INFRA-2283) ci.jenkins.io just falls over when we try.
// Running in parallel by plugin but serially by line works, albeit slowly, since workflow-cps is a bottleneck.
// So we try to manually constrain parallelism.
def semaphore = 50 // 50× parallelism seems to work reliably; 84× seems to fail reliably.
branches = [failFast: true]
plugins.each { plugin ->
lines.each {line ->
lines.each {line ->
plugins.each { plugin ->
branches["pct-$plugin-$line"] = {
mavenEnv {
deleteDir()
unstash 'pct.sh'
unstash 'pct'
unstash "megawar-$line"
withEnv(["PLUGINS=$plugin", "LINE=$line"]) {
sh 'mv megawar-$LINE.war megawar.war && bash pct.sh'
waitUntil {if (semaphore > 0) {semaphore--; true} else {false}} // see JENKINS-27127
assert semaphore >= 0
try {
mavenEnv {
deleteDir()
unstash 'pct.sh'
unstash 'pct'
unstash "megawar-$line"
withEnv(["PLUGINS=$plugin", "LINE=$line"]) {
sh 'mv megawar-$LINE.war megawar.war && bash pct.sh'
}
}
} finally {
semaphore++
}
}
}
Expand Down

0 comments on commit b099e6b

Please sign in to comment.