Do jobs in self-hosted runners run in parallel?

[DaHuoKolmostar]

The document says "Jobs run in parallel by default. ". When I tried workflow in github-hosted runners, jobs do run in parallel. However, when I try to run the workflow in self-hosted runners, the jobs do not seem to run in parallel. I’ve tried in both macos and centos with the latest version runner v2.165.2.

Is it that jobs only run in parallel in github-hosted runners?

My work flow is as follows:

on: [push]
jobs:

Synthesize:

runs-on: self-hosted

steps:

- uses: actions/checkout@v2
  - run: |
      ./code_generation.sh

  - run: |
      ./basic_verification.sh

  - run: |
      ./synthesis.sh

Estimation:

runs-on: self-hosted

needs: Synthesize

steps:

- name: check

run: |

pwd

ls
  - name: start time
    run: |
      date

  - name: sleep
    run: |
      sleep 8

  - name: end time
    run: |
      date

Deeper_Verification:

runs-on: self-hosted

needs: Synthesize

steps:

- name: check

run: |

pwd

ls
  - name: start time
    run: |
      date

  - name: sleep
    run: |
      sleep 8

  - name: end time
    run: |
      date

And the runner output in CentOS is as follows:

2020-03-06 05:41:01Z: Listening for Jobs
2020-03-06 05:42:38Z: Running job: Synthesize
2020-03-06 05:42:46Z: Job Synthesize completed with result: Succeeded
2020-03-06 05:42:51Z: Running job: Deeper_Verification
2020-03-06 05:43:02Z: Job Deeper_Verification completed with result: Succeeded
2020-03-06 05:43:04Z: Running job: Estimation
2020-03-06 05:43:15Z: Job Estimation completed with result: Succeeded

[weide-zhou]

Hi @dahuokolmostar ,

Jobs should run in parallel for both hosted runner and self-hosted runner.

But for self-hosted runner, there are some comments:

• One self-hosted runner can only run one job at a time, when no available runners are idle, the subsequent jobs will be in queueing until available runners are idle.
• If have enough available self-hosted runners for a workflow run, each job will be assigned a self-hosted runner, and the jobs will run in parallel.
• When multiple workflows are running, if the workflows are more than the available self-hosted runners, each workflow will be assigned only one self-hosted runner.

Thanks.

[sebastiaanluca]

Well that's a bummer finding out after setting everything up 🥲

[crohr]

If you are on AWS you might be interested in RunsOn, which handles the runner autoscaling for you.

[reukiodo]

So... if a single running can only run a single job at a time, consecutively and NOT concurrently, then "Jobs should run in parallel for both hosted runner and self-hosted runner." seems like a generally false or at best misleading statement. Each single runner will only run a single job at a time. The only concurrency which can occur is if there are multiple runners, one for each job. This is not just annoying but counter-intuitive to all the current documentation.

[allir]

Are there any plans to change this? That is so that a runner can run multiple jobs at the same time (concurrently).

It would be really nice, since a lot of the time the jobs are small and don’t require the full machine to run and could run multiple jobs at the same time. Plus most of the time the jobs are spun up in containers so just having fewer runner machines with multiple cpus to spin those up vs having multiple smaller machines or having to run multiple instances of the actions runner service per machine to achieve this…

For more cpu/resource intensive jobs you could select a runner via labels so that it would go onto a machine that is dedicated to that job?

[faddat]

Fact is that currently, the GitLab runner is much better than the Github actions runnner.

Hopefully this will change.

[penso]

Yes, those are years apart, I was using gitlab and it's way better...

[sohanyadav]

@allir i agree with you man!!!

[sohanyadav]

We can run Runner in docker and then we can run multiple runners on same server!!!

[diaristadeveloper]

any details on this?
It would be necessary to create a docker-compose/docker run exposing docker socket. This is a fast way to deploy multiple runners on a cluster by using for example a docker service / docker stack deploye.

[ivan-klass]

Thanks @weide-zhou for the details. I think it makes a lot of sense to add these comments into the official documentation.
Before I’ve found this discussion I’ve realized that all our repo workflows share the same default workspace directory on self-hosted runners, and it really made my hair stands on end. I was searching the docs to find out whether it’s actually a problem and there was no answer.

[devbro1]

if you want to have multiple runners on the same machine you can run them under different users:

useradd runner1
usermod -a -G docker runner1
usermod -a -G rvm runner1
usermod -a -G wheel runner1
sudo su runner1

cd ~

#from https://github.com/actions/runner/releases
mkdir actions-runner && cd actions-runner
curl -O -L https://github.com/actions/runner/releases/download/v2.296.1/actions-runner-linux-x64-2.296.1.tar.gz
tar xzf ./actions-runner-linux-x64-2.296.1.tar.gz

#from https://github.com/financeit/centah/settings/actions/runners/new?arch=x64&os=linux
./config.sh --url https://github.com/financeit/centah --token ??? --name runner1 --unattended 
sudo ./svc.sh install
sudo ./svc.sh start

[AllanOricil]

This doc should include this information.
"parallelization is only possible when multiple runners are available"

[srid]

The doc you linked to no longer contains this statement.

[blafasel42]

should be possible to install the runner multiple times on the same machine with different users, no?

[LucBerge]

You can install multiple runners on the same machine with the same user. Just use a different folder location.
I currently have 6 runners running in parallel.

[clarkmcc]

I've installed multiple runners in different folders, but this is quite frustrating to maintain between all the systemctl service definitions and special configs and permissions for each. I would love to see a single runner instance be able to run multiple jobs concurrently.

[crohr]

Using ARC on a single node with microk8s might be a solution?

[clarkmcc]

I did try that but found the runner performance to be significantly worse than running on bare metal. Might be just me though...

[sebyx07]

here is a script to automate adding runners on a single server

#!/bin/bash
if [ "$#" -ne 2 ]; then
    echo "Usage: $0 <runner-number> <github-token>"
    exit 1
fi
RUNNER_NUMBER=$1
TOKEN=$2
RUNNER_DIR="/home/$USER/actions-runner-${RUNNER_NUMBER}"
SERVICE_FILE="/tmp/github-actions-${RUNNER_NUMBER}.service"

# Remove existing directory if exists
rm -rf "$RUNNER_DIR"

# Create fresh runner directory
mkdir -p "$RUNNER_DIR"
cd "$RUNNER_DIR" || exit

# Download and extract runner
curl -o actions-runner-linux-x64-2.321.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz
tar xzf ./actions-runner-linux-x64-2.321.0.tar.gz

# Configure runner
./config.sh --url https://github.com/tesote/treasury \
    --token "$TOKEN" \
    --name "runner-${RUNNER_NUMBER}" \
    --unattended \
    --replace

# Create service file in temp location
cat > "$SERVICE_FILE" << EOL
[Unit]
Description=GitHub Actions Runner ${RUNNER_NUMBER}
After=network.target

[Service]
Type=simple
User=${USER}
Group=${USER}
WorkingDirectory=${RUNNER_DIR}
ExecStart=${RUNNER_DIR}/run.sh
Restart=always
RestartSec=10
KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target
EOL

# Move service file and configure systemd (these need sudo)
sudo mv "$SERVICE_FILE" "/etc/systemd/system/"
sudo systemctl daemon-reload
sudo systemctl enable "github-actions-${RUNNER_NUMBER}"
sudo systemctl start "github-actions-${RUNNER_NUMBER}"

echo "Runner ${RUNNER_NUMBER} has been created and started"

run the script as

./add-runner 1 TOKEN_1
./add-runner 2 TOKEN_2
./add-runner 3 TOKEN_3