[self-hosted] A hook that is fired before job starts

[gajus]

We are using the self-hosted runners and we would like to perform various clean up jobs before every job, e.g. cleaning up the workdir. Is there any hook / event that we could subscribe to?

[ethomson]

Yeah, I can understand where you're coming from - do you have other work that you want to perform in this hook, or is it really just cleaning up the workdir? I ask because maybe there's an option to always clean the workdir instead of always running a hook. 🤔

[omsmith]

Not affiliated with @gajus, but I believe a pre-job hook would work well for our organization as well.

Our runners are ephemeral, launched into a "warm pool" of registered runners via an AWS Auto Scaling Group. We don't need to do any cleanup, but would use the hook to detach the instance from the pool once it picks up a job so that it can be replaced by a new one.

Currently we do this with a background loop checking for the worker process (as below), but a supported --before-job-script ./detach.sh or similar would certainly be preferable!

while true; do
  WORKER_PROC=$( pgrep -P "$(pgrep Runner.Listener)" Runner.Worker )

  if [[ -n "$WORKER_PROC" ]]; then
    echo "Detaching from ASG..."

    INSTANCE_ID=$(ec2metadata --instance-id)

    ASG_NAME=$( aws autoscaling describe-auto-scaling-instances \
      --instance-ids "$INSTANCE_ID" \
      --query 'AutoScalingInstances[0].AutoScalingGroupName' --output text )

    aws autoscaling detach-instances \
      --instance-ids "$INSTANCE_ID" \
      --auto-scaling-group-name "$ASG_NAME" \
      --no-should-decrement-desired-capacity

    break
  fi

  sleep 2
done

[gajus]

We ended up tapping into logs.

[gajus]

Yeah, I can understand where you're coming from - do you have other work that you want to perform in this hook, or is it really just cleaning up the workdir? I ask because maybe there's an option to always clean the workdir instead of always running a hook. 🤔

Sorry, I just noticed the question –

It's mostly clean up, but we are now considering pre-populating directory with repository specific build tools too.

The approach of tapping into logs works Okay, but it is super hacky.

[gajus]

@ethomson An alternative that I would really appreciate is if there was a way to prepend GitHub Actions steps to every workflow running on the machine.

[asangas]

We have the exact same requirement, where we need to ensure a runner is in a clean state before a new job can start execution.

Perhaps a post-job hook would be slightly better for this use case since the new job does not have to wait for cleanup?

@ethomson would you be able to elaborate on the alternative option that you have mentioned?

[tyrken]

I have a PR for this already at #1469 which we're using to get improved docker caching (so needs to run before the runner starts doing any docker stuff like pulling/building action images). Would love to get some feedback from GH devs @ethomson

[jmendiara]

Cleanup (for root owned files) is our use case.

What is that option, @ethomson ?

Yeah, I can understand where you're coming from - do you have other work that you want to perform in this hook, or is it really just cleaning up the workdir? I ask because maybe there's an option to always clean the workdir instead of always running a hook. 🤔

[thboop]

We recently published an ADR for Job Started / Job Completed hooks for self hosted runners, feel free to provide your feedback.

In particular we would love to hear what (if anything ) else you would need to support your use case, and if the interface makes sense for you.

[thboop]

Closing this as a dupe of #699