Passing Environment Variables to Reusable Workflow

[michaely-ln]

It’s my understanding that the environment from main workflow does not extend to the reusable workflow it calls. When I dump the env from both jobs to the stdout, indeed, the env variables I set in the main job do not appear in the env of the reusable workflow I have called.

When I pass a variable other than an ‘env’ variable from the main workflow to the reusable workflow with the following syntax (an expression), it works:

  with:
    my-variable: ${{ needs.MyJobId.outputs.MyVariable }}
    ref: ${{ github.GITHUB_REF_NAME }}

When I attempt to do this with an environment variable I get an error:
Unrecognized named-value: 'env’
Here is the offending syntax:

  with:
    my-variable: ${{ env.MY_VARIABLE }}

So ‘with’ doesn’t like to evaluate the ‘env’ context in an expression.

I’ve tried bash (I’m on an ubuntu runner) notation which doesn’t work, either:

  with:
    $MY_VARIABLE

I’ve tried about everything I can think of including stupid things like quotes, using github context, etc. What is the magic cookie I need here?

[jsoref]

I don’t think you can use env, the problem is that env is set for stages based on the nested thing and that’s firewalled off in this case. (It needs to know what kind of runner to live in amongst other things.)

You could have a step that converts an env item into an output and then use the step output was input to the reusable workflow.

There are other similar restrictions, I tripped on not being able to use env at the job level (in an if). It’s the same restriction/problem. I could either replicate the if for each step, or I could make a step with it and somehow use that step.

[michaely-ln]

Yeah, I’ve dealt with the env scope problem and used outputs to get around it. I just thought this was within scope since I set the env value at the workflow level and the “With” is down in a step. Unless someone has an official position on it, I’m going to submit a feature request and see if it goes anywhere.

I think the problem is specific to reusable workflows, but I’ll test.

[pavel-kostromitinov_wartsila]

Can you please provide an example how did you do this? I cannot figure it out for hours already.
I tried creating a separate job setting its output to needed value - but apparently I cannot reference ‘jobs’ as well when calling reusable workflow…

[jsoref]

https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability

sorry, it’s job not jobs.

It seems like there may be a typo in their documentation.

[michaely-ln]

name: Project Specific Workflow
on:
  push:
    branches: [<which branches to run on>]
  pull_request:
    branches: [<which branches to run on>]
  workflow_dispatch:
    # Defaults here are for worflow_dispatch only, all other events are set in EnvSetup.setup-output-defaults:
    inputs:
      build-and-verify:
        description: 'Do a build and verify'
        required: true
        default: 'true'    
jobs:
  EnvSetup:
    name: Setup Dynamic Environment Variables
    runs-on: <your-awesome-runner>
    outputs:
      build-and-verify: ${{ steps.set-output-defaults.outputs.build-and-verify }}
    steps:
      - name: set outputs with default values
        id: set-output-defaults
        run: |    
          # If workflow_dispatch, use inputs (left), if other trigger, use default env (right)
          echo "::set-output name=build-and-verify::${{ github.event.inputs.build-and-verify || 'true' }}"
  invoke-workflow:
    needs: [EnvSetup]
    uses: <my-awesome-org>/<my-awesome-repo>/.github/workflows/<my-resuable-workflow>@<the-branch-to-invoke>
    with:
      build-and-verify: ${{needs.EnvSetup.outputs.build-and-verify}}
    secrets:
      token: "${{ secrets.PERSONAL_ACCESS_TOKEN }}"

Some important things to note about this code snippet that calls a reusable workflow:

1. The “uses” statement that calls the reusable workflow can’t evaluate expressions yet. That means the repo name, branch, etc. must be hardcoded.
2. The “with” statement that supplies arguments can only take elements from the github context and the “needs” object, so you MUST have a line that “needs” the job where you set the values and return the outputs.
3. the echo "::set-output name=::${{}} syntax is where the variable is set and exposed as a step output, then it is returned with the job output that references it. Pay attention to the names of each object to see how they fit together.
4. Because you can’t pass ENV variables to the reusable workflow, they are almost useless in this pattern.
5. I’m hoping GitHub comes up with a better way to assign values inline or pass them into reusable workflows, because handling each parameter three or more times just to pass it into a reusable workflow is cumbersome.
6. I included some extra trimmings to show how to pass secrets to the reusable workflow and evaluate the default vs. an input inline during variable assignment. The problem is that your default from the “inputs” section of “workflow_dispatch” only gets assigned on workflow_dispatch, and not on any other trigger. So the ${{ $var || value }} syntax lets you assign the value on the right if the one on the left is empty.

[azamfir]

Acording to https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

echo "::set-output name=build-and-verify::${{ github.event.inputs.build-and-verify || 'true' }}"

will become

echo "build-and-verify=${{ github.event.inputs.build-and-verify || 'true' }}" >> $GITHUB_OUTPUT

but let's hope that an official workaround will be provided by then :)

[vitalykarasik]

4. Because you can’t pass ENV variables to the reusable workflow, they are almost useless in this pattern.
5. I’m hoping GitHub comes up with a better way to assign values inline or pass them into reusable workflows, because handling each parameter three or more times just to pass it into a reusable workflow is cumbersome.
   @michaely-ln - many thanks!
   But I really hope that Github will add a feature for using reusable workflows with dynamic parameters in a simple way.

[jsoref]

You can use toJSON and fromJSON to serialize a larger set of things.

name: Project Specific Workflow
on:
  push:
    branches: [<which branches to run on>]
  pull_request:
    branches: [<which branches to run on>]
  workflow_dispatch:
    # Defaults here are for workflow_dispatch only, all other events are set in EnvSetup.setup-output-defaults:
    inputs:
      build-and-verify:
        description: 'Do a build and verify'
        required: true
        default: 'true'    
      something:
        description: 'Some thing'
        required: true
        default: else

jobs:
  EnvSetup:
    name: Setup Dynamic Environment Variables
    runs-on: <your-awesome-runner>
    outputs:
      wrapped: ${{ steps.set-output-defaults.outputs.as-json }}
    steps:
      - name: set outputs with default values
        id: set-output-defaults
        env:
          inputs: ${{ github.event.inputs != null && toJSON(github.event.inputs) || '{"build-and-verify":"true", "something":"else"}' }}
        run: |    
          # If workflow_dispatch, use inputs (left), if other trigger, use default env (right)
          echo "as-json=$inputs" >> $GITHUB_OUTPUT

  invoke-workflow:
    needs: [EnvSetup]
    uses: <my-awesome-org>/<my-awesome-repo>/.github/workflows/<my-resuable-workflow>@<the-branch-to-invoke>
    with:
      env-variables-as-json: ${{needs.EnvSetup.outputs.wrapped}}
    secrets:
      token: "${{ secrets.PERSONAL_ACCESS_TOKEN }}"

[justinmchase]

The toJson approach does not work for me because my org has so many secrets/vars in it that it causes a string max length error when trying to pass it as input or output.

This hack is not a reliable solution, we need a way to dynamically resolve variables inside the workflow, in the context of the environment without having to trigger multiple deployments.

[robertDespegar]

Hi there.

I know that the following fix doesn't cover all cases, but in case of "local" called workflows, I just applied this:

jobs:
  someJob:
    uses: ./.github/workflows/workflow_x.yml

being workflow_x.yml a yml file existing in the current branch and it just worked for me.

I mean I did't need to apply:
uses: current_organization/current_repository/.github/workflows/workflow_x.yml

[timharris777]

Has anyone opened an issue for this? This seems like a no brainer feature/bug fix

[renny-oh]

would this relevant ? actions/toolkit#931

[jakehilton]

I found something like this to work for me. Nothing fancy just exposes an env var to the "with" block. Of course you could add multiple output values for more env vars.

jobs:
  vars:
    runs-on: ubuntu-22.04
    outputs:
      user: ${{ env.USER }}
    steps:
      - run: echo "Exposing env vars"

  invoke-workflow:
    needs: vars
    name: "Workflow that needs env vars"
    uses: <my-awesome-org>/<my-awesome-repo>/.github/workflows/<my-resuable-workflow>@<the-branch-to-invoke>
    with:
      user: ${{ needs.vars.outputs.user }}

[romikoops]

Everything is good, but Github Actions charges 1 extra minute of time for 1 sec of work with this workaround :(

[julienbonastre]

Micro$oft are struggling though for money, every cent counts, and in this case: every second 🤭

Get Rich or Die Tryin'

[paulogodinhoaq]

Yes, I need another workaround for something that should be a no-brainer, had to do the same to use env/inputs with runs-on

It saddens me to see stuff like this, sometimes I think Github Actions is run by a skeleton crew.

[hcastrofactored]

@github-staff Is this on the roadmap? It would be beneficial to avoid running these workarounds and be able to leverage reusable workflows with environments.

[andrea-cassioli-maersk]

This is one more time a weird an not intuitive way github actions are setup. Any person would expect to be able to use an env variable as an input of a reusable workflow, but instead there is no clean way to do it....

... and you get a weird error message as well!

In the docs

https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations

it is surely stated, but what is the alternative?

Please fix this.

[sanjacob]

You expect this from the people who make Windows? 😆

[janosh]

Can we please have the ability to pass env to reusable workflows? I can't re-use workflows on ~50% of my repos because of this one limitation!

[paulogodinhoaq]

Inject the env in a output, that is how we all doing, it is ridiculous, but works :(

[mattdavis0351]

even with injecting the env in an output the problem isn't solved. Sure you can pass around inputs from that injection but if you rely on a lot of secrets you still can't use secrets: inherit.

[reed-hanger]

This has also caused a lot of issues and a several weeks delay in implementing reusable workflows for my company. The workarounds here only get us so far. When we need to add an arbitrary number of secrets to the env of the reusable workflow from the caller workflow, options are really limited, and with those limited options there are a lot of drawbacks. We were able to mostly work around this by passing a map of secrets in a JSON string to another secret then let the reusable workflow process those. However, that loses the automatic secret masking inside the reusable workflow (since secrets are no longer inherited), and we have to manually re-mask secret values. Even with that, we've run into the issue where passing toJson(secrets) causes { and } to be masked, and multiline secrets are problematic.

Injecting the secrets as an output doesn't work for us since we run the reusable worfklow in a matrix of GitHub Environments, and each environment has its own environment secrets.

Even if we just had more GitHub expressions available that could run a substring, replace, or convert objects to a non-prettified JSON would greatly help.

[yash-dxt]

I guess the use of ::set-output is now deprecated

src: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

so - the accepted answer might not work...

[oconstantin]

The solution for that is in this comment above.

[BlazeIsClone]

I hope GitHub has planned a method like secrets: inherit to pass through environment variables as well.

[rdhar]

BACKGROUND CONTEXT

Although there are now docs on masking and passing secrets between jobs or workflows (as @reed-hanger linked), it didn't meet requirements like:

• Storing and retrieving from a secret store seems a tad over-the-top (as @jonashartwig.
  • The same goes for encrypted artifact upload/download.
• For ephemeral, temporary OIDC tokens, it's not feasible to dynamically store/retrieve those secrets (as @jnahmias raised).
• Passing any N number of variables as secrets in bulk, since secrets: inherit scope doesn't work in this context (as @mattdavis0351, @janosh, and @BlazeIsClone brought up).

PROBLEM STATEMENTS

With these factors in mind, the main blockers are:

• Per docs, to prevent accidental disclosures, GitHub Actions redacts any configured secrets as well as common encodings of those values, like Base64.
• For dynamically-generated secrets, those have to remain masked from the caller workflow to the reusable workflow to avoid exposing their their values in the workflow logs.

PROPOSED SOLUTION

1. Before going to output secrets from the caller workflow, Base64 encode the values twice.
   1. This enables GitHub Actions to output the secrets without exposing them.
2. Per docs, pass the encoded values to the reusable workflow as secret inputs.
   1. This enables the reusable workflow accept and treat the encoded values as secrets.
3. In the reusable workflow, decode the values from Base64 twice before masking them.
   1. This ensures that the secrets remain masked in the workflow logs throughout.
   2. They can now be populated as environment variables for use in the subsequent steps of the reusable workflow.

WORKING EXAMPLE

These methods are sourced from DevSecTop/TF-via-PR (permalink) repository, which hosts a reusable workflow to run Terraform commands via PR comments, like a CLI.

As a bonus, any number of secrets can be securely passed into the reusable workflow to be used as environment variables, for example. The repository also contains recent GitHub Actions workflow runs to verify that the secrets remain masked throughout.

# caller-workflow.yml

jobs:
  credentials:
    runs-on: ubuntu-latest
    outputs:
      CREDENTIAL1: ${{ steps.credentials.outputs.CREDENTIAL1 }}
      CREDENTIAL2: ${{ steps.credentials.outputs.CREDENTIAL2 }}
    steps:
      - name: Output encoded credentials
        id: credentials
        env:
          CREDENTIAL1: ${{ secrets.CREDENTIAL1 }}
          CREDENTIAL2: ${{ secrets.CREDENTIAL2 }}
        run: |
          echo "CREDENTIAL1=$(echo $CREDENTIAL1 | base64 -w0 | base64 -w0)" >> $GITHUB_OUTPUT
          echo "CREDENTIAL2=$(echo $CREDENTIAL2 | base64 -w0 | base64 -w0)" >> $GITHUB_OUTPUT

  reusable-workflow:
    needs: credentials
    uses: reusable-workflow.yml
    secrets:
      env_vars: |
        CREDENTIAL1=${{ needs.credentials.outputs.CREDENTIAL1 }}
        CREDENTIAL2=${{ needs.credentials.outputs.CREDENTIAL2 }}

# reusable-workflow.yml

on:
  workflow_call:
    secrets:
      env_vars:
        required: true
jobs:
  parse-credentials:
    runs-on: ubuntu-latest
    env:
      env_vars: ${{ secrets.env_vars }}
    steps:
      - name: Decode credentials as environment variables
        run: |
          for i in $env_vars; do
            i=$(echo $i | sed 's/=.*//g')=$(echo ${i#*=} | base64 -di | base64 -di)
            echo ::add-mask::${i#*=}
            printf '%s\n' "$i" >> $GITHUB_ENV
          done
      - name: Validate credentials
        run: |
          # Secrets are now available as masked environment variable.
          echo $CREDENTIAL1 # or ${{ env.CREDENTIAL1 }}
          echo $CREDENTIAL2 # or ${{ env.CREDENTIAL2 }}

Where:

• base64 -w0: Ensures that the encoded values are outputted in a single line, regardless of the length of the input.
• base64 -di: Decodes the values from Base64, ignoring any newlines from echo'd strings.
• echo ${i#*=}: Removes the key of the key-value pair using parameter expansion, leaving only the value.
• sed 's/=.*//g': Removes the value of the key-value pair, leaving only the key.
• echo ::add-mask::${i#*=}: Masks the value of the key-value pair without outputting.

FLAWS

• If the workflow is (re-)run in debug mode, then secrets from credentials job specifically are output in the workflow log.
• The reusable workflow needs to be configured with the steps under parse-credentials job.

[stempler]

@rdhar Thanks a lot, this is quite useful.

I had a problem when there is a space in one of the values.
To also support that case I needed to replace printf '%s\n' $i >> $GITHUB_ENV with printf '%s\n' "$i" >> $GITHUB_ENV. Maybe that helps others that run into the same issue.

[rdhar]

@stempler That's a great call-out, thank you! I've amended the snippet accordingly as well as appended a little "Flaws" section to ensure others are aware of the limitations for this unofficial workaround.

[dupski]

This is really dumb and the workaround with the extra "set environment variables" step and double base64 encoding is nasty.

Has anyone raised an issue for this?!

All I need to be able to do is use "environment" and "uses" together to call my reusable workflow, and pass some vars.abc and secrets.xyz parameters as inputs. I can't see a good reason why this shouldn't be allowed?

[boxrick]

I have similar problems and have come up with a workaround.

My aim is to get inside of my re-usable workflow with a number of defined 'Environment Variables' ready for consumption by things like Terraform I do not wish to define every one.

After messing about, the best way is to define Repository Variables ( This also works for 'environment variables' )

Then after this iterate overall the variables, format them correctly and echo them into $GITHUB_ENV for usage in subsequent steps. The big advantage here is not needing to define every environment variable. They all just get slurped up and updated.

The below is the reusable workflow

      - name: Loading repository variables to real env vars
        id: environment
        run: |
          ## Iterate over the input variables
          ## output them in a format so we can run and update the env variables 
          ## making use of $GITHUB_ENV
          echo "$ALLMYVARS" | jq -r 'keys[] as $k | "echo \"\($k)=\(.[$k] )\" >> $GITHUB_ENV"' > ./update_env.sh
          chmod +x ./update_env.sh
          ./update_env.sh
        shell: bash
        env:
          ALLMYVARS: ${{ toJSON(vars) }}
          
      - name: Output testing
         id: output
         run: |
           echo "Dynamic Var ${{ env.VAR1 }}"
           echo "Dynamic Var ${ VAR1 }"

Hope this helps someone!

[abd-rehman-sid]

2 years 2 months since this thread. Do we have a feature request?

[Erica-West]

any update?? a simplistic solution would be nice

[doublethink13]

When possible, use a composite action instead of reusable workflows.

That being said, we should be able to define a reusable workflow env from the workflow that is calling the reusable workflow.

[gtjoseph]

When possible, use a composite action instead of reusable workflows.

Unfortunately, there are things you can do from a reusable workflow that you can't do from a composite action. Creating and using service containers being one of them.

[doublethink13]

But you can create a job, create a service container, then use that service container in the composite action that is called in one of the job's steps. I think, haven't tested 👼

[gtjoseph]

We tried that but unfortunately calling a composite action from a job that creates a service container seems to break the link to the service container so any docker containers the composite action may start can't use the service container's name as a hostname.

[jsoref]

Unfortunately, that feels like a (desirable) security boundary.

Depending on what you're doing, you could probably stitch together a "preamble" action and "reliant" action -- that'd at least allow you to "reuse" the code from your "preamble" action in multiple "reliant" actions instead of having to maintain n copies of the "preamble".

[doublethink13]

If you create an action called test-debian with these three files:

# Dockerfile
FROM debian:bookworm-20240423-slim

RUN apt update && apt install curl jq --yes

COPY entrypoint.sh /scripts/entrypoint.sh

ENTRYPOINT ["/scripts/entrypoint.sh"]

# action.yml
description: test curling from container to service

inputs:
  http-bin-url:
    description: http bin url to query
    required: true

name: test-debian

outputs:
  response:
    description: response from http bin

runs:
  args:
    - ${{ inputs.http-bin-url }}
  image: Dockerfile
  using: docker

# entrypoint.sh

#!/bin/bash

export HTTP_BIN_URL="$1"

response=$(curl -s $HTTP_BIN_URL/get | jq .headers.Host)

echo "response=$response"
echo "response=$response" >> "$GITHUB_OUTPUT"

And then reuse the test-debian action in this workflow:

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      httpbin:
        image: kennethreitz/httpbin
        ports:
          - 80:80
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - id: test-debian
        uses: ./.github/actions/test-debian
        with:
          http-bin-url: http://httpbin
      - run: echo "${{ steps.test-debian.outputs.response }}"

name: test

on: push

You should see httpbin echoed in the last step.

Using this composite action instead:

description: test curling from composite action

inputs:
  http-bin-url:
    description: http bin url to query
    required: true

name: test-debian

outputs:
  response:
    description: response from http bin
    value: ${{ steps.response.outputs.response }}

runs:
  steps:
    - id: response
      run: |
        response=$(curl ${{ inputs.http-bin-url }}/get | jq .headers.Host)

        echo "response=$response" >> "$GITHUB_OUTPUT"
      shell: bash
  using: composite

Needs a workflow like this:

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      httpbin:
        image: kennethreitz/httpbin
        ports:
          - 80:80
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - id: test-debian
        uses: ./.github/actions/test-debian
        with:
          http-bin-url: http://127.0.0.1
      - run: echo "${{ steps.test-debian.outputs.response }}"

name: test

on: push

Note that http-bin-url changed from httpbin (docker action) to 127.0.0.1 (composite action).

Docs explain it better than me:

• https://docs.github.com/en/actions/using-containerized-services/about-service-containers#running-jobs-in-a-container
• https://docs.github.com/en/actions/using-containerized-services/about-service-containers#running-jobs-on-the-runner-machine

[lbornov2paymentology]

+1 for this feature request.

[qtnx]

Migrated from GitLab CI, but this feature has still not been available for over 2 years. What a disappointment.

[WilliamWatsonEAI]

Where is this feature? Surely this is required by so many different projects...

[ABOBAKAR-IT]

on:
push:
branches:
- main

jobs:
deploy:
name: Deploy to EC2
runs-on: ubuntu-latest
env:
VITE_API_BASE_URL: ${{ secrets.VITE_API_BASE_URL }}

steps:
  - name: Checkout Repository
    uses: actions/checkout@v4

  - name: Setup Node.js
    uses: actions/setup-node@v4
    with:
      node-version: '20'
  


  - name: Install Dependencies
    run: npm install

  - name: env setup
    run: | 
      export VITE_API_BASE_URL=${{secrets.VITE_API_BASE_URL}} 

  - name: Build Project
    run: npm run build