Support the secret context in a reusable workflow input

[toast-gear]

I'm actually shocked I've got to raise a feature request for something as basic as this for reusable workflows. As it stands for our organisation reusable workflows are almost entirely useless.

One of the main use cases for reusable workflows is to provide a centralised definition of a generic workflow, for example a something that invovles a docker build. As a result, a need for dynamism around inputs is critical to even entertain using a reusable workflow e.g. an output from a job, a environment variable. Without the ability to do this the reusable workflow must be extremely rigid.

Take this extremely stripped down basic example, in this scenario I am letting people define the build arguments as an input:

# some reusable workflow
on:
  workflow_call:
    inputs:
      docker_build_arguments:
        required: true
        type: string

jobs:
  build:
    runs-on: self-hosted
    steps:
      - name: Build
        uses: docker/build-push-action@v2
        with:
          context: .
          file: Dockerfile
          push: false
          build-args: |
            ${{ inputs.docker_build_arguments }}
          tags: |
             image:latest

If I consume this reusable workflow like below I can't do anything around setting the values in the key-value pairs for the docker_build_arguments input like below:

# some workflow consuming the reusable workflow
jobs:          
  job2:
    uses: owner/repo/.github/workflows/my-workflow.yaml@master
    with:
      docker_build_arguments: |
        BUNDLE_GITHUB__COM=${{ secrets.FOO }}
        BUNDLE_RUBYGEMS__PKG__GITHUB__COM=${{ env.BAR }}
        BRANCH_NAME=${{ needs.job1.outputs.BAZ }}
        SOURCE_CODE_VERSION=${{ github.head_ref }}

I am honestly at a loss as how anyone is using reusable workflows with such a limitation. I hope I am missing something as Jenkins can do better than this and it's a decade old.

I'm aware you can pass in secrets via the dedicated key or just implicitly inherit all accessible secrets which allows some aspect to be worked around but then actually using that said results in:

• a complicated workflow as you end up with lots of conditionals and duplication just to be able to account for potential secrets to subsequently map them to their corresponding key
• having to bump your reusable workflow to introduce support for some new secret
  If you can't use them directly like I've shown as an example at least let me process in an initial job and then pass them in as outputs or environment variables
• completely prevents generic key-value inputs as your reusable workflow needs to hard code up front what key will go with what value (the secret)
• a still very rigid reusable workflow making it ... not very reusable

Reusable workflows as it stands only work for a predefined extremely rigid flow which, whilst is a use case and my org certainly has pipelines that fit within that limited scope, makes the reusable feature offering of incredibly limited value and hard to justify opting for GitHub Actions as a CI/CD platform.

[toast-gear]

output is supported, the docs make it very unclear. The inability to pass in secrets via an input like in composite actions is atrocious (you're allowed in 1 abstraction but not in another?!?!) but it's not quite as limited as I first thought

[HariSekhon]

I worked around this problem for a similar docker build reusable workflow.

I used a job to define outputs and tested conditionals on those, but I totally agree that env and secrets should be supported in conditionals.

You can see my solution which is working in a similar reusable workflow here:

https://github.com/HariSekhon/GitHub-Actions/blob/master/.github/workflows/docker_build.yaml

[toast-gear]

Yeh job outputs solve some of the problems but the lack of "conventional" (the same as composite actions) secret support really limits how reusable reusable workflows are. With job outputs you can expose all the other contexts e.g. the github context (it's a bit of a pain as you have to use an additional runner in a separate job to set-up everything and output it for consumption but it works) but the way GitHub have implemented secret support is mental (on the surface at least, perhaps there are great reasons for this at the infrastructure level but that doesn't really help me as a consumer). You can kinda do workarounds but not if that secret is part of a key value pair and you don't want to pre-define what that key will be, you want the consumer to be able ot define that as part of their use of the reusable workflow.

What concerns me most is that the way secrets work atm aren't listed as a limitation making me think this is just how it's going to work for a long long time

I'll update the title to better reflect the real problem (secrets although all the contexts would be great)

[Teakowa]

That's what should be right. We compilation process requires variables that aren't exactly consistent, and are too cumbersome for a CI/CD platform if they can't be easily passed by the caller.

Many of our projects are using reusable workflows, but they are only used to regulate the workflow of each repository execution, not quite at 100% reusable, It's actually true that it's not exactly a reusable workflow.

Although GitHub Action is already much more convenient and simple than Jenkins and GitLab CI, I still wish it could be better ❤️

[glloyd2010f]

I would say that composite actions and and reusable workflows are two features that need to have the same functionality.

The biggest two are pass workflow env vars to the with block and being able leverage a secrets block when calling from the parent workflow.

[crazy-matt]

I would add being able to call its own secrets from the called workflow within the called workflow. At the moment it seems we can pass all secrets from caller to called (secrets: inherit) or more granularly passing the secrets you want to use. But you can't use the called workflow context secrets themselves.
I do understand the need for security, but there are some use cases where the reusable workflow doesn't do anything critical and you trust your teams internally. Why Github being so opinionated?

I found a very clear description of the problem here. So the product team took the wrong decision in my humble opinion. If the idea is to avoid confusion, then the Organization secrets should be emptied too within the called workflow? (unless it's already the case, I haven't tested it)

[badbort]

Encountered the same limitation and truly baffled why its not possible. Reusable workflows seem so rigid and painful at times. I think I understand they want to limit what secrets the reusable workflow has access to, but this use case isnt uncommon.

I have a reusable workflow that runs terraform with arguments. There's an input that lets you define custom arguments, e.g. for setting -var and -var-file, but if I want to provide it with a value from a github repo/org secret, I'm out of luck. This particular variable I'm passing in is product specific, and thus doesn't make sense to introduce a reusable workflow secret. It should be flexible to other use cases

I've tried various approaches including:

• using needs.<job>.output.<my output> - fail: Skip output 'tf-args' since it may contain secret.
• Above but b64 encoding the secret - fail: Skip output 'tf-args' since it may contain secret.

# reusabale workflow
on:
  workflow_call:
    inputs:
      username:
        required: true
        type: string
        
    secrets:
      SECRET_TF_OPTIONS_PLAN:
        description: "Optional tf args for the tf plan that may contains secrest"
        required: false
      SECRET_TF_OPTIONS_APPLY:
        description: "Optional tf args for the tf apply that may contains secrest"
        required: false

jobs:
  blah:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - run: |        
        echo "Hello ${{inputs.username}} ${{secrets.SECRET_TF_OPTIONS_PLAN}}"

# main workflow
on:
  workflow_dispatch:
        
jobs:
  call-workflow:
    uses: bortington/github-common/.github/workflows/test-reusable-workflow.yml@main
    with:
      username: Bert
    secrets:
      SECRET_TF_OPTIONS_PLAN: -var "beep=${{secrets.NOT_A_SECRET}}"

This works, but my secret is leaked and is not masked at all:

[rdhar]

Curiously, I'm in a very similar position trying to run Terraform commands+arguments in a reusable workflow, while passing credentials/secret values from the caller workflow. Eventually came across a solution here which meets the requirements.

These methods are sourced from DevSecTop/TF-via-PR (permalink) repository, which hosts a reusable workflow to run Terraform commands via PR comments, like a CLI.

As a bonus, any number of secrets can be securely passed into the reusable workflow to be used as environment variables, for example. The repository also contains recent GitHub Actions workflow runs to verify that the secrets remain masked throughout.