Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add low-level expression language support #2826

Merged
merged 2 commits into from
Oct 28, 2024

Conversation

krancour
Copy link
Member

@krancour krancour commented Oct 23, 2024

This PR adds a new expressions package, which currently exports just one useful function -- EvaluateJSONTemplate(). This will take an array of bytes, evaluate them, and return the evaluated bytes.

The idea is that as we continue to build out expression language support in promotion steps, this can be called by an Engine implementation to pre-process the raw config for each step.

This is all accomplished with a combination of fasttemplate and expr-lang, similar to how Argo Workflows does.

Our implementation varies a bit from Argo Workflows' in some key ways.

  1. We use ${{ }} to offset expressions. This was chosen for the sake of being more similar to GitHub Actions, with which many users are already familiar.

  2. We don't process the JSON in one shot because I wished to avoid expressions being used in keys. We unmarshal JSON into a map[string]any and recurse over that, treating any string value containing ${{ as a mini-template.

  3. We can play nicely with all valid JSON types.

    For a template to be valid and thus transmissible as a *apiextensionsv1.JSON, expressions must be enclosed within quotes. If you do not unmarshal the JSON template into a map first and simply evaluate all ${{ }} in the raw JSON, the surrounding quotes will always remain, making it impossible to use expressions to construct values of other JSON types. i.e. You're stuck with string values only.

    By unpacking the template into a map, recusing over it as we do, and inspecting the result, we avoid this limitation. This means that an expression like ${{ 42 }} correctly evaluates as a JSON number and ${{ true }} correctly evaluates as a JSON boolean. In the event that you wish to force an evaluated expression to be treated like a string, you can do something like: ${{ quote(42) }}.

    All of the above should be intuitive to anyone who has worked with YAML before where anything that resembles a number or boolean is inferred to be such unless explicitly quoted.

Intended usage will (after a few follow-ups) be something along these lines:

promotionTemplate:
  spec:
    consts:
      gitRepo: https://github.com/example/repo.git
      imageRepo: example/image
      srcPath: "./src"
      outPath: "./out"
    steps:
    - uses: git-clone
      config:
        repoURL: ${{ consts.gitRepo }}
        checkout:
        - fromFreight: true
          path: ${{ consts.srcPath }}
        - branch: stage/${{ ctx.Stage }}
          create: true
          path: ${{ consts.outPath }}
    - uses: git-clear
      config:
        path: ${{ consts.outPath }}
    - uses: kustomize-set-image
      as: update-image
      config:
        path: ${{ consts.srcPath }}/base
        images:
        - image: ${{ consts.imageRepo }}
    - uses: kustomize-build
      config:
        path: ${{ consts.srcPath }}/stages/${{ ctx.Stage }}
        outPath: ${{ consts.outPath }}
    - uses: git-commit
      as: commit
      config:
        path: ${{ consts.outPath }}
        messageFragments:
        - ${{ outputs.updateImage.commitMessage }}
    - uses: git-push
      config:
        path: ${{ consts.outPath }}
    - uses: argocd-update
      config:
        apps:
        - name: kargo-demo-${{ ctx.Stage }}
          sources:
          - repoURL: ${{ consts.gitRepo }}
            desiredCommit: ${{ outputs.commit.commit }}

An interesting dimension to this is that the promotion template begins to look sufficiently generic that it could be useful in the not-too-distant-future to create a top-level PromotionTemplate CRD (and maybe ClusterPromotionTemplate, too) so Stages that all follow the same generic recipe can reference that recipe instead of repeating it.

Copy link

netlify bot commented Oct 23, 2024

Deploy Preview for docs-kargo-io ready!

Name Link
🔨 Latest commit aaeaafc
🔍 Latest deploy log https://app.netlify.com/sites/docs-kargo-io/deploys/671ff120c701210008be2208
😎 Deploy Preview https://deploy-preview-2826.docs.kargo.io
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Oct 23, 2024

Codecov Report

Attention: Patch coverage is 65.68627% with 35 lines in your changes missing coverage. Please review.

Project coverage is 49.02%. Comparing base (f2ee9a8) to head (aaeaafc).
Report is 22 commits behind head on main.

Files with missing lines Patch % Lines
internal/expressions/json_templates.go 65.68% 26 Missing and 9 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2826      +/-   ##
==========================================
+ Coverage   48.79%   49.02%   +0.23%     
==========================================
  Files         270      271       +1     
  Lines       23962    24010      +48     
==========================================
+ Hits        11692    11772      +80     
+ Misses      11639    11598      -41     
- Partials      631      640       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Kent Rancourt <kent.rancourt@gmail.com>
Copy link
Contributor

@hiddeco hiddeco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like you are off to a great start. I have no real objections to anything, and it quite perfectly matches what I had envisioned the functionality to look like.

My only minor nit based on your "future" example would be to consts -> const, as I can see myself mistyping the former frequently. The fluent API then also becomes nicer, as you can refer to const.myValue.

Also: tiny suggestion attached at the bottom.

internal/expressions/json_templates_test.go Outdated Show resolved Hide resolved
Co-authored-by: Hidde Beydals <hiddeco@users.noreply.github.com>
@krancour krancour merged commit a25dc52 into akuity:main Oct 28, 2024
16 of 17 checks passed
@krancour krancour deleted the krancour/express-yoself branch October 28, 2024 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants