Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pebble filters applied to JSON inputs produce non-JSON strings #4340

Closed
Tracked by #4656
aku opened this issue Jul 16, 2024 · 1 comment · Fixed by #4979
Closed
Tracked by #4656

Pebble filters applied to JSON inputs produce non-JSON strings #4340

aku opened this issue Jul 16, 2024 · 1 comment · Fixed by #4979
Assignees
Labels
bug Something isn't working

Comments

@aku
Copy link

aku commented Jul 16, 2024

Describe the issue

If you have a JSON field, e.g.

inputs:
  - id: payload
    type: JSON
    defaults: |-
      [{"name": "single quote in string '"}]

and want to apply pebble filters such as replace or escapeChar the resulting string will loose JSON formatting.

"{{ inputs.payload | escapeChar('single') }}" -> [{name=single quote in string \'}]

if you do the same against STRING field it will produce expected result [{"name":"single quote in string \'"}]

I.e. when you use pebble filters with JSON it gets serialized to string which is non-json compliant

Context and examples: https://kestra-io.slack.com/archives/C03FQKXRK3K/p1720800171410189

Example:

id: json_vs_string_filtering
namespace: dev
inputs:
  - id: payload_json
    type: JSON
    defaults: |-
      [{"name": "single quote in string '"}]
  - id: payload_string
    type: STRING
    defaults: "[{\"name\":\"single quote\" in string '}]"
tasks:

  - id: debug
    type: io.kestra.plugin.core.debug.Echo
    format: "{{ inputs.payload_json | escapeChar('single') }} \n {{ inputs.payload_string | escapeChar('single') }}"

Environment

  • Kestra Version: 17.8
  • Operating System (OS/Docker/Kubernetes):
  • Java Version (if you don't run kestra in Docker):
@aku aku added the bug Something isn't working label Jul 16, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Issues Jul 16, 2024
@aku aku changed the title Pebble filters against JSON inputs produce non-JSON strings Pebble filters applied against JSON inputs produce non-JSON strings Jul 16, 2024
@aku aku changed the title Pebble filters applied against JSON inputs produce non-JSON strings Pebble filters applied to JSON inputs produce non-JSON strings Jul 19, 2024
@smunteankestra
Copy link
Contributor

reproducing, will be prioritised in 0.19.0 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
4 participants