Feat/add json process tool

[scotlandowl]

Description

The existing template-transform node is quite complex when it comes to processing JSON data. For example, to extract just the URL from {"result": "{"url": "https://github.com/langgenius/dify"}\n"}, the complex code {{ result | replace('\n','') | replace('{"url": "','') | replace('"}','') }} needs to be used. To address this, a tool that uses jsonpath-ng for JSON processing has been introduced. This tool can parse, insert, delete and replace JSON content using simple instructions, greatly improving the ease of handling JSON. For the above example, simply inputting 'url' in json_filter is enough to obtain "https://github.com/langgenius/dify". This tool has significantly enhanced the convenience of JSON processing.

20240624_214703.mp4

parse

Given a JSON content and a json_filter, return the parsed result.
When the parsed result is a JSON structure, return it in JSON format. In other cases such as strings, numbers, etc., return a string.

insert

Given a JSON content, a query path, and a new_value:

• If the query path exists:
  1. If the matched item is a dictionary, update an existing key-value pair or add a new one.
  2. If the matched item is a list, insert the new_value at the specified index, or append it to the end.
  3. For other types, directly replace the original value with the new_value.
• If the query path does not exist and create_path is true, create the new path and insert the new_value.
• Otherwise, do not make any changes.

delete

Given a JSON content and query. Based on the type of the query match, perform different deletion operations:

• For dictionaries, delete the specified key.
• For lists, remove the specified element.
• For other cases, try to delete the corresponding key in the parent element.

replace

Given the JSON content, query, replace_value, and replace_model:

• If replace_model is "pattern", for each result matching the query, replace the replace_pattern in the input string with replace_value.
• If replace_model is "key", replace the key of each match with replace_key.
• If replace_model is "value", replace the value of each match with replace_value.

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update, included: Dify Document
• Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
• Dependency upgrade

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• TODO

Suggested Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
• optional I have made corresponding changes to the documentation
• optional I have added tests that prove my fix is effective or that my feature works
• optional New and existing unit tests pass locally with my changes

[crazywoola]

Please fix the errors in CI, and make sure the commit messages are in English.

[scotlandowl]

Please fix the errors in CI, and make sure the commit messages are in English.

The python and yaml styles have been fixed.

[scotlandowl]

@Yeuoly I have submitted the requested changes. Please review when you have a moment. Thank you.

[Yeuoly]

hello, pls rebase to main to fix conflicts.

[scotlandowl]

hello, pls rebase to main to fix conflicts.

The branch is now up-to-date with the main branch and includes the latest dependency updates.

[scotlandowl]

@Yeuoly hello, the python style error in /tools/insert.py has been fixed.

[Yeuoly]

LGTM, Thanks for your amazing tools!

[scotlandowl]

LGTM, Thanks for your amazing tools!

Thank you. And I want to ask if I need to resolve conflicts all the time？

[Yeuoly]

LGTM