Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Remove ingest processor doesn't handle ignore_missing parameter correctly #10066

Closed
gaobinlong opened this issue Sep 15, 2023 · 1 comment
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing untriaged

Comments

@gaobinlong
Copy link
Collaborator

gaobinlong commented Sep 15, 2023

Describe the bug
Same to #9687, by checking all of the ingest processors which support template snippets for the field parameter and have ignore_missing parameter, I found that remove ingest processor also has the same problem with rename ingest processor, when setting field to a template snippet {{field_a}} and set ignore_missing to true, if field_a doesn't exist then remove processor throws illegal_argument_exception rather than exiting quietly.

Only remove and rename processor have that problem, other processors work well.

To Reproduce
Steps to reproduce the behavior:
Request:

POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "_description",
    "processors": [
      {
        "remove": {
          "field": "{{key_xxxx}}",
          "ignore_missing": true
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "key":"foo",
        "foo": "bar=x",
        "target_field":"zoo"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "key":"foo",
        "foo": "rab=x",
        "target_field":"zoo"
      }
    }
  ]
}

Response:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "illegal_argument_exception",
            "reason": "path cannot be null nor empty"
          }
        ],
        "type": "illegal_argument_exception",
        "reason": "path cannot be null nor empty"
      }
    },
    {
      "error": {
        "root_cause": [
          {
            "type": "illegal_argument_exception",
            "reason": "path cannot be null nor empty"
          }
        ],
        "type": "illegal_argument_exception",
        "reason": "path cannot be null nor empty"
      }
    }
  ]
}

Expected behavior
Remove ingest processor can handle ignore_missing parameter correctly, when the resolved template snippet of the field parameter is empty and ignore_missing is true, the processor exits quietly and do nothing.

Host/Environment (please complete the following information):

  • OS: [MacOS]
  • Version [OpenSearch 2.9]
@gaobinlong gaobinlong added bug Something isn't working untriaged labels Sep 15, 2023
@kotwanikunal kotwanikunal added the Indexing Indexing, Bulk Indexing and anything related to indexing label Sep 19, 2023
@gaobinlong
Copy link
Collaborator Author

Closed by #10089.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing untriaged
Projects
None yet
Development

No branches or pull requests

2 participants