You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Data Prepper opensearch sink can write empty DLQ objects to S3 in certain case.
From what I can tell, this mostly occurs from high-level errors from the _bulk request which do not result in underlying items provided with failures.
To Reproduce
Steps to reproduce the behavior:
Configure an index that you do not have permission to write to.
Run Data Prepper and provide input.
I see the following errors:
2023-11-13T18:09:23,902 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - Bulk Operation Failed. Number of retries 5. Retrying...
2023-11-13T18:09:23,906 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - operation = Index, error = OpenSearch exception [type=authorization_exception, reason=User does not have permissions for the requested resource]
2023-11-13T18:09:23,909 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - operation = Index, error = OpenSearch exception [type=authorization_exception, reason=User does not have permissions for the requested resource]
2023-11-13T18:09:50,577 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - Bulk Operation Failed. Number of retries 10. Retrying...
2023-11-13T18:09:50,579 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - operation = Index, error = OpenSearch exception [type=authorization_exception, reason=User does not have permissions for the requested resource]
2023-11-13T18:09:50,579 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - operation = Index, error = OpenSearch exception [type=authorization_exception, reason=User does not have permissions for the requested resource]
I also got 8 S3 objects in the DLQ. Each looked like:
{"dlqObjects":[]}
Expected behavior
The DLQ should include one item per failed document. It should include the high-level error instead of the individual item error.
Environment (please complete the following information):
Data Prepper main working toward 2.6.0.
Additional context
N/A
The text was updated successfully, but these errors were encountered:
I am getting this error with an Amazon OpenSearch Serverless collection. In particular, I can see it when the role I use has access to the collection, but not the necessary permissions to create an index.
dlvenable
added a commit
to dlvenable/data-prepper
that referenced
this issue
Nov 14, 2023
Describe the bug
The Data Prepper
opensearch
sink can write empty DLQ objects to S3 in certain case.From what I can tell, this mostly occurs from high-level errors from the
_bulk
request which do not result in underlyingitems
provided with failures.To Reproduce
Steps to reproduce the behavior:
Configure an index that you do not have permission to write to.
Run Data Prepper and provide input.
I see the following errors:
I also got 8 S3 objects in the DLQ. Each looked like:
Expected behavior
The DLQ should include one item per failed document. It should include the high-level error instead of the individual item error.
Environment (please complete the following information):
Data Prepper
main
working toward 2.6.0.Additional context
N/A
The text was updated successfully, but these errors were encountered: