Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid data stream configuration when using dlq_custom_codes option #1135

Closed
ckauf opened this issue May 22, 2023 · 0 comments · Fixed by #1144
Closed

Invalid data stream configuration when using dlq_custom_codes option #1135

ckauf opened this issue May 22, 2023 · 0 comments · Fixed by #1144
Assignees

Comments

@ckauf
Copy link

ckauf commented May 22, 2023

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version)
    8.7.1

  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
    tar.gz

  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
    via CLI

  4. How was the Logstash Plugin installed
    Bundled

JVM (e.g. java -version):
Bundled

OS version (uname -a if on a Unix-like system):
MacOS 13.4

Description of the problem including expected versus actual behavior:
Actual behavior: When adding dlq_custom_codes to an output with data_stream => true the pipeline does not start but instead displays the following error message:

Invalid data stream configuration, the following parameters are not supported: {"dlq_custom_codes"=>[401]}

Expected behavior: Configuring custom codes works with Data Streams

Steps to reproduce:
Create a pipeline config that looks like this:

input {
  generator {}
}

output {
  elasticsearch {
    hosts => ["https://localhost:9200/"]
    user => "elastic"
    password => "test"
    dlq_custom_codes => [401]
    data_stream => "true"
  }
}

Provide logs (if relevant):

[2023-05-22T16:10:21,618][ERROR][logstash.outputs.elasticsearch][main] Invalid data stream configuration, the following parameters are not supported: {"dlq_custom_codes"=>[401]}
[2023-05-22T16:10:21,619][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Invalid data stream configuration: ["dlq_custom_codes"]>, :backtrace=>["/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:76:in `check_enabled_data_stream_config!'", "/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:59:in `check_data_stream_config!'", "/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:32:in `data_stream_config?'", "/Users/user/Documents/elastic/logstash-8.7.1/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.13.1-java/lib/logstash/outputs/elasticsearch.rb:307:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:69:in `register'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'", "org/jruby/RubyArray.java:1865:in `each'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:607:in `maybe_setup_out_plugins'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:249:in `start_workers'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["/Users/user/Documents/elastic/logstash-8.7.1/config/logstash-dlq.conf"], :thread=>"#<Thread:0x245522bb@/Users/user/Documents/elastic/logstash-8.7.1/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants