[Agent] Inject index with sane defaults to configuration #16328
Conversation
|
Pinging @elastic/ingest (Project:fleet) |
There was a problem hiding this comment.
The generated index doesn't use the right format, elastic/kibana#56132
I have added a question for @ruflin, I am not sure what is the behavior we should have if the fields namespace or the dataset is missing. We use default values or we fail some validation?
| output: | ||
| elasticsearch: | ||
| hosts: | ||
| - 127.0.0.1:9200 | ||
| - 127.0.0.1:9300 | ||
| username: elastic | ||
| password: changeme | ||
| api_key: TiNAGG4BaaMdaH1tRfuU:KnR6yE41RrSowb0kQ0HWoA | ||
| ca_sha256: 7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y= |
There was a problem hiding this comment.
it does indeed
| @@ -4,10 +4,13 @@ filebeat: | |||
| paths: | |||
| - /var/log/hello1.log | |||
| - /var/log/hello2.log | |||
| index: filebeat-generic-default | |||
There was a problem hiding this comment.
This is incorrect, it should be:
logs-{dataset}-{namespace}
@ruflin When no dataset and namespace are defined what are the default values for them? The elastic/kibana#56132 doesn't mention it?
There was a problem hiding this comment.
There was a problem hiding this comment.
this issue mentions defaults #15691
for NS it is default for dataset generic so this follows logs-dataset-namespace correctly
There was a problem hiding this comment.
The above should be logs-generic-default.
There was a problem hiding this comment.
I believe I've read it too fast. thanks
| - module: apache | ||
| setting: two | ||
| metricsets: [info] | ||
| index: metricbeat-generic-testing |
There was a problem hiding this comment.
should be metrics-{dataset}-{namespace}
There was a problem hiding this comment.
namespace==testing in this case so i think it is correct.
| fleet: | ||
| kibana_url: https://kibana.mydomain.com:5601 | ||
| ca_hash: 7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y= | ||
| checkin_interval: 5m |
There was a problem hiding this comment.
Something weird with the same space in a few place, I think YAML will strip it?
There was a problem hiding this comment.
this is weird i have a single space here
x-pack/agent/_meta/agent.docker.yml
Outdated
| - enabled: false | ||
| dataset: sample.error | ||
| paths: /var/log/sample/error.log | ||
| - type: metrics/system |
There was a problem hiding this comment.
it should be system/metrics as defined in https://github.com/elastic/beats/blob/feature-ingest/x-pack/agent/docs/agent_configuration_example.yml#L56
| // Apply injects index into input. | ||
| func (r *InjectIndexRule) Apply(ast *AST) error { | ||
| const defaultNamespace = "default" | ||
| const defaultDataset = "generic" |
There was a problem hiding this comment.
++ to have theses constants in the function.
michalpristas
force-pushed
the
agent-new-config
branch
from
0906c4f
to
1d35566
Compare
|
going to test with kibana, either with working UI or manual API calls |
|
created new PR against master here: #16903 |
Migrated from: #15852
Build on top of: #15809
Added one extra rule for injecting index into inputs based on provided type and detected namespace/dataset type.
Other option was to use decorator but at the time of decorating there are already some pieces of config missing.
Draft until final configuration format is settled and mentioned PR is in, but READY for review.
It overwrites index specified in input.
Fixes: #15690
Fixes: #15691