Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Manager] New structure of agent configuration #19128

Merged
merged 10 commits into from
Jun 17, 2020
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
phase 2
  • Loading branch information
michalpristas committed Jun 11, 2020
commit a044e9c1864522cbb55109411c3c7ff6f627a88d
24 changes: 12 additions & 12 deletions x-pack/elastic-agent/pkg/agent/program/program_test.go
Original file line number Diff line number Diff line change
@@ -428,18 +428,18 @@ func TestConfiguration(t *testing.T) {
programs: []string{"filebeat"},
expected: 1,
},
// // "audit_config": {
// // programs: []string{"auditbeat"},
// // expected: 1,
// // },
// // "journal_config": {
// // programs: []string{"journalbeat"},
// // expected: 1,
// // },
// // "monitor_config": {
// // programs: []string{"heartbeat"},
// // expected: 1,
// // },
// "audit_config": {
// programs: []string{"auditbeat"},
// expected: 1,
// },
// "journal_config": {
// programs: []string{"journalbeat"},
// expected: 1,
// },
// "monitor_config": {
// programs: []string{"heartbeat"},
// expected: 1,
// },
"enabled_true": {
programs: []string{"filebeat"},
expected: 1,
2 changes: 1 addition & 1 deletion x-pack/elastic-agent/pkg/agent/program/supported.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ inputs:
- type: apache/metrics
constraints:
- "validate_version(%{[agent.version]}, '1.0.0 - 7.0.0')"
namespace: testing
dataset.namespace: testing
streams:
- enabled: true
metricset: info
Original file line number Diff line number Diff line change
@@ -24,9 +24,9 @@ inputs:
use_output: default
streams:
- metricset: status
dataset: docker.status
dataset.name: docker.status
- metricset: info
dataset: ""
dataset.name: ""
hosts: ["http://127.0.0.1:8080"]
- type: logs
use_output: default
@@ -37,7 +37,7 @@ inputs:
vars:
var: value
- type: apache/metrics
namespace: testing
dataset.namespace: testing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably good to add one of these tests files to use:

dataset:
  namespace: testing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean with invalid key?
keys we dont know are passed to input by default

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I mean add a test to ensure that dataset.namespace: testing or

dataset:
  namespace: testing

Both work

use_output: default
processors:
- add_fields:
16 changes: 8 additions & 8 deletions x-pack/elastic-agent/pkg/agent/transpiler/rules.go
Original file line number Diff line number Diff line change
@@ -369,7 +369,7 @@ func (r *FixStreamRule) Apply(ast *AST) error {
}

for _, inputNode := range inputsNodeList.value {
nsNode, found := inputNode.Find("namespace")
nsNode, found := inputNode.Find("dataset.namespace")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will Find("dataset.namespace") handle the fact that dataset could be a dictionary in the configuration:

dataset:
  namespace: default
  name: other

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no in fact it will not, i added support for both just to be sure

if found {
nsKey, ok := nsNode.(*Key)
if ok {
@@ -383,7 +383,7 @@ func (r *FixStreamRule) Apply(ast *AST) error {
continue
}
datasourceMap.value = append(datasourceMap.value, &Key{
name: "namespace",
name: "dataset.namespace",
value: &StrVal{value: defaultNamespace},
})
}
@@ -404,7 +404,7 @@ func (r *FixStreamRule) Apply(ast *AST) error {
continue
}

dsNode, found := streamNode.Find("dataset")
dsNode, found := streamNode.Find("dataset.name")
if found {
dsKey, ok := dsNode.(*Key)
if ok {
@@ -414,7 +414,7 @@ func (r *FixStreamRule) Apply(ast *AST) error {
}
} else {
streamMap.value = append(streamMap.value, &Key{
name: "dataset",
name: "dataset.name",
value: &StrVal{value: defaultDataset},
})
}
@@ -455,7 +455,7 @@ func (r *InjectIndexRule) Apply(ast *AST) error {

for _, inputNode := range inputsList.value {
namespace := defaultNamespace
nsNode, found := inputNode.Find("namespace")
nsNode, found := inputNode.Find("dataset.namespace")
if found {
nsKey, ok := nsNode.(*Key)
if ok {
@@ -483,7 +483,7 @@ func (r *InjectIndexRule) Apply(ast *AST) error {

dataset := defaultDataset

dsNode, found := streamNode.Find("dataset")
dsNode, found := streamNode.Find("dataset.name")
if found {
dsKey, ok := dsNode.(*Key)
if ok {
@@ -535,7 +535,7 @@ func (r *InjectStreamProcessorRule) Apply(ast *AST) error {

for _, inputNode := range inputsList.value {
namespace := defaultNamespace
nsNode, found := inputNode.Find("namespace")
nsNode, found := inputNode.Find("dataset.namespace")
if found {
nsKey, ok := nsNode.(*Key)
if ok {
@@ -563,7 +563,7 @@ func (r *InjectStreamProcessorRule) Apply(ast *AST) error {

dataset := defaultDataset

dsNode, found := streamNode.Find("dataset")
dsNode, found := streamNode.Find("dataset.name")
if found {
dsKey, ok := dsNode.(*Key)
if ok {
4 changes: 2 additions & 2 deletions x-pack/elastic-agent/spec/filebeat.yml
Original file line number Diff line number Diff line change
@@ -48,9 +48,9 @@ rules:
- remove_key:
key: use_output
- remove_key:
key: namespace
key: dataset.namespace
- remove_key:
key: dataset
key: dataset.name

- filter_values:
selector: inputs
4 changes: 2 additions & 2 deletions x-pack/elastic-agent/spec/metricbeat.yml
Original file line number Diff line number Diff line change
@@ -64,9 +64,9 @@ rules:
- remove_key:
key: enabled
- remove_key:
key: dataset
key: dataset.name
- remove_key:
key: namespace
key: dataset.namespace
- remove_key:
key: use_output