Nil pointer dereference in Agent config transpiler #23734
Comments
|
Pinging @elastic/agent (Team:Agent) |
|
I though we had unit test for this kind of scenario in the AST, I believe using |
|
i'm having issues reproducing this both using tests and using 7.10.2 release, can you attach whole config as you have it? |
|
OK so this took me a while to reproduce. This is on ECK so Agent is running in standalone mode. The trick is that you first create the agent without the offending So to begin we have an Elasticsearch cluster (and a Kibana which I omitted here): apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: elasticsearch
spec:
version: 7.10.2
nodeSets:
- name: default
count: 3
config:
node.store.allow_mmap: falseThen I create the Elastic Agent: ---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
name: elastic-agent
spec:
version: 7.10.2
elasticsearchRefs:
- name: elasticsearch
daemonSet:
podTemplate:
spec:
serviceAccountName: agent
automountServiceAccountToken: true
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true # Allows to provide richer host metadata
containers:
- name: agent
securityContext:
runAsUser: 0
# If using Red Hat OpenShift uncomment this:
#privileged: true
volumeMounts:
- name: varlogcontainers
mountPath: /var/log/containers
- name: varlogpods
mountPath: /var/log/pods
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumes:
- name: varlogcontainers
hostPath:
path: /var/log/containers
- name: varlogpods
hostPath:
path: /var/log/pods
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
config:
id: 2d70a6f0-33a5-11eb-bb2f-418d0388a8cf
revision: 2
agent:
monitoring:
enabled: true
use_output: default
logs: true
metrics: true
inputs:
- id: d8b4d520-615b-11eb-847e-6bd07fa09e69
name: log-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: log
version: 0.4.6
data_stream:
namespace: default
streams:
- id: logfile-log.log
data_stream:
dataset: generic
type: container
paths:
- /var/log/containers/*.log
symlinks: true
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: agent
rules:
- apiGroups: [""] # "" indicates the core API group
resources:
- namespaces
- pods
verbs:
- get
- watch
- list
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: agent
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: agent
subjects:
- kind: ServiceAccount
name: agent
namespace: default
roleRef:
kind: ClusterRole
name: agent
apiGroup: rbac.authorization.k8s.ioThen I add the apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
name: elastic-agent
spec:
version: 7.10.2
elasticsearchRefs:
- name: elasticsearch
daemonSet:
podTemplate:
spec:
serviceAccountName: agent
automountServiceAccountToken: true
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true # Allows to provide richer host metadata
containers:
- name: agent
securityContext:
runAsUser: 0
# If using Red Hat OpenShift uncomment this:
#privileged: true
volumeMounts:
- name: varlogcontainers
mountPath: /var/log/containers
- name: varlogpods
mountPath: /var/log/pods
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumes:
- name: varlogcontainers
hostPath:
path: /var/log/containers
- name: varlogpods
hostPath:
path: /var/log/pods
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
config:
id: 2d70a6f0-33a5-11eb-bb2f-418d0388a8cf
revision: 2
agent:
monitoring:
enabled: true
use_output: default
logs: true
metrics: true
inputs:
- id: d8b4d520-615b-11eb-847e-6bd07fa09e69
name: log-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: log
version: 0.4.6
data_stream:
namespace: default
streams:
- id: logfile-log.log
data_stream:
dataset: generic
type: container
paths:
- /var/log/containers/*.log
symlinks: true
processors:
- add_kubernetes_metadata:
and it reliably panics. |
|
thanks this is helpful |
|
running this on master and 7.10.2 the only difference is that you're using k8s and i'm not. |
|
ECK generates the output configuration. If it helps I can later extract the effective configuration from k8s. |
|
it would be helpful as well, thank you |
|
Actually I think this might be partially caused on the ECK end. We use https://github.com/elastic/go-ucfg to render the configuration. And it turns this processor statement into a null value: agent:
monitoring:
enabled: true
logs: true
metrics: true
use_output: default
id: 2d70a6f0-33a5-11eb-bb2f-418d0388a8cf
inputs:
- data_stream:
namespace: default
id: d8b4d520-615b-11eb-847e-6bd07fa09e69
meta:
package:
name: log
version: 0.4.6
name: log-1
revision: 1
streams:
- data_stream:
dataset: generic
id: logfile-log.log
paths:
- /var/log/containers/*.log
processors:
- null
symlinks: true
type: container
type: logfile
use_output: default
outputs:
default:
hosts:
- https://elasticsearch-es-http.default.svc:9200
password: <redacted>
ssl:
certificate_authorities:
- /mnt/elastic-internal/elasticsearch-association/default/elasticsearch/certs/ca.crt
type: elasticsearch
username: default-elastic-agent-default-elasticsearch-agent-user
revision: 2Apologies I should have checked the rendered configuration right away 😞 We are documenting this behaviour for Beats and probably should do the same for Agent. However I think it is probably still a reasonable bugfix on the Agent side that even a malformed configuration like this should not let the process crash. |
|
this seems like something we can handle definitely, will raise a PR soon |
While exploring the standalone Elastic Agent support through the ECK operator I ran into this sigsegv:
when using the following input:
It is the null valued
-add_kubernetes_metadata:that leads to the panic.Affected version: 7.10.2
The text was updated successfully, but these errors were encountered: