Agent v2 #458
Agent v2 #458
Conversation
|
This pull request does not have a backport label. Could you fix it @olegsu? 🙏
|
olegsu
force-pushed
the
agent-v2
branch
2 times, most recently
from
3120de9
to
49e8a62
Compare
Cloudbeat CI 🤖Allure Report: http://csp-allure-reports.s3.amazonaws.com/allure_reports/cloudbeat/prs/458/index.html |
|
So, it looks like you're missing the config transformation callback, like what we have here: https://github.com/elastic/beats/blob/feature-arch-v2/x-pack/metricbeat/cmd/agent.go Trying to figure out the cloudbeat setup, will see if I can push to this PR. |
|
So, a few things: As mentioned earlier, you'll need a custom config transform, like this, which should go in Theoretically, that should be the only missing piece to get this running.
Under the root elastic-agent directory, it's I imagine the issue might be fixed by properly installing the binary. If we're still running into issues, can you post the logs from |
|
Thank you @fearful-symmetry, I have tried to add the The Elastic Agent logs say that `input is not supported when I am adding KSPM integration. The Elastic-Agent complied on my machine and the output of |
olegsu
force-pushed
the
agent-v2
branch
2 times, most recently
from
31e07d6
to
1301328
Compare
That's odd, and I think it's an issue with how the policy is getting handled by elastic-agent. Looking at the KSPM integration, it returns an input value of The specfile for cloudbeat is here: https://github.com/elastic/elastic-agent/blob/feature-arch-v2/specs/cloudbeat.spec.yml And I don't see any aliases listed for the inputs, which most other beats seem to have. @michalpristas is there a reason why cloudbeat doesn't have any alternate inputs listed in the specfile I linked to? |
I assume the issue here is that elastic-agent isn't actually running? That or there's some container-specific issue with how we're reaching out to the RPC socket. |
|
Alright, because everyone dealing with this is on opposite sides of the planet, I've gone ahead and put in a PR to fix what I think is the issue: elastic/elastic-agent#1596 Don't have any cloudbeat experience, so input from other folks would probably be appreciated with that PR. |
|
Thanks, yes if the agent is complaining it doesn't recognize the input type it's because none of the spec files known to the agent declare support for that input type. In this case the cloudbeat spec file needs to be updated in the agent's v2 branch. |
Elastic Agent is running in kind cluster. root@kind-control-plane:/usr/share/elastic-agent# elastic-agent status
State: DEGRADED
Message: 1 or more components/units in a failed state
Components:
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
Thank you, I will try to recompile and run again, in both Kind and EKS. |
|
I have pulled the last changes compiled it again and tried in both EKS cluster and Kind cluster. root@ip-172-31-8-150:/usr/share/elastic-agent# elastic-agent status
State: DEGRADED
Message: 1 or more components/units in a failed state
Components:
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
* (FAILED)
input not supported
root@ip-172-31-8-150:/usr/share/elastic-agent# elastic-agent version
Binary: 8.6.0-SNAPSHOT (build: 96e071e16f49194ab1c6a01a7e88707986afbad2 at 2022-10-25 09:39:46 +0000 UTC)
Daemon: <failed to communicate>
could not get version. failed to communicate with running daemon: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix /usr/share/elastic-agent/data/tmp/elastic-agent-control.sock: connect: no such file or directory"
Use --binary-only flag to skip trying to retrieve version from running daemonAnd the logs: {"log.level":"info","@timestamp":"2022-10-25T10:46:41.672Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":322},"message":"New component created","component":{"id":"cloudbeat/cis_eks-default │
│ ","state":"Failed","message":"input not supported","inputs":[{"id":"cloudbeat/cis_eks-default-fe3be4a3-32af-40d1-ba65-b1a1e649c7a1","state":"Failed","message":"input not supported"}],"output":{"id":"cloudbeat/c │
│ is_eks-default","state":"Failed","message":"input not supported"}},"ecs.version":"1.6.0"} │
│ {"log.level":"info","@timestamp":"2022-10-25T10:46:41.975Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":322},"message":"New component created","component":{"id":"logfile-default","state": │
│ "Failed","message":"input not supported","inputs":[{"id":"logfile-default-logfile-system-449c9421-29fe-4a93-8f29-2497d7e7aef5","state":"Failed","message":"input not supported"}],"output":{"id":"logfile-default" │
│ ,"state":"Failed","message":"input not supported"}},"ecs.version":"1.6.0"} │
│ {"log.level":"info","@timestamp":"2022-10-25T10:46:42.168Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":322},"message":"New component created","component":{"id":"winlog-default","state":" │
│ Failed","message":"input not supported","inputs":[{"id":"winlog-default-winlog-system-449c9421-29fe-4a93-8f29-2497d7e7aef5","state":"Failed","message":"input not supported"}],"output":{"id":"winlog-default","st │
│ ate":"Failed","message":"input not supported"}},"ecs.version":"1.6.0"}Might be that I am compiling it in a wrong way (I follow the README) ? |
|
it seems like we prefixing Line 105 in 2ebaff9 so our output is not exactly cloudbeat/cis_k8s but cloudbeat/cis_k8s-default, though I'm not sure if it's related, since the fix was about the configuration (which is the input). Do we also need to configure the output explicitly? @cmacknz @fearful-symmetry
@olegsu I think in any case, it's worth testing cloudbeat without the |
|
Yah, this isn't an issue with cloudbeat I don't think, I'm 90% sure this is something happening between the integration and elastic-agent. This is really interesting:
That seems to indicate that the failure is in |
|
@olegsu precisely what platform is this running on? What OS/arch/cloud environment/etc |
|
Also @olegsu can I see the full integration config? In the fleet UI, if you go to the page for the agent config, there should be a button called Actions > View Config. |
I tried to run it two ways, in both the elastic stack deployed in the cloud o 8.6 snapshot version
The configurationid: ccd0ab20-5070-11ed-8602-11c37f297830
revision: 16
outputs:
default:
type: elasticsearch
hosts:
- REDACTED
output_permissions:
default:
_elastic_agent_monitoring:
indices:
- names:
- logs-elastic_agent.apm_server-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.apm_server-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.auditbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.auditbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.cloudbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.cloudbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.elastic_agent-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.endpoint_security-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.endpoint_security-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.filebeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.filebeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.fleet_server-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.fleet_server-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.heartbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.heartbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.metricbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.metricbeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.osquerybeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.osquerybeat-default
privileges:
- auto_configure
- create_doc
- names:
- logs-elastic_agent.packetbeat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-elastic_agent.packetbeat-default
privileges:
- auto_configure
- create_doc
_elastic_agent_checks:
cluster:
- monitor
e2150a54-25fd-49f6-8d95-0079bacd934d:
indices:
- names:
- logs-system.auth-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.syslog-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.application-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.security-default
privileges:
- auto_configure
- create_doc
- names:
- logs-system.system-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.cpu-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.diskio-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.filesystem-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.fsstat-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.load-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.memory-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.network-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.process-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.process.summary-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.socket_summary-default
privileges:
- auto_configure
- create_doc
- names:
- metrics-system.uptime-default
privileges:
- auto_configure
- create_doc
a72d4c42-1bb0-4b94-9985-6ed6d5b106a4:
indices:
- names:
- logs-cloud_security_posture.findings-default
privileges:
- auto_configure
- create_doc
agent:
download:
source_uri: 'https://artifacts.elastic.co/downloads/'
monitoring:
enabled: true
use_output: default
namespace: default
logs: true
metrics: true
inputs:
- id: logfile-system-e2150a54-25fd-49f6-8d95-0079bacd934d
name: system-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: system
version: 1.20.4
data_stream:
namespace: default
package_policy_id: e2150a54-25fd-49f6-8d95-0079bacd934d
streams:
- id: logfile-system.auth-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.auth
type: logs
ignore_older: 72h
paths:
- /var/log/auth.log*
- /var/log/secure*
exclude_files:
- .gz$
multiline:
pattern: ^\s
match: after
tags:
- system-auth
processors:
- add_locale: null
- id: logfile-system.syslog-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.syslog
type: logs
paths:
- /var/log/messages*
- /var/log/syslog*
exclude_files:
- .gz$
multiline:
pattern: ^\s
match: after
processors:
- add_locale: null
ignore_older: 72h
- id: winlog-system-e2150a54-25fd-49f6-8d95-0079bacd934d
name: system-1
revision: 1
type: winlog
use_output: default
meta:
package:
name: system
version: 1.20.4
data_stream:
namespace: default
package_policy_id: e2150a54-25fd-49f6-8d95-0079bacd934d
streams:
- id: winlog-system.application-e2150a54-25fd-49f6-8d95-0079bacd934d
name: Application
data_stream:
dataset: system.application
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: winlog-system.security-e2150a54-25fd-49f6-8d95-0079bacd934d
name: Security
data_stream:
dataset: system.security
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: winlog-system.system-e2150a54-25fd-49f6-8d95-0079bacd934d
name: System
data_stream:
dataset: system.system
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: system/metrics-system-e2150a54-25fd-49f6-8d95-0079bacd934d
name: system-1
revision: 1
type: system/metrics
use_output: default
meta:
package:
name: system
version: 1.20.4
data_stream:
namespace: default
package_policy_id: e2150a54-25fd-49f6-8d95-0079bacd934d
streams:
- id: system/metrics-system.cpu-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.cpu
type: metrics
metricsets:
- cpu
cpu.metrics:
- percentages
- normalized_percentages
period: 10s
- id: system/metrics-system.diskio-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.diskio
type: metrics
metricsets:
- diskio
diskio.include_devices: null
period: 10s
- id: system/metrics-system.filesystem-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.filesystem
type: metrics
metricsets:
- filesystem
period: 1m
processors:
- drop_event.when.regexp:
system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.fsstat-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.fsstat
type: metrics
metricsets:
- fsstat
period: 1m
processors:
- drop_event.when.regexp:
system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.load-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.load
type: metrics
metricsets:
- load
condition: '${host.platform} != ''windows'''
period: 10s
- id: system/metrics-system.memory-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.memory
type: metrics
metricsets:
- memory
period: 10s
- id: system/metrics-system.network-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.network
type: metrics
metricsets:
- network
period: 10s
network.interfaces: null
- id: system/metrics-system.process-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.process
type: metrics
metricsets:
- process
period: 10s
process.include_top_n.by_cpu: 5
process.include_top_n.by_memory: 5
process.cmdline.cache.enabled: true
process.cgroups.enabled: false
process.include_cpu_ticks: false
processes:
- .*
- id: >-
system/metrics-system.process.summary-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.process.summary
type: metrics
metricsets:
- process_summary
period: 10s
- id: >-
system/metrics-system.socket_summary-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.socket_summary
type: metrics
metricsets:
- socket_summary
period: 10s
- id: system/metrics-system.uptime-e2150a54-25fd-49f6-8d95-0079bacd934d
data_stream:
dataset: system.uptime
type: metrics
metricsets:
- uptime
period: 10s
- id: a72d4c42-1bb0-4b94-9985-6ed6d5b106a4
name: cloud_security_posture-3
revision: 2
type: cloudbeat/cis_k8s
use_output: default
meta:
package:
name: cloud_security_posture
version: 1.0.3
data_stream:
namespace: default
package_policy_id: a72d4c42-1bb0-4b94-9985-6ed6d5b106a4
streams:
- id: >-
cloudbeat/cis_k8s-cloud_security_posture.findings-a72d4c42-1bb0-4b94-9985-6ed6d5b106a4
name: Findings
data_stream:
dataset: cloud_security_posture.findings
type: logs
processors:
- add_cluster_id: null
fetchers:
- name: kube-api
- name: process
processes:
kube-apiserver: null
kubelet:
config-file-arguments:
- config
kube-scheduler: null
etcd: null
kube-controller: null
directory: /hostfs
- name: file-system
patterns:
- /hostfs/etc/kubernetes/scheduler.conf
- /hostfs/etc/kubernetes/controller-manager.conf
- /hostfs/etc/kubernetes/admin.conf
- /hostfs/etc/kubernetes/kubelet.conf
- /hostfs/etc/kubernetes/manifests/etcd.yaml
- /hostfs/etc/kubernetes/manifests/kube-apiserver.yaml
- /hostfs/etc/kubernetes/manifests/kube-controller-manager.yaml
- /hostfs/etc/kubernetes/manifests/kube-scheduler.yaml
- /hostfs/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
- /hostfs/etc/kubernetes/pki/*
- /hostfs/var/lib/kubelet/config.yaml
- /hostfs/var/lib/etcd
- /hostfs/etc/kubernetes/pki
runtime_cfg:
activated_rules:
cis_k8s:
- cis_1_2_18
- cis_1_2_20
- cis_1_2_19
- cis_1_2_16
- cis_1_2_32
- cis_1_1_15
- cis_4_2_6
- cis_4_1_10
- cis_1_1_12
- cis_1_1_3
- cis_5_2_8
- cis_5_2_5
- cis_1_2_15
- cis_1_2_29
- cis_1_1_14
- cis_5_1_5
- cis_4_2_2
- cis_1_1_2
- cis_1_2_24
- cis_4_1_5
- cis_1_1_11
- cis_1_2_14
- cis_5_2_4
- cis_1_1_20
- cis_2_3
- cis_4_2_12
- cis_1_2_25
- cis_1_4_2
- cis_4_2_4
- cis_1_2_27
- cis_1_1_18
- cis_4_2_8
- cis_2_6
- cis_1_2_5
- cis_1_1_21
- cis_1_4_1
- cis_4_1_9
- cis_1_1_1
- cis_1_1_5
- cis_2_1
- cis_1_2_2
- cis_1_1_16
- cis_5_1_6
- cis_2_4
- cis_4_1_6
- cis_5_2_9
- cis_1_1_17
- cis_1_1_19
- cis_1_2_7
- cis_1_1_6
- cis_1_3_5
- cis_4_1_2
- cis_5_2_3
- cis_1_2_21
- cis_1_2_4
- cis_5_1_3
- cis_4_2_9
- cis_4_1_1
- cis_1_2_12
- cis_1_2_6
- cis_1_2_13
- cis_1_2_26
- cis_4_2_1
- cis_4_2_3
- cis_2_2
- cis_1_3_6
- cis_2_5
- cis_5_2_7
- cis_5_2_6
- cis_1_2_23
- cis_1_1_8
- cis_5_2_2
- cis_5_2_10
- cis_1_2_10
- cis_1_2_8
- cis_1_2_11
- cis_4_2_11
- cis_4_2_5
- cis_4_2_10
- cis_4_2_7
- cis_1_2_17
- cis_1_3_4
- cis_1_2_28
- cis_1_2_22
- cis_1_2_9
- cis_1_3_2
- cis_1_3_7
- cis_1_3_3
- cis_1_1_4
- cis_1_1_7
- cis_4_2_13
- cis_1_1_13
fleet:
hosts:
- REDACTED
|
|
So, I did a little bit of testing, and there's a few ways to reproduce the
Looking at the original instructions for reproducing this at the top of the PR, I have a strong suspicion that the specfile isn't actually there. However, based on the error messages posted above, I'm guessing we're somehow missing even more specfiles:
This is...kind of baffling. I'm guessing there's something very wrong with the whole install. @olegsu can I see the contents of the components directory you're trying to run from? It's at If you see a components directory with a bunch of |
|
Alright, the more I look at the build code, I'm not even sure how "non-packaged" components like cloudbeat are supposed to get their specfile, since they're not getting packaged with the binary as far as I can tell. In the real world, would elastic-agent download them along with the binary? @michalpristas / @blakerouse might see something obvious here, but I assume the issue is that something is wrong with the components directory, or elastic-agent's data paths. |
|
Alright, I can reproduce this with the Perhaps there's some issue with how we're setting the data directory under |
When running the image that is shipped with the release it seems that the cloudbeat is embedded and not downloaded. @oren-zohar am I, right? Comparing two filesystems content: 8.5This is the filesystem content of Elastic-Agent taking from image elastic-agent@f018d455f7f8:~/data/elastic-agent-0e4f48/install$ ls -la
total 40
drwxr-xr-x 10 elastic-agent elastic-agent 4096 Oct 27 04:51 .
drwxrwx--- 1 root root 4096 Oct 27 04:51 ..
drwxr-xr-x 2 elastic-agent elastic-agent 4096 Oct 26 06:22 apm-server-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 2 elastic-agent elastic-agent 4096 Oct 27 04:51 cloudbeat-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 2 elastic-agent elastic-agent 4096 Oct 27 04:51 endpoint-security-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 5 elastic-agent elastic-agent 4096 Oct 27 04:51 filebeat-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 2 elastic-agent elastic-agent 4096 Oct 26 05:53 fleet-server-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 4 elastic-agent elastic-agent 4096 Oct 27 04:51 heartbeat-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 5 elastic-agent elastic-agent 4096 Oct 27 04:51 metricbeat-8.5.0-SNAPSHOT-linux-arm64
drwxr-xr-x 3 elastic-agent elastic-agent 4096 Oct 27 04:51 osquerybeat-8.5.0-SNAPSHOT-linux-arm64feature-arch-v2root@kind-control-plane:/usr/share/elastic-agent/data/elastic-agent-96e071/components# ls -la
total 932856
drwxrwx--- 1 root root 4096 Oct 25 06:29 .
drwxrwx--- 1 root root 4096 Oct 25 06:31 ..
-rw-rw---- 1 root root 41 Oct 25 06:28 .build_hash.txt
-rw-rw---- 1 root root 13675 Oct 25 06:28 LICENSE.txt
-rw-rw---- 1 root root 2566303 Oct 25 06:28 NOTICE.txt
-rw-rw---- 1 root root 840 Oct 25 06:28 README.md
drwxrwx--- 2 root root 4096 Oct 25 06:28 certs
-rw-r--r-- 1 root root 1303 Oct 25 06:28 checksum.yml
-rw-r--r-- 1 root root 389399 Oct 25 06:28 fields.yml
-rwxr-xr-x 1 root root 179136435 Oct 25 06:28 filebeat
-rw-r--r-- 1 root root 174363 Oct 25 06:28 filebeat.reference.yml
-rw-r--r-- 1 root root 3743 Oct 25 06:28 filebeat.spec.yml
-rw-r--r-- 1 root root 8622 Oct 25 06:28 filebeat.yml
-rwxr-xr-x 1 root root 161979497 Oct 25 06:28 heartbeat
-rw-r--r-- 1 root root 67937 Oct 25 06:28 heartbeat.reference.yml
-rw-r--r-- 1 root root 1057 Oct 25 06:28 heartbeat.spec.yml
-rw-r--r-- 1 root root 7276 Oct 25 06:28 heartbeat.yml
drwxrwx--- 4 root root 4096 Oct 25 06:29 kibana
-rwxr-xr-x 1 root root 239604625 Oct 25 06:28 metricbeat
-rw-r--r-- 1 root root 103498 Oct 25 06:28 metricbeat.reference.yml
-rw-r--r-- 1 root root 3998 Oct 25 06:28 metricbeat.spec.yml
-rw-r--r-- 1 root root 6899 Oct 25 06:28 metricbeat.yml
drwxrwx--- 84 root root 4096 Oct 25 06:29 module
drwxrwx--- 2 root root 4096 Oct 25 06:29 modules.d
drwxrwx--- 2 root root 4096 Oct 25 06:29 monitors.d
-rw-rw---- 1 root root 5526616 Oct 25 06:28 osquery-extension.ext
-rwxr-xr-x 1 root root 147685279 Oct 25 06:28 osquerybeat
-rw-r--r-- 1 root root 43600 Oct 25 06:28 osquerybeat.reference.yml
-rw-r--r-- 1 root root 584 Oct 25 06:28 osquerybeat.spec.yml
-rw-r--r-- 1 root root 6504 Oct 25 06:28 osquerybeat.yml
-rw-rw---- 1 root root 217818752 Oct 25 06:28 osqueryd
I have tried also to download the |
To my knowledge, yes that should be the behavior |
Adding cloudbeat to the If this doesn't work we can fix it. The only other alternative is updating the agent packaging step to know how to fetch cloudbeat specifically the way it does for binaries from the main Beats repository. I don't think we want to special case binaries like this in the long term. Really the agent build system is due for a redesign to make this whole process easier. |
|
@olegsu the fix for the input not found bug has now been merged to the agent feature-arch-v2 branch: elastic/elastic-agent#1653 if you want to retest it. We will be planning to merge these changes to main next week so that agent v2 is available in the 8.6 snapshot images. We'll send an email once we decide on a specific date. |
|
Thank you @cmacknz I will try running it and update
Update logsSo I tried another thing, building the cloudbeat assets and then copying them into the expected directory ( This process worked and all the binaries were copied to the final image and cloudbeat started as expected. I think that it is possible to overcome it by doing one of:
|
|
That's odd, none of the latest changes include changes to the build system, so I'm not sure what would suddenly cause the |
|
So, the behavior of |
Thank you for the answer, @fearful-symmetry. |
|
This pull request is now in conflicts. Could you fix it? 🙏 |
|
I agree that it would be nice to separate flag that treats the contents of AGENT_DROP_PATH as additional binaries to be bundled with the standard set, rather than expecting it to be an alternative source for all binaries. I think there is an existing workaround for this in the current cloudbeat magefile that will pack filebeat, metricbeat, heartbeat, and and osquerybeat in addition to cloudbeat: Lines 271 to 276 in b84e3bd The behaviour here didn't change in v2, but I do agree the agent build and packaging system could be improved regardless. |
olegsu
force-pushed
the
agent-v2
branch
3 times, most recently
from
1db972d
to
697c6bf
Compare
| cp_to_pod $POD $LOCAL_DIR/cloudbeat $DEST | ||
| cp_to_pod $POD $LOCAL_DIR/cloudbeat.yml $DEST/cloudbeat.yml | ||
|
|
||
| # Start with COPY_BUNDLE=true to move also the opa bundle to the agent |
scripts/remote_replace_cloudbeat.sh
Outdated
| ROOT=/usr/share/elastic-agent/data/elastic-agent-$SHA | ||
| DEST=$ROOT/components | ||
| cp_to_pod $POD $LOCAL_DIR/cloudbeat $DEST | ||
| cp_to_pod $POD $LOCAL_DIR/cloudbeat.yml $DEST/cloudbeat.yml |
There was a problem hiding this comment.
not sure we'll always want to copy cloudbeat.yml, wdyt?
There was a problem hiding this comment.
Right, this is not the behavior we have today.
|
@cmacknz Thank you! So I changed it to 1e3bdad. This is not a blocker, but might be something worth looking on. |
olegsu
force-pushed
the
agent-v2
branch
2 times, most recently
from
cd313be
to
f7de335
Compare
|
This pull request is now in conflicts. Could you fix it? 🙏 |
Assuming this is reproducible, this is a bug. All of the elastic-agent commands should keep working. I'll try to quickly reproduce this and then open an issue to get it fixed. |
|
Confirmed restart is broken, along with a few other things: elastic/elastic-agent#1709 |
What does this PR do?
Based on @eyalkraft work. This will align with the new agent v2 architecture.
State: OK 👍
Changes
Elastic-Agent V2 has a few changes which impact us. I will try to list here all the changes that I found during this work.
The most official document can be found here
Filesystem
The file system structure had changed:
logsdirectory moved to/usr/share/elastic-agent/data/elastic-agent-{AGENT_SHA}/logs/downloaddirectory are no longs existsinstalldirectory is no longer exists/usr/share/elastic-agent/data/elastic-agent-{AGENT_SHA}/components/API
Cloudbeat will no longer get a list of
inputsthat contain an array ofstreamsbut every time a singlestreamfromstreamsElastic-Agent Behavior
When compiling the Elastic-Agent locally
cloudbeatwill not be there. To copy it we need to build them both (https://github.com/elastic/security-team/blob/main/docs/cloud-security-posture-team/Onboarding/deploy-agent-cloudbeat-on-eks.mdx).After the image is deployed and the assets are copied, the process will still wont start (even in the integration is connected). This is due to component registration on startup in the Agent (#458 (comment)). A quick agent restart will work around this.
Reference - #458 (comment)
Run locally
feature-arch-v2branchDEV=true SNAPSHOT=true PLATFORMS=linux/arm64 PACKAGES=docker mage packageDEV=true PLATFORMS=linux/arm64 SNAPSHOT=true mage -v package4.1 From Kibana UI - add new agent.
4.2 Download the agent manifests for Kubernetes
4.3 Add "Kubernetes Security Posture Management" integration to the policy
5.1 Update the manifests with the new agent image
./scripts/remote_replace_cloudbeat.shto copy and restart the agent