[SLES 15]: No "system.auth" logs for system integration under Data Streams tab for SLES 15 linux agent.

[amolnater-qasource]

Kibana version: 7.15.0 BC-2 Kibana cloud environment

Host OS and Browser version: All, All

Build details:

Build: 43742
Commit: 113d5f2a745ab9be7d567d13ff906dfd0f5b7a9a
Artifact Link: https://staging.elastic.co/7.15.0-d7816905/downloads/beats/elastic-agent/elastic-agent-7.15.0-linux-x86_64.tar.gz

Preconditions:

1. 7.15.0 BC-2 Kibana cloud environment should be available.
2. An SLES 15 linux agent must be installed with Default policy having System integration.

Steps to reproduce:

1. Login to Kibana environment.
2. Navigate to Data Streams tab.
3. Select System from Integration dropdown and logs from type dropdown.

Logs:
logs.zip

Expected Result:
"system.auth" logs should be generated for system integration under Data Streams tab for SLES 15 agent.

Screenshot:

Note:
This issue is not observed for Ubuntu 20 machines with .tar artifact.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[amolnater-qasource]

@manishgupta-qasource Please review.

[manishgupta-qasource]

Reviewed & Assigned to @fearful-symmetry

[fearful-symmetry]

The auth module tends to depend a lot on system configuration. Have we confirmed that:

• There's populated log files in /var/log/auth.log* or /var/log/secure*
• That this otherwise works in 7.14?

[amolnater-qasource]

Hi @fearful-symmetry
As per feedback we have revalidated this on 7.14.1 BC-2 and found it reproducible there too.

• No system.auth logs type data under Data Streams for SLES 15 linux.tar agent.

Build details:
Build: 42892
Commit: 196ec3974d4c725a3d937725419e5ed7d8fdb104
Artifact Link: https://staging.elastic.co/7.14.1-96b6cf91/downloads/beats/elastic-agent/elastic-agent-7.14.1-linux-x86_64.tar.gz

Screenshot:

Logs:
logs.zip

Please let us know if anything else is required from our end.
cc: @EricDavisX
Thanks

[fearful-symmetry]

I don't see any errors in the logs. Can we verify if:

• There's populated log files in /var/log/auth.log* or /var/log/secure* on the host machine?
• Can we enable debug level logging?

[EricDavisX]

hi - sorry I missed this, even 7 days ago, but I'm not certain SLES 15 is going to be cited as supported. Let's hold off until we get that decision. Our original manual test was a fact finding mission, here is a good fact to consider. We can ask the leadership group to confirm and circle back.

Having said that, I would prefer to run the same test of versions against one of our confirmed supported OSes to see, like a CentOS or Debian distro. If it is reproduced there, I'd log a separate bug to make it clear - but we can link them, in case any logging / info here is helpful. @amolnater-qasource @dikshachauhan-qasource we don't need to do any SLES 15 tests further until confirmed, except to test other OSes for potential bugs that may actually be evidenced elsewhere.

[amolnater-qasource]

Hi @fearful-symmetry
Please find below attached debug level logs for SLES 15:
logs.zip

@EricDavisX thanks for the update, we will hold our testing on SLES 15 till further confirmation.

Please let us know if anything else is required from our end.
Thanks

[fearful-symmetry]

Having said that, I would prefer to run the same test of versions against one of our confirmed supported OSes to see, like a CentOS or Debian distro.

Seconded.

[amolnater-qasource]

Hi @EricDavisX
As per feedback we have validated this issue on 7.15.0 BC-4 with Centos8 using .tar and .rpm agents.

Observations:

• We have observed that system.auth datasets are available under Data Streams tab for both .tar and .rpm agents installed on Centos8.

Build details:
Build: 43886
Commit: 29a6969dc230abf16dc65a41c535cb534ae64fa7
Artifact Link: https://staging.elastic.co/7.15.0-9e0972b3/summary-7.15.0.html

Screenshot:

Please let us know if anything else is required from our end.
Thanks

[botelastic]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[ghost]

Hi @fearful-symmetry @EricDavisX,

We have re-validated this issue on the latest 8.6.0 BC8 Kibana Staging environment and found the issue is still reproducible.

Build details:

Version: 8.6.0 BC8
Build: 58807	
Commit: 335b86a633af9ce1d9c8a92187ed1c8697ab139f

Below are the observations:

• "system.auth" logs is not shown for System integration under Data Streams tab and Discover tab under logs-* index for SLES 15 Linux agent.

Screenshots:

Please let us know if we are missing anything.

Thanks!

[dikshachauhan-qasource]

Hi @fearful-symmetry

We have observed that this issue is still reproducible and occuring on 8.5 BC5 build.

Observations:

• "system.auth" logs not available for system integration under Data Streams tab for SLES 15 linux agent.

Screenshots:

Build details:

VERSION: 8.7.0 BC5
BUILD: 61017
COMMIT: e70452f9697a18a8458f3d4618e001b7e3681bed
Artifact: https://staging.elastic.co/8.7.0-69b36bf6/summary-8.7.0.html

Please let us know if we are missing anything.

Thanks!

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[cmacknz]

@dikshachauhan-qasource can you attach agent diagnostics from when this problem was most recently reproduced?

[dikshachauhan-qasource]

Hi @cmacknz

Please find the required daignostics log in attached zip folder reproduced today on below build details.

Build details:

VERSION: 8.7.0 BC6
BUILD: 61051
COMMIT: 04ef24287f26854ad99a46ae983854c6184717cb
Artifact: https://staging.elastic.co/8.7.0-a7fb3750/summary-8.7.0.html

Screenshot

Logs:
elastic-agent-diagnostics-2023-03-22T09-01-23Z-00.zip

Thanks
QAS

[botelastic]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[amolnater-qasource]

Hi Team,

We have revalidated this issue on latest 8.14.0 Snapshot kibana cloud environment and found it still reproducible on SLES 15.

Observations:

• No system.auth logs for system integration under Data Streams tab for SLES 15 linux agent.

Screenshot:

Build details:
VERSION: 8.14.0 SNAPSHOT
BUILD: 72779
COMMIT: ea309fdbf7ed02d5a8872daf56132b9e40e60982

Agent Logs:
elastic-agent-diagnostics-2024-03-28T07-32-42Z-00.zip

Please let us know if anything else is required from our end.
Thanks!!

[amolnater-qasource]

JFI @pierrehilbert

[pierrehilbert]

For me this is the same case than #3650 will be covered when elastic/beats#37086 will be done.