Permission issues on some EventLog Channels

[tfriesen]

I first noticed that despite Sysmon running and generating plenty of events in Microsoft->Windows->Sysmon/Operational log, none were showing up in Kibana. In fact, hardly any logs were showing up at all.

Digging further, I found a Warning in Eventlog-ForwardingPlugin: The subscription lme is created, but one or more channels in the query could not be read at this time.

Details:

<t:QueryStatus xmlns:t="http://schemas.microsoft.com/wbem/wsman/1/windows/EventLog"><t:Channel Name="Application" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-AppLocker/EXE and DLL" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-AppLocker/MSI and Script" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-AppLocker/Packaged app-Deployment" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-AppLocker/Packaged app-Execution" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-SmartCard-Audit/Authentication" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-SMBClient/Operational" ErrorCode="5"/><t:Channel Name="Microsoft-Windows-Sysmon/Operational" ErrorCode="5"/><t:Channel Name="Microsoft-Windows-TaskScheduler/Operational" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-TerminalServices-RDPClient/Operational" ErrorCode="0"/><t:Channel Name="Microsoft-Windows-Windows Defender/Operational" ErrorCode="15007"/><t:Channel Name="Security" ErrorCode="5"/><t:Channel Name="System" ErrorCode="0"/></t:QueryStatus>

Followed by another Event in Eventlog-ForwardingPlugin Information: The subscription lme is unsubscribed.

I am no expert on Windows access permissions and rights, but here are some of the rights for two of the channels which return different error codes:

c:\Windows\sysmon>wevtutil gl "Microsoft-Windows-Sysmon/Operational"
name: Microsoft-Windows-Sysmon/Operational
enabled: true
type: Operational
owningPublisher: Microsoft-Windows-Sysmon
isolation: Custom
channelAccess: O:BAG:SYD:(A;;0xf0007;;;SY)(A;;0x7;;;BA)(A;;0x1;;;BO)(A;;0x1;;;SO
)(A;;0x1;;;S-1-5-32-573)
logging:
logFileName: %SystemRoot%\System32\Winevt\Logs\Microsoft-Windows-Sysmon%4Opera
tional.evtx
retention: false
autoBackup: false
maxSize: 67108864
publishing:
fileMax: 1

c:\Windows\sysmon>wevtutil gl "System"
name: System
enabled: true
type: Admin
owningPublisher:
isolation: System
channelAccess: O:BAG:SYD:(A;;0xf0007;;;SY)(A;;0x7;;;BA)(A;;0x3;;;BO)(A;;0x5;;;SO
)(A;;0x1;;;IU)(A;;0x3;;;SU)(A;;0x1;;;S-1-5-3)(A;;0x2;;;S-1-5-33)(A;;0x1;;;S-1-5-
32-573)
logging:
logFileName: %SystemRoot%\System32\Winevt\Logs\System.evtx
retention: false
autoBackup: false
maxSize: 20971520
publishing:
fileMax: 1

S-1-5-32-573 is the Event Log Readers group, which contains one member, NT AUTHORITY\NETWORK SERVICE

[duncan-ncc]

Hi @tfriesen
Some Windows event logs have stricter permissions by standard than others, For example Security logs. We add Network Service to event log readers in order to be able to read and forward it.
This is set by the GPO and often requires a reboot for these permissions to take effect.
Can you give one of the machines on which you have logs forwarding from a reboot and see if this does indeed fix the issue?

[tfriesen]

Hi @duncan-ncc

Thanks for the reply. Yes, restarting the machine seems to have resolved the issue (don't frown, power down!)

I have some hosts which can't be restarted quite as readily. Is there a way to apply these changes without restarting them?

[duncan-ncc]

Hi @tfriesen
I do not know of a easy method to apply these changes without a host reboot.
Microsoft do not seem to recommend anything on this page either https://support.microsoft.com/en-gb/help/323076/how-to-set-event-log-security-locally-or-by-using-group-policy

I will reach out to them for a response.

[tfriesen]

Addendum: It appears that restarting the host does not work with Domain Controllers, though it does with other machines. Domain Controllers still return ErrorCode=5 for a number of channels, despite multiple reboots. Unsure as to why just as yet; a colleague suggested that the use of local groups is problematic on Domain Controllers.

[gm-mitacom]

Hi @tfriesen

Did you manage to resolve the issue with the Domain Controller? I am testing LME and have 6 clients configured via GPO. 5 are forwarding logs as expected and 1 does not. It's a Domain Controller and reports the following error:

Log Name: Microsoft-Windows-Forwarding/Operational
Source: Microsoft-Windows-Forwarding
Date: 23/01/2020 17:56:12
Event ID: 105
Task Category: None
Level: Error
Keywords:
User: NETWORK SERVICE
Computer: xxx.yyy.zzz
Description:
The forwarder is having a problem communicating with subscription manager at address http://aaa.bbb.ccc:5985/wsman/SubscriptionManager/WEC. Error code is 5 and Error Message is <f:WSManFault xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="5" Machine="xxx.yyy.zzz"><f:Message>Access is denied. </f:Message></f:WSManFault>.

[tfriesen]

Hi @gm-mitacom

No, I'm still experiencing the issue. Last time I checked, anyway. Was waiting to hear back from @duncan-ncc if he had heard anything from MS.

Sounds like your issue is slightly different, though? The Denied Access is happening earlier in the subscription creation process, maybe? Check the rights of the user in the GPO(s). You might also be affected by this: #23

[gm-mitacom]

Hi @tfriesen,

Thanks for the reply. I've added another couple of machines to my LME-Clients security group (one a W10 machine and the other a domain controller) and the domain controller fails to connect. I'm almost certain that this issue is only affecting our domain controllers. All other machines are forwarding logs as expected.

I did run the commands identified in the troubleshooting guide (and highlighted by you) but that hasn't solved the problem.

I have domain network firewalls on the collector and client turned off - so I don't think that was an issue.

I wonder if anyone has successfully managed to get this working with domain controllers - and if so what steps did they take?

@duncan-ncc,

Have you had anything back from Microsoft regarding domain controllers and/or can you offer any other advice?

[ghost]

I use WEF with DCs. Your GPOs linked to OUs or sites that will hit your DCs? You can use gpresult on a domain controller to see group policy results for example to ensure it is receiving the settings required. Does your subscription on the WEC have domain controllers added under the computer security groups?

…

On Jan 28, 2020, at 8:24 AM, gm-mitacom ***@***.***> wrote:  Hi @tfriesen, Thanks for the reply. I've added another couple of machines to my LME-Clients security group (one a W10 machine and the other a domain controller) and the domain controller fails to connect. I'm almost certain that this issue is only affecting our domain controllers. All other machines are forwarding logs as expected. I did run the commands identified in the troubleshooting guide (and highlighted by you) but that hasn't solved the problem. I have domain network firewalls on the collector and client turned off - so I don't think that was an issue. I wonder if anyone has successfully managed to get this working with domain controllers - and if so what steps did they take? @duncan-ncc, Have you had anything back from Microsoft regarding domain controllers and/or can you offer any other advice? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[ghost]

Is your WEC Server Computer Object added to the built in DOMAIN Event Log Readers Group?

[gm-mitacom]

@1NF053C

In reply to your questions:

1. [Are]Your GPOs linked to OUs or sites that will hit your DCs?
   Yes, the GPO is linked with the OU that holds all the domain controllers.

2. The gpresult output for the domain controller in question shows the Event Forwarding Subscription Manager configured as expected.

3. Does your subscription on the WEC have domain controllers added under the computer security groups?
   We might be onto something here...
   I checked the subscriptions on the WEC and it had the following OU's listed:
   [DOMAIN]\Domain Computers; and
   [DOMAIN]\Domain Controllers.

I added one of the domain controllers as a specific computer (not as part of an OU or SG) and it connected. Progress...
I then removed it and added the SG that lists all the devices that I am currently testing and, hey presto, all 8 computers in that SG are counted. I also have logs being forwarded for 7 machines and appearing in Kibana.

4. Yes, the WEC Server computer object has been added to the DOMAIN\Builtin Event Log Readers Group.

So, many thanks for your pointers! Onwards and upwards!

Best regards

Gus

[ghost]

Excellent!

…

On Feb 5, 2020, at 8:47 AM, gm-mitacom ***@***.***> wrote:  @1NF053C In reply to your questions: [Are]Your GPOs linked to OUs or sites that will hit your DCs? Yes, the GPO is linked with the OU that holds all the domain controllers. The gpresult output for the domain controller in question shows the Event Forwarding Subscription Manager configured as expected. Does your subscription on the WEC have domain controllers added under the computer security groups? We might be onto something here... I checked the subscriptions on the WEC and it had the following OU's listed: [DOMAIN]\Domain Computers; and [DOMAIN]\Domain Controllers. I added one of the domain controllers as a specific computer (not as part of an OU or SG) and it connected. Progress... I then removed it and added the SG that lists all the devices that I am currently testing and, hey presto, all 8 computers in that SG are counted. I also have logs being forwarded for 7 machines and appearing in Kibana. Yes, the WEC Server computer object has been added to the DOMAIN\Builtin Event Log Readers Group. So, many thanks for your pointers! Onwards and upwards! Best regards Gus — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[njwjordan]

Hi all,

We've just installed this and we encountered this problem when adding our domain controllers as LME clients.

It appears that the LME clients push events to the collector using the NETWORK SERVICE user of each domain controller. The NETWORK SERVICE user doesn't have access to read the security event log of a domain by default, so you need to add it to the ACL for the security event log using wevtutil.

The command for this was:
wevtutil sl security /ca:O:BAG:SYD:(A;;0xf0005;;;SY)(A;;0x5;;;BA)(A;;0x1;;;S-1-5-32-573)(A;;0x1;;;S-1-5-20)

this effectively appends SID S-1-5-20 (which is for NETWORK SERVICE) to the ACL with read-only permissions for the security event log.

This had to be run from an elevated prompt on each domain controller. It also required us to then reboot the domain controllers to see the logs arriving at Kibana, which arrived almost immediately after reboot.

We found this out by reading this:
https://rockyprogress.wordpress.com/2011/12/04/security-event-log-collection-from-a-domain-controller/

Hope this helps!
Nick

[ghost]

Nice Nick!

…

On Jun 8, 2020, at 8:55 AM, njwjordan ***@***.***> wrote:  Hi all, We've just installed this and we encountered this problem when adding our domain controllers as LME clients. It appears that the LME clients push events to the collector using the NETWORK SERVICE user of each domain controller. The NETWORK SERVICE user doesn't have access to read the security event log of a domain by default, so you need to add it to the ACL for the security event log using wevtutil. The command for this was: wevtutil sl security /ca:O:BAG:SYD:(A;;0xf0005;;;SY)(A;;0x5;;;BA)(A;;0x1;;;S-1-5-32-573)(A;;0x1;;;S-1-5-20) this effectively appends SID S-1-5-20 (which is for NETWORK SERVICE) to the ACL with read-only permissions for the security event log. This had to be run from an elevated prompt on each domain controller. It also required us to then reboot the domain controllers to see the logs arriving at Kibana, which arrived almost immediately after reboot. We found this out by reading this: https://rockyprogress.wordpress.com/2011/12/04/security-event-log-collection-from-a-domain-controller/ Hope this helps! Nick — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[5eanT]

This overall issue is true, and an automated script for wevutil that at least may be optional for people to run can help with security permissions on specific event log channels in addition to security. (there are operational log channels for example)
If the script were to be run, a comment for why and what each security permission is would need to be provided for clarity.
I was originally 1nf053c in this thread, maybe I can try to help with this aspect if it is seen as needed.