Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vCenter Parser: Error in parsing events of type UserEventsLogoutEvent when the value of "number of API invocations" within the raw event message is not an integer. #11542

Open
kevintamlsWork opened this issue Dec 9, 2024 · 15 comments
Assignees
Labels
Parser Parser specialty review needed

Comments

@kevintamlsWork
Copy link

Describe the bug
The vCenter Parser fails to parse events of type UserEventsLogout when the value of "number of API invocations" within the raw event message is not an integer.

To Reproduce
Steps to reproduce the behavior:

  1. Go to Sentinel or Log Analytics
  2. Attempt to perform a search using the KQL query "vCenter () | where Message has ("UserLogoutSessionEvent").
  3. Locate events where in the raw message, has a number containing a comma after "number of API invocations:"
  4. See error

Expected behavior
Events that contain a comma within the APIInvocationCount field to be parsed correctly in all fields.

Screenshots
Some values of fields within events has been redacted.

Working:
Image

Not Working:
Image

Desktop (please complete the following information):

  • OS: Windows 11
  • Browser: Google Chrome
  • Version: 131.0.6778.86

Additional context
N/A

@v-sudkharat
Copy link
Contributor

Hi @kevintamlsWork, Thanks for flagging this issue, we will investigate this issue and get back to you with some updates. Thanks!

@v-sudkharat
Copy link
Contributor

@kevintamlsWork, could you please share the sample logs with us on mail id - v-sudkharat@microsoft.com Thanks!

@kevintamlsWork
Copy link
Author

Hi @v-sudkharat, just sent a selection to your email. Do let me know if you'd like further information or clarifications. Thanks.

@v-sudkharat
Copy link
Contributor

@kevintamlsWork, Received the mail, Will investigate on it and get back to you with some updates. Thanks!

@v-visodadasi v-visodadasi removed their assignment Dec 18, 2024
@kevintamlsWork
Copy link
Author

@v-sudkharat Thank you, will await you response. Thanks.

@v-sudkharat
Copy link
Contributor

Hi @kevintamlsWork, Thanks for sharing the RAW logs which help us to address this issue.
We have updated the parser to meet your expected results, as for the Number of API Invocations values are in comma separated format. Could you please run/test the below shared parser in LAW and let's know if still it not shows the expected results :

    let vCenter_Login =() {
        vcenter_CL
        | where Message has ("UserLoginSessionEvent")
        | parse Message  with * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity "]" * "[User " DomainName:string "\\" Username:string "@" SourceIP " logged in as " UserAgent:string "]" *
    };
    let vCenter_Logout =() {
        vcenter_CL
        | where Message has ("UserLogoutSessionEvent")
        | parse Message with * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity "]" * "[User" DomainName:string "\\" Username:string "@" SourceIP " logged out (login time:" LoginTime:string ", number of API invocations: " APIInvocationCount:dynamic ", user agent:" UserAgent:string ")]" *
    };
    let vCenter_Role=() {
        vcenter_CL
        | where Message has_any("RoleAddedEvent","RoleRemovedEvent")
        | parse Message with * " " * " " Hostname:string "vpxd" * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity:string "]" * "[New role " RoleName:string " " Operation:string  "]" *
    };
    let vCenter_RoleModified=() {
        vcenter_CL
        | where Message has ("RoleUpdatedEvent")
        | parse Message with * " " * " " Hostname:string "vpxd" * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity:string "]" * "Previous name: " OldRoleName:string ", new name "NewRoleName:string " Added privileges: " AddedPriviledges:string " Removed privileges: " RemovedPriviledges:string "]" *
    };
    union vCenter_Login,vCenter_Logout,vCenter_Role,vCenter_RoleModified
    | extend ClientIP = SourceIP

Thanks!

@kevintamlsWork
Copy link
Author

@v-sudkharat that has worked for this specific parsing issue. However, we are encountered another parsing issue separate from this issue. Shall I keep using this issue or open a new one? Thanks

@v-sudkharat
Copy link
Contributor

@kevintamlsWork, Could you please share what new issue your facing? and it's due to the above shared parser or it due to the any filed not parsing correctly?

@kevintamlsWork
Copy link
Author

@v-sudkharat It's not related to the above shared parser, this issue affects both Login and Logout events and results in all fields not being parsed correctly. This issue affects the original parser also so is unrelated to the shared parser above.

Describe the bug
The vCenter Parser fails to parse all fields in events of type UserLoginSessionEvent and UserLogoutSessionEvent when either (this requires investigation on your end)

  • More Likely: The [User] message field value format is not in "domain\userName@ip", or possibly;
  • Less Likely: If there is a value in between the [domainName\userName] and [threadNumber] fields

To Reproduce
Steps to reproduce the behavior:

  1. Go to Sentinel or Log Analytics
  2. Attempt to perform a search using the KQL query

vCenterV2
| where EventType != "vim.event.UserLogoutSessionEvent"
| where EventType != "vim.event.UserLoginSessionEvent"

  1. See error

Expected behavior
There should be no events, any events that displays shows that the eventType and other fields have not been extracted properly, thus matching the conditions in this KQL search query.

Screenshots
Some values of fields within events has been redacted.

Working:

Image

Not working:

Image

Sample logs below, do let me know if you would like more in an email alongside a more detailed description of the issue, I have put the areas we think may be the cause of the issue in Bold and Italics. OS, and browser used are the same.

Login Events

Not Working

Dec 18 14:36:56 SERVERNAME vpxd[7058] Event [50297246] [1-1] [2024-12-18T14:36:47.241233Z] [vim.event.UserLoginSessionEvent] [info] [username] [xxx] [50297246] [User userName@xxx.xxx.xxx.xxx logged in as VMware-client/6.5.0]

Working

Dec 19 10:30:27 SERVERNAME vpxd[7058] Event [50319750] [1-1] [2024-12-19T10:30:27.644457Z] [vim.event.UserLoginSessionEvent] [info] [domainName\userName] [] [50319750] [User domainName\userName@xxx.xxx.xxx.xxx logged in as VMware vim-java 1.0]

Logout Events

Not Working

Dec 19 10:41:18 SERVERNAME vpxd[7058] Event [50319923] [1-1] [2024-12-19T10:41:00.843279Z] [vim.event.UserLogoutSessionEvent] [info] [username] [xxx] [50319923] [User userName@xxx.xxx.xxx.xxx logged out (login time: Thursday, 19 December, 2024 10:10:30 AM, number of API invocations: 6, user agent: VMware-client/6.5.0)]

Working

Dec 19 09:50:02 SERVERNAME vpxd[7058] Event [50319033] [1-1] [2024-12-19T09:50:02.514923Z] [vim.event.UserLogoutSessionEvent] [info] [domainName\userName] [] [50319033] [User domainName\userName@xxx.xxx.xxx.xxx logged out (login time: Thursday, 19 December, 2024 09:50:02 AM, number of API invocations: 1, user agent: Apache-CXF/3.4.10)]

@v-sudkharat
Copy link
Contributor

@kevintamlsWork, Ok, let us investigate on it.

@kevintamlsWork
Copy link
Author

@v-sudkharat Thank you, much appreciated.

@v-sudkharat
Copy link
Contributor

@kevintamlsWork, the Parser not parsing the below RAW format -

Dec 18 14:36:56 SERVERNAME vpxd[7058] Event [50297246] [1-1] [2024-12-18T14:36:47.241233Z] [vim.event.UserLoginSessionEvent] [info] [username] [xxx] [50297246] [User userName@xxx.xxx.xxx.xxx logged in as VMware-client/6.5.0]

And

Dec 19 10:41:18 SERVERNAME vpxd[7058] Event [50319923] [1-1] [2024-12-19T10:41:00.843279Z] [vim.event.UserLogoutSessionEvent] [info] [username] [xxx] [50319923] [User userName@xxx.xxx.xxx.xxx logged out (login time: Thursday, 19 December, 2024 10:10:30 AM, number of API invocations: 6, user agent: VMware-client/6.5.0)]
can you fill some dummy values in above fields and share.

@kevintamlsWork
Copy link
Author

@v-sudkharat yes sure

Dec 18 14:36:56 VMWARE-P-01-01 vpxd[7058] Event [50297246] [1-1] [2024-12-18T14:36:47.241233Z] [vim.event.UserLoginSessionEvent] [info] [test] [GH1] [50297246] [User test@127.0.0.1 logged in as VMware-client/6.5.0]

Dec 19 10:41:18 VMWARE-P-01-01 vpxd[7058] Event [50319923] [1-1] [2024-12-19T10:41:00.843279Z] [vim.event.UserLogoutSessionEvent] [info] [test] [GH1] [50319923] [User test@127.0.0.1 logged out (login time: Thursday, 19 December, 2024 10:10:30 AM, number of API invocations: 6, user agent: VMware-client/6.5.0)]

@v-sudkharat
Copy link
Contributor

@kevintamlsWork, Updated the below parser to meet your requirements: -

Could you please test and let us know if it meets the results:

    let vCenter_Login =() {
        vcenter_CL
        | where Message has ("UserLoginSessionEvent")
        | parse Message with * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity "]" * "[User " DomainName:string "\\" Username:string "@" SourceIP " logged in as " UserAgent:string "]" *
        | extend DomainName = iff(isnull(DomainName), "", DomainName)
        | extend Username = iff(isnull(Username), DomainName, Username)
        | parse Message with * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity "]" * "[User " Username:string "@" SourceIP " logged in as " UserAgent:string "]" *
        | extend Username = iff(isnull(Username), DomainName, Username)
    };
    let vCenter_Logout =() {
        vcenter_CL
        | where Message has ("UserLogoutSessionEvent")
        | parse Message with * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity "]" * "[User " DomainName:string "\\" Username:string "@" SourceIP " logged out (login time:" LoginTime:string ", number of API invocations: " APIInvocationCount:dynamic ", user agent:" UserAgent:string ")]" *
        | extend DomainName = iff(isnull(DomainName), "", DomainName)
        | extend Username = iff(isnull(Username), DomainName, Username)
        | parse Message with * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity "]" * "[User " Username:string "@" SourceIP " logged out (login time:" LoginTime:string ", number of API invocations: " APIInvocationCount:dynamic ", user agent:" UserAgent:string ")]" *
        | extend Username = iff(isnull(Username), DomainName, Username)
    };
    let vCenter_Role=() {
        vcenter_CL
        | where Message has_any("RoleAddedEvent","RoleRemovedEvent")
        | parse Message with * " " * " " Hostname:string "vpxd" * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity:string "]" * "[New role " RoleName:string " " Operation:string  "]" *
    };
    let vCenter_RoleModified=() {
        vcenter_CL
        | where Message has ("RoleUpdatedEvent")
        | parse Message with * " " * " " Hostname:string "vpxd" * "Event [" EventId:string "] [1-1] [" EventTime:datetime "] [" EventType:string "] [" EventSeverity:string "]" * "Previous name: " OldRoleName:string ", new name "NewRoleName:string " Added privileges: " AddedPriviledges:string " Removed privileges: " RemovedPriviledges:string "]" *
    };
    union vCenter_Login,vCenter_Logout,vCenter_Role,vCenter_RoleModified
    | extend ClientIP = SourceIP

@kevintamlsWork
Copy link
Author

@v-sudkharat Will do, do expect a reply after Christmas though as I will be busy over it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Parser Parser specialty review needed
Projects
None yet
Development

No branches or pull requests

4 participants