Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[libbeat] Fingerprint processor cannot access @timestamp #28683

Closed
andrewkroh opened this issue Oct 27, 2021 · 2 comments · Fixed by #28705
Closed

[libbeat] Fingerprint processor cannot access @timestamp #28683

andrewkroh opened this issue Oct 27, 2021 · 2 comments · Fixed by #28705
Labels
bug libbeat :Processors Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@andrewkroh
Copy link
Member

The fingerprint processor directly accesses beat.Event.Fields to get data for the fingerprint. Therefore it cannot see the @timestamp because that is stored separately. It should use beat.Event.GetValue and then this would work as expected.

processors:
- fingerprint:
    fields: [winlog.computer, winlog.channel, winlog.record_id, '@timestamp']
    method: sha256
    target_field: '@metadata._id'

"failed applying processor fingerprint=[method=[0x1f460000]]: failed to compute fingerprint: failed to find field [@timestamp] in event: key not found"
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 27, 2021
andrewkroh added a commit to andrewkroh/beats that referenced this issue Oct 28, 2021
@andrewkroh
Allow fingerprint processor to read @timestamp
34f0922 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes elastic#28683
@andrewkroh andrewkroh mentioned this issue Oct 28, 2021
Merged
6 tasks
@jsoriano
Copy link
Member

There are similar inconsistencies on processors that add fields #25425, perhaps processors should not have access to the underlying fields and use only the GetFields/SetFields interface.

@jsoriano jsoriano added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Oct 29, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 29, 2021
andrewkroh added a commit that referenced this issue Nov 2, 2021
@andrewkroh
Allow fingerprint processor to read @timestamp (#28705)
6390852 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683
mergify bot pushed a commit that referenced this issue Nov 2, 2021
@andrewkroh
Allow fingerprint processor to read @timestamp (#28705)
0e38c73 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683

(cherry picked from commit 6390852)
mergify bot pushed a commit that referenced this issue Nov 2, 2021
@andrewkroh
Allow fingerprint processor to read @timestamp (#28705)
8368860 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683

(cherry picked from commit 6390852)
mergify bot pushed a commit that referenced this issue Nov 2, 2021
@andrewkroh
Allow fingerprint processor to read @timestamp (#28705)
ae734f5 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683

(cherry picked from commit 6390852)
andrewkroh added a commit that referenced this issue Nov 2, 2021
@mergify @andrewkroh
Allow fingerprint processor to read @timestamp (#28705) (#28766)
4f8d721 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683

(cherry picked from commit 6390852)

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
andrewkroh added a commit that referenced this issue Nov 2, 2021
@mergify @andrewkroh
Allow fingerprint processor to read @timestamp (#28705) (#28767)
835a241 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683

(cherry picked from commit 6390852)

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
andrewkroh added a commit that referenced this issue Nov 2, 2021
@mergify @andrewkroh
Allow fingerprint processor to read @timestamp (#28705) (#28768)
523cfb9 
Use `beat.Event.GetValue` so that @timestamp and @metadata are accessible to
be hashed.

Also clarify the documentation to explain how the value being hashed is constructed.

Fixes #28683

(cherry picked from commit 6390852)

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug libbeat :Processors Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[libbeat] Allow fingerprint processor to read @timestamp andrewkroh/beats
3 participants
@jsoriano @andrewkroh @elasticmachine