Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] convert netflow input to API v2 #37901

Merged
merged 13 commits into from
Feb 12, 2024
Merged

[Filebeat] convert netflow input to API v2 #37901

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
00ceb28
feat: migrate netflow input to v2.Plugin
pkoutsovasilis Feb 7, 2024
eb59d65
fix: replace deprecated ioutil usage
pkoutsovasilis Feb 7, 2024
c51ba25
fix: remove unnecessary type conversions
pkoutsovasilis Feb 7, 2024
59a57f9
doc: update CHANGELOG.next.asciidoc
pkoutsovasilis Feb 7, 2024
9b64dca
ci: install missing libpcap-dev for linux lint step
pkoutsovasilis Feb 9, 2024
86f40dc
fix: utilise require.Eventually to wait for conditions to become true…
pkoutsovasilis Feb 9, 2024
63be274
fix: proper log verbosity for errors during stopping netflow input
pkoutsovasilis Feb 9, 2024
9449fb1
fix: remove redundant config creation
pkoutsovasilis Feb 9, 2024
d34cac2
fix: utilise a std context in statsLoop
pkoutsovasilis Feb 9, 2024
f64f697
fix: add server in udp regarding log messages
pkoutsovasilis Feb 9, 2024
a06c8c7
fix: remove redundant empty line in CHANGELOG.next.asciidoc
pkoutsovasilis Feb 9, 2024
1e5e4d1
fix: enrich more the messages of eventual condition in netflow tests
pkoutsovasilis Feb 9, 2024
a1d60d0
Merge remote-tracking branch 'beats/main' into pkoutsovasilis/netflow_v2
pkoutsovasilis Feb 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/golangci-lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ jobs:
with:
go-version-file: .go-version

- name: Install Apt Package
run: sudo apt-get update && sudo apt-get install -y libpcap-dev

- name: golangci-lint
env:
GOOS: ${{ matrix.GOOS }}
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ fields added to events containing the Beats version. {pull}37553[37553]

*Filebeat*

- Convert netflow input to API v2 and disable event normalisation {pull}37901[37901]


*Heartbeat*

Expand Down
8 changes: 8 additions & 0 deletions x-pack/dockerlogbeat/pipelinemock/pipelines.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,11 @@ func (pc *MockPipelineConnector) ConnectWith(beat.ClientConfig) (beat.Client, er

return c, nil
}

// HasConnectedClients returns true if there are clients connected.
func (pc *MockPipelineConnector) HasConnectedClients() bool {
pc.mtx.Lock()
defer pc.mtx.Unlock()

return len(pc.clients) > 0
}
3 changes: 1 addition & 2 deletions x-pack/dockerlogbeat/pipelinemock/reader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import (
"bytes"
"encoding/binary"
"io"
"io/ioutil"
"testing"

"github.com/docker/docker/api/types/plugins/logdriver"
Expand All @@ -33,7 +32,7 @@ func CreateTestInputFromLine(t *testing.T, line string) io.ReadCloser {
writer := new(bytes.Buffer)

encodeLog(t, writer, exampleStruct)
return ioutil.NopCloser(writer)
return io.NopCloser(writer)
}

func encodeLog(t *testing.T, out io.Writer, entry *logdriver.LogEntry) {
Expand Down
2 changes: 2 additions & 0 deletions x-pack/filebeat/input/default-inputs/inputs_other.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"github.com/elastic/beats/v7/x-pack/filebeat/input/http_endpoint"
"github.com/elastic/beats/v7/x-pack/filebeat/input/httpjson"
"github.com/elastic/beats/v7/x-pack/filebeat/input/lumberjack"
"github.com/elastic/beats/v7/x-pack/filebeat/input/netflow"
"github.com/elastic/beats/v7/x-pack/filebeat/input/o365audit"
"github.com/elastic/beats/v7/x-pack/filebeat/input/shipper"
"github.com/elastic/elastic-agent-libs/logp"
Expand All @@ -39,5 +40,6 @@ func xpackInputs(info beat.Info, log *logp.Logger, store beater.StateStore) []v2
awscloudwatch.Plugin(),
lumberjack.Plugin(),
shipper.Plugin(log, store),
netflow.Plugin(log),
}
}
3 changes: 1 addition & 2 deletions x-pack/filebeat/input/netflow/case.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,8 @@ func CamelCaseToSnakeCase(in string) string {
}

out := make([]rune, 0, len(in)+4)
runes := []rune(in)
upperCount := 1
for _, r := range runes {
for _, r := range in {
lr := unicode.ToLower(r)
isUpper := lr != r
if isUpper {
Expand Down
6 changes: 3 additions & 3 deletions x-pack/filebeat/input/netflow/definitions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ package netflow
import (
"errors"
"fmt"
"io/ioutil"
"io"
"math"
"os"
"strconv"
Expand Down Expand Up @@ -95,7 +95,7 @@ func LoadFieldDefinitionsFromFile(path string) (defs fields.FieldDict, err error
return nil, err
}
defer file.Close()
contents, err := ioutil.ReadAll(file)
contents, err := io.ReadAll(file)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -169,7 +169,7 @@ func loadFields(def map[interface{}]interface{}, pem uint32, dest fields.FieldDi
return fmt.Errorf("bad field ID %d: should have two items (type, name) or one (:skip) (Got %+v)", fieldID, list)
}
key := fields.Key{
EnterpriseID: uint32(pem),
EnterpriseID: pem,
FieldID: uint16(fieldID),
}
if _, exists := dest[key]; exists {
Expand Down
Loading
Loading