Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linux capabilities: normalization and auditbeat process support #37453

Merged
merged 22 commits into from
Feb 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
bc80df8
Capabilities normalization and auditbeat support
haesbaert Dec 4, 2023
593afcc
Error messages should be lowercase, thanks matt@
haesbaert Dec 13, 2023
810f4c2
Turn allMask into a package var, thanks to matt
haesbaert Dec 13, 2023
cac206b
range()->range, thanks to matt
haesbaert Dec 13, 2023
a3e7b8b
typo
haesbaert Dec 18, 2023
b6859ab
Add process.thread.capabilities to ECS fields
haesbaert Dec 18, 2023
1456e53
Make gofmt happy
haesbaert Dec 18, 2023
17d94fa
Curb journald test to linux
haesbaert Dec 18, 2023
62ade13
Update libbeat/common/capabilities/capabilities_linux_test.go
haesbaert Dec 20, 2023
4e33d51
Update libbeat/common/capabilities/capabilities_linux_test.go
haesbaert Dec 20, 2023
d7a0459
Update libbeat/common/capabilities/capabilities_other.go
haesbaert Dec 20, 2023
51ce44e
CHANGELOG.next.asciidoc entry
haesbaert Dec 19, 2023
c60b637
Fix the remaining empty slices cases
haesbaert Dec 20, 2023
020e2c5
Shuffle things around like Dan suggested
haesbaert Dec 20, 2023
28c241d
Rename 'v' to 'enabled' as suggested by Lee
haesbaert Jan 9, 2024
77218a7
Make the linter happy about error documentation
haesbaert Jan 11, 2024
2f2e444
Handle SetFlag error
haesbaert Jan 11, 2024
676cc5a
Merge remote-tracking branch 'origin/main' into newcaps.capall
haesbaert Jan 29, 2024
a008721
Remove CAP_ALL
haesbaert Jan 26, 2024
1271f79
Progress has arrived, go 1.21 has a generic ErrUnsupported
haesbaert Jan 26, 2024
7b717d8
Don't bother fetching capabilities of kernel processes
haesbaert Jan 26, 2024
6b158f9
Update CHANGELOG.next.asciidoc
haesbaert Jan 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d

*Auditbeat*

- Add linux capabilities to processes in the system/process. {pull}37453[37453]

*Filebeat*

Expand Down
22 changes: 22 additions & 0 deletions auditbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -18925,6 +18925,28 @@ type: keyword
--


*`process.thread.capabilities.effective`*::
+
--
This is the set of capabilities used by the kernel to perform permission checks for the thread.

type: keyword

example: ["CAP_BPF", "CAP_SYS_ADMIN"]

--

*`process.thread.capabilities.permitted`*::
+
--
This is a limiting superset for the effective capabilities that the thread may assume.

type: keyword

example: ["CAP_BPF", "CAP_SYS_ADMIN"]

--

[float]
=== hash

Expand Down
65 changes: 3 additions & 62 deletions filebeat/input/journald/pkg/journalfield/conv.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@

import (
"fmt"
"math/bits"
"regexp"
"strconv"
"strings"

"github.com/elastic/beats/v7/libbeat/common/capabilities"
"github.com/elastic/elastic-agent-libs/logp"
"github.com/elastic/elastic-agent-libs/mapstr"
)
Expand Down Expand Up @@ -190,72 +190,13 @@
if !ok {
return
}
w, err := strconv.ParseUint(c, 16, 64)
if err != nil {
return
}
if w == 0 {
caps, err := capabilities.FromString(c, 16)
if err != nil || len(caps) == 0 {
return
}
caps := make([]string, 0, bits.OnesCount64(w))
for i := 0; w != 0; i++ {
if w&1 != 0 {
if i < len(capTable) {
caps = append(caps, capTable[i])
} else {
caps = append(caps, strconv.Itoa(i))
}
}
w >>= 1
}
fields.Put("process.thread.capabilities.effective", caps)
}

// include/uapi/linux/capability.h
var capTable = [...]string{
0: "CAP_CHOWN",
1: "CAP_DAC_OVERRIDE",
2: "CAP_DAC_READ_SEARCH",
3: "CAP_FOWNER",
4: "CAP_FSETID",
5: "CAP_KILL",
6: "CAP_SETGID",
7: "CAP_SETUID",
8: "CAP_SETPCAP",
9: "CAP_LINUX_IMMUTABLE",
10: "CAP_NET_BIND_SERVICE",
11: "CAP_NET_BROADCAST",
12: "CAP_NET_ADMIN",
13: "CAP_NET_RAW",
14: "CAP_IPC_LOCK",
15: "CAP_IPC_OWNER",
16: "CAP_SYS_MODULE",
17: "CAP_SYS_RAWIO",
18: "CAP_SYS_CHROOT",
19: "CAP_SYS_PTRACE",
20: "CAP_SYS_PACCT",
21: "CAP_SYS_ADMIN",
22: "CAP_SYS_BOOT",
23: "CAP_SYS_NICE",
24: "CAP_SYS_RESOURCE",
25: "CAP_SYS_TIME",
26: "CAP_SYS_TTY_CONFIG",
27: "CAP_MKNOD",
28: "CAP_LEASE",
29: "CAP_AUDIT_WRITE",
30: "CAP_AUDIT_CONTROL",
31: "CAP_SETFCAP",
32: "CAP_MAC_OVERRIDE",
33: "CAP_MAC_ADMIN",
34: "CAP_SYSLOG",
35: "CAP_WAKE_ALARM",
36: "CAP_BLOCK_SUSPEND",
37: "CAP_AUDIT_READ",
38: "CAP_PERFMON",
39: "CAP_BPF",
40: "CAP_CHECKPOINT_RESTORE",
}

func getStringFromFields(key string, fields mapstr.M) string {
value, _ := fields.GetValue(key)
str, _ := value.(string)
Expand All @@ -276,7 +217,7 @@
}

// helpers for creating a field conversion table.
var ignoredField = Conversion{Dropped: true}

Check failure on line 220 in filebeat/input/journald/pkg/journalfield/conv.go

View workflow job for this annotation

GitHub Actions / lint (windows)

var `ignoredField` is unused (unused)

func text(names ...string) Conversion {
return Conversion{Names: names, IsInteger: false, Dropped: false}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
// specific language governing permissions and limitations
// under the License.

//go:build linux && cgo

package journalfield

import (
Expand Down Expand Up @@ -228,8 +230,8 @@ var expandCapabilitiesTests = []struct {
"CAP_PERFMON",
"CAP_BPF",
"CAP_CHECKPOINT_RESTORE",
"41",
"42",
"CAP_41",
"CAP_42",
},
},
},
Expand Down
161 changes: 161 additions & 0 deletions libbeat/common/capabilities/capabilities_linux.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

//go:build linux

package capabilities

import (
"errors"
"math/bits"
"strconv"
"strings"

"kernel.org/pub/linux/libs/security/libcap/cap"
)

var (
// errInvalidCapability expresses an invalid capability ID: x < 0 || x >= 64.
errInvalidCapability = errors.New("invalid capability")
)

// The capability set flag/vector, re-exported from
// libcap(3). Inherit, Bound & Ambient not exported since we have no
// use for it yet.
type Flag = cap.Flag

const (
// aka CapEff
Effective = cap.Effective
// aka CapPrm
Permitted = cap.Permitted
)

// Fetch the capabilities of pid for a given flag/vector and convert
// it to the representation used in ECS. cap.GetPID() fetches it with
// SYS_CAPGET.
// Returns errors.ErrUnsupported on "not linux".
func FromPid(flag Flag, pid int) ([]string, error) {
set, err := cap.GetPID(pid)
if err != nil {
return nil, err
}
empty, err := isEmpty(flag, set)
if err != nil {
return nil, err
}
if empty {
return []string{}, nil
}

sl := make([]string, 0, cap.MaxBits())
for i := 0; i < int(cap.MaxBits()); i++ {
c := cap.Value(i)
enabled, err := set.GetFlag(flag, c)
if err != nil {
return nil, err
}
if !enabled {
continue
}
s, err := toECS(i)
// impossible since MaxBits <= 64
if err != nil {
return nil, err
}
sl = append(sl, s)
}

return sl, err
}

// Convert a uint64 to the capabilities representation used in ECS.
// Returns errors.ErrUnsupported on "not linux".
func FromUint64(w uint64) ([]string, error) {
sl := make([]string, 0, bits.OnesCount64(w))
for i := 0; w != 0; i++ {
if w&1 != 0 {
s, err := toECS(i)
// impossible since MaxBits <= 64
if err != nil {
return nil, err
}
sl = append(sl, s)
}
w >>= 1
}

return sl, nil
}

// Convert a string to the capabilities representation used in
// ECS. Example input: "1ffffffffff", 16.
// Returns errors.ErrUnsupported on "not linux".
func FromString(s string, base int) ([]string, error) {
w, err := strconv.ParseUint(s, 16, 64)
if err != nil {
return nil, err
}

return FromUint64(w)
}

// True if sets are equal for the given flag/vector, errors out in
// case any of the sets is malformed.
func isEqual(flag Flag, a *cap.Set, b *cap.Set) (bool, error) {
d, err := a.Cf(b)
if err != nil {
return false, err
}

return !d.Has(flag), nil
}

// Convert the capability ID to a string suitable to be used in
// ECS.
// If capabiliy ID X is unknown, but valid (0 <= X < 64), "CAP_X"
// will be returned instead. Fetches from an internal table built at
// startup.
var toECS = makeToECS()

// Make toECS() which creates a map of every possible valid capability
// ID on startup. Returns errInvalidCapabilty for an invalid ID.
func makeToECS() func(int) (string, error) {
ecsNames := make(map[int]string)

for i := 0; i < 64; i++ {
c := cap.Value(i)
if i < int(cap.MaxBits()) {
ecsNames[i] = strings.ToUpper(c.String())
} else {
ecsNames[i] = strings.ToUpper("CAP_" + c.String())
}
}

return func(b int) (string, error) {
s, ok := ecsNames[b]
if !ok {
return "", errInvalidCapability
}
return s, nil
}
}

// Like isAll(), but for the empty set, here for symmetry.
func isEmpty(flag Flag, set *cap.Set) (bool, error) {
return isEqual(flag, set, cap.NewSet())
}
87 changes: 87 additions & 0 deletions libbeat/common/capabilities/capabilities_linux_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package capabilities

import (
"os"
"testing"

"github.com/stretchr/testify/assert"
"kernel.org/pub/linux/libs/security/libcap/cap"
)

func TestEmpty(t *testing.T) {
sl, err := FromString("0", 16)
assert.Nil(t, err)
assert.Equal(t, len(sl), 0)

sl, err = FromUint64(0)
assert.Nil(t, err)
assert.Equal(t, len(sl), 0)

// assumes non root has no capabilities
if os.Geteuid() != 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it must be true that euid != 0 has no capabilities. If that were to be not true in CI, that could lead to an interesting debugging scenario. Can this be reworked or mock out the os.Geteuid() call?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, you can still have capabilities as a non-root user, I've added the comment to reflect that, I guess it's just rare enough that we could live with it.
I can't think of a good way to mock it, one way would be to drop all capabilities that are "Effective & Permitted", then test Effective for empty and restore the old set, but this is too impure and doesn't sound very appealing to me.
I'd either leave it and improve the comment or just remove it all together, let me know what you think.

empty := cap.NewSet()
self := cap.GetProc()
d, err := self.Cf(empty)
assert.Nil(t, err)
assert.False(t, d.Has(cap.Effective))
assert.False(t, d.Has(cap.Permitted))
assert.False(t, d.Has(cap.Inheritable))
}
}

func TestOverflow(t *testing.T) {
sl, err := FromUint64(^uint64(0))
assert.Nil(t, err)
assert.Equal(t, len(sl), 64)

for _, cap := range []string{
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_DAC_READ_SEARCH",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_KILL",
"CAP_SETGID",
"CAP_SYS_MODULE",
"CAP_SYS_RAWIO",
"CAP_IPC_LOCK",
"CAP_MAC_OVERRIDE",
} {
assertHasCap(t, sl, cap)
}
if cap.MaxBits() <= 62 {
assertHasCap(t, sl, "CAP_62")
}
if cap.MaxBits() <= 63 {
assertHasCap(t, sl, "CAP_63")
}
}

func assertHasCap(t *testing.T, sl []string, s string) {
var found int

for _, s2 := range sl {
if s2 == s {
found++
}
}

assert.Equal(t, found, 1, s)
}
Loading
Loading