Skip to content

Commit

Permalink
ebpf: check for known faulty Ubuntu kernel
Browse files Browse the repository at this point in the history
The Ubuntu Xenial update to kernel 4.4.0-119.143 from 4.4.0-116.140 did
include a regression in the eBPF code. A basic `bpf_map_lookup_elem`
call as found in the tcptracer-bpf library used by Scope leads to a
kernel panic. As a result, Scope / the system crashes during startup
when the tcptracer is initialized. The Scope bug report can be found
here:

weaveworks#3131

To avoid crashes and gently fallback to procfs (as Scope already does
for systems not supporting eBPF), update `isKernelSupported()` and
explicitly check for Ubuntu Kernel versions with the problem.

Once the bug is fixed and an update published, the `abiNumber` check in
`isKernelSupported()` can and should be updated with an upper limit.

The Ubuntu bug report can be found here:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454
  • Loading branch information
schu committed Apr 13, 2018
1 parent e2b4b3e commit c75700f
Showing 1 changed file with 33 additions and 3 deletions.
36 changes: 33 additions & 3 deletions probe/endpoint/ebpf.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,28 @@ type EbpfTracker struct {
closedDuringInit map[fourTuple]struct{}
}

var releaseRegex = regexp.MustCompile(`^(\d+)\.(\d+).*$`)
// releaseRegex should match all possible variations of a common Linux
// version string:
// - 4.1
// - 4.22-foo
// - 4.1.2-foo
// - 4.1.2-33.44+bar
// - etc.
// For example, on a Ubuntu system the vendor specific release part
// (after the first `-`) could look like:
// '<ABI number>.<upload number>-<flavour>' or
// '<ABI number>-<flavour>'
// See https://wiki.ubuntu.com/Kernel/FAQ
var releaseRegex = regexp.MustCompile(`^(\d+)\.(\d+)\.?(\d*)-?(\d*)(.*)$`)

func isKernelSupported() error {
release, _, err := host.GetKernelReleaseAndVersion()
release, version, err := host.GetKernelReleaseAndVersion()
if err != nil {
return err
}

releaseParts := releaseRegex.FindStringSubmatch(release)
if len(releaseParts) != 3 {
if len(releaseParts) != 6 {
return fmt.Errorf("got invalid release version %q (expected format '4.4[.2-1]')", release)
}

Expand All @@ -84,6 +96,24 @@ func isKernelSupported() error {
return fmt.Errorf("got kernel %s but need kernel >=4.4", release)
}

if strings.Contains(version, "Ubuntu") {
// Check for specific Ubuntu kernel versions with
// known issues.

abiNumber, err := strconv.Atoi(releaseParts[4])
if err != nil {
// By now we know it's at least kernel 4.4 and
// not "119-ish", so allow it.
return nil
}
// TODO: give the check an upper limit once the bug is fixed
if major == 4 && minor == 4 && abiNumber >= 119 {
// https://github.com/weaveworks/scope/issues/3131
// https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454
return fmt.Errorf("got Ubuntu kernel %s with known bug", release)
}
}

return nil
}

Expand Down

0 comments on commit c75700f

Please sign in to comment.