Add versions to product names in MSRC bulletins to aid Windows vulnerability matching #24172

sgress454 · 2024-11-26T17:55:42Z

This PR addresses an issue that can cause Windows vulnerability checks to fail (possibly causing false negatives). We determine whether a vulnerability in an MSRC bulletin applies to any hosts in a Fleet instance by attempting to matching the data in each row of the operating_systems table with at least one "product" in a bulletin, including matching architecture and "display version". However a subset of products listed in these bulletins do not include the display version, so for example a host whose OS was listed as Microsoft Windows Server 2022 Datacenter 21H2 (21H2 being the "display version") would match nothing in the bulletins because no listed Server 2022 products include "21H2" in their names.

The fix made here is to add relevant version info to the products list when we do our ETL of the MSRC bulletins. The version info was gleaned from https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions.

We see logs related to this issue a lot, so cleaning this up will alleviate some noise and infra costs as well.

sgress454 · 2024-11-26T17:57:02Z

server/vulnerabilities/msrc/parsed/product.go

+	p := Product(fullName)
+	if p.HasDisplayVersion() {
+		return p
+	}


This ensures this function is still idempotent and won't keep adding version strings.

sgress454 · 2024-11-26T17:57:55Z

server/vulnerabilities/msrc/parsed/security_bulletin.go

+	for pID, name := range bulletin.Products {
+		bulletin.Products[pID] = NewProductFromFullName(string(name))
+	}


Utilize the new logic in NewProductFromFullName to add version strings when needed.

i'm not sure this is needed here as NewProductFromFullName is already modifying the published feed here:

fleet/server/vulnerabilities/msrc/parser.go

Line 50 in cf6e15f

bulletins[name].Products[pID] = parsed.NewProductFromFullName(p.FullName)

I think I needed it here for when we process existing .json files that were saved without the versions added to the product names.

sgress454 · 2024-11-26T18:09:19Z

server/vulnerabilities/msrc/parsed/product.go

+		versionString = "1809"
+
+	case strings.Contains(fullName, "Windows 8.1"):
+		versionString = "6.3 / NT 6.3"


I've seen it both ways (6.3 and NT 6.3) and it's hard to say what the devices report, so putting both to be safe.

lukeheath · 2024-12-03T22:18:26Z

@mostlikelee Please give this a review when you have a moment. Thanks!

mostlikelee

it seems like the general assumption we're making is that by adding a "display version" into our msrc feed, more operating system "products" will start matching against Fleet operating systems. We may need more data to see if osquery does indeed report display versions on these systems.

mostlikelee · 2024-12-11T14:08:14Z

server/vulnerabilities/msrc/parsed/product.go

+	case strings.Contains(fullName, "Windows Server 2019"):
+		versionString = "1809"
+
+	case strings.Contains(fullName, "Windows 8.1"):


a solution here could be to drop the 8.1 feed as it is EOL by microsoft last year. Fleet generally follows the EOL lifecycle of the vendor. There may be a few more in this list.

Yeah we have Windows 7 in there too, could definitely drop those two if we want. The bigger culprits are the different Windows Server versions, which tend to not have version info in the product name.

sgress454 · 2024-12-11T15:43:49Z

it seems like the general assumption we're making is that by adding a "display version" into our msrc feed, more operating system "products" will start matching against Fleet operating systems. We may need more data to see if osquery does indeed report display versions on these systems.

That's definitely the assumption I'm making here. I dumped the OS table from dogfood into this spreadsheet, and for the versions in there the display version was always reported. I can't guarantee it for other versions without checking against VMs for different systems, which I'm happy to do if we think it's worth the effort (but would still be far from exhaustive given all the different Windows flavors). As it stands, this update won't make things worse, and if it turns out that some systems don't report their version # then we'll have to put more serious thought into how to do this matching.

Another idea would be to update the "no product matches" error with the name of the OS we're trying to match, to gather datapoints that way. I can put that in this PR.

mostlikelee

agree if we're getting consistent display versions, this should help although we will have some maintenance to do as new os versions are released. We may also want to consider reverting some of this change as it addressed an issue on first release before hosts had a chance to report display versions. reverting it will match os versions against the "display name"-less product in the list.

sgress454 · 2024-12-12T22:35:44Z

cmd/fleet/cron.go

@@ -309,7 +309,7 @@ func checkWinVulnerabilities(
 				"found new", len(r))
 			results = append(results, r...)
 			if err != nil {
-				errHandler(ctx, logger, "analyzing hosts for Windows vulnerabilities", err)
+				errHandler(ctx, kitlog.With(logger, "os name", o.Name, "display version", o.DisplayVersion), "analyzing hosts for Windows vulnerabilities", err)


this should help identify any OSs that still aren't matching.

codecov · 2024-12-12T22:42:34Z

Codecov Report

Attention: Patch coverage is 97.22222% with 1 line in your changes missing coverage. Please review.

Project coverage is 63.55%. Comparing base (3d671f1) to head (1611db6).
Report is 48 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/fleet/cron.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24172      +/-   ##
==========================================
- Coverage   63.56%   63.55%   -0.02%     
==========================================
  Files        1601     1595       -6     
  Lines      151671   151243     -428     
  Branches     3898     3713     -185     
==========================================
- Hits        96416    96119     -297     
+ Misses      47585    47463     -122     
+ Partials     7670     7661       -9

Flag	Coverage Δ
backend	`64.37% <97.22%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sgress454 · 2024-12-12T22:57:19Z

@mostlikelee had to push an update as some tests were failing. I also updated the error log to show exactly what OS isn't matching, so if the logs don't drop completely after this change we can at least see where the gaps are.

We may also want to consider reverting some of this change as it addressed an issue on first release before hosts had a chance to report display versions

I'll defer to you on this. Could removing this lead to false positives for systems that match on the OS name aren't one of those builds? Or are you saying that the systems we were concerned with when that fix was put in will now report a display version?

add versions to product names in bulletins

76f5bac

sgress454 requested a review from a team as a code owner November 26, 2024 17:55

sgress454 temporarily deployed to Docker Hub November 26, 2024 17:55 — with GitHub Actions Inactive

sgress454 changed the title ~~add versions to product names in bulletins~~ Add versions to product names in MSRC bulletins to aid Windows vulnerability matching Nov 26, 2024

sgress454 commented Nov 26, 2024

View reviewed changes

sgress454 requested a review from mostlikelee November 26, 2024 18:09

Merge branch 'main' into 24041-fix-win-vuln-job

507bfac

sgress454 temporarily deployed to Docker Hub December 3, 2024 14:46 — with GitHub Actions Inactive

mostlikelee reviewed Dec 11, 2024

View reviewed changes

mostlikelee self-assigned this Dec 11, 2024

mostlikelee previously approved these changes Dec 12, 2024

View reviewed changes

sgress454 added 2 commits December 12, 2024 13:27

fix test

e7cefe0

update log

c4ffe7b

sgress454 dismissed mostlikelee’s stale review via c4ffe7b December 12, 2024 19:27

sgress454 temporarily deployed to Docker Hub December 12, 2024 19:27 — with GitHub Actions Inactive

Merge branch 'main' into 24041-fix-win-vuln-job

205aefd

sgress454 temporarily deployed to Docker Hub December 12, 2024 21:42 — with GitHub Actions Inactive

Merge branch 'main' into 24041-fix-win-vuln-job

1611db6

sgress454 temporarily deployed to Docker Hub December 12, 2024 22:02 — with GitHub Actions Inactive

sgress454 commented Dec 12, 2024

View reviewed changes

mostlikelee approved these changes Dec 16, 2024

View reviewed changes

sgress454 merged commit edc68d3 into main Dec 17, 2024
22 checks passed

sgress454 deleted the 24041-fix-win-vuln-job branch December 17, 2024 15:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add versions to product names in MSRC bulletins to aid Windows vulnerability matching #24172

Add versions to product names in MSRC bulletins to aid Windows vulnerability matching #24172

sgress454 commented Nov 26, 2024 •

edited

Loading

sgress454 Nov 26, 2024

sgress454 Nov 26, 2024

mostlikelee Dec 11, 2024

sgress454 Dec 12, 2024

sgress454 Nov 26, 2024

lukeheath commented Dec 3, 2024

mostlikelee left a comment

mostlikelee Dec 11, 2024

sgress454 Dec 11, 2024

sgress454 commented Dec 11, 2024

mostlikelee left a comment

sgress454 Dec 12, 2024

codecov bot commented Dec 12, 2024 •

edited

Loading

sgress454 commented Dec 12, 2024

Add versions to product names in MSRC bulletins to aid Windows vulnerability matching #24172

Add versions to product names in MSRC bulletins to aid Windows vulnerability matching #24172

Conversation

sgress454 commented Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukeheath commented Dec 3, 2024

mostlikelee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgress454 commented Dec 11, 2024

mostlikelee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Dec 12, 2024 • edited Loading

Codecov Report

sgress454 commented Dec 12, 2024

sgress454 commented Nov 26, 2024 •

edited

Loading

codecov bot commented Dec 12, 2024 •

edited

Loading