Add --cvss-fail-threshold

See README for general description.

New option is mutually exclusive to `--fail-on-result`; if both are
specified, clj-watson fails fast with usage error and help.

Conservatively derives score when missing or suspicious looking:
- When severity is available, conservatively converts to score
- Since we don't know if if score is CVSS2 or CVSS3/4 derives, High and
Critical to 10.0, Medium and Low are converted to upper bound of their
ranges.
- The experimental github-advisory strategy seems to regularly populate score
with `0.0` but with a valid looking severity; we treat a score of 0.0 as
suspicious.
- I've not seen cases of invalid severities in the wild, but we handle
them just the same, when we can't make sense of things we derive to the
most critical score which is 10.0.

Also:
- factored out table support from cli-spec ns to new table ns to reuse it
for summary table.
- renamed summarize fn to final-summary to better distinguish from our new
summary fn
- a new utils ns for `assoc-some` fn (cribbed clj-kondo which cribbed from medley).

Closes clj-holmes#114

CHANGELOG.md

@@ -1,6 +1,7 @@
 # CHANGELOG
 
 * Unreleased
+  * Add `--cvss-fail-threshold` to fail when a vulnerability meets or exceeds a given CVSS score [#114](https://github.com/clj-holmes/clj-watson/issues/114)
   * Fix: `--output json` now renders correctly & JSON output now pretty-printed [#116](https://github.com/clj-holmes/clj-watson/issues/116)
   * Recognize CVSS2 and CVSS4 scores when available [#112](https://github.com/clj-holmes/clj-watson/issues/112)
   * Show short summary of findings [#87](https://github.com/clj-holmes/clj-watson/issues/87)

README.md

@@ -329,6 +329,11 @@ OPTIONS:
   -s, --suggest-fix                                          Include dependency remediation suggestions in vulnurability findings [false]
   -f, --fail-on-result                                       When enabled, exit with non-zero on any vulnerability findings
                                                              Useful for CI/CD [false]
+  -c, --cvss-fail-threshold <score>                          Exit with non-zero when any vulnerability's CVSS base score is >= threshold
+                                                             CVSS scores range from 0.0 (least severe) to 10.0 (most severe)
+                                                             We interpret a score of 0.0 as suspicious
+                                                             Missing or suspicious CVSS base scores are conservatively derived
+                                                             Useful for CI/CD
   -h, --help                                                 Show usage help
 
 OPTIONS valid when database-strategy is dependency-check:
@@ -437,6 +442,59 @@ See the [NVD NIST website description for details](https://nvd.nist.gov/vuln-met
 > - It only populates scores from a single CVSS version.
 > - It does not always populate the CVSS score, or populates it with `0.0`.
 
+# Failing on Findings
+
+By default, `clj-watson` exits with `0`.
+
+You can opt to have `clj-watson` exit with a non-zero value when it detects vulnerabilities, which can be useful when running from a continuous integration (CI) server or service.
+
+Specify `--fail-on-result` (or `-f`) to exit with non-zero when any vulnerabilities are detected.
+
+Example usages:
+
+```
+clojure -M:clj-watson --deps-edn-path deps.edn --fail-on-result
+clojure -Tclj-watson :deps-edn-path deps.edn :fail-on-result true
+```
+
+For finer control use `--cvss-fail-threshold` (or `-c`) to specify a CVSS score at which to fail.
+When any detected vulnerability has a score equal to or above the threshold, `clj-watson` will summarize vulnerabilities that have met the threshold and exit with non-zero.
+
+Example usages:
+```
+clojure -M:clj-watson --deps-edn-path deps.edn --cvss-fail-threshold 5.8
+clojure -Tclj-watson :deps-edn-path deps.edn :cvss-fail-threshold 5.8
+```
+
+Example summary:
+
+```
+CVSS fail score threshold of 5.8 met for:
+
+  Dependency                                     Version Identifiers      CVSS Score
+  org.apache.httpcomponents/httpclient           4.1.2   CVE-2014-3577    5.8 (version 2.0)
+  com.fasterxml.jackson.core/jackson-annotations 2.4.0   CVE-2018-1000873 6.5 (version 3.1)
+  com.fasterxml.jackson.core/jackson-core        2.4.2   CVE-2018-1000873 6.5 (version 3.1)
+  org.jsoup/jsoup                                1.6.1   CVE-2021-37714   7.5 (version 3.1)
+  com.fasterxml.jackson.core/jackson-databind    2.4.2   CVE-2020-9548    9.8 (version 3.1)
+  org.clojure/clojure                            1.8.0   CVE-2017-20189   9.8 (version 3.1)
+  org.codehaus.plexus/plexus-utils               3.0     CVE-2017-1000487 9.8 (version 3.1)
+```
+
+When the score is missing or suspicious-looking, `clj-watson` will conservatively derive a score and indicate how it has done so (see `httpclient` below):
+
+```
+CVSS fail score threshold of 5.8 met for:
+
+  Dependency                                  Version Identifiers                          CVSS Score
+  org.jsoup/jsoup                             1.6.1   GHSA-m72m-mhq2-9p6c CVE-2021-37714   7.5 (version 3.1)
+  com.fasterxml.jackson.core/jackson-databind 2.4.2   GHSA-qxxx-2pp7-5hmx CVE-2017-7525    9.8 (version 3.1)
+  com.mchange/c3p0                            0.9.5.2 GHSA-q485-j897-qc27 CVE-2018-20433   9.8 (version 3.0)
+  org.clojure/clojure                         1.8.0   GHSA-jgxc-8mwq-9xqw CVE-2017-20189   9.8 (version 3.1)
+  org.codehaus.plexus/plexus-utils            3.0     GHSA-8vhq-qq4p-grq3 CVE-2017-1000487 9.8 (version 3.1)
+  org.apache.httpcomponents/httpclient        4.1.2   GHSA-2x83-r56g-cv47 CVE-2012-6153    10.0 (score 0.0 suspicious - derived from High severity)
+```
+
 # Output & Logging
 
 `clj-watson` uses [SLFJ4](https://www.slf4j.org/) and [Logback](https://logback.qos.ch) to collect and filter meaningful log output from its dependencies.

src/clj_watson/cli_spec.clj

@@ -1,6 +1,7 @@
 (ns clj-watson.cli-spec
   (:require
    [babashka.cli :as cli]
+   [clj-watson.logic.table :as table]
    [clojure.java.io :as io]
    [clojure.string :as str]))
 
@@ -59,6 +60,16 @@
     :desc (str "When enabled, exit with non-zero on any vulnerability findings\n"
                "Useful for CI/CD")}
 
+   :cvss-fail-threshold
+   {:alias :c
+    :ref "<score>"
+    :coerce :double
+    :desc (str "Exit with non-zero when any vulnerability's CVSS base score is >= threshold\n"
+               "CVSS scores range from 0.0 (least severe) to 10.0 (most severe)\n"
+               "We interpret a score of 0.0 as suspicious\n"
+               "Missing or suspicious CVSS base scores are conservatively derived\n"
+               "Useful for CI/CD")}
+
    :usage-help-style
    {:coerce :keyword
     :default :cli
@@ -100,50 +111,6 @@
   [kw]
   (subs (str kw) 1))
 
-(defn- cell-widths [rows]
-  (reduce
-   (fn [widths row]
-     (map max (map count row) widths)) (repeat 0) rows))
-
-(defn- pad-cells
-  "Adapted from bb cli"
-  [rows widths]
-  (let [pad-row (fn [row]
-                  (map (fn [width cell] (cli/pad width cell)) widths row))]
-    (map pad-row rows)))
-
-(defn- expand-multilines
-  "Expand last column cell over multiple rows if it contains newlines"
-  [rows]
-  (reduce (fn [acc row]
-            (let [[line & extra-lines] (-> row last str/split-lines)
-                  cols (count row)]
-              (if (seq extra-lines)
-                (apply conj acc
-                       (assoc (into [] row) (dec cols) line)
-                       (map #(conj (into [] (repeat (dec cols) ""))
-                                   %)
-                            extra-lines))
-                (conj acc row))))
-          []
-          rows))
-
-(defn- format-table
-  "Modified from bb cli format-table. Allow pre-computed widths to be passed in."
-  [{:keys [rows indent widths] :or {indent 2}}]
-  (let [widths (or widths (cell-widths rows))
-        rows (expand-multilines rows)
-        rows (pad-cells rows widths)
-        fmt-row (fn [leader divider trailer row]
-                  (str leader
-                       (apply str (interpose divider row))
-                       trailer))]
-    (->> rows
-         (map (fn [row]
-                (fmt-row (apply str (repeat indent " ")) " " "" row)))
-         (map str/trimr)
-         (str/join "\n"))))
-
 (defn styled-long-opt [longopt {:keys [usage-help-style]}]
   (if (= :clojure-tool usage-help-style)
     longopt
@@ -183,15 +150,15 @@
   [{:as cfg
     :keys [groups]}]
   (if (not groups)
-    (format-table {:rows (opts->table cfg) :indent 2})
+    (table/format-table {:rows (opts->table cfg) :indent 2})
     (let [groups (mapv #(assoc % :rows
                                (opts->table (assoc cfg :order (:order %))))
                        groups)
-          widths (cell-widths (mapcat :rows groups))]
+          widths (table/cell-widths (mapcat :rows groups))]
       (->> groups
            (reduce (fn [acc {:keys [heading rows]}]
                      (conj acc (str heading "\n"
-                                    (format-table {:rows rows :widths widths}))))
+                                    (table/format-table {:rows rows :widths widths}))))
                    [])
            (str/join "\n\n")))))
 
@@ -219,15 +186,15 @@
   (println
    (format-opts {:spec spec-scan-args :opts opts
                  :groups [{:heading "OPTIONS:"
-                           :order [:deps-edn-path :output :aliases :database-strategy :suggest-fix :fail-on-result :help]}
+                           :order [:deps-edn-path :output :aliases :database-strategy :suggest-fix :fail-on-result :cvss-fail-threshold :help]}
                           {:heading "OPTIONS valid when database-strategy is dependency-check:"
                            :order [:clj-watson-properties :run-without-nvd-api-key]}]})))
 
 (defn- usage-error [{:keys [spec type cause msg option opts] :as data}]
   (report-deprecations opts)
   (case type
     :clj-watson/cli
-    (println (error msg))
+    (println (str (error msg) "\n"))
 
     :org.babashka/cli
     (let [error-desc (cause {:require "Missing required argument"
@@ -277,7 +244,15 @@
 
       :else
       (let [opts (cli/parse-opts orig-args {:spec spec-scan-args :error-fn usage-error :restrict true})]
-        (report-deprecations opts)
+        (if (and (:cvss-fail-threshold opts) (:fail-on-result opts))
+          (usage-error {:type :clj-watson/cli
+                        :msg (format "Invalid usage, specificy only one of: %s"
+                                     (->> [:fail-on-result :cvss-fail-threshold]
+                                          (mapv #(styled-long-opt % opts))
+                                          (str/join ", ")))
+                        :spec spec-scan-args
+                        :opts opts})
+          (report-deprecations opts))
         opts))))
 
 (defn validate-tool-opts [opts]

src/clj_watson/controller/output.clj

@@ -2,6 +2,7 @@
   (:require
    [cheshire.core :as json]
    [clj-watson.logic.sarif :as logic.sarif]
+   [clj-watson.logic.table :as table]
    [clj-watson.logic.template :as logic.template]
    [clojure.java.io :as io]
    [clojure.pprint :as pprint]
@@ -54,3 +55,53 @@
                      (if (seq details)
                        (str " " (str/join ", " details))
                        "")))))
+
+(defn- cvss-threshold-summary->rows [opts data]
+  {:rows
+   (into [(mapv #(:title %) opts)]
+         (mapv (fn [s]
+                 (mapv (fn [{:keys [key]}]
+                         (-> s key str))
+                       opts))
+               data))})
+
+(defn cvss-threshold-summary [{:keys [threshold scores-met]}]
+  ;; CVSS fail score threshold of 2.0 met for:
+  ;;
+  ;;  Dependency Version Identifiers CVSS Score
+  ;;  foo/bar    1.2.3   CVE-1231    5.2 (version 3.1)
+  ;;  foo/bar1   7.7a    CVE-123134  6.9 (score missing - derived from Medium severity)
+  ;;  foo/bar2   2.0     CVE-12313   10.0 (score missing - derived from High severity)
+  ;;  foo/bar3   24.2    CVE-188     10.0 (score missing - derived from Critical severity)
+  ;;  foo/bar4   8.2     CVE-123132  10.0 (score missing and severity Foo unrecognized)
+  ;;  foo/bar5   1.12    CVE-12313   10.0 (score and severity missing)
+  ;;  foo/bar6   1.12    CVE-1232    10.0 (score 0.0 suspicious and severity missing)
+  ;;  foo/bar6   1.12    CVE-1237    10.0 (score foo suspicious and severity missing)
+  ;;  foo/bar6   1.12    CVE-1238    10.0 (score 11.2 suspicious and severity missing)
+  (if-not (seq scores-met)
+    (println (format "No scores met CVSS fail threshold of %s" threshold))
+    (do
+      (println (format "CVSS fail score threshold of %s met for:\n" threshold))
+      (->> scores-met
+           (mapv (fn [{:keys [identifiers score severity score-version score-derivation suspicious-score suspicious-severity] :as m}]
+                   (assoc m
+                          :identifiers (str/join " " identifiers)
+                          :score-desc
+                          (format "%s (%s)"
+                                  score
+                                  (if-let [[score-analysis severity-analysis] score-derivation]
+                                    (format "%s %s"
+                                            (case score-analysis
+                                              :missing-score "score missing"
+                                              :suspicious-score (format "score %s suspicious" suspicious-score))
+                                            (case severity-analysis
+                                              :valid-severity (format "- derived from %s severity" severity)
+                                              :missing-severity "and severity missing"
+                                              :suspicious-severity (format "and severity %s unrecognized" suspicious-severity)))
+                                    (format "version %s" (or score-version "<missing>")))))))
+           (cvss-threshold-summary->rows [{:key :dependency  :title "Dependency"}
+                                          {:key :version     :title "Version"}
+                                          {:key :identifiers :title "Identifiers"}
+                                          {:key :score-desc  :title "CVSS Score"}])
+           table/format-table
+           println))))

src/clj_watson/entrypoint.clj

@@ -32,7 +32,7 @@
 (defmethod scan* :default [opts]
   (scan* (assoc opts :database-strategy :dependency-check)))
 
-(defn do-scan [{:keys [fail-on-result output deps-edn-path aliases suggest-fix] :as opts}]
+(defn do-scan [{:keys [fail-on-result cvss-fail-threshold output deps-edn-path aliases suggest-fix] :as opts}]
   (logging-config/init)
   (let [{:keys [deps dependencies]} (controller.deps/parse deps-edn-path aliases)
         repositories (select-keys deps [:mvn/repos])
@@ -43,9 +43,19 @@
                    (controller.remediate/scan findings deps)
                    findings)]
     (controller.output/generate findings deps-edn-path output)
-    (-> result summarize/summarize controller.output/final-summary)
-    (if (and (seq findings) fail-on-result)
+    (-> result summarize/final-summary controller.output/final-summary)
+    (cond
+      (and fail-on-result (seq findings))
       (System/exit 1)
+
+      cvss-fail-threshold
+      (let [{:keys [scores-met] :as cvss-summary} (summarize/cvss-threshold-summary cvss-fail-threshold result)]
+        (controller.output/cvss-threshold-summary cvss-summary)
+        (if (seq scores-met)
+          (System/exit 1)
+          (System/exit 0)))
+
+      :else
       (System/exit 0))))
 
 #_{:clj-kondo/ignore [:clojure-lsp/unused-public-var]}