skip file / directory patterns or globs

[wyardley]

It would be useful to be able to skip directories or files using globs or simple regex type patterns.

Use case: Using trivy to scan initialized Terraform config directories with modules that have embedded examples can take a long time to scan, and turn up false positives

For example, if a module like https://github.com/terraform-google-modules/terraform-google-kubernetes-engine is included
Presumably, while trivy should be smart enough to follow actual module code that's used, it shouldn't naively scan all tf files in the .terraform directory of an initialized repo. (side note: if you believe this behavior is a bug, I can also open a bug report for this separately).

While the --skip-dirs option exists, it doesn't seem to support ignoring a glob pattern or regex, so you can't do, e.g.,
--skip-dirs **/.terraform or --skip-files **/.terraform/**/*.tf

simple repro:

module "gke" {
  source                            = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster"
  version                           = "25.0.0"
  project_id                        = "foo-bar"
  name                              = "test-cluster"
  regional                          = true
  region                            = "us-west2"
  zones                             = ["us-west2-a", "us-west2-b", "us-west2-c"]
  network                           = "default"

  create_service_account            = true
  dns_cache                         = true
  enable_private_endpoint           = false
  enable_private_nodes              = true
  disable_legacy_metadata_endpoints = true
  master_ipv4_cidr_block            = "10.5.0.0/28"
  master_global_access_enabled      = false
  subnetwork                        = "foo-subnet-1"
  ip_range_pods                     = "10.1.0.0/16"
  ip_range_services                 = "10.2.0.0/16"
  kubernetes_version                = "1.22.8-gke.200"
  http_load_balancing               = true
  horizontal_pod_autoscaling        = true
  network_policy                    = false
  remove_default_node_pool          = true
  initial_node_count                = 1
  gce_pd_csi_driver                 = true

  cluster_resource_labels = {
    foo = "bar"
  }
}

LOW: container should drop all
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Containers must drop ALL capabilities, and are only permitted to add back the NET_BIND_SERVICE capability.

See https://avd.aquasec.com/misconfig/ksv106
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 gke/.terraform/modules/gke/examples/acm-terraform-blog-part3/config-root/wordpress-bundle.yaml:103-145
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 103 ┌         - name: wordpress
 104 │           image: wordpress:5.2.2-apache
 105 │           ports:
 106 │             - containerPort: 80
 107 │           env:
 108 │             - name: WORDPRESS_DB_HOST
 109 │               value: 127.0.0.1:3306
 110 │             - name: WORDPRESS_DB_USER
 111 └               valueFrom:
 ...   

[AnaisUrlichs]

Hi there, have you looked into trivy.yaml config file -- you could create a config file for each scan e.g. one for config scans that says which files should be excluded & one for fs scans etc. https://aquasecurity.github.io/trivy/v0.38/docs/references/customization/config-file/
Here is an example:

trivy/examples/trivy-conf/trivy.yaml

Line 11 in 3357ed0

skip-dirs:

it doesn't seem to support ignoring a glob pattern or regex,

If it is just about ignoring files etc. you could use a .trivyignore file -- tho I think there is a bug so let me follow up on that

[wyardley]

Skipping specific files / dirs is already possible. This is asking about skipping patterns or globs of files or directories, which I don’t think would be possible with the config file suggestion or trivyignore files either?

[AnaisUrlichs]

Yeah, my bad, I thought trivyignore would make it possible

[wyardley]

This will be less of a problem if scanning initialized .terraform directories is unintentional. I can make a more limited issue for that and maybe a narrower fix could be found, though I suspect this would still be a useful general purpose feature.

[simar7]

hi @wyardley you could try the following

1. Add a trivy ignore to ignore all issues in a remote terraform module

// #trivy:ignore:*
module "gke" {
  source    = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster

2. Explicitly ignore the .terraform directory if you don't wish to scan it.
   trivy config --skip-dirs=".terraform"

Having both of them should not report any errors in remote modules or files found within .terraform directory.

$ trivy config --skip-dirs=".terraform" .
2023-03-16T00:08:32.496-0700	INFO	Misconfiguration scanning is enabled
2023-03-16T00:08:35.497-0700	INFO	Detected config files: 2

$ tree
.
└── main.tf

0 directories, 1 file

cat main.tf
// #trivy:ignore:*
module "gke" {
  source                            = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster" 
  version                           = "25.0.0"
  project_id                        = "foo-bar"
  name                              = "test-cluster"
  regional                          = true
  region                            = "us-west2"
  zones                             = ["us-west2-a", "us-west2-b", "us-west2-c"]
  network                           = "default"

  create_service_account            = true
  dns_cache                         = true
  enable_private_endpoint           = false
  enable_private_nodes              = true
  disable_legacy_metadata_endpoints = true
  master_ipv4_cidr_block            = "10.5.0.0/28"
  master_global_access_enabled      = false
  subnetwork                        = "foo-subnet-1"
  ip_range_pods                     = "10.1.0.0/16"
  ip_range_services                 = "10.2.0.0/16"
  kubernetes_version                = "1.22.8-gke.200"
  http_load_balancing               = true
  horizontal_pod_autoscaling        = true
  network_policy                    = false
  remove_default_node_pool          = true
  initial_node_count                = 1
  gce_pd_csi_driver                 = true

  cluster_resource_labels = {
    foo = "bar"
  }
}

[wyardley]

@simar7 Thanks!

Add a trivy ignore to ignore all issues in a remote terraform module

So, I don't think this solves the issue, since I believe will also probably ignore any findings from the module itself? The goal should be to not blindly scan all files in .terraform after modules are initialized, but to still analyze the actual terraform that's being included / executed.

I'm not sure if tfsec does this in a more smart way (based on https://github.com/aquasecurity/tfsec/issues/1684, I think it does also scan the directory), or if this is just more of an issue because trivy currently doesn't have a way to disable scanning for other types of configuration besides terraform configs.

Explicitly ignore the .terraform directory if you don't wish to scan it.

Based on my understanding of the --skip-dirs option (and the reason for this feature request), I don't think this will work.

For example, imagine recursively scanning a structure like this:

bar/gke/module.tf
foo/bar/someotherstate/xyz.tf
foo/gke/module.tf
wizzle/foo.tf

where directories may be added or removed over time. It's not practical to ignore each .terraform directory explicitly, which is the whole point of this feature request.

IMO, either trivy needs to be smarter about which stuff it parses and which it doesn't (i.e., don't scan the .terraform directory post initialization, but rather only follow the exact terraform code that's actually being included), and / or to include some way to ignore the contents of a glob of files or directories.

Also, I don't think not initializing the code will work either, because then some findings will (in my experience) be missed. Once terraform init is run, there will be quite a bit of code in .terraform, and in the case of a module being downloaded, a lot of it will not be actual code paths that are "live" / included.

[simar7]

@simar7 Thanks!

Add a trivy ignore to ignore all issues in a remote terraform module

So, I don't think this solves the issue, since I believe will also probably ignore any findings from the module itself? The goal should be to not blindly scan all files in .terraform after modules are initialized, but to still analyze the actual terraform that's being included / executed.

I'm not sure if tfsec does this in a more smart way (based on aquasecurity/tfsec#1684, I think it does also scan the directory), or if this is just more of an issue because trivy currently doesn't have a way to disable scanning for other types of configuration besides terraform configs.

Explicitly ignore the .terraform directory if you don't wish to scan it.

Based on my understanding of the --skip-dirs option (and the reason for this feature request), I don't think this will work.

For example, imagine recursively scanning a structure like this:

bar/gke/module.tf
foo/bar/someotherstate/xyz.tf
foo/gke/module.tf
wizzle/foo.tf

where directories may be added or removed over time. It's not practical to ignore each .terraform directory explicitly, which is the whole point of this feature request.

IMO, either trivy needs to be smarter about which stuff it parses and which it doesn't (i.e., don't scan the .terraform directory post initialization, but rather only follow the exact terraform code that's actually being included), and / or to include some way to ignore the contents of a glob of files or directories.

Also, I don't think not initializing the code will work either, because then some findings will (in my experience) be missed. Once terraform init is run, there will be quite a bit of code in .terraform, and in the case of a module being downloaded, a lot of it will not be actual code paths that are "live" / included.

Thanks understood. I've created a PR to take care of glob support. #3866

[itaysk]

@simar7 is this issue closed with #3866?

[ianchesal]

@simar7 is this issue closed with #3866?

I can't find a way to write a glob pattern in trivy.yaml that successfully ignores .terraform/ directories that appear all over my infra monorepo.

What pattern did you envision using to do this?

So far I've tried:

scan:
  skip-dirs:
    - '*/.terraform'
    - '*/.terraform/'
    - '*/.terraform/*'
    - '**/.terraform'
    - '**/.terraform/'
    - '**/.terraform/*'

There were probably a few others but none of them work. Trivy still scans into .terraform directories all over my repo. For example, I would have expected this directory and everything under it to have been ignored:

terraform/google/security/twingate/.terraform/modules/gce-container/test/fixtures/shared/network.tf (terraform)

Tests: 1 (SUCCESSES: 0, FAILURES: 1, EXCEPTIONS: 0)
Failures: 1 (CRITICAL: 1)

CRITICAL: Firewall rule allows ingress traffic from multiple addresses on the public internet.
══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Network security rules should not use very broad subnets.

Where possible, segments should be broken into smaller subnets and avoid using the <code>/0</code> subnet.

See https://avd.aquasec.com/misconfig/avd-gcp-0027
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 terraform/google/security/twingate/.terraform/modules/gce-container/test/fixtures/shared/network.tf:47
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  37   resource "google_compute_firewall" "ssh" {
  ..
  47 [   source_ranges = ["0.0.0.0/0"]
  ..
  49   }
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

[simar7]

@simar7 is this issue closed with #3866?

I can't find a way to write a glob pattern in trivy.yaml that successfully ignores .terraform/ directories that appear all over my infra monorepo.

What pattern did you envision using to do this?

So far I've tried:

scan:

  skip-dirs:

    - '*/.terraform'

    - '*/.terraform/'

    - '*/.terraform/*'

    - '**/.terraform'

    - '**/.terraform/'

    - '**/.terraform/*'

There were probably a few others but none of them work. Trivy still scans into .terraform directories all over my repo. For example, I would have expected this directory and everything under it to have been ignored:

terraform/google/security/twingate/.terraform/modules/gce-container/test/fixtures/shared/network.tf (terraform)



Tests: 1 (SUCCESSES: 0, FAILURES: 1, EXCEPTIONS: 0)

Failures: 1 (CRITICAL: 1)



CRITICAL: Firewall rule allows ingress traffic from multiple addresses on the public internet.

══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════

Network security rules should not use very broad subnets.



Where possible, segments should be broken into smaller subnets and avoid using the <code>/0</code> subnet.



See https://avd.aquasec.com/misconfig/avd-gcp-0027

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

 terraform/google/security/twingate/.terraform/modules/gce-container/test/fixtures/shared/network.tf:47

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

  37   resource "google_compute_firewall" "ssh" {

  ..

  47 [   source_ranges = ["0.0.0.0/0"]

  ..

  49   }

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

The yaml config and CLI options are equivalent. Can you try to use the CLI options to see if it works in this case?

[wyardley]

@ianchesal fwiw, while I haven't switched over the thing that initially prompted me to ask for this feature yet, I did a quick test and the feature seems to work for me:

trivy config --skip-dirs='**/.terraform' --debug .

Using debug mode, I can confirm that given a simple structure like

├── aws
│   ├── .terraform
│   ├── main.tf
├── gcp
│   ├── .terraform

trivy will ignore the .terraform directories with the command above. However, it doesn't follow the usual shell glob convention of **/.terraform matching a .terraform directory at any depth

% ls -d **/.terraform
aws/.terraform/          aws/test22/.terraform/   gcp/.terraform/          test/foo/bar/.terraform/

That is, if you have

├── aws
│   ├── .terraform
│   ├── main.tf
├── gcp
│   ├── .terraform
[...]
└── test
    └── foo
        └── bar
            ├── .terraform

there doesn't seem to be an obvious glob pattern that will match .terraform directories at any depth, though there might be some way to do it? You can repeat it multiple times at each depth, but this doesn't seem ideal

% trivy config --skip-dirs='*/**/.terraform' --skip-dirs='**/.terraform/' --skip-dirs='*/*/**/.terraform' --debug .
[...]
2023-07-20T20:47:53.964-0700	DEBUG	Walk the file tree rooted at '.' in parallel
2023-07-20T20:47:53.964-0700	DEBUG	Skipping directory: aws/.terraform
2023-07-20T20:47:53.964-0700	DEBUG	Skipping directory: gcp/.terraform
2023-07-20T20:47:53.964-0700	DEBUG	Skipping directory: test/foo/bar/.terraform
2023-07-20T20:47:53.964-0700	DEBUG	Skipping directory: aws/test22/.terraform

@simar7 is there a way to get the expected type of globbing behavior (if so, I'd be happy to make a PR to enhance the docs)

[nikpivkin]

@simar7 I have the same output when using the --skip-dirs flag cli or when using the configuration file

using the --skip-dirs flag:

trivy config -d . --skip-dirs ".terraform/**"
2023-07-21T09:48:38.988+0600    INFO    Loaded trivy.yaml
2023-07-21T09:48:38.989+0600    DEBUG   Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-07-21T09:48:38.999+0600    DEBUG   cache dir:  /Users/tososomaru/Library/Caches/trivy
2023-07-21T09:48:38.999+0600    DEBUG   Module dir: /Users/tososomaru/.trivy/modules
2023-07-21T09:48:38.999+0600    INFO    Misconfiguration scanning is enabled
2023-07-21T09:48:39.000+0600    DEBUG   Policies successfully loaded from disk
2023-07-21T09:48:39.026+0600    DEBUG   Walk the file tree rooted at '.' in parallel
2023-07-21T09:48:39.026+0600    DEBUG   Skipping directory: .terraform/providers
2023-07-21T09:48:39.026+0600    DEBUG   Skipping directory: .terraform/modules
2023-07-21T09:48:39.026+0600    DEBUG   Scanning Helm files for misconfigurations...
2023-07-21T09:48:39.026+0600    DEBUG   Scanning Terraform files for misconfigurations...
2023-07-21T09:48:42.835+0600    DEBUG   OS is not detected.
2023-07-21T09:48:42.835+0600    INFO    Detected config files: 3
2023-07-21T09:48:42.835+0600    DEBUG   Scanned config file: .
2023-07-21T09:48:42.835+0600    DEBUG   Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/sa.tf
2023-07-21T09:48:42.835+0600    DEBUG   Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf

git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf (terraform)

Tests: 18 (SUCCESSES: 16, FAILURES: 2, EXCEPTIONS: 0)
Failures: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 1, CRITICAL: 0)

MEDIUM: Cluster does not have a network policy enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling a network policy allows the segregation of network traffic by namespace

See https://avd.aquasec.com/misconfig/avd-gcp-0056
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:38
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  22   resource "google_container_cluster" "primary" {
  ..   
  38 [       enabled  = network_policy.value.enabled
 ...   
 441   }
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────


HIGH: Cluster does not have master authorized networks enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling authorized networks means you can restrict master access to a fixed set of CIDR ranges

See https://avd.aquasec.com/misconfig/avd-gcp-0061
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:22-441
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  22 ┌ resource "google_container_cluster" "primary" {
  23 │   provider = google-beta
  24 │ 
  25 │   name            = var.name
  26 │   description     = var.description
  27 │   project         = var.project_id
  28 │   resource_labels = var.cluster_resource_labels
  29 │ 
  30 └   location          = local.location
  ..   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

using the configuration file:

scan:
  skip-dirs:
    - .terraform/**

trivy config -d . --config trivy.yaml
2023-07-21T09:49:07.860+0600    INFO    Loaded trivy.yaml
2023-07-21T09:49:07.861+0600    DEBUG   Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-07-21T09:49:07.870+0600    DEBUG   cache dir:  /Users/tososomaru/Library/Caches/trivy
2023-07-21T09:49:07.870+0600    DEBUG   Module dir: /Users/tososomaru/.trivy/modules
2023-07-21T09:49:07.870+0600    INFO    Misconfiguration scanning is enabled
2023-07-21T09:49:07.870+0600    DEBUG   Policies successfully loaded from disk
2023-07-21T09:49:07.891+0600    DEBUG   Walk the file tree rooted at '.' in parallel
2023-07-21T09:49:07.892+0600    DEBUG   Skipping directory: .terraform/providers
2023-07-21T09:49:07.892+0600    DEBUG   Skipping directory: .terraform/modules
2023-07-21T09:49:07.892+0600    DEBUG   Scanning Helm files for misconfigurations...
2023-07-21T09:49:07.892+0600    DEBUG   Scanning Terraform files for misconfigurations...
2023-07-21T09:49:11.355+0600    DEBUG   OS is not detected.
2023-07-21T09:49:11.355+0600    INFO    Detected config files: 3
2023-07-21T09:49:11.355+0600    DEBUG   Scanned config file: .
2023-07-21T09:49:11.355+0600    DEBUG   Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf
2023-07-21T09:49:11.355+0600    DEBUG   Scanned config file: git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/sa.tf

git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf (terraform)

Tests: 18 (SUCCESSES: 16, FAILURES: 2, EXCEPTIONS: 0)
Failures: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 1, CRITICAL: 0)

MEDIUM: Cluster does not have a network policy enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling a network policy allows the segregation of network traffic by namespace

See https://avd.aquasec.com/misconfig/avd-gcp-0056
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:38
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  22   resource "google_container_cluster" "primary" {
  ..   
  38 [       enabled  = network_policy.value.enabled
 ...   
 441   }
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────


HIGH: Cluster does not have master authorized networks enabled.
═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Enabling authorized networks means you can restrict master access to a fixed set of CIDR ranges

See https://avd.aquasec.com/misconfig/avd-gcp-0061
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 git::https:/github.com/terraform-google-modules/terraform-google-kubernetes-engine?ref=v25.0.0/modules/beta-private-cluster/cluster.tf:22-441
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  22 ┌ resource "google_container_cluster" "primary" {
  23 │   provider = google-beta
  24 │ 
  25 │   name            = var.name
  26 │   description     = var.description
  27 │   project         = var.project_id
  28 │   resource_labels = var.cluster_resource_labels
  29 │ 
  30 └   location          = local.location
  ..   
──────────────────────────────────────────────────────────────────────────

[wyardley]

Go's path.filepath.Glob() doesn't seem to support extended glob:

% cat test.go 
package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	files, _ := filepath.Glob("./" + "**/.terraform")
	fmt.Println(files)
}
% go run test.go
[aws/.terraform gcp/.terraform]

Looks like https://github.com/ganbarodigital/go_glob doesn't support extended globbing either. golang/go#11862 has some discussion about it.

Based on the suggestion there, something like the below could work:

% cat test.go 
package main

import (
	"fmt"
	"os"

        "github.com/bmatcuk/doublestar/v4"
)

func main() {
	fs := os.DirFS("./")
	files, _ := doublestar.Glob(fs, "**/.terraform")
	fmt.Println(files)
}
% go run test.go
[aws/.terraform aws/test22/.terraform gcp/.terraform test/foo/bar/.terraform]

[wyardley]

A gitignore type syntax would also work quite well — some existing golang interpretations, e.g., https://pkg.go.dev/github.com/sabhiram/go-gitignore

[nikpivkin]

@simar7 I opened a issue

@ianchesal at the moment, you can use the skip-files flag to skip all files in .terraform: trivy config . --skip-files "**/.terraform/**/*". Or add this pattern to the config:

scan:
  skip-files:
    - "**/.terraform/**/*"

[ianchesal]

@nikpivkin ah! Thank you! That seems to do it.

For certain, the skip-dirs globbing doesn't seem to have a way to accomplish this. As @wyardley notes, it doesn't seem to match .terraform dirs at arbitrary depths.

If I can help with a patch here I'm more than happy to pitch in.

Thanks all for the work on this!

[nikpivkin]

@ianchesal thanks, I opened PR with fixes