Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Take patterns into account when computing hash #169

Conversation

pdecat
Copy link
Contributor

@pdecat pdecat commented Jul 2, 2021

Description

This PR takes patterns into account when computing hash.

This should resolve #53.

Motivation and Context

Without this change, files that are excluded from Zip packages with ! prefixed patterns are still considered during hash computing, which triggers diff when plans are run on workspaces from different users.

Example plan of the examples/build-package configuration from master branch:

# terraform plan -target module.package_with_patterns.local_file.archive_plan[0]
[1600662] prepare: QUERY: {
  "artifacts_dir": "builds",
  "hash_extra": "",
  "hash_extra_paths": "[]",
  "paths": "{\"cwd\":\"/home/patrick/workspaces/terraform/terraform-aws-modules/terraform-aws-lambda/examples/build-package\",\"module\":\"../..\",\"root\":\".\"}",
  "runtime": "python3.8",
  "source_path": "[{\"pip_requirements\":\"./../fixtures/python3.8-app1/requirements.txt\"},\"./../fixtures/python3.8-app1/index.py\",{\"path\":\"./../fixtures/python3.8-app1/index.py\",\"patterns\":\"            # To use comments in heredoc patterns set the env var:\\n            # export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true\\n            # The intent for comments just to use in examples and demo snippets.\\n            # To write a comment start it with double spaces and the # sign\\n            # if it follows a pattern.\\n\\n            !index\\\\.py         # Exclude a file with 'index.py' name\\n            index\\\\..*          # Re-include\\n\"},{\"path\":\"./../fixtures/dirtree\",\"patterns\":\"            # To see step by step output how files and folders was skipped or added\\n            # set an env var in your shell by a command similar to:\\n            # export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG\\n\\n            # To play with this demo pattern you can go to a ../fixtures folder\\n            # and run there a dirtree.sh script to create a demo dirs tree\\n            # matchable by following patterns.\\n\\n            !abc/.*           # Filter out everything in an abc folder with the folder itself\\n            !abc/.+           # Filter out just all folder's content but keep the folder even if it's empty\\n            abc/def/.*        # Re-include everything in abc/def sub folder\\n            !abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder\\n            !.*/fiv.*\\n            !.*/zi--.*\\n\\n            ########################################################################\\n            # Special cases\\n\\n            # !.*/  # Removes all explicit directories from a zip file, mean while\\n                    # the zip file still valid and unpacks correctly but will not\\n                    # contain any empty directories.\\n\\n            # !.*/bar/  # To remove just some directory use more qualified path to it.\\n\\n            # !([^/]+/){3,}  # Remove all directories more than 3 folders in depth.\\n\",\"prefix_in_zip\":\"vendor\"}]"
}
[1600662] prepare: BUILD_PLAN: [
  [
    "pip",
    "python3.8",
    "./../fixtures/python3.8-app1/requirements.txt",
    null
  ],
  [
    "zip",
    "./../fixtures/python3.8-app1/index.py",
    null
  ],
  [
    "set:filter",
    [
      "# To use comments in heredoc patterns set the env var:",
      "# export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true",
      "# The intent for comments just to use in examples and demo snippets.",
      "# To write a comment start it with double spaces and the # sign",
      "# if it follows a pattern.",
      "!index\\.py         # Exclude a file with 'index.py' name",
      "index\\..*          # Re-include"
    ]
  ],
  [
    "zip",
    "./../fixtures/python3.8-app1/index.py",
    null
  ],
  [
    "clear:filter"
  ],
  [
    "set:filter",
    [
      "# To see step by step output how files and folders was skipped or added",
      "# set an env var in your shell by a command similar to:",
      "# export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG",
      "# To play with this demo pattern you can go to a ../fixtures folder",
      "# and run there a dirtree.sh script to create a demo dirs tree",
      "# matchable by following patterns.",
      "!abc/.*           # Filter out everything in an abc folder with the folder itself",
      "!abc/.+           # Filter out just all folder's content but keep the folder even if it's empty",
      "abc/def/.*        # Re-include everything in abc/def sub folder",
      "!abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder",
      "!.*/fiv.*",
      "!.*/zi--.*",
      "########################################################################",
      "# Special cases",
      "# !.*/  # Removes all explicit directories from a zip file, mean while",
      "# the zip file still valid and unpacks correctly but will not",
      "# contain any empty directories.",
      "# !.*/bar/  # To remove just some directory use more qualified path to it.",
      "# !([^/]+/){3,}  # Remove all directories more than 3 folders in depth."
    ]
  ],
  [
    "zip",
    "./../fixtures/dirtree",
    "vendor"
  ],
  [
    "clear:filter"
  ]
]
[1600662] prepare: Computing content hash on files...
[1600662] prepare.hash: ./../fixtures/python3.8-app1/requirements.txt
[1600662] prepare.hash: ./../fixtures/python3.8-app1/index.py
[1600662] prepare.hash: ./../fixtures/python3.8-app1/index.py
[1600662] prepare.hash.ls: abc/a2
[1600662] prepare.hash.ls: abc/a1
[1600662] prepare.hash.ls: abc/def/ghk/bbb9
[1600662] prepare.hash.ls: abc/def/ghk/b1
[1600662] prepare.hash.ls: abc/def/ghk/bbb8
[1600662] prepare.hash.ls: abc/def/ghk/b2
[1600662] prepare.hash.ls: abc/def/ghk/mul/bar/fiv99
[1600662] prepare.hash.ls: abc/def/ghk/mul/bar/zi---11
[1600662] prepare.hash.ls: abc/def/zum/bar0
[1600662] prepare.hash.ls: abc/def/zum/bar9
[1600662] prepare.hash.ls: abc/def/foo/bar1
[1600662] prepare.hash.ls: abc/def/foo/bar2
[1600662] prepare.hash: ./../fixtures/dirtree/abc/a1
[1600662] prepare.hash: ./../fixtures/dirtree/abc/a2
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/foo/bar1
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/foo/bar2
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/ghk/b1
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/ghk/b2
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/ghk/bbb8
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/ghk/bbb9
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/ghk/mul/bar/fiv99
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/ghk/mul/bar/zi---11
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/zum/bar0
[1600662] prepare.hash: ./../fixtures/dirtree/abc/def/zum/bar9

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.package_with_patterns.local_file.archive_plan[0] will be created
  + resource "local_file" "archive_plan" {
      + content              = jsonencode(
            {
              + artifacts_dir = "builds"
              + build_plan    = [
                  + [
                      + "pip",
                      + "python3.8",
                      + "./../fixtures/python3.8-app1/requirements.txt",
                      + null,
                    ],
                  + [
                      + "zip",
                      + "./../fixtures/python3.8-app1/index.py",
                      + null,
                    ],
                  + [
                      + "set:filter",
                      + [
                          + "# To use comments in heredoc patterns set the env var:",
                          + "# export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true",
                          + "# The intent for comments just to use in examples and demo snippets.",
                          + "# To write a comment start it with double spaces and the # sign",
                          + "# if it follows a pattern.",
                          + "!index\\.py         # Exclude a file with 'index.py' name",
                          + "index\\..*          # Re-include",
                        ],
                    ],
                  + [
                      + "zip",
                      + "./../fixtures/python3.8-app1/index.py",
                      + null,
                    ],
                  + [
                      + "clear:filter",
                    ],
                  + [
                      + "set:filter",
                      + [
                          + "# To see step by step output how files and folders was skipped or added",
                          + "# set an env var in your shell by a command similar to:",
                          + "# export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG",
                          + "# To play with this demo pattern you can go to a ../fixtures folder",
                          + "# and run there a dirtree.sh script to create a demo dirs tree",
                          + "# matchable by following patterns.",
                          + "!abc/.*           # Filter out everything in an abc folder with the folder itself",
                          + "!abc/.+           # Filter out just all folder's content but keep the folder even if it's empty",
                          + "abc/def/.*        # Re-include everything in abc/def sub folder",
                          + "!abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder",
                          + "!.*/fiv.*",
                          + "!.*/zi--.*",
                          + "########################################################################",
                          + "# Special cases",
                          + "# !.*/  # Removes all explicit directories from a zip file, mean while",
                          + "# the zip file still valid and unpacks correctly but will not",
                          + "# contain any empty directories.",
                          + "# !.*/bar/  # To remove just some directory use more qualified path to it.",
                          + "# !([^/]+/){3,}  # Remove all directories more than 3 folders in depth.",
                        ],
                    ],
                  + [
                      + "zip",
                      + "./../fixtures/dirtree",
                      + "vendor",
                    ],
                  + [
                      + "clear:filter",
                    ],
                ]
              + filename      = "builds/b71054c2838c7286f0dbcba3a44735c3c9e7164466bf2eda7c1335dd30c41fb7.zip"
              + runtime       = "python3.8"
            }
        )
      + directory_permission = "0755"
      + file_permission      = "0644"
      + filename             = "builds/b71054c2838c7286f0dbcba3a44735c3c9e7164466bf2eda7c1335dd30c41fb7.plan.json"
      + id                   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Warning: Resource targeting is in effect
│
│ You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by the current configuration.
│
│ The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use
│ it as part of an error message.
╵

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

Example plan of the examples/build-package configuration from my take_patterns_into_account_for_hash branch:

# terraform plan -target module.package_with_patterns.local_file.archive_plan[0]
[1967741] prepare: QUERY: {
  "artifacts_dir": "builds",
  "hash_extra": "",
  "hash_extra_paths": "[]",
  "paths": "{\"cwd\":\"/home/patrick/workspaces/terraform/terraform-aws-modules/terraform-aws-lambda/examples/build-package\",\"module\":\"../..\",\"root\":\".\"}",
  "runtime": "python3.8",
  "source_path": "[{\"pip_requirements\":\"./../fixtures/python3.8-app1/requirements.txt\"},\"./../fixtures/python3.8-app1/index.py\",{\"path\":\"./../fixtures/python3.8-app1/index.py\",\"patterns\":\"            # To use comments in heredoc patterns set the env var:\\n            # export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true\\n            # The intent for comments just to use in examples and demo snippets.\\n            # To write a comment start it with double spaces and the # sign\\n            # if it follows a pattern.\\n\\n            !index\\\\.py         # Exclude a file with 'index.py' name\\n            index\\\\..*          # Re-include\\n\"},{\"path\":\"./../fixtures/dirtree\",\"patterns\":\"            # To see step by step output how files and folders was skipped or added\\n            # set an env var in your shell by a command similar to:\\n            # export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG\\n\\n            # To play with this demo pattern you can go to a ../fixtures folder\\n            # and run there a dirtree.sh script to create a demo dirs tree\\n            # matchable by following patterns.\\n\\n            !abc/.*           # Filter out everything in an abc folder with the folder itself\\n            !abc/.+           # Filter out just all folder's content but keep the folder even if it's empty\\n            abc/def/.*        # Re-include everything in abc/def sub folder\\n            !abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder\\n            !.*/fiv.*\\n            !.*/zi--.*\\n\\n            ########################################################################\\n            # Special cases\\n\\n            # !.*/  # Removes all explicit directories from a zip file, mean while\\n                    # the zip file still valid and unpacks correctly but will not\\n                    # contain any empty directories.\\n\\n            # !.*/bar/  # To remove just some directory use more qualified path to it.\\n\\n            # !([^/]+/){3,}  # Remove all directories more than 3 folders in depth.\\n\",\"prefix_in_zip\":\"vendor\"}]"
}
zip: filter pattern: # To use comments in heredoc patterns set the env var:
zip: filter pattern: # export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true
zip: filter pattern: # The intent for comments just to use in examples and demo snippets.
zip: filter pattern: # To write a comment start it with double spaces and the # sign
zip: filter pattern: # if it follows a pattern.
zip: filter pattern: !index\.py         # Exclude a file with 'index.py' name
zip: filter pattern: index\..*          # Re-include
zip: filter pattern: # To see step by step output how files and folders was skipped or added
zip: filter pattern: # set an env var in your shell by a command similar to:
zip: filter pattern: # export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG
zip: filter pattern: # To play with this demo pattern you can go to a ../fixtures folder
zip: filter pattern: # and run there a dirtree.sh script to create a demo dirs tree
zip: filter pattern: # matchable by following patterns.
zip: filter pattern: !abc/.*           # Filter out everything in an abc folder with the folder itself
zip: filter pattern: !abc/.+           # Filter out just all folder's content but keep the folder even if it's empty
zip: filter pattern: abc/def/.*        # Re-include everything in abc/def sub folder
zip: filter pattern: !abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder
zip: filter pattern: !.*/fiv.*
zip: filter pattern: !.*/zi--.*
zip: filter pattern: ########################################################################
zip: filter pattern: # Special cases
zip: filter pattern: # !.*/  # Removes all explicit directories from a zip file, mean while
zip: filter pattern: # the zip file still valid and unpacks correctly but will not
zip: filter pattern: # contain any empty directories.
zip: filter pattern: # !.*/bar/  # To remove just some directory use more qualified path to it.
zip: filter pattern: # !([^/]+/){3,}  # Remove all directories more than 3 folders in depth.
zip: skip:   vendor/abc/def/ghk/mul/bar/fiv99
zip: skip:   vendor/abc/def/ghk/mul/bar/zi---11
[1967741] prepare: BUILD_PLAN: [
  [
    "pip",
    "python3.8",
    "./../fixtures/python3.8-app1/requirements.txt",
    null
  ],
  [
    "zip",
    "./../fixtures/python3.8-app1/index.py",
    null
  ],
  [
    "set:filter",
    [
      "# To use comments in heredoc patterns set the env var:",
      "# export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true",
      "# The intent for comments just to use in examples and demo snippets.",
      "# To write a comment start it with double spaces and the # sign",
      "# if it follows a pattern.",
      "!index\\.py         # Exclude a file with 'index.py' name",
      "index\\..*          # Re-include"
    ]
  ],
  [
    "zip",
    "./../fixtures/python3.8-app1/index.py",
    null
  ],
  [
    "clear:filter"
  ],
  [
    "set:filter",
    [
      "# To see step by step output how files and folders was skipped or added",
      "# set an env var in your shell by a command similar to:",
      "# export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG",
      "# To play with this demo pattern you can go to a ../fixtures folder",
      "# and run there a dirtree.sh script to create a demo dirs tree",
      "# matchable by following patterns.",
      "!abc/.*           # Filter out everything in an abc folder with the folder itself",
      "!abc/.+           # Filter out just all folder's content but keep the folder even if it's empty",
      "abc/def/.*        # Re-include everything in abc/def sub folder",
      "!abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder",
      "!.*/fiv.*",
      "!.*/zi--.*",
      "########################################################################",
      "# Special cases",
      "# !.*/  # Removes all explicit directories from a zip file, mean while",
      "# the zip file still valid and unpacks correctly but will not",
      "# contain any empty directories.",
      "# !.*/bar/  # To remove just some directory use more qualified path to it.",
      "# !([^/]+/){3,}  # Remove all directories more than 3 folders in depth."
    ]
  ],
  [
    "zip",
    "./../fixtures/dirtree",
    "vendor"
  ],
  [
    "clear:filter"
  ]
]
[1967741] prepare: Computing content hash on files...
[1967741] prepare.hash: ./../fixtures/python3.8-app1/requirements.txt
[1967741] prepare.hash: ./../fixtures/python3.8-app1/index.py
[1967741] prepare.hash: ../fixtures/python3.8-app1/index.py
[1967741] prepare.hash.ls: a2
[1967741] prepare.hash.ls: a1
[1967741] prepare.hash.ls: def/ghk/bbb9
[1967741] prepare.hash.ls: def/ghk/b1
[1967741] prepare.hash.ls: def/ghk/bbb8
[1967741] prepare.hash.ls: def/ghk/b2
[1967741] prepare.hash.ls: def/ghk/mul/bar/fiv99
[1967741] prepare.hash.ls: def/ghk/mul/bar/zi---11
[1967741] prepare.hash.ls: def/zum/bar0
[1967741] prepare.hash.ls: def/zum/bar9
[1967741] prepare.hash.ls: def/foo/bar1
[1967741] prepare.hash.ls: def/foo/bar2
[1967741] prepare.hash: ../fixtures/dirtree/abc/a1
[1967741] prepare.hash: ../fixtures/dirtree/abc/a2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb8
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb9
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/fiv99
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/zi---11
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar0
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar9
[1967741] prepare.hash: ../fixtures/dirtree/abc/a2
[1967741] prepare.hash: ../fixtures/dirtree/abc/a1
[1967741] prepare.hash.ls: ghk/bbb9
[1967741] prepare.hash.ls: ghk/b1
[1967741] prepare.hash.ls: ghk/bbb8
[1967741] prepare.hash.ls: ghk/b2
[1967741] prepare.hash.ls: ghk/mul/bar/fiv99
[1967741] prepare.hash.ls: ghk/mul/bar/zi---11
[1967741] prepare.hash.ls: zum/bar0
[1967741] prepare.hash.ls: zum/bar9
[1967741] prepare.hash.ls: foo/bar1
[1967741] prepare.hash.ls: foo/bar2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb8
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb9
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/fiv99
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/zi---11
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar0
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar9
[1967741] prepare.hash.ls: bbb9
[1967741] prepare.hash.ls: b1
[1967741] prepare.hash.ls: bbb8
[1967741] prepare.hash.ls: b2
[1967741] prepare.hash.ls: mul/bar/fiv99
[1967741] prepare.hash.ls: mul/bar/zi---11
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb8
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb9
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/fiv99
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/zi---11
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb9
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/bbb8
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/b2
[1967741] prepare.hash.ls: bar/fiv99
[1967741] prepare.hash.ls: bar/zi---11
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/fiv99
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/zi---11
[1967741] prepare.hash.ls: fiv99
[1967741] prepare.hash.ls: zi---11
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/fiv99
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/ghk/mul/bar/zi---11
[1967741] prepare.hash.ls: bar0
[1967741] prepare.hash.ls: bar9
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar0
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar9
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar0
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/zum/bar9
[1967741] prepare.hash.ls: bar1
[1967741] prepare.hash.ls: bar2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar2
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar1
[1967741] prepare.hash: ../fixtures/dirtree/abc/def/foo/bar2

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.package_with_patterns.local_file.archive_plan[0] will be created
  + resource "local_file" "archive_plan" {
      + content              = jsonencode(
            {
              + artifacts_dir = "builds"
              + build_plan    = [
                  + [
                      + "pip",
                      + "python3.8",
                      + "./../fixtures/python3.8-app1/requirements.txt",
                      + null,
                    ],
                  + [
                      + "zip",
                      + "./../fixtures/python3.8-app1/index.py",
                      + null,
                    ],
                  + [
                      + "set:filter",
                      + [
                          + "# To use comments in heredoc patterns set the env var:",
                          + "# export TF_LAMBDA_PACKAGE_PATTERN_COMMENTS=true",
                          + "# The intent for comments just to use in examples and demo snippets.",
                          + "# To write a comment start it with double spaces and the # sign",
                          + "# if it follows a pattern.",
                          + "!index\\.py         # Exclude a file with 'index.py' name",
                          + "index\\..*          # Re-include",
                        ],
                    ],
                  + [
                      + "zip",
                      + "./../fixtures/python3.8-app1/index.py",
                      + null,
                    ],
                  + [
                      + "clear:filter",
                    ],
                  + [
                      + "set:filter",
                      + [
                          + "# To see step by step output how files and folders was skipped or added",
                          + "# set an env var in your shell by a command similar to:",
                          + "# export TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG",
                          + "# To play with this demo pattern you can go to a ../fixtures folder",
                          + "# and run there a dirtree.sh script to create a demo dirs tree",
                          + "# matchable by following patterns.",
                          + "!abc/.*           # Filter out everything in an abc folder with the folder itself",
                          + "!abc/.+           # Filter out just all folder's content but keep the folder even if it's empty",
                          + "abc/def/.*        # Re-include everything in abc/def sub folder",
                          + "!abc/def/ghk/.*   # Filter out again in abc/def/ghk sub folder",
                          + "!.*/fiv.*",
                          + "!.*/zi--.*",
                          + "########################################################################",
                          + "# Special cases",
                          + "# !.*/  # Removes all explicit directories from a zip file, mean while",
                          + "# the zip file still valid and unpacks correctly but will not",
                          + "# contain any empty directories.",
                          + "# !.*/bar/  # To remove just some directory use more qualified path to it.",
                          + "# !([^/]+/){3,}  # Remove all directories more than 3 folders in depth.",
                        ],
                    ],
                  + [
                      + "zip",
                      + "./../fixtures/dirtree",
                      + "vendor",
                    ],
                  + [
                      + "clear:filter",
                    ],
                ]
              + filename      = "builds/43548b361681b338cb58499a7f0952415a0bcab083b4c3ccbd9d7e6660f2e6b7.zip"
              + runtime       = "python3.8"
            }
        )
      + directory_permission = "0755"
      + file_permission      = "0644"
      + filename             = "builds/43548b361681b338cb58499a7f0952415a0bcab083b4c3ccbd9d7e6660f2e6b7.plan.json"
      + id                   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Warning: Resource targeting is in effect
│
│ You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by the current configuration.
│
│ The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use
│ it as part of an error message.
╵

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

Breaking Changes

This should not be a breaking change.

How Has This Been Tested?

  • I have tested and validated these changes using one or more of the provided examples/* projects

Tested this change with examples/build-package.

I have also tested this change with my own terraform configurations for now.

@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from 9713664 to b851a7b Compare July 2, 2021 14:10
package.py Outdated Show resolved Hide resolved
@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from b851a7b to f5771c1 Compare July 2, 2021 15:11
@pdecat pdecat marked this pull request as draft July 2, 2021 15:19
@pdecat
Copy link
Contributor Author

pdecat commented Jul 2, 2021

Marking as draft as not everything works as expected (e.g. folders are lost in the target zip).

@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch 4 times, most recently from 3cfac03 to 98eea82 Compare July 2, 2021 15:36
@pdecat pdecat marked this pull request as ready for review July 2, 2021 15:41
@pdecat
Copy link
Contributor Author

pdecat commented Jul 2, 2021

Marking as draft as not everything works as expected (e.g. folders are lost in the target zip).

Issue resolved, path value was taken from the inner loop when patterns was defined.

package.py Outdated Show resolved Hide resolved
@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from 98eea82 to b447a7c Compare July 15, 2021 11:02
@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from b447a7c to 9f5d7fb Compare July 22, 2021 12:08
@pdecat pdecat changed the title Take patterns into account when computing hash fix: Take patterns into account when computing hash Jul 22, 2021
@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from 0b9e8af to 58da4af Compare August 21, 2021 07:41
@antonbabenko
Copy link
Member

Have you finished this PR? I try to run it locally now and I don't see it working (same behavior as in master).

@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from 6029948 to 4afb710 Compare August 30, 2021 12:58
@pdecat pdecat force-pushed the take_patterns_into_account_for_hash branch from 4afb710 to 3ed5d7e Compare August 30, 2021 13:07
@pdecat
Copy link
Contributor Author

pdecat commented Aug 30, 2021

Have you finished this PR? I try to run it locally now and I don't see it working (same behavior as in master).

The only thing that changes with this PR is the computed hash when files are excluded, package content stays the same.

Don't you see differences in the plan with TF_LAMBDA_PACKAGE_LOG_LEVEL=DEBUG? Most notably, there should be some zip: skip ... lines.

@antonbabenko
Copy link
Member

Right, I can see the difference in your output but not when I ran it a couple of hours ago. Let me try it again.

@antonbabenko antonbabenko merged commit e2d5ddd into terraform-aws-modules:master Aug 30, 2021
@antonbabenko
Copy link
Member

Now I can see the difference. Thank you very much for this fix!

v2.14.0 has been just released.

@pdecat pdecat deleted the take_patterns_into_account_for_hash branch August 30, 2021 13:30
@github-actions
Copy link

github-actions bot commented Nov 9, 2022

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build plan changes don't re triggering a zip package rebuild
2 participants