Bug: Lambda custom policies not working #8392

osddeitf · 2021-10-08T10:32:40Z

Before opening, please confirm:

I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
I have searched for duplicate or closed issues.
I have read the guide for submitting bug reports.
I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?

No response

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

6.2.1

What operating system are you using?

Debian Bullseye

Amplify Categories

function

Amplify Commands

push

Describe the bug

I have added custom policy for my lambda function in <function_name>-cloudformation-template.json:

"CustomLambdaExecutionPolicy": {
      "Type": "AWS::IAM::Policy",
      "DependsOn": [
        "LambdaExecutionRole"
      ],
      "Properties": {
        "PolicyName": "custom-lambda-execution-policy",
        "Roles": [
          {
            "Ref": "LambdaExecutionRole"
          }
        ],
        "PolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": "cloudfront:*",
              "Resource": [
                "*"
              ]
            }
          ]
        }
      }
    },

When I updated Amplify CLI from v6.1.1 to v6.2.1, when I run amplify push, it remove my custom policy.
Then I noticed a change in v6.2.0:

6.2.0 (2021-10-06)
Features
Custom policies IAM Policies for Lambda and Containers (#8068) (3e1ce0d)

Then I create custom-policies.json according to the updated documentation, with the following content:

[
  {
    "Effect": "Allow",
    "Action": ["cloudfront:*"],
    "Resource": ["arn:aws:cloudfront:::*"]
  }
]

Then run amplify push, the error showed up as below:

? Are you sure you want to continue? Yes
strict mode: missing type "object" for keyword "additionalProperties" at "#" (strictTypes)
✖ An error occurred when pushing the resources to the cloud
🛑 
strict mode: unknown keyword: "optionalProperties"
An error occurred during the push operation: strict mode: unknown keyword: "optionalProperties"

Expected behavior

It should work. I tried the example in the docs, also won't work:

[
  {
    "Action": ["s3:CreateBucket"],
    "Resource": ["arn:aws:s3:::*"]
  }
]

Reproduction steps

As described above

GraphQL schema(s)

# Put schemas below this line

Log output

# Put your logs below this line

Additional information

I have tried to clean all of my workspaces, run amplify init then amplify pull again, the error still persisted.

The text was updated successfully, but these errors were encountered:

ammarkarachi · 2021-10-08T17:12:55Z

@osddeitf Can you share your custom policies file? I tried it with 6.2.1 and don't see that error

osddeitf · 2021-10-09T03:34:48Z

I already shared it above, but this is how I reproduced it just now:

Update amplify-cli from v6.1.1 to v6.2.1
Run amplify push, it was success, but the custom policy I added to <function-name>-cloudformation-template.json get deleted:

"CustomLambdaExecutionPolicy": {
      "Type": "AWS::IAM::Policy",
      "DependsOn": [
        "LambdaExecutionRole"
      ],
      "Properties": {
        "PolicyName": "custom-lambda-execution-policy",
        "Roles": [
          {
            "Ref": "LambdaExecutionRole"
          }
        ],
        "PolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": "cloudfront:*",
              "Resource": [
                "*"
              ]
            }
          ]
        }
      }
    }

I add custom-policies.json with the following content:

[
  {
    "Effect": "Allow",
    "Action": ["cloudfront:*"],
    "Resource": ["arn:aws:cloudfront:::*"]
  }
]

Run amplify push again, after Uploading files... log, it resulted in the following error:

? Are you sure you want to continue? Yes
strict mode: missing type "object" for keyword "additionalProperties" at "#" (strictTypes)
✖ An error occurred when pushing the resources to the cloud
🛑 
strict mode: unknown keyword: "optionalProperties"
An error occurred during the push operation: strict mode: unknown keyword: "optionalProperties"

I tried with different custom-policies.json, even copy examples from docs, but unfortunately only the following worked:

[]

I run amplify inside this Docker image:

FROM node:14-bullseye-slim

RUN npm i -g @aws-amplify/cli

USER node

CMD [ "bash" ]

osddeitf · 2021-10-09T03:37:15Z

Anyway, the way it removed the existing custom policy in the template files is definitely a breaking change.

osddeitf · 2021-10-09T03:52:35Z

This may be due to wrong usage of ajv, in packages/amplify-cli-core/src/customPoliciesUtils.ts:

export const CustomIAMPoliciesSchema = {
  type : 'array',
  minItems: 1,
  items: {
    type: 'object',
    properties: {
      Action: { type: 'array', items: { type: 'string' }, minItems: 1, nullable: false },
      Resource: { type: 'array', items: { type: 'string' }, minItems: 1, nullable: false }
    },
    optionalProperties: {
      Effect: { type: 'string', enum:['Allow', 'Deny'], default: 'Allow' },
    },
    required: ['Resource', 'Action'],
    additionalProperties: true
  },
  additionalProperties: false
}

According to ajv docs:
Array type has no additionalProperties, and object type has no optionalProperties.

osddeitf · 2021-10-09T04:10:48Z

The above schema CustomIAMPoliciesSchema works fine with ajv@6.12.6, but not with ajv@8.6.3. Looks like somehow the dependencies were wrong.

osddeitf · 2021-10-09T05:40:10Z

I think I found the problem:

CustomIAMPoliciesSchema defined in packages/amplify-cli-core/src/customPoliciesUtils.ts, and amplify-cli-core packages has ajv@^6.12.3 as dependencies.
CustomIAMPoliciesSchema is used in packages/amplify-provider-awscloudformation/src/pre-push-cfn-processor/cfn-pre-processor.ts, but amplify-provider-awscloudformation has no ajv dependencies.
So module cfn-pre-processor.ts, will use ajv dependencies in @aws-amplify/cli/node_modules/ajv, which is v8.6.3.

We should pin ajv version of amplify-provider-awscloudformation to the same as amplify-cli-core.

grovejc · 2021-10-20T15:10:14Z

I am having this same issue using 6.3.1 cli.

osddeitf · 2021-10-21T01:55:20Z

@grovejc I created a PR for fixing this already, hopefully it will soon be landed in a release.

bensewell · 2021-11-02T11:40:22Z

Is there a workaround for this? It's blocking my CI at the moment, as Amplify's built-in CI auto-patches to the latest version.

osddeitf · 2021-11-02T12:32:09Z

@bensewell As I wrote in the PR, for linux, specifically inside official node Docker image:

I tried to go to /usr/local/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation, run npm install ajv@^6.12.3, then the issue is resolved.

dudzin · 2021-11-09T16:21:18Z

Hello, I have the same issue on CI but not when I execute it in localhost. On localhost I have amplify-cli 6.3.1.
What can I do to fix the CI?
The actual error is
- Uploading files... 2021-11-09T16:01:14.522Z [WARNING]: strict mode: missing type "object" for keyword "additionalProperties" at "#" (strictTypes) 2021-11-09T16:01:14.523Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud 2021-11-09T16:01:14.524Z [WARNING]: ✖ There was an error initializing your environment. 2021-11-09T16:01:14.583Z [INFO]: �[0mError: strict mode: unknown keyword: "optionalProperties"�[0m �[0m at checkStrictMode (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/util.ts:211:28)�[0m �[0m at checkUnknownRules (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/util.ts:27:22)�[0m �[0m at alwaysValidSchema (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/util.ts:17:3)�[0m �[0m at Object.code (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/vocabularies/applicator/items.ts:16:26)�[0m �[0m at keywordCode (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:523:9)�[0m �[0m at /root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:265:9�[0m �[0m at CodeGen.code (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:525:33)�[0m �[0m at CodeGen.block (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:680:20)�[0m �[0m at iterateKeywords (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:262:7)�[0m �[0m at groupKeywords (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:241:7)�[0m �[0m at /root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:233:38�[0m �[0m at CodeGen.code (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:525:33)�[0m �[0m at CodeGen.block (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:680:20)�[0m �[0m at schemaKeywords (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:232:7)�[0m �[0m at typeAndKeywords (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:161:3)�[0m �[0m at /root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:100:5�[0m �[0m at CodeGen.code (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:525:33)�[0m �[0m at /root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:61:45�[0m �[0m at CodeGen.code (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:525:33)�[0m �[0m at CodeGen.func (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/codegen/index.ts:699:24)�[0m �[0m at validateFunction (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:60:9)�[0m �[0m at topSchemaObjCode (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:94:3)�[0m �[0m at validateFunctionCode (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/validate/index.ts:42:7)�[0m �[0m at Ajv.compileSchema (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/compile/index.ts:163:25)�[0m �[0m at Ajv._compileSchemaEnv (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/core.ts:718:24)�[0m �[0m at Ajv.compile (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/ajv/lib/core.ts:370:34)�[0m �[0m at validateCustomPolicies (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/pre-push-cfn-processor/cfn-pre-processor.ts:132:30)�[0m �[0m at Object.writeCustomPoliciesToCFNTemplate (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/pre-push-cfn-processor/cfn-pre-processor.ts:63:9)�[0m �[0m at updateS3Templates (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/push-resources.ts:759:7)�[0m �[0m at Object.run (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/push-resources.ts:198:5)�[0m

osddeitf · 2021-11-10T02:02:48Z

@dudzin, and for anyone having issue with CI.
This is how i mitigate the issue currently, basically opt out of the new feature:

don't use custom-policies.json.
rename resource name of any manually added policies in <function_name>-cloudformation-template.json to make sure CustomLambdaExecutionPolicy won't appear in Resource section. (e.g. rename CustomLambdaExecutionPolicy to MyCustomLambdaExecutionPolicy).

They are having a lot of works going on in transform v2 or some sorts. So I think newer version of amplify-cli than 6.3.1 won't come out soon.

bensewell · 2021-11-10T17:29:34Z

I think I've found a way of applying the patch above in CI for Amplify.

In your amplify.yml (which is either in the root of your repo, or under Build Settings in the console), add a backend section - I've put my amplify.yml here for reference: https://gist.github.com/bensewell/60b9aafa23c156e98735b66b702aaf53

osddeitf · 2021-11-11T01:45:55Z

v6.4.0 released, it's time to test it.

josefaidt · 2021-12-08T22:05:24Z

Hey @osddeitf 👋 just wanted to follow-up here and see if you're still experiencing this issue after the merge?

osddeitf · 2021-12-09T04:42:50Z

Sorry, i forgot to close this issue.

josefaidt · 2021-12-09T14:40:24Z

No worries @osddeitf ! Thank you for the contribution! 🚀

github-actions · 2022-02-08T01:47:27Z

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels for those types of questions.

yuth added pending-triage Issue is pending triage functions Issues tied to the functions category labels Oct 8, 2021

osddeitf mentioned this issue Oct 11, 2021

fix: Lambda custom policies not working #8415

Merged

2 tasks

yuth added bug Something isn't working and removed pending-triage Issue is pending triage labels Oct 11, 2021

josefaidt mentioned this issue Oct 20, 2021

migrate manual Lambda CFN changes to custom-policies.json aws-amplify/discord-bot#21

Closed

osddeitf closed this as completed Dec 9, 2021

github-actions bot locked as resolved and limited conversation to collaborators Feb 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: Lambda custom policies not working #8392

Bug: Lambda custom policies not working #8392

osddeitf commented Oct 8, 2021

ammarkarachi commented Oct 8, 2021 •

edited

Loading

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

grovejc commented Oct 20, 2021 •

edited

Loading

osddeitf commented Oct 21, 2021

bensewell commented Nov 2, 2021

osddeitf commented Nov 2, 2021

dudzin commented Nov 9, 2021

osddeitf commented Nov 10, 2021 •

edited

Loading

bensewell commented Nov 10, 2021 •

edited

Loading

osddeitf commented Nov 11, 2021

josefaidt commented Dec 8, 2021

osddeitf commented Dec 9, 2021

josefaidt commented Dec 9, 2021

github-actions bot commented Feb 8, 2022

Bug: Lambda custom policies not working #8392

Bug: Lambda custom policies not working #8392

Comments

osddeitf commented Oct 8, 2021

Before opening, please confirm:

How did you install the Amplify CLI?

If applicable, what version of Node.js are you using?

Amplify CLI Version

What operating system are you using?

Amplify Categories

Amplify Commands

Describe the bug

Expected behavior

Reproduction steps

GraphQL schema(s)

Log output

Additional information

ammarkarachi commented Oct 8, 2021 • edited Loading

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

osddeitf commented Oct 9, 2021

grovejc commented Oct 20, 2021 • edited Loading

osddeitf commented Oct 21, 2021

bensewell commented Nov 2, 2021

osddeitf commented Nov 2, 2021

dudzin commented Nov 9, 2021

osddeitf commented Nov 10, 2021 • edited Loading

bensewell commented Nov 10, 2021 • edited Loading

osddeitf commented Nov 11, 2021

josefaidt commented Dec 8, 2021

osddeitf commented Dec 9, 2021

josefaidt commented Dec 9, 2021

github-actions bot commented Feb 8, 2022

ammarkarachi commented Oct 8, 2021 •

edited

Loading

grovejc commented Oct 20, 2021 •

edited

Loading

osddeitf commented Nov 10, 2021 •

edited

Loading

bensewell commented Nov 10, 2021 •

edited

Loading