[aws-lambda-nodejs] .parcel-cache permission denied in BitBucket Pipelines #9312

SergKam · 2020-07-28T23:05:15Z

Bitbucket Pipeline failed to create NodejsFunction in docker. When I attempt to create a lambda using that construct and synthesize, I get an error [Error: EACCES: permission denied, mkdir '/asset-input/.parcel-cache']

Reproduction Steps

import * as cdk from '@aws-cdk/core';
import * as nodejs from '@aws-cdk/aws-lambda-nodejs';

export class BugStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new nodejs.NodejsFunction(this, 'handler');
  }
}

The handler a bit complex to show here, but it is irrelevant.

Error Log

> zolar-cdk@1.0.0 deploy /opt/atlassian/pipelines/agent/build/.cdk
> cdk --trace deploy "-a" "projects/sfbridge/create.js"
Bundling asset dev-serg-test-sfbridge/dev-serg-test-sfbridge-sf-zpc-bridge/Code/Stage...
[Error: EACCES: permission denied, mkdir '/asset-input/.parcel-cache'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'mkdir',
  path: '/asset-input/.parcel-cache'
}
/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/core/lib/asset-staging.js:123
            throw new Error(`Failed to run bundling Docker image for asset ${this.node.path}: ${err}`);
            ^
Error: Failed to run bundling Docker image for asset dev-serg-test-sfbridge/dev-serg-test-sfbridge-sf-zpc-bridge/Code/Stage: Error: docker exited with status 1
    at AssetStaging.bundle (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/core/lib/asset-staging.js:123:19)
    at new AssetStaging (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/core/lib/asset-staging.js:38:35)
    at new Asset (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-s3-assets/lib/asset.js:21:25)
    at AssetCode.bind (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-lambda/lib/code.js:137:26)
    at new Function (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-lambda/lib/function.js:80:33)
    at new NodejsFunction (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-lambda-nodejs/lib/function.js:30:13)
    at ZolarSFBridge.scheduleLambda (/opt/atlassian/pipelines/agent/build/.cdk/projects/sfbridge/ZolarSFBridge.js:34:24)
    at new ZolarSFBridge (/opt/atlassian/pipelines/agent/build/.cdk/projects/sfbridge/ZolarSFBridge.js:29:14)
    at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/.cdk/projects/sfbridge/create.js:7:19)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
Subprocess exited with error 1

Environment

CLI Version : 1.53.0
Framework Version: 1.53.0
Node.js Version: 12
OS : lambci/lambda:build-nodejs12.x
Language (Version): TypeScript (3.8.3)

Other

looks related to #8757

This is 🐛 Bug Report

The text was updated successfully, but these errors were encountered:

jogold · 2020-07-29T06:50:25Z

Hi @SergKam,

Have you tried playing with the cacheDir option?

SergKam · 2020-07-29T08:54:31Z

@jogold No, but I found a workaround for now. I'm creating directory .parcel-cache with 0777 permissions in the current working directory before running the CDK.
I think it doesn't matter what cacheDir I set they all should be inside my working dir (restriction by BitBacket docker daemon).
To make it work "out of the box" cdk should create this directory with the right permissions before running parcel in docker.

anilchinnam · 2020-08-14T20:28:17Z

I am also running into this issue and unable to upgrade my cdk version from 1.36.1 to 1.58.0. I tried couple of previous versions to upgrade but the issue is been there for quite some time.

Please some one help, what is the work around on this, I tried providing the 777 permission to both .cdk.staging and .parcel-cache folders and deleted these folders before running the synth. Still getting the same error.
This is on my Mac and i have the docker running (not sure why it is looking for docker in first place for bundling)

Bundling asset hello-world-dev-ApiDynamoDbStack/dlqS3UtilFunction/dlqS3UtilFunction/Code/Stage...
[Error: EACCES: permission denied, mkdir '/.parcel-cache'] {
errno: -13,
code: 'EACCES',
syscall: 'mkdir',
path: '/.parcel-cache'
}
Error: Failed to bundle asset hello-world-dev-ApiDynamoDbStack/dlqS3UtilFunction/dlqS3UtilFunction/Code/Stage: Error: docker exited with status 1
at AssetStaging.bundle (/Users/achinnam/Applications/hello-world/hello-world-app/node_modules/@aws-cdk/core/lib/asset-staging.ts:196:13)
at new AssetStaging (/Users/achinnam/Applications/hello-world/hello-world-app/node_modules/@aws-cdk/core/lib/asset-staging.ts:93:29)

SergKam · 2020-08-17T12:39:30Z

@anilchinnam In my case, I'm using docker to build and deploy lambdas with cdk.
my Dockerfile looks like this

FROM lambci/lambda:build-nodejs12.x
WORKDIR /cdk/
COPY bin bin
COPY config config
COPY lib lib
COPY projects projects
COPY index.ts .
COPY package.json .
COPY package-lock.json .
COPY tsconfig.json .
COPY cdk.json .
ENV NODE_ENV production
# Parcel need this to find project root for NodeJsLambda
RUN mkdir --parents .git/keep && mkdir --mode=0777 .parcel-cache && chown 1000:1000 .parcel-cache && \
    npm i && npm run build && npm run build:lambdas
WORKDIR /app/

aws cdk uses another docker image with "parcel" inside to build lambda, and inside this "parcel docker" it uses a user with user id 1000.
so i added chown 1000:1000 .parcel-cache just to be sure. I don't know is it actually needed since mode 777 allows write to everyone.

ps: my setup only works up to cdk version 1.54 (i cannot upgrade because another issue #9351)

anilchinnam · 2020-08-18T16:25:19Z

@SergKam Thank you for the response.
Our app currently not using docker with the CDK 1.36.1 version. This is one of the main reason we could not upgrade past 1.36.1 as nodejs-lambda bundle functionality is changed since 1.37.0 . I do not see if there is any AWS documentation on how to organize our code base for the lambda functions in order to build with the docker file.

Any input on the folder structure or documentation for building multiple lambdas in a project is helpful for us to upgrade until 1.54 version. Thanks Again.

mbonig · 2020-08-25T23:17:22Z

So, not sure if it helps anyone else, but I found I was having this similar issue. I had a direct "dependency" in my package.json to parcel and parcel-bundler. Removing those (and a rm -rf node_modules && npm i) resolved the issue for me.

Hope that helps someone else.

beanworks-eng-shared · 2020-09-10T21:45:42Z

☝️ This helped me fix my problem as well. To be clear I have to run npm uninstall parcel to remove parcel from my packages.json But make sure you have parcel installed globally: npm install -g parcel@next

rv-rmiranda · 2020-09-15T18:08:45Z

I am using AWS CDK 1.63.0, and Docker 9.03.8, in my Mac I have no issue creating the bundles and deploy the Lambdas. But in CircleCI I am getting the error:

[Error: EACCES: permission denied, mkdir '/asset-input/src/lambdas/proxy/.parcel-cache'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'mkdir',
  path: '/asset-input/src/lambdas/proxy/.parcel-cache'
}

Also, I have no cache, so I am installing all node modules on each deployment, so I don't think is an NPM Dependenciesissue in my case. Any ideas on how to fix this?

jogold · 2020-09-16T08:04:41Z

@rv-rmiranda can you share your folder structure and also how you're instantiating the NodejsFunction construct?

jogold · 2020-09-16T13:57:28Z

@eladb I wonder if we should always run docker bundling as root when in CI (process.env.CI). Not only for aws-lambda-nodejs but in core:

We switched to a non root user only to avoid creating content that is owned by root (see #8489, #8492). But in CI we don't care about this, no?

eladb · 2020-09-16T16:34:42Z

I wonder if we should always run docker bundling as root when in CI

What's the reason /asset-input/src/lambdas/proxy/.parcel-cache is not writable by the CI user?

rv-rmiranda · 2020-09-16T21:59:19Z

@rv-rmiranda can you share your folder structure and also how you're instantiating the NodejsFunction construct?

Folder Structure

./infrastructure
 |— bin/
 |———— type_script_infrastructure.ts
 |— lib/
 |———— proxy-stack.ts
 |— cdk.json
 |— package.json
 |— ...
./src
|—— lambdas/
|———— proxy/
|—————— proxy.js
|—————— package.json

Lambda Stack:

import * as core from "@aws-cdk/core";
import { Runtime } from "@aws-cdk/aws-lambda";
import { NodejsFunction } from "@aws-cdk/aws-lambda-nodejs";

export class ProxyStack extends core.Stack {
  public proxy: NodejsFunction;

  constructor(scope: core.App, id: string, props?: core.StackProps) {
    super(scope, id, props);

    const stage = process.env.STAGE || "development";

    this.proxy = new NodejsFunction(this, "LambdaProxy", {
      entry: "../src/lambdas/proxy/proxy.js",
      functionName: `lambda-name-${stage}`,
      handler: "handler",
      memorySize: 1048,
      runtime: Runtime.NODEJS_12_X,
      timeout: core.Duration.minutes(15),
      minify: (stage === "production")? true : false,
    });
  }
}

Please, let me know if you need anything else.

Thank you!

jogold · 2020-09-17T07:27:32Z

Please, let me know if you need anything else.

Are you using the docker executor or machine executor? https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI and https://circleci.com/docs/2.0/executor-types/#using-machine

rv-rmiranda · 2020-09-17T14:53:18Z

Please, let me know if you need anything else.

Are you using the docker executor or machine executor? https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI and https://circleci.com/docs/2.0/executor-types/#using-machine

@jogold — I was using docker executor, I change it to machine executor, and now is working! 🎉 🍻

Working Code

version: 2.1
jobs:
  build_and_deploy:
    machine:
      image: ubuntu-1604:202007-01
    working_directory: ~/repo
    steps:
      - checkout
      - attach_workspace:
          at: .
   
... MORE CODE ...

Not Working Code

version: 2.1
jobs:
  build_and_deploy:
    docker:
      - image: cimg/python:3.8.4
    working_directory: ~/repo
    steps:
      - checkout
      - attach_workspace:
          at: .

      - setup_remote_docker:
          version: 19.03.12
          docker_layer_caching: true

... MORE CODE ...

Thank you for the help!

eladb · 2020-09-17T18:17:27Z

Please, let me know if you need anything else.

Are you using the docker executor or machine executor? https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI and https://circleci.com/docs/2.0/executor-types/#using-machine

@jogold — I was using docker executor, I change it to machine executor, and now is working! 🎉 🍻

Working Code
version: 2.1

jobs:

  build_and_deploy:

    machine:

      image: ubuntu-1604:202007-01

    working_directory: ~/repo

    steps:

      - checkout

      - attach_workspace:

          at: .

   

... MORE CODE ...
Not Working Code
version: 2.1

jobs:

  build_and_deploy:

    docker:

      - image: cimg/python:3.8.4

    working_directory: ~/repo

    steps:

      - checkout

      - attach_workspace:

          at: .



      - setup_remote_docker:

          version: 19.03.12

          docker_layer_caching: true



... MORE CODE ...
Thank you for the help!

Should we add a note in our docs?

jogold · 2020-09-17T18:26:29Z

@eladb yes, will do.

@SergKam were you able to find a solution by changing your pipeline configuration (similar to the CircleCI solution) or do you still rely on creating the folder beforehand?

SergKam · 2020-09-17T21:53:16Z

@jogold As far as I know, there is no option in bitbucket pipelines to switch to anything else except Docker
https://support.atlassian.com/bitbucket-cloud/docs/use-docker-images-as-build-environments/
So no, we still need to create this folder before cdk command.

Replace Parcel with esbuild for bundling. esbuild offers [impressive performances](https://esbuild.github.io/) compared to Parcel. Moreover everything can be configured via the CLI. This means that we don't need to play with the user `package.json` file anymore. Add full Windows support for local bundling. Refactor and clean-up. Closes #10286 Closes #9130 Closes #9312 Resolves #11222 BREAKING CHANGE: local bundling now requires `esbuild` to be installed. * **lambda-nodejs**: `projectRoot` has been replaced by `depsLockFilePath`. It should point to your dependency lock file (`package-lock.json` or `yarn.lock`) * **lambda-nodejs**: `parcelEnvironment` has been renamed to `bundlingEnvironment` * **lambda-nodejs**: `sourceMaps` has been renamed to `sourceMap`

github-actions · 2020-11-18T10:03:48Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

SergKam added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 28, 2020

github-actions bot assigned eladb Jul 28, 2020

github-actions bot added the @aws-cdk/aws-lambda-nodejs label Jul 28, 2020

eladb added effort/small Small work item – less than a day of effort p2 labels Aug 4, 2020

eladb assigned jogold Aug 4, 2020

SomayaB removed the needs-triage This issue or PR still needs to be triaged. label Sep 16, 2020

jogold mentioned this issue Nov 11, 2020

feat(lambda-nodejs): esbuild bundling #11289

Merged

eladb closed this as completed in #11289 Nov 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[aws-lambda-nodejs] .parcel-cache permission denied in BitBucket Pipelines #9312

[aws-lambda-nodejs] .parcel-cache permission denied in BitBucket Pipelines #9312

SergKam commented Jul 28, 2020

jogold commented Jul 29, 2020

SergKam commented Jul 29, 2020

anilchinnam commented Aug 14, 2020

SergKam commented Aug 17, 2020

anilchinnam commented Aug 18, 2020

mbonig commented Aug 25, 2020

beanworks-eng-shared commented Sep 10, 2020

rv-rmiranda commented Sep 15, 2020 •

edited

Loading

jogold commented Sep 16, 2020

jogold commented Sep 16, 2020

eladb commented Sep 16, 2020

rv-rmiranda commented Sep 16, 2020 •

edited

Loading

jogold commented Sep 17, 2020

rv-rmiranda commented Sep 17, 2020

eladb commented Sep 17, 2020

jogold commented Sep 17, 2020

SergKam commented Sep 17, 2020

github-actions bot commented Nov 18, 2020

[aws-lambda-nodejs] .parcel-cache permission denied in BitBucket Pipelines #9312

[aws-lambda-nodejs] .parcel-cache permission denied in BitBucket Pipelines #9312

Comments

SergKam commented Jul 28, 2020

Reproduction Steps

Error Log

Environment

Other

jogold commented Jul 29, 2020

SergKam commented Jul 29, 2020

anilchinnam commented Aug 14, 2020

SergKam commented Aug 17, 2020

anilchinnam commented Aug 18, 2020

mbonig commented Aug 25, 2020

beanworks-eng-shared commented Sep 10, 2020

rv-rmiranda commented Sep 15, 2020 • edited Loading

jogold commented Sep 16, 2020

jogold commented Sep 16, 2020

eladb commented Sep 16, 2020

rv-rmiranda commented Sep 16, 2020 • edited Loading

jogold commented Sep 17, 2020

rv-rmiranda commented Sep 17, 2020

eladb commented Sep 17, 2020

jogold commented Sep 17, 2020

SergKam commented Sep 17, 2020

github-actions bot commented Nov 18, 2020

⚠️COMMENT VISIBILITY WARNING⚠️

rv-rmiranda commented Sep 15, 2020 •

edited

Loading

rv-rmiranda commented Sep 16, 2020 •

edited

Loading