AWS Beanstalk deploy permission error - node-pre-gyp

[BHare1985]

The pull request #658 solves the issue;
I am documenting that this node-pre-gyp permission issue was caused from preinstalling node-pre-gyp.

Node 4.4.3 on Linux 4.4.8-20.46.amzn1.x86_64 (AWS Beanstalk) and mapnik 3.5.13
I am getting permission errors when running the preinstall of mapnik.

It seems as if all the other node modules get installed into /tmp/npm- with root as the owner but node-pre-gyp gets it's own folder but ec2-user as the owner. ec2-user is the default user for AWS beanstalk, so I making the assumption that preinstall commands get ran as the non-root user (ec2-user) but node dependencies get ran as root.

Therefore, you cannot install mapnik/node-pre-gyp on Amazon beanstalk because of permission issues. Possible work around is to include a ebextention script setting /tmp as 777 (not recommended)

commands:
    01_set_file_permissions:
        command: "chmod 777 -R /tmp"

The other work around I have success with is to fork node-mapnik and revert back to when node-pre-gyp wasn't preinstalled but bundled (Essentially what #658 is doing). This has been confirmed as installing without issue.

Issue was originally created at : mapbox/node-pre-gyp#220

[darwinserrano]

Hi. I'm experiencing a similar error, but I can see that in my install has an error when try to download this package https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.5.13/Release/node-v48-linux-x64.tar.gz

I can see this message

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>4F5017D96B8D7CF2</RequestId><HostId>0mtiZgjmuHcexj0nbtJ4tD8mImgyWEvqzVHrKYyU0zgj170/c06fEuYVHeQM+HVp8gXqsZ6S37c=</HostId></Error>

The error log.

sudo npm install mapnik
npm WARN lifecycle mapnik@3.5.13~preinstall: cannot run in wd %s %s (wd=%s) mapnik@3.5.13 npm install node-pre-gyp /home/darwin/proyectos/node/testmapnik/node_modules/.staging/mapnik-fe363e44

mapnik@3.5.13 install /home/darwin/proyectos/node/testmapnik/node_modules/mapnik
node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download: https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.5.13/Release/node-v48-linux-x64.tar.gz 
node-pre-gyp ERR! Pre-built binaries not found for mapnik@3.5.13 and node@6.2.2 (node-v48 ABI) (falling back to source compile with node-gyp) 
/bin/sh: 1: mapnik-config: not found
gyp: Call to 'mapnik-config --cflags' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:305:16)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)
gyp ERR! System Linux 4.4.0-24-generic
gyp ERR! command "/usr/bin/nodejs" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/home/darwin/proyectos/node/testmapnik/node_modules/mapnik/lib/binding/node-v48-linux-x64/mapnik.node" "--module_name=mapnik" "--module_path=/home/darwin/proyectos/node/testmapnik/node_modules/mapnik/lib/binding/node-v48-linux-x64"
gyp ERR! cwd /home/darwin/proyectos/node/testmapnik/node_modules/mapnik
gyp ERR! node -v v6.2.2
gyp ERR! node-gyp -v v3.3.1
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/nodejs /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/home/darwin/proyectos/node/testmapnik/node_modules/mapnik/lib/binding/node-v48-linux-x64/mapnik.node --module_name=mapnik --module_path=/home/darwin/proyectos/node/testmapnik/node_modules/mapnik/lib/binding/node-v48-linux-x64' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/home/darwin/proyectos/node/testmapnik/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at emitTwo (events.js:106:13)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:852:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)
node-pre-gyp ERR! System Linux 4.4.0-24-generic
node-pre-gyp ERR! command "/usr/bin/nodejs" "/home/darwin/proyectos/node/testmapnik/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /home/darwin/proyectos/node/testmapnik/node_modules/mapnik
node-pre-gyp ERR! node -v v6.2.2
node-pre-gyp ERR! node-pre-gyp -v v0.6.29
node-pre-gyp ERR! not ok 
Failed to execute '/usr/bin/nodejs /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/home/darwin/proyectos/node/testmapnik/node_modules/mapnik/lib/binding/node-v48-linux-x64/mapnik.node --module_name=mapnik --module_path=/home/darwin/proyectos/node/testmapnik/node_modules/mapnik/lib/binding/node-v48-linux-x64' (1)
npm ERR! Linux 4.4.0-24-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "mapnik"
npm ERR! node v6.2.2
npm ERR! npm  v3.9.5
npm ERR! code ELIFECYCLE

npm ERR! mapnik@3.5.13 install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the mapnik@3.5.13 install script 'node-pre-gyp install --fallback-to-build'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the mapnik package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-pre-gyp install --fallback-to-build
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs mapnik
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls mapnik
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/darwin/proyectos/node/testmapnik/npm-debug.log

Regards.

[BHare1985]

node-v48 is node 6, I am not sure node-mapnik has precompiled versions for node 6 yet. If you try node 4 you should be able to download https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.5.13/Release/node-v46-linux-x64.tar.gz no problem. I don't think your issue is identical to this one

[tehnrd]

I also came across this original issue when updating the Elastic Beanstalk configuration to v2.1.3 which also required me to update npm and node versions to resolve some other unrelated errors. The chmod 777 fix works but wouldn't it also be possible to create a group, add ec2-user to that group and give that group access to the /tmp folder? My command line knowledge is seriously lacking but I'll try to take a stab a this.

[tehnrd]

FWIW I tried running the command:

01_set_folder_owner:
    command: "chown -R ec2-user /tmp"

...with the thought this would change the owner to the ec2-user and then this user would have access to the folder during the preinstall, but this did not work. :-/

[tehnrd]

I have spent way too much time trying to debug this issue due to the painfully slow deployments on Elastic Beanstalk. This is not specifically a mapnik issue, but is more of an issue on the AWS Elastic Beanstalk side. The chmod 777 command works, sort of. All of my testing was done with a "Load Balancing, auto scaling" Environment type. The deployment behavior is very strange. If you add that command and make a deployment it will fail, at first. On failure when you are using the "Load Balancing, auto scaling" type it will instantly try to re-deploy, and this re-deploy will succeed. This can be seen by setting the minimum number of instances in the Scaling section to two, terminated an instance, and then watch a new one be added to take its place. Deployment to the new instance on initial bootup will fail, and then succeed in quick succession. Any new deployments to this box moving forward will then succeed on the first try.

This also makes Immutable updates impossible because initial deployment to the new instances always fails, as there is no retry, and AWS stops the deployment. Rolling updates attempt to retry the deployment and it is this second retry deploy that works.

Node: 4.4.6
NPM: 2.15.5
Elastic Beanstalk Config: 64bit Amazon Linux 2016.03 v2.1.3

At some point I will try to create a post on the AWS support forums, but given this work around sort of works, I'm not sure when I may get to it. EDIT: Looks like BHare1985 already created a post on AWS forums, but it didn't get much attention. https://forums.aws.amazon.com/thread.jspa?messageID=734162&#734162

[springmeyer]

This is not specifically a mapnik issue, but is more of an issue on the AWS Elastic Beanstalk side

I agree, but thanks everyone for detailing the issue. i'm going to close this now since there is no clear action to be taken in node-mapnik. Bundling is not desirable and if it triggers other bugs, then those need to be solved in the tools causing them: npm or aws.

[BHare1985]

@tehnrd

In regards to the command failing the first time around , I believe commands are ran post-app deploy. I had luck with this. Also better to limit /tmp to ec2-user rather than 777

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/00_set_tmp_permissions.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      chown -R ec2-user /tmp

[holyjak]

I believe that an alternative (better?) solution to changing the /tmp permissions is to set unsafe-perm=true in .npmrc of the application being deployed to Beanstalk - see https://stackoverflow.com/questions/46001516/beanstalk-node-js-deployment-node-gyp-fails-due-to-permission-denied/46001517#46001517

[Lepozepo]

Recently ran into this issue as well. We tried setting permissions but that didn't seem to work, it seems the main reason is that the 50npm.sh file runs a python script that does not have permissions to write. Running this script with sudo passes root permissions down to the script so that it doesn't fail. Some people may gasp at using sudo but unless you want to rewrite the python script I'm not sure what other path you can take.

Here is what we did:

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/bash
      #==============================================================================
      # Copyright 2013 Amazon.com, Inc. or its affiliates. All Rights Reserved.
      #
      # Licensed under the Amazon Software License (the "License"). You may not use
      # this file except in compliance with the License. A copy of the License is
      # located at
      #
      #       http://aws.amazon.com/asl/
      #
      # or in the "license" file accompanying this file. This file is distributed on
      # an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
      # implied. See the License for the specific language governing permissions
      # and limitations under the License.
      #==============================================================================

      chmod 777 -R /tmp
      set -xe

      sudo /opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install

[vimmisinghani]

it is still not working
ERR! sharp 'darwin-x64' binaries cannot be used on the 'linux-x64' platform. Please remove the 'node_modules/sharp/vendor' directory and run 'npm install'.
info sharp Attempting to build from source via node-gyp but this may fail due to the above error
info sharp Please see https://sharp.pixelplumbing.com/install for required dependencies
gyp ERR! clean error
gyp ERR! stack Error: EACCES: permission denied, rmdir 'build'
gyp ERR! System Linux 4.14.171-105.231.amzn1.x86_64
gyp ERR! command "/opt/elasticbeanstalk/node-install/node-v12.16.1-linux-x64/bin/node" "/opt/elasticbeanstalk/node-install/node-v12.16.1-linux-x64/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /tmp/deployment/application/node_modules/sharp

[Lepozepo]

@vimmisinghani it looks like that's not a permissions error. sharps binaries might not work on your VM because it is linux. You'll have to go through something like this