Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not swallow errors in case npm ls fails without an empty stdout and use spawn for child processes #373

Merged
merged 4 commits into from
Apr 25, 2018

Conversation

HyperBrain
Copy link
Member

@HyperBrain HyperBrain commented Apr 25, 2018

What did you implement:

If npm ls fails the plugin tries to analyze stderr for npm errors that can be ignored. If none are found, it will pass stdout to JSON.parse().
The case that any non-npm error was happening was not handled correctly (e.g. if the stdout maxbuffer size got exceeded by a very big dependency section). The plugin passed the empty stdout to JSON parse and failed there without telling the exact reason for the failure.
Additionally, Node's buffered stdio streams do not work reliably and the exec method is not meant to be used for processes that might return data of arbitrary length.

How did you implement it:

The packager executables are now executed with child_process.spawn which allows for stdio streams of arbitrary length. On execution failure, a proper SpawnError is now thrown that contains the full stdout and stderr streams.
Additionally this eliminates the need of the packExternalModulesMaxBuffer buffer configuration which has been removed from the README.
The implementation should seamlessly work as before.

How can we verify it:

Use a package.json with many dependencies, so that the output of npm ls exceeds the max stdout buffer default of 200k with the old serverless-webpack version.
It should work with this branch now.

Todos:

  • Write tests
  • Write documentation
  • Fix linting errors
  • Make sure code coverage hasn't dropped
  • Provide verification config / commands / resources
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

@HyperBrain HyperBrain changed the title Do not swallow errors in case npm ls fails without an empty stdout Do not swallow errors in case npm ls fails without an empty stdout and use spawn for child processes Apr 25, 2018
@HyperBrain HyperBrain added this to the 5.1.2 milestone Apr 25, 2018
Enhance error message

Removed deprecated option from README

Convert all outstanding changes (yarn and package)

Adjusted NPM unit tests

Use spawn in any npm commands

Use spawn instead of exec

Fix unit test description

Do not swallow errors in case npm ls fails without a proper stdout
@HyperBrain HyperBrain force-pushed the error-on-stdout-empty-with-npm branch from c9548bc to b379de9 Compare April 25, 2018 16:05
@HyperBrain HyperBrain merged commit 81f6696 into master Apr 25, 2018
@HyperBrain HyperBrain deleted the error-on-stdout-empty-with-npm branch April 25, 2018 16:38
@HyperBrain
Copy link
Member Author

Hi @franciscocpg . Could you do me a favor and test if the current master still works properly for you? I switched the packager invocations to use child_process.spawn instead of exec here, but only verified Windows yet. It would be great if I had an additional success report before I release the fix 😃 . Thanks.

@franciscocpg
Copy link
Member

hi @HyperBrain
I'll do it.

A sls package would be enough?

@HyperBrain
Copy link
Member Author

Thanks 😏 . Yes I think so, because it runs through the whole npm or yarn processes. There's no need to actually deploy it.

@franciscocpg
Copy link
Member

franciscocpg commented Apr 25, 2018

@HyperBrain
It works seamlessly in one of our biggest project with npm process.

I'm unable to run using yarn. See #370 (comment).

@HyperBrain
Copy link
Member Author

Thanks for the tests 👍 . I commented on the issue with yarn. Maybe the plugin could emit a better error message for that case.

@franciscocpg
Copy link
Member

Was able to test with yarn using the workaround I've mentioned in #370 (comment). It worked seamlessly too.

jamesmbourne pushed a commit to jamesmbourne/serverless-webpack that referenced this pull request Oct 15, 2019
…-on-stdout-empty-with-npm

Do not swallow errors in case npm ls fails without an empty stdout and use spawn for child processes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants