Error Building phar #714
Comments
|
This can be related to file open limits. For a successful build this limit needs to be raised from the system standard configuration. I experienced this on my own already, it's a bit of a pity this is still not very well documented. So the references are a bit scattered:
Please let me know if you tried the How much this counts as an actual flaw in PHP is not totally clear to me, I can confirm you can prevent the error by raising the limits. Here some resources from the PHP bug tracker, perhaps it's worth to file another report as there still might be an underlying flaw with the Phar PHP extension:
In any case, please try with the limits. 500 000 should be sufficient, these worked for me. |
|
I tried with both the master and develop branch and encountered the same error. Added the lines as suggested (in |
|
@pocallaghan Thanks for the detailed feedback and good to see you got it running. |
Comporessing a phar file needs a lot of open file descriptors/file handles, roughly as many as compressed files it contains. As that limit is (for magerun normally) below the hard limit (of an assumed 4096), *build.sh* raises the soft-limit to the hard-limit prior running the phing build. - The *build.sh* script raises the openfiles soft-limit to the maximum possible size (the hard-limit). - The *build.sh* script did not exit with a non-zero exit status when it ran into an error on build. Fix by keeping the exit status and returning it. - The patched phing task to create the phar file shows the number of files that are compressed. - The patched phing task catches BadMethodCallException on Phar::compressFiles(), checks for the exception message and inserts a more-saying exception message in the middle. This should alleviate the problem reported in #714. In case the error happens again, more information should be available from the exception message then.
|
@pocallaghan: Good news, I reviewed this again and could find out more. First of all, on a fresh Ubuntu 14.04 64Bit system, I've got the following openfiles limits: The build.sh script needs an openfiles limit of the number of files to compress plus a little offset for the script and PHP itself. A limit of 1839 (that is 1834 files to be compressed in the phar file plus the offset of five for the script itself) did it for me then. As this is between soft and hard limit, raising the soft-limit to the maximum should be a good "fix" on this issue. Thanks again for taking the time to report this and the feedback you provided! |
Compressing a phar file needs a lot of open file descriptors/file handles, roughly as many as compressed files it contains. As that limit is (for magerun normally) below the hard limit (of an assumed 4096), *build.sh* raises the soft-limit to the hard-limit prior running the phing build. - The *build.sh* script raises the openfiles soft-limit to the maximum possible size (the hard-limit). - The *build.sh* script did not exit with a non-zero exit status when it ran into an error on build. Fix by keeping the exit status and returning it. - The patched phing task to create the phar file shows the number of files that are compressed. - The patched phing task catches BadMethodCallException on Phar::compressFiles(), checks for the exception message and inserts a more-saying exception message in the middle. This should alleviate the problem reported in #714. In case the error happens again, more information should be available from the exception message then.
|
This should be fixed now, at least up to the hard-limit the system is configured for. I'll close this for now. I only write should because this can't be tested for all circumstances easily (for example, the travis server didn't have this problem on the openfiles limit so it got unnoticed as the system I wrote build.sh on was already "mis"configured (@pocallaghan 500 000 is quite high for this build.sh, so sorry for making this suggestion first. it works but is far off at least for magerun phar building. might be OK for a dev machine anyway)). |
|
I only changed it on a vagrant box I spun up specifically for the test so no bother. I had no idea such limits existed before this, every days a lesson. :) |
|
Yes, good teamwork! Also for me I now finally understood what's going on with the phar compression and why it triggers these limits. Everyday a lesson, that's a good saying (-: |
Compressing a phar file needs a lot of open file descriptors/file handles, roughly as many as compressed files it contains. As that limit is (for magerun normally) below the hard limit (of an assumed 4096), *build.sh* raises the soft-limit to the hard-limit prior running the phing build. - The *build.sh* script raises the openfiles soft-limit to the maximum possible size (the hard-limit). - The *build.sh* script did not exit with a non-zero exit status when it ran into an error on build. Fix by keeping the exit status and returning it. - The patched phing task to create the phar file shows the number of files that are compressed. - The patched phing task catches BadMethodCallException on Phar::compressFiles(), checks for the exception message and inserts a more-saying exception message in the middle. This should alleviate the problem reported in #714. In case the error happens again, more information should be available from the exception message then.
|
(found some older notes which is better to placed here) Some internals: For the concrete exception message:
I suspect it is triggered by phar/phar.c#2846 ff which leads to I can't say whether this construes a flaw or not. It looks to me that each file can be handeled with a different compression (no compression, gzip or bzip2) and that file-pointer holds the compressed data (as a pointer to a tempfile) until the phar file is written out later on phar/phar.c#3073 ff. This at least is expensive on the number of open file handles and doesn't scale well for a large number of files. |
Pretty sure it's a localised issue, just hoping somebody else may have ran into the issue and have a solution. Every time I try and use
./build.shI get the error:I'm running a MBP with El Capitan and PHP 5.5.27. But I also tried a build on a ubuntu vagrant box with PHP 5.3 and got the same error.
The text was updated successfully, but these errors were encountered: