Composer timeouts and slowness due to default use of vboxsf on Windows

[fb-multimedia]

Issue Type

• Bug Report / Support Request

Your Environment

vagrant 1.9.2
Virtualbox 5.1.10

Your OS

• Windows 8.1

Full console output

Summary

Hello,
I know it's not the first report for this error, but I haven't find solution.
After many try I can't use composer to create a new drupal 8 project :
composer create-project drupal-composer/drupal-project:8.x-dev drupal8 --stability dev --no-interaction
I always have a composer timeout and the message "couldn't delete web/core"
I have try to install in another directory instead of the share folder but same problem.
I have change the COMPOSER_PROCESS_TIMEOUT: 1200
Can somebody confirm that this command actually work good with drupal vm under windows with default share folder ?
Thanks.

[geerlingguy]

@fb-multimedia - If you're on an older computer (especially if you don't have an SSD), or if you're using native shared folders, or if you have a slower Internet connection, or if your Internet connection is behind a firewall or proxy, you may have issues with the command.

Are you having trouble running that command manually, or does Drupal VM show a failure during vagrant up or vagrant provision when it builds a Drupal codebase as per its default configuration?

Did you set the timeout like: COMPOSER_PROCESS_TIMEOUT=2000 composer create-project drupal-composer/drupal-project... or some other way?

And does it do anything, or does it just hang after you run the command? If it's doing stuff, and failing at 1200 seconds, then maybe you need to increase the timeout to 2000, 3000, or even more (again, depends on your environment).

[fb-multimedia]

Thanks for your answer.
It fails on vagrant up or if I run the command under the VM. Finally the composer command work under /var but not in /var/www (my default shared folder)
So I think the problem come from the shared folder type.
I have try to change for nfs but It doesn't work (can't see any winnfsd.exe process launch with vagrant-winnfsd installed).
If I change type to smb vagrant up fails too. Should I install samba on the VM before changing the type on config.yml ? (I haven't found explainations for smb share)
Thanks

[geerlingguy]

@fb-multimedia - Ah, in that case it was definitely the shared folder type. The default vboxsf type is extremely slow in Windows 7/8 (and even in 10), and that will definitely lead to a Composer timeout.

The best option for speed is to use rsync as the type, but that has its own tradeoffs. Other than that, smb should 'just work' out of the box, but you have to use a terminal emulator like Cmder or Cygwin, and you have to run it as an administrator (otherwise the directory permissions get broken).

See these two guides for more information:

• http://docs.drupalvm.com/en/latest/getting-started/syncing-folders/
• http://docs.drupalvm.com/en/latest/getting-started/installation-windows/#troubleshooting-vagrant-synced-folders

Another option—though it won't help with performance—is to extend Composer's timeout.

[oxyc]

The setup that I use is to have rsync for everything except the configuration sync directory where I use NFS for two-way sync.

vagrant_synced_folders:
  - local_path: .
    destination: /var/www/drupal
    id: drupal
    type: rsync
    create: true
    excluded_paths: ['.git/', '.tmp/', 'node_modules/', 'sites/default/files/', 'config/drupal']
    options_override:
      owner: vagrant
      group: www-data
  # Use slower but two-way NFS sync for configuration sync directory.
  - local_path: config/drupal
    destination: /var/www/drupal/config/drupal
    type: "" # On Windows, on macOS "nfs"
    create: true

Note this example only works in drupal vm master because of a bug where owner/group wasnt picked up by Vagrant

[geerlingguy]

@oxyc - That's... brilliant!

It's slightly obtuse, but if you're on Windows 7 or Windows 8, that's definitely the best way to go, unless your codebase is many GB (which it hopefully isn't).

We need to get this added to the docs. And I'm going to rebuild my Windows 7 PC again to see if it'll work well that way (with a BLT project).

I definitely need to tag a new release, probably another minor version bump due to a few improvements. (Update: see #1257)

[ashabed]

@geerlingguy FYI I tried out the smb solution with my class and it worked for composer, but failed when the website had to write anything.

Using smb:

• ssh'ed into vagrant
• git cloned project repo within a synced folder
• ran composer install (successfully)
• Visited site/install.php in my browser
• Can't get through the installation. It seems it isn't able to edit settings.php, or that the edits don't sync back.

I'm going to try @oxyc 's recommendation right now, with the (maybe) additional restriction of also syncing custom modules + themes over nfs.

[geerlingguy]

@ashabed - Another note—when using SMB, you should right click on your terminal environment and "Run as administrator"—else some things get screwed up.

It might be worth recommending people use makefiles if they're on Windows (if they're just trying to get something up and running and aren't building an entire new Drupal project for ultimate deployment to a production environment). Drush Make is a lot faster than Composer for this kind of thing.

[fb-multimedia]

I have to investigate, cause when using smb as type sharing (using powershell as admin) I get errors on vagrant up.
Another solution for performances is to use smb inside the guest and share to the host
I know I'll have to backup my shared folder with this solution cause it only stay in the vbox and could have trouble with a vagrant destroy.
But unfortunatly it doesn't work with basic samba install (I post another ticket #1265)
So for the moment I'm doing it with my actual from scratch ubuntu vbox while I can use composer drupal-project or loud drupal 7 projetcs and have a very good performances with guest samba sharing.
Drupal vm offers many features and I really like to using it...

[AMDAndy]

@fb-multimedia if you can manage to get guest-to-host samba sharing to work please let me know how. I tried doing this but could never figure it out.

[geerlingguy]

Updating issue title to more accurately reflect the problem.

[geerlingguy]

@AMDAndy - Please see #1265 for discussion on SMB shares from guest to host. I know some other people have gotten it working, and though it's not the most optimal way to easily build the environment (you have to manually mount the shared folder on your host after each vagrant up or otherwise mess around with things to get this to happen), it's a ton faster than the default/reverse way to do it.

[AMDAndy]

@geerlingguy you're absolutely killing it today! I don't mind complication as I get a fast dev environment.

[geerlingguy]

@AMDAndy - You're in luck, because I'm currently helping a client in a Windows 7 (!) environment try to find a way to make development bearable. It's been about 3 years since the last time I was looking into this issue, and back then my answer was to use rsync. But nowadays, with Drupal and configuration management... some form of two-way sync is absolutely essential.

[AMDAndy]

We're stuck on Windows 7 and we're working within the VMs completely on our work laptops currently. We did steal some iMacs from our design group for our desks and I'm now running Ubuntu on my home desktop so it's not the most urgent thing for me. I'm glad you're stuck working on it, though. 👍

[geerlingguy]

@AMDAndy - rsync is a bit of a non-starter for me... I've even tried rsync with https://github.com/smerrill/vagrant-rsync-back to get files back from the VM... and that's just horribly clunky and error-prone :(

I also tried SSHFS and found there were too many bugs with Windows to be very reliable (it seems like once those bugs are fixed it might be the best option since it could work with Vagrant automatically).

The winner so far is to create a reverse Samba (SMB) share, from the Ubuntu guest to the Windows host, and I'm working on docs. I'll be pushing those over to #1265.

But basically, to tie off this particular issue: Composer stuff is going to be extraordinarily slow on Windows when using standard vboxsf (default) or even SMB shares. So my current recommendation is to create a reverse mount if you need speed.

If I were stuck on Windows, I would definitely do that (reverse mount) even though it (currently) requires a few manual steps whenever you build a new VM the first time.

Let's move that discussion over to #1265

[fb-multimedia]

With the new samba reverse parameters in #1265 I can now use composer to install drupal 8 without error in 3 minutes ! Thank you geerlinguy

[geerlingguy]

@fb-multimedia - Wonderful! Glad I can help :)