File access in mounted volumes extremely slow

[schmunk42]

continued from https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/178

Expected behavior

File-system performance in the container not significantly slower than on the host or docker-machine.

Actual behavior

Running a composer (PHP) update in a container takes much longer than on the host. In the container we usually hit timeouts when updating huge git repos like twbs/bootstrap.

Information & Steps to reproduce the behavior

See link above

[Vanuan]

Same for npm install, bundle install, browserify, etc.

[cyb3rd4d]

Hi

Ok, that's why a Symfony application is very slow in a Docker container!
I load a page in ~ 150 ms when I'm running my local Symfony server (on my host) VS ~ 2000 ms when the config files have been serialized in the cache directory within a Docker container.
It seems the IO in the vendors and the cache / logs directories are very slow...

[lfv89]

Same here with rails and its friends (rails, rake, rspec...)

Please let me know what information I can provide to help you guys out.

[mduller]

Same here as well. We're looking into using IntelliJ IDEA and git on the Mac and compiling & running our product in a container, from the shared filesystem. Development cycles are considerably longer and our devs not happy with the proposed switch. Currently evaluating workarounds to still enable the switch to Docker and in particular Docker for Mac.

[wadjeroudi]

For those who look for a workaround and didn't read the whole thread, here is the solution :
https://github.com/EugenMayer/docker-sync

[ZZromanZZ]

Same with PHP (Symfony, Nette, Composer deps...). I can confirm https://github.com/EugenMayer/docker-sync is usable workaround.

[iwaffles]

We're experiencing this as well. We have our API dockerized and sometimes endpoints time out on our local machines (only in docker for mac). We're using Rails 5.

[cyb3rd4d]

In my team, some developers use a Mac, others use Linux... Docker sync is not the solution for me because the project configuration will not be the same for all environments. I hope that the OSXFS issues will be fixed quickly.

[wadjeroudi]

@MartialGeek Off topic : https://github.com/EugenMayer/docker-sync/wiki/Keep-you-docker-compose.yml-portable
But you should create a makefile to handle the environment properly.

[cyb3rd4d]

@wadjeroudi Thank you, I will read that ASAP ;)

[petergombos]

Same issue here, takes +30 sec to run my babel build using a mounted volume vs. 5 sec using container's fs.

[jsamouh]

👍 Same issue with Git in docker, very slow ! Unusable compare to Docker Machine (with NFS)
Docker For Mac released too fast ! Need to be reviews !!!

[jzahedieh]

Whole team is having issues with Magento, 3 minute page loads vs 8 seconds using docker-sync

[BenMcH]

I'm also experiencing this issue. Hitting the cache takes no less than 1 second for each file.

[apahne]

Will this issue eventually be addressed?

[Vanuan]

Here's the latest reply from Docker Team:
https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/158

It's been quite a while since then. We all hope that they have something new to share.

[joemewes]

Just checked on latest Docker for Mac update : Version 1.12.1 (build: 12133)

seems to be still an issue.

[so0k]

Latest posts on the alternatives are interesting - https://forums.docker.com/t/alternatives-to-osxfs-performant-shares-under-osx/19565/14 - which points to https://github.com/cweagans/docker-bg-sync (minimal changes to compose file for developer allow a significant speed improvement for mounted source files)

[joemewes]

Yeah, I've gone the UNISON approach (also mentioned a couple times in the original thread) : see here https://github.com/4AllDigital/DockerDrupal-lite/blob/master/docker-compose.yml#L7

using this container : https://github.com/4AllDigital/drupaldev-unison

if anyone needs an example to copy and want to use UNISON too.

[piotrekkaminski]

Same issue with Magento. Major performance issues. We are moving to Unison now as well.

[BenMcH]

I just got around to testing the bg-sync workaround with our in-house rails app and it worked perfectly! Thanks for the suggestion @so0k

[jeanlaurent]

@aLkRicha yes, it's called edge now, and only starting on Docker 17.04 CE. See the FAQ entry on Stable and Edge if you want to switch.

[Vanuan]

@barat I think currently it should be like this

  volumes:
    - .:/symfony:ro,cached
    - app_cache_volume:/symfony/app/cache
    - vendor_volume:/symfony/vendor
    - logs_volume:/symfony/app/logs

volumes:
  app_cache_volume:
  vendor_volume:
  logs_volume:

Yes, if you need to look into vendor files in your IDE you should wait for delegated.

For now you can use duplicate vendor in both docker and out of docker with manual synchronization. So that PHPStorm uses local version and docker uses volume in VM.

[norio-nomura]

I tried :cached on building Swift toolchain for Linux that source codes are placed in mac volume.
It seems that it certainly got faster, but the memory consumption of osxfs was so great that it affected other applications.
I will continue to use the combination of docker-machine and nfs.

[andrerom]

@andrerom I understand that this is a new feature, but what about people that want to support development on multiple OSes. RHEL 6 and 7 are going to be around for a long time still, so you're saying people will need to keep multiple configs around for years just so :cached can be specified on their Macs?

@daveisfera Well, yes and no. Docker is despite all the hype pushing for it's use early adopter stuff, it is currently not mature enough for early majority users to use it broadly yet, has some known stability issues and a fragmented market. RHEL 6 is even late majority users, they by nature don't want to test out new things, so don't make them. RHEL 7 however is supported by docker packages, so once they migrate there they can take advantage of this. In other words research usage of docker for future version of your own software, not for older versions. At least not under all possible supported platforms you have there, that won't blend.

[asteinlein]

I got unknown option: cached when using this in a compose file when deploying to a stack (docker stack deploy foo --compose-file=docker-compose.yml (I'm on 17.06.0-ce-rc1-mac8 edge). When will that be supported?

[andrerom]

@asteinlein did you set version to 3.2?`, at least that is the thing needed on docker-compose.

---

For PHP/Symfony users posted numbers for :cached with eZ Platform. TL;DR works, but for any heavy REST based app doing parallel requests needed to set SYMFONY_ENV=prod.

[toooni]

@andrerom you may just change the cache and log dir in AppKernel.php to a non shareded dir (for dev):

    public function getCacheDir()
    {
        if (in_array($this->environment, ['dev', 'test'])) {
            return '/var/www/symfony/cache/'.$this->environment;
        }

        return parent::getCacheDir();
    }

    public function getLogDir()
    {
        if (in_array($this->environment, ['dev', 'test'])) {
            return '/var/www/symfony/logs';
        }

        return parent::getLogDir();
    }

[andrerom]

@toooni I did test with /tmp/symfony/ folder for cache and log but got better response times without for some reason, but I'll have a second look when I can.

[asteinlein]

@andrerom Thanks for the tip, but same error about unknown option: cached with the compose file version set to 3.2 when deploying as a stack, unfortunately.

[rivaros]

@asteinlein maybe still wrong version of docker. here's mine

ross@MAC-ROSS:~/projects/DockerPack $ docker version
Client:
 Version:      17.05.0-ce-rc1
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   2878a85
 Built:        Tue Apr 11 20:55:05 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.05.0-ce-rc1
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   2878a85
 Built:        Tue Apr 11 20:55:05 2017
 OS/Arch:      linux/amd64
 Experimental: true

Btw your docker-compose should start like:

version: "3"

[rivaros]

Personally staying with inotify/rsync solutions because of gulp watchers/builders inside containers
So waiting for "delegate". :)

[MetalArend]

@rivaros could you share that inotify/rsync solution?

[carn1x]

Is there any high level estimate for when :delegate might start to appear? Just wondering if I should try and get all my devs converted to something like docker-sync which has a lot more initial/ongoing overhead, especially for our devs who are just putting up with our decision to enforce docker rather than enthusiastically embracing it!

[sanbor]

I'm also wondering this. I'm considering switching back to DockerToolbox until Docker for Mac get's acceptable speed.`docker-compose` it's almost unusable.

…

On Fri, Apr 28, 2017 at 6:03 AM, carn1x ***@***.***> wrote: Is there any high level estimate for when :delegate might start to appear? Just wondering if I should try and get all my devs converted to something like docker-sync which has a lot more initial/ongoing overhead, especially for our devs who are just putting up with our decision to enforce docker rather than enthusiastically embracing it! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#77 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADJOIjhSEv4kRZG4-9zOEYuHCOBzo_Gks5r0atlgaJpZM4Ja5LY> .

[bilby91]

Is there any solution for mac that haves a decent speed ? Running docker in vagrant solves the write speed issue ?

[andrerom]

@sanbor @bilby91 As found earlier up in this thread, there are two main choices if :cached does not solve it for you:

• nfs either using Docker for Mac via d4m-nfs or with DockerToobox (cleanest, only thing you need to take care of is paths which gets different)
• docker-sync or similar rsync/event based syncing (probably fastest, but you'll need to change your setup for it, and you might run into sync issues)

[ianks]

Why does the docker for mac engine not just mount NFS?

[aimakun]

@ianks There are several reasons for not using NFS. The issue is current one Docker for Mac using still have own problems. Give the team more time to solve that.

As @andrerom mentioned above, you still can use workarounds. I'm still using docker/toolbox and it's working fine.

Even Docker for Windows has long way to go, currently it still only available on Windows 10 pro.

[lox]

Could mods close this thread to prevent more comments like @ianks? Lots of us want to keep tabs on updates on the issue and the sheer volume of "use nfs or docker-sync" comments makes that very annoying.

[luckydonald]

@lox, You shouldn't kill discussion however.
Don't be so grumpy 😃

[ianks]

@lox It was a question, not a statement.

[lox]

Search on this page for NFS @ianks. If that doesn't clarify your question, we can provide more detail.

[EugenMayer]

To add something for the discussion, i made some benchmarks with :cached and the newest edge release today https://github.com/EugenMayer/docker-sync/wiki/Performance-Tests-2017

In the end, :cached will not really help with 17s, thats even far below the fusion based shares with 12s, not to mention when you compare it with native_osxm which has only 0.20s, near hte native speed with 0.19s.

we are talking about :cached beeing 50x slower then a native docker container or alternatives

[lephuongbg]

@EugenMayer, :cached is used for improving read speed, and you're testing write speed. You will have to wait for :delegate for that benchmark.

[Vanuan]

Looks like cached/delegated wouldn't solve the issue. People wouldn't know when to apply what.

[EugenMayer]

For sure it would not, and not even because this implicates, that the container has the superior over the host - which is right the opposite what you need when you actually develop. The container will hardly ever surprise you with something "new interesting" - and if anything, its generated and reproducible.

Very very different for code changes, which are not reproducible and very hard to write again ( mentally alone ). Thats why docker-sync uses always "hosts wins" while still using the buffer. Losing code is just not funny - losing generated assets (in case it ever happens to be a conflict) is a no care.

so :delegate is just something you do never want to touch - even if i do get the general idea.

[yallop]

Subscribers to this issue might be interested to read the following blog post:

    User-guided caching in Docker for Mac

which explains how and when to use cached, along with some details about the implementation.

Since there are now significant improvements in the use cases that prompted this issue, we're closing the issue to further conversation. We'll continue to post occasional updates here when further performance improvements are available. There's a new, more focused issue open to track the progress of the caching implementation (i.e. the implementation of delegated, and further improvements to cached):

    File system performance improvements

If you'd like to follow developments closely, we recommend subscribing there. Thanks to everyone for contributing to this conversation!