Automated build fails on 'tar' with: "Directory renamed before its status could be extracted"

[0o-de-lally]

Hello I'm trying to move a repository into Docker Cloud. All my
repositories are failing with a 'tar' command error.

The Error:
tar: .meteor/packages/babel-compiler: Directory renamed before its
status could be extracted
tar: Exiting with failure status due to previous errors

I've searched online and there are no answers. Even on Docker's github
moby/moby#19647

[pierreozoux]

I have the same here!

Here is my repo https://hub.docker.com/r/pierreozoux/tioapp/builds/bsnpyaasmo46p962p9ikbhu/

I can build fine on my machine, but can't build on the hub.

Did you move to some distributed filesyteme backend?
Or overlayFS?

I see these issues that might be related:
meteor/meteor#5762
meteorhacks/meteord#64

[0o-de-lally]

@pierreozoux Have you tried it on Quay.io ? It seems that my automated builds work there.

[pkennedyr]

Hi @pierreozoux, @keyscores: Thanks for the feedback. We are indeed on OverlayFS which might be the source of the trouble. We are happy to file an FR for selectable file drivers as an advanced feature if enough people are seeing this issue.

Please +1 if you are interested in this feature.

[0o-de-lally]

@pkennedyr @borjaburgos I have no need to set file system drivers. I think that this is a docker bug that needs to be fixed. I would recommend fixing that bug rather than adding a feature.

I just need my images to build predictably. For now I will use Quay.io, and I will recommend that others use the Quay automated build service over Docker Cloud for that reason.

[pkennedyr]

@keyscores, We value your feedback and are investigating this issue further to find a more versatile solution.

[0o-de-lally]

@pkennedyr This issue is still occurring, even though I received an email from support saying it was closed.

The error I receive:
tar: .meteor/packages/coffeescript/.1.2.3.m88qvm++os+web.browser+web.cordova: Directory renamed before its status could be extracted

[karalabe]

My repos started failing with this error about a month ago, then last week it seem to have worked a few times, then it started failing again. I'm extracting a simple tar file, nothing fancy.

[cioc]

Same issue impacting our builds : https://hub.docker.com/r/hammerlab/biokepi-run/builds/bz4d2t8qk2fttro8vdcfxpm/

[mattmcd]

Same issue with tar in my builds: https://hub.docker.com/r/mattmcd/naclbuildbase/builds/blfctp7gaqtlmgkeq3twjse/
Vaguely interesting that the previous unzip step worked fine in this build.

[0o-de-lally]

@pchico83 @borjaburgos This is still happening. I got a support message saying this was solved months ago.

Building works fine on quay.io, but dockercloud produces authentication errors when pulling from quay. So I can't use docker cloud.

[0o-de-lally]

@borjaburgos , I've written multiple times this week, with no response.

Tar files are not opening correctly during builds on certain infrastructure. I think this is an old issue that should have been solved a while ago.

Try with docker hub image: jshimko/meteor-launchpad:latest

Full error:
tar: .meteor/packages/babel-compiler/.6.13.0.1qw8sow++os+web.browser+web.cordova/npm/node_modules/.bin: Directory renamed before its status could be extracted

[0o-de-lally]

@pkennedyr When I do a 'build on your own infrastructure' option in docker cloud. The issue goes away. I think it has to do with your docker version on certain machines in your build pool. See this issue: moby/moby#19647

[pkennedyr]

@keyscores We apologize for the delay wrt this issue. Indeed, the build failure appears related to moby/moby#19647 and once the upstream overlayfs kernel update is available, Docker Cloud and Docker Hub will apply the the corresponding version update to address this issue.

[0o-de-lally]

@pkennedyr To be clear: this is intermittent. It seems like you have some servers in the pool with the correct specs, and other which are not. Sometimes I get the error, other times not.

[emilgoldsmith]

Yeah I'm also experiencing this problem right now, and even rebuilding is not working for me though it used to work and also worked intermittently but my last 15 attempts or so have all failed in a row. Do we have an estimate on when this can be fixed?

[Bregor]

Same here.

Kernel:

$ uname -r
4.9.5-040905-generic

Docker:

# docker info
Containers: 46
 Running: 15
 Paused: 0
 Stopped: 31
Images: 17
Server Version: 1.12.6
Storage Driver: overlay2
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.8.17-040817-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.796 GiB
Name: kube04
ID: BSSV:EKQL:B646:2JMV:JEPX:PPVG:GEPF:IOZF:WF3H:QDND:YLHQ:UEYN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Eventually tar returns following:

tar: ./node_modules/webpack/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./node_modules/webpack/node_modules: Directory renamed before its status could be extracted
tar: ./node_modules/webpack: Directory renamed before its status could be extracted
tar: ./node_modules/envify/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./node_modules/envify/node_modules: Directory renamed before its status could be extracted
tar: ./node_modules/envify: Directory renamed before its status could be extracted
tar: ./node_modules/agent-base/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./node_modules/agent-base/node_modules: Directory renamed before its status could be extracted
tar: ./node_modules/agent-base: Directory renamed before its status could be extracted
...

[jbernard1]

So to those of you wondering how to fix this. I followed @pommi 's direction of using bsdtar instead of tar. This is what I added to my docker file.

RUN apt-get install -y --no-install-recommends bsdtar
RUN export tar='bsdtar'

[Dan23]

Do we know when this issue will be resolved in either the edge or stable release?

[markshust]

overlayfs is the root of all these issues. convert to aufs and it should resolve this.

[mmuman]

Anyone actually tried to fix the cause of this? Seems it's been 2 years and everyone has to workaround with bsdtar or other things…
And, sorry but why is it tagged as "feature-request" while it actually really is a bug???

[beenishkh]

This is 2019 and the issue is still there, on macOS. And by now, docker desktop doesn't support 'ausf' at all, and I can't use bdstar. Wasted whole day trying to work around it. Team : You had 3 years to solve this problem :). Your work impact people's live, you could've done better here. I mean, 3 years! And still no fix. :( In the meanwhile, how do I get around it ? :'(

[kammoh]

I'm surprised this has been tagged as a "feature-request". It looks like a nasty critical bug to me; a serious bug which has been pretty much overlooked for almost 3 years!

[jonathanstiansen]

Happening for me on Mac Mojave:

$ uname -a
Darwin JonosMaokRetina.hitronhub.home 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64

[AntonioSchmidt]

For those who are having this issue with meteor, I was able to make it work following this script here:
https://github.com/jshimko/meteor-launchpad/blob/master/scripts/install-meteor.sh

I basically added these lines to my Dockerfile:

RUN apt-get update
RUN apt-get install -y --no-install-recommends bsdtar
RUN curl -v https://install.meteor.com -o /tmp/install_meteor.sh
RUN sed -i.bak "s/tar -xzf.*/bsdtar -xf \"\$TARBALL_FILE\" -C \"\$INSTALL_TMPDIR\"/g" /tmp/install_meteor.sh
RUN sh /tmp/install_meteor.sh

I hope this helps someone.

[rpavlovicz]

Either rebuilding my images or setting "RUN export tar='bsdtar'" in the Dockerfile previously worked for me when i had this problem (i was trying too many things at once to know what the fix was), but now the tar issue has presented itself again. Maybe due to an OSX update? Either way, nothing is helping now.

[MrCavallo]

I'm relatively new to Docker, running Docker Desktop on a Mac, and with my very first project, I hit this bug. Tar is completely useless, bsdtar will not work in my case, and aufs is no longer an option. What this means is that for me, Docker is completely useless.

Given that this has been an extant bug for literally years with no solution, I think I'll be deleting Docker as a purely practical matter. If the devs don't have time to deal with this, then I certainly don't.

[ahomatthew]

For those who are having this issue with meteor, I was able to make it work following this script here:
https://github.com/jshimko/meteor-launchpad/blob/master/scripts/install-meteor.sh

I basically added these lines to my Dockerfile:

RUN apt-get update
RUN apt-get install -y --no-install-recommends bsdtar
RUN curl -v https://install.meteor.com -o /tmp/install_meteor.sh
RUN sed -i.bak "s/tar -xzf.*/bsdtar -xf \"\$TARBALL_FILE\" -C \"\$INSTALL_TMPDIR\"/g" /tmp/install_meteor.sh
RUN sh /tmp/install_meteor.sh

I hope this helps someone.

@AntonioSchmidt you did help someone

[Heermosi]

This is 2019 and the issue is still there, on macOS. And by now, docker desktop doesn't support 'ausf' at all, and I can't use bdstar. Wasted whole day trying to work around it. Team : You had 3 years to solve this problem :). Your work impact people's live, you could've done better here. I mean, 3 years! And still no fix. :( In the meanwhile, how do I get around it ? :'(

As a Chinese, I simply agree you on this......
If such things happen in China, the group would be fired if they cannot find a way to resolve it in weeks.
I'm sure, 3 years is enough to fire 70+ groups of maintainers.

I got stuck on this issue for weeks, carefully checked all the limits. Fun, I've learnt a lot of things about linux(or damm it, why should people wasting time searching for a reason?) , and my boss shall send some one to assist me(or replace me?).

[darran-kelinske-fivestars]

What is a practical fix for this in 2019 on macOS running Docker Desktop

[beenishkh]

Run linux on mac via a vagrant VM and run your code there.

[mg262]

Working on a Mac, I found this issue but resolved it by upgrading my Dockerfile base image from Ubuntu 16.04 to 18.04. (See https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728489) . I'm a bit confused because I thought it was the overlayfs in the host OS that caused the issue, not the guest OS, but hey, it works.

Edit: it makes the error less frequent, but it still happens. I think the reason is that Docker for Mac is on kernel 4.9, and the issue is fixed in kernel 4.11. I recommend downgrading tothe 25 July 2018 stable release and then using aufs instead of overlayfs. [In Preferences/Daemon/Advanced, replace the contents with {"storage-driver" : "aufs"}.]

[supershabam]

I'm running into this issue running latest stable docker for mac. Hoping somebody gets assigned to this bug.

[ishandrov]

@keyscores We apologize for the delay wrt this issue. Indeed, the build failure appears related to moby/moby#19647 and once the upstream overlayfs kernel update is available, Docker Cloud and Docker Hub will apply the the corresponding version update to address this issue.

I'm still getting the bug on Mac. When are you going to fix it?! I sense it won't be fixed at all, right?!

[ryanmickler]

supposedly fixed in ubuntu xenial:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728489

[geissonator]

Didn't fix it for me on my mac with any of the recent ubuntu's (eoan, bionic, xenial).

[leafbot]

This solved it:

WORKDIR /
RUN curl https://install.meteor.com/ | /bin/sh
WORKDIR /dir/to/app
COPY . .
RUN meteor npm install

or.. its just that I increased memory and swap in docker settings.

[pruthvi6767]

On mac : used gtar 
On RHEL: installed libarchive and export tar=bsdtar

[gaebor]

Same error :( on Windows Docker Desktop

FROM ubuntu:18.04

RUN apt update && apt -y upgrade
RUN apt -y install python libpng-dev wget
RUN cd /opt \
&& wget -O - -o /dev/null ftp://ftp.fu-berlin.de/unix/misc/sage/linux/64bit/sage-8.9-Ubuntu_18.04-x86_64.tar.bz2 | tar -xjf -

bsdtar works though

[huguesbrunelle]

As @leafbot said : Increased memory and swap in docker settings worked for me.

[gaebor]

@huguesbrunelle I tried extracting a 1.7GB tar file into ~6GB data, with 4.5GB ram and 1GB swap still the same error. Unfortunately, I cannot give more memory to the VM :(

[abhinavsingi]

I deleted disk image from - Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw
and then restarted Docker - it fixed the issue for me.