Shell server terminated (code: 1, signal: null)

[NixBiks]

• VSCode Version: 1.62
• Local OS Version: macOS Monterey v12.0.1 M1 chip
• Remote OS Version: mcr.microsoft.com/vscode/devcontainers/base:0-bullseye
• Remote Extension/Connection Type: Docker Desktop 4.3.0
• Logs:

[2021-12-10T16:04:17.847Z] Remote-Containers 0.205.2 in VS Code 1.62.3 (ccbaa2d27e38e5afa3e5c21c1c7bef4657064247).
[2021-12-10T16:04:17.847Z] Start: Resolving Remote
[2021-12-10T16:04:17.849Z] Setting up container for folder or workspace: /Users/nix/Projects/plx/RabbitSystem
[2021-12-10T16:04:17.854Z] Start: Check Docker is running
[2021-12-10T16:04:17.854Z] Start: Run: docker version --format {{.Server.APIVersion}}
[2021-12-10T16:04:17.958Z] Stop (104 ms): Run: docker version --format {{.Server.APIVersion}}
[2021-12-10T16:04:17.959Z] Server API version: 1.41
[2021-12-10T16:04:17.959Z] Stop (105 ms): Check Docker is running
[2021-12-10T16:04:17.966Z] Start: Run: git rev-parse --show-cdup
[2021-12-10T16:04:17.971Z] Stop (5 ms): Run: git rev-parse --show-cdup
[2021-12-10T16:04:17.972Z] Start: Run: docker ps -q -a --filter label=vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem --filter label=vsch.quality=stable
[2021-12-10T16:04:18.082Z] Stop (110 ms): Run: docker ps -q -a --filter label=vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem --filter label=vsch.quality=stable
[2021-12-10T16:04:18.083Z] Start: Run: docker inspect --type container 6b9a6f764045
[2021-12-10T16:04:18.196Z] Stop (113 ms): Run: docker inspect --type container 6b9a6f764045
[2021-12-10T16:04:18.196Z] Start: Removing Existing Container
[2021-12-10T16:04:18.196Z] Start: Run: docker rm -f 6b9a6f76404561cff4b81d578d976c13b2a143744c32d20769b5e2d36d41cdae
[2021-12-10T16:04:18.332Z] Stop (136 ms): Run: docker rm -f 6b9a6f76404561cff4b81d578d976c13b2a143744c32d20769b5e2d36d41cdae
[2021-12-10T16:04:18.332Z] Stop (136 ms): Removing Existing Container
[2021-12-10T16:04:18.333Z] Start: Run: docker build -f /Users/nix/Projects/plx/RabbitSystem/.devcontainer/Dockerfile -t vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57 --build-arg UPGRADE_PACKAGES=true /Users/nix/Projects/plx/RabbitSystem/.devcontainer
[2021-12-10T16:04:18.684Z] [+] Building 0.0s (0/1)                                                         
[2021-12-10T16:04:18.806Z] [+] Building 0.1s (3/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for mcr.microsoft.com/vscode/devcontainers/b  0.1s
[2021-12-10T16:04:18.806Z] 
[2021-12-10T16:04:18.822Z] [+] Building 0.1s (8/8) FINISHED                                                
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for mcr.microsoft.com/vscode/devcontainers/b  0.1s
 => [internal] load build context                                          0.0s
[2021-12-10T16:04:18.823Z]  => => transferring context: 152B                                          0.0s
 => [1/3] FROM mcr.microsoft.com/vscode/devcontainers/base:0-bullseye@sha  0.0s
 => CACHED [2/3] COPY library-scripts/*.sh /tmp/library-scripts/           0.0s
 => CACHED [3/3] RUN apt-get update     && /bin/bash /tmp/library-scripts  0.0s
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:469cb6a73ae40298a1c693b3140a2ee14e6446bde5707  0.0s
 => => naming to docker.io/library/vsc-rabbitsystem-2fa42f726273f93e353d1  0.0s
[2021-12-10T16:04:19.046Z] 
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[2021-12-10T16:04:19.064Z] Stop (731 ms): Run: docker build -f /Users/nix/Projects/plx/RabbitSystem/.devcontainer/Dockerfile -t vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57 --build-arg UPGRADE_PACKAGES=true /Users/nix/Projects/plx/RabbitSystem/.devcontainer
[2021-12-10T16:04:19.064Z] Fetching remote container features: Disabled
[2021-12-10T16:04:19.069Z] Start: Run: docker inspect --type image vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57
[2021-12-10T16:04:19.185Z] Stop (116 ms): Run: docker inspect --type image vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57
[2021-12-10T16:04:19.186Z] local container features stored at: /Users/nix/.vscode/extensions/ms-vscode-remote.remote-containers-0.205.2/dist/node_modules/vscode-dev-containers/container-features
[2021-12-10T16:04:19.186Z] Start: Run: tar --no-same-owner -x -f -
[2021-12-10T16:04:19.197Z] Stop (11 ms): Run: tar --no-same-owner -x -f -
[2021-12-10T16:04:19.197Z] Start: Run: docker build -t vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features --build-arg BASE_IMAGE=vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57 --build-arg IMAGE_USER=root /var/folders/ck/sm_0mymn3x55m23w9z2s8zl40000gn/T/vsch/container-features/0.205.2-1639152259064
[2021-12-10T16:04:19.589Z] 
[2021-12-10T16:04:19.589Z] [+] Building 0.0s (0/1)                                                         
[2021-12-10T16:04:19.690Z] [+] Building 0.1s (7/7)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 578B                                       0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/vsc-rabbitsystem-2fa42  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 237.62kB                                      0.0s
 => [1/3] FROM docker.io/library/vsc-rabbitsystem-2fa42f726273f93e353d13e  0.0s
 => CACHED [2/3] COPY . /tmp/build-features/                               0.0s
[2021-12-10T16:04:19.690Z]  => CACHED [3/3] RUN cd /tmp/build-features/f_microsoft-vscode-dev-contai  0.0s
[2021-12-10T16:04:19.701Z] [+] Building 0.1s (8/8) FINISHED                                                
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 578B                                       0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/vsc-rabbitsystem-2fa42  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 237.62kB                                      0.0s
 => [1/3] FROM docker.io/library/vsc-rabbitsystem-2fa42f726273f93e353d13e  0.0s
[2021-12-10T16:04:19.701Z]  => CACHED [2/3] COPY . /tmp/build-features/                               0.0s
 => CACHED [3/3] RUN cd /tmp/build-features/f_microsoft-vscode-dev-contai  0.0s
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:2f002bd7d32d49a5598a6ae8d1d26329f2503cd96bd91  0.0s
 => => naming to docker.io/library/vsc-rabbitsystem-2fa42f726273f93e353d1  0.0s
[2021-12-10T16:04:19.755Z] 
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[2021-12-10T16:04:19.773Z] Stop (576 ms): Run: docker build -t vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features --build-arg BASE_IMAGE=vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57 --build-arg IMAGE_USER=root /var/folders/ck/sm_0mymn3x55m23w9z2s8zl40000gn/T/vsch/container-features/0.205.2-1639152259064
[2021-12-10T16:04:19.774Z] Start: Run: docker events --format {{json .}} --filter event=start
[2021-12-10T16:04:19.775Z] Start: Starting container
[2021-12-10T16:04:19.775Z] Start: Run: docker run --sig-proxy=false -a STDOUT -a STDERR --mount type=bind,source=/Users/nix/Projects/plx/RabbitSystem,target=/workspaces/RabbitSystem,consistency=cached --mount type=volume,src=vscode,dst=/vscode --mount type=volume,src=minikube-config,dst=/home/vscode/.minikube -l vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem -l vsch.quality=stable -l vsch.remote.devPort=0 -e SYNC_LOCALHOST_GCLOUD=true -e NEXT_PUBLIC_apiKey=needed-to-build-artemis -e NEXT_PUBLIC_authDomain=needed-to-build-artemis -e NEXT_PUBLIC_databaseURL=needed-to-build-artemis -e NEXT_PUBLIC_projectId=needed-to-build-artemis -e NEXT_PUBLIC_storageBucket=needed-to-build-artemis -e NEXT_PUBLIC_messagingSenderId=needed-to-build-artemis -e NEXT_PUBLIC_appId=needed-to-build-artemis -e NEXT_PUBLIC_measurementId=needed-to-build-artemis -e NEXT_PUBLIC_BACKEND_BASE_URL=needed-to-build-artemis --init --privileged --init vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features
[2021-12-10T16:04:20.194Z] Stop (419 ms): Starting container
[2021-12-10T16:04:20.194Z] Start: Run: docker ps -q -a --filter label=vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem --filter label=vsch.quality=stable
[2021-12-10T16:04:20.207Z] sed: couldn't flush stdout: Device or resource busy
[2021-12-10T16:04:20.308Z] Stop (533 ms): Run: docker run --sig-proxy=false -a STDOUT -a STDERR --mount type=bind,source=/Users/nix/Projects/plx/RabbitSystem,target=/workspaces/RabbitSystem,consistency=cached --mount type=volume,src=vscode,dst=/vscode --mount type=volume,src=minikube-config,dst=/home/vscode/.minikube -l vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem -l vsch.quality=stable -l vsch.remote.devPort=0 -e SYNC_LOCALHOST_GCLOUD=true -e NEXT_PUBLIC_apiKey=needed-to-build-artemis -e NEXT_PUBLIC_authDomain=needed-to-build-artemis -e NEXT_PUBLIC_databaseURL=needed-to-build-artemis -e NEXT_PUBLIC_projectId=needed-to-build-artemis -e NEXT_PUBLIC_storageBucket=needed-to-build-artemis -e NEXT_PUBLIC_messagingSenderId=needed-to-build-artemis -e NEXT_PUBLIC_appId=needed-to-build-artemis -e NEXT_PUBLIC_measurementId=needed-to-build-artemis -e NEXT_PUBLIC_BACKEND_BASE_URL=needed-to-build-artemis --init --privileged --init vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features
[2021-12-10T16:04:20.312Z] Stop (118 ms): Run: docker ps -q -a --filter label=vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem --filter label=vsch.quality=stable
[2021-12-10T16:04:20.312Z] Start: Run: docker inspect --type container f770875ee654
[2021-12-10T16:04:20.423Z] Stop (111 ms): Run: docker inspect --type container f770875ee654
[2021-12-10T16:04:20.424Z] Start: Inspecting container
[2021-12-10T16:04:20.424Z] Start: Run: docker inspect --type container f770875ee6542719fcaadfb1a530d1902c5db6b8a725341583421b62c7eb8869
[2021-12-10T16:04:20.539Z] Stop (115 ms): Run: docker inspect --type container f770875ee6542719fcaadfb1a530d1902c5db6b8a725341583421b62c7eb8869
[2021-12-10T16:04:20.540Z] Stop (116 ms): Inspecting container
[2021-12-10T16:04:20.540Z] Start: Run in container: /bin/sh
[2021-12-10T16:04:20.541Z] Start: Run in container: uname -m
[2021-12-10T16:04:20.669Z] Stop (129 ms): Run in container: /bin/sh
[2021-12-10T16:04:20.670Z] Start: Run in container: cat /etc/passwd
[2021-12-10T16:04:20.670Z] Stdin closed!
[2021-12-10T16:04:20.673Z] Shell server terminated (code: 1, signal: null)

Error response from daemon: Container f770875ee6542719fcaadfb1a530d1902c5db6b8a725341583421b62c7eb8869 is not running

2. devcontainer.json

{
  "name": "RabbitSystem",
  "dockerFile": "Dockerfile",
  "runArgs": [
    "--init",
    "--privileged"
  ],
  "mounts": [
  ],
  "overrideCommand": false,
  "settings": {},
  "extensions": [],
  "remoteUser": "vscode",
  "build": {
    "args": {
      "UPGRADE_PACKAGES": "true"
    },
  },
  "features": {
    "kubectl-helm-minikube": "1.22",
    "python": "3.9",
    "git": "latest",
    "node": "16"
  }
}

2. Dockerfile

# Note: You can use any Debian/Ubuntu based image you want. 
FROM mcr.microsoft.com/vscode/devcontainers/base:0-bullseye

# [Option] Install zsh
ARG INSTALL_ZSH="true"
# [Option] Upgrade OS packages to their latest versions
ARG UPGRADE_PACKAGES="false"
# [Option] Enable non-root Docker access in container
ARG ENABLE_NONROOT_DOCKER="true"
# [Option] Use the OSS Moby Engine instead of the licensed Docker Engine
ARG USE_MOBY="true"
# [Option] Engine/CLI Version
ARG DOCKER_VERSION="latest"

# Enable new "BUILDKIT" mode for Docker CLI
ENV DOCKER_BUILDKIT=1

# Install needed packages and setup non-root user. Use a separate RUN statement to add your
# own dependencies. A user of "automatic" attempts to reuse an user ID if one already exists.
ARG USERNAME=vscode
ARG USER_UID=1000
ARG USER_GID=$USER_UID
COPY library-scripts/*.sh /tmp/library-scripts/
RUN apt-get update \
    && /bin/bash /tmp/library-scripts/common-debian.sh "${INSTALL_ZSH}" "${USERNAME}" "${USER_UID}" "${USER_GID}" "${UPGRADE_PACKAGES}" "true" "true" \
    # Use Docker script from script library to set things up
    && /bin/bash /tmp/library-scripts/docker-in-docker-debian.sh "${ENABLE_NONROOT_DOCKER}" "${USERNAME}" "${USE_MOBY}" "${DOCKER_VERSION}" \
    # Clean up
    && apt-get autoremove -y && apt-get clean -y && rm -rf /var/lib/apt/lists/* /tmp/library-scripts/

VOLUME [ "/var/lib/docker" ]

# Setting the ENTRYPOINT to docker-init.sh will start up the Docker Engine 
# inside the container "overrideCommand": false is set in devcontainer.json. 
# The script will also execute CMD if you need to alter startup behaviors.
ENTRYPOINT [ "/usr/local/share/docker-init.sh" ]
CMD [ "sleep", "infinity" ]

# [Optional] Uncomment this section to install additional OS packages.
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
#     && apt-get -y install --no-install-recommends <your-package-list-here>

Does this issue occur when you try this locally?: Yes
Does this issue occur when you try this locally and all extensions are disabled?: Yes

[chrmarti]

It looks like the container stopped and that causes the shell server to be stopped too. Does the container keep running when you start it yourself from the command line?

[NixBiks]

Does this answer your question?

╰─❯ docker run vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features                  
mount: /sys/kernel/security: permission denied.
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: /tmp: permission denied.

Not sure how to run the container from the command line other than simply run the image

[NixBiks]

Also tried this

╰─❯ docker run --sig-proxy=false -a STDOUT -a STDERR --mount type=bind,source=/Users/nix/Projects/plx/RabbitSystem,target=/workspaces/RabbitSystem,consistency=cached --mount type=volume,src=vscode,dst=/vscode --mount type=volume,src=minikube-config,dst=/home/vscode/.minikube -l vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem -l vsch.quality=stable -l vsch.remote.devPort=0 --init --privileged --init vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features
sed: couldn't flush stdout: Device or resource busy

[chrmarti]

Also tried this

╰─❯ docker run --sig-proxy=false -a STDOUT -a STDERR --mount type=bind,source=/Users/nix/Projects/plx/RabbitSystem,target=/workspaces/RabbitSystem,consistency=cached --mount type=volume,src=vscode,dst=/vscode --mount type=volume,src=minikube-config,dst=/home/vscode/.minikube -l vsch.local.folder=/Users/nix/Projects/plx/RabbitSystem -l vsch.quality=stable -l vsch.remote.devPort=0 --init --privileged --init vsc-rabbitsystem-2fa42f726273f93e353d13eb90ab5d57-features
sed: couldn't flush stdout: Device or resource busy

Did this continue running or did it return to the shell prompt? It appears to be an issue with the image/container, but it's hard to tell where to look next from what we currently have.

[afterthought]

I have been struggling with this for the last few days. Problems out of the blue on containers that I have been using for months.

Setting "overrideCommand": true gets past the immediate issue. However, I then have problems with docker-in-docker. Something that had been working fine.

[afterthought]

Another observation. The devcontainers that I have been using for the last few months looked very similar to @mr-bjerre sample above. To get around the docker-in-docker not working, I have been updating to the experimental "features" to add docker-in-docker. I have noticed a limitation though. The feature mounts a volume named dind-var-lib-docker and if I try to start a second devcontainer it won't activate docker-in-docker due to the conflict.

[NixBiks]

Interesting @afterthought

I also just noticed that if I use ubuntu focul or debian buster instead of debian bullseye then I get this error instead

Failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[chrmarti]

@afterthought The volume name conflict is tracked as #5679.

[chrmarti]

@mr-bjerre Tried with your configuration and it works for me. Does it stop with 'Shell server terminated' every time or only sometimes?

@afterthought What OS and hardware are you running with?

[afterthought]

I've been testing on both x86 and apple silicon Macs. The latest error mentioned by @mr-bjerre is what I get on apple silicon (using bullseye) but only if using the "features": {} config. Thanks for the tracking issue. We're trying to transition into Codespaces anyway, so we were going to move forward with the conflict.

[NixBiks]

My latest finding:

I'm running on a Macbook Pro with M1 and I need to work in a devcontainer with x86_64 architecture until this is resolved.

So I'm creating my devcontainers with FROM --platform=x86_64 .... That works fine unless I use features in devcontainer.json - If I use features then I get the following error

Failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Workaround

Build my own Dockerfile that supports docker in docker, python and node. I almost have it working but unfortunately I'm getting an error when trying to run docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (maybe it isn't possible to run in x86_64 arch and have docker in docker? Out of my knowledge).

[NixBiks]

I just tested a fresh docker in docker on my M1 mac. If I use the default buster` (x86_64) then I get the following

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

If I change my Dockerfile to use bullseye (arm64) then it works. I guess that means no docker in docker in a x86_64 container on a M1 mac?

[chrmarti]

@mr-bjerre Not sure if mounting a socket across architectures could cause problems. Does it work with an ARM container? /cc @Chuxel

[NixBiks]

@mr-bjerre Not sure if mounting a socket across architectures could cause problems. Does it work with an ARM container? /cc @Chuxel

@chrmarti Yes, did you see my last comment?

If I change my Dockerfile to use bullseye (arm64) then it works. I guess that means no docker in docker in a x86_64 container on a M1 mac?

[Chuxel]

Yeah docker-in-docker requires your chip architecture to match given how things work. We mention this here: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md

To see what is going on with docker startup, you can type cat /tmp/dockerd.log once you're in the container.

[github-actions]

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!