Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Cannot run as unprivileged container #340

Closed
4 tasks done
ghost opened this issue Nov 18, 2021 · 18 comments
Closed
4 tasks done

[BUG] Cannot run as unprivileged container #340

ghost opened this issue Nov 18, 2021 · 18 comments
Assignees
Labels
🐛 Bug [ISSUE] Ticket describing something that isn't working 📌 Keep Open [ISSUE][PR] Prevent auto-closing

Comments

@ghost
Copy link

ghost commented Nov 18, 2021

Environment

Self-Hosted (Docker)

Version

Latest

Describe the problem

Despite what the documentation says, it seems that Dashy fails to both drop privileges and run under an unprivileged user (via --user argument or in compose).

The compose file below can be used to reproduce the issue.

---
version: "3.8"
services:
  dashy:
    # To build from source, replace 'image: lissy93/dashy' with 'build: .'
    # build: .
    image: lissy93/dashy
    container_name: Dashy
    user: '2222:2222'
    # Pass in your config file below, by specifying the path on your host machine
    volumes:
      - ./my-config.yml:/app/public/conf.yml:Z
    ports:
      - 8088:80
    # Set any environmental variables
    environment:
      - NODE_ENV=production
    # Specify your user ID and group ID. You can find this by running `id -u` and `id -g`
      - UID=2222
      - GID=2222
    # Specify restart policy
    restart: unless-stopped
    # Configure healthchecks
    healthcheck:
      test: ['CMD', 'node', '/app/services/healthcheck']
      interval: 1m30s
      timeout: 20s
      retries: 3
      start_period: 40s

The expected result is that the UID and GID values, matching the user argument, would work properly together to launch the container in a fully unprivileged state.

The actual result is that it fails to start and the entire build/prep process depends on root privileges being effective at launch.

Additional info

No response

Please tick the boxes

@ghost ghost added the 🐛 Bug [ISSUE] Ticket describing something that isn't working label Nov 18, 2021
@ghost ghost assigned Lissy93 Nov 18, 2021
@liss-bot
Copy link
Collaborator

Welcome to Dashy 👋
It's great to have you here, but unfortunately your ticket has been closed to prevent spam. Before reopening this issue, please ensure the following criteria are met.

Issues are sometimes closed when users:

  • Have only recently joined GitHub
  • Have not yet stared this repository
  • Have not previously interacted with the repo

Before you reopen this issue, please also ensure that:

  • You have checked that a similar issue does not already exist
  • You have checked the documentation for an existing solution
  • You have completed the relevant sections in the Issue template

Once you have verified the above standards are met, you may reopen this issue. Sorry for any inconvenience caused, I'm just a bot, and sometimes make mistakes 🤖

@Lissy93
Copy link
Owner

Lissy93 commented Nov 18, 2021

Sudo privileges are not be required to run the app. But let me look into it, I run it with a non-privileged user, so haven't come across this issue before. Could you share the console output?

@ghost
Copy link
Author

ghost commented Nov 18, 2021

Lissy, here you go:

# docker-compose logs -f
Attaching to Dashy
Dashy    | yarn run v1.22.15
Dashy    | warning Skipping preferred cache folder "/.cache/yarn" because it is not writable.
Dashy    | warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-3005".
Dashy    | $ npm-run-all --parallel build-watch start
Dashy    | warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
Dashy    | warning Skipping preferred cache folder "/.cache/yarn" because it is not writable.
Dashy    | warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-3005".
Dashy    | warning Skipping preferred cache folder "/.cache/yarn" because it is not writable.
Dashy    | warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-3005".
Dashy    | $ node server
Dashy    | $ vue-cli-service build --watch --mode production
Dashy    | warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
Dashy    | warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
Dashy    | 
Dashy    | Checking config file against schema...
Dashy    | No issues found, your configuration is valid :)
Dashy    | 
Dashy    | SSL Not Enabled: Public key not present
Dashy    | 
Dashy    | 
Dashy    |  ██████╗  █████╗ ███████╗██╗  ██╗██╗   ██╗
Dashy    |  ██╔══██╗██╔══██╗██╔════╝██║  ██║╚██╗ ██╔╝
Dashy    |  ██║  ██║███████║███████╗███████║ ╚████╔╝
Dashy    |  ██║  ██║██╔══██║╚════██║██╔══██║  ╚██╔╝
Dashy    |  ██████╔╝██║  ██║███████║██║  ██║   ██║
Dashy    |  ╚═════╝ ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝   ╚═╝
Dashy    | 
Dashy    | *******************************************************************************************
Dashy    | Welcome to Dashy! 🚀
Dashy    | Your new dashboard is now up and running with Docker
Dashy    | After updating your config file, run  'docker exec -it [container-id] yarn build' to rebuild
Dashy    | *******************************************************************************************
Dashy    | 
Dashy    | 
Dashy    | Using Dashy V-1.9.2. Update Check Complete
Dashy    | ✅ Dashy is Up-to-Date
Dashy    | 
Dashy    | 
Dashy    | -  Building for production...
Dashy    |  WARN  A new version of sass-loader is available. Please upgrade for best experience.
Dashy    |  ERROR  Error: EACCES: permission denied, rmdir '/app/dist'
Dashy    | Error: EACCES: permission denied, rmdir '/app/dist'
Dashy    | error Command failed with exit code 1.
Dashy    | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Dashy    | ERROR: "build-watch" exited with 1.
Dashy    | error Command failed with exit code 1.
Dashy    | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Tested on a RHEL 8 based host (functionally the same as CentOS 8, Rocky Linux 8, etc) with Docker CE. Also verified that the issue exists with SELinux policy enforcement disabled.

Could you please provide a sanitized copy of your compose file or Docker options? Also a procps/ps output displaying the container processes, which is the only effective way to prove that the container is indeed running unprivileged.

Also, you must take into account that the only real safe way to operate a Docker container under the privileges of an unprivileged user is to start it already as that user. Dropping privileges and capabilities on runtime is a tricky game, and often any leftover privileges can be used to re-escalate.

You can find a very extensive look at this issue here:
https://wiki.sei.cmu.edu/confluence/display/c/POS36-C.+Observe+correct+revocation+order+while+relinquishing+privileges

Now, back to our issue with Dashy: so far, nothing I have observed, using the compose file suggested, or the arguments in the documentation you provide, results in effective privilege removal:

root      281366  0.0  0.2 711960  9380 ?        Sl   12:01   0:00 /usr/bin/containerd-shim-runc-v2 -namespace moby -id XXX -address /run/containerd/containerd.sock
root      281386  3.7  1.8 320356 69944 ?        Ssl  12:01   0:00 node /opt/yarn-v1.22.15/bin/yarn.js build-and-start
root      281456  3.0  1.1 288832 42332 ?        Sl   12:01   0:00 /usr/local/bin/node /app/node_modules/.bin/npm-run-all --parallel build-watch start
root      281467  4.4  1.8 320392 69520 ?        Sl   12:01   0:00 /usr/local/bin/node /opt/yarn-v1.22.15/bin/yarn.js run build-watch
root      281468  4.4  1.8 320140 70016 ?        Sl   12:01   0:00 /usr/local/bin/node /opt/yarn-v1.22.15/bin/yarn.js run start
root      281489  7.7  1.4 305500 56384 ?        Sl   12:01   0:00 /usr/local/bin/node server
root      281490  118  5.1 432836 196540 ?       Rl   12:01   0:13 /usr/local/bin/node /app/node_modules/.bin/vue-cli-service build --watch --mode production
root      281513 48.0  2.9 358140 113284 ?       Ssl  12:01   0:03 /usr/local/bin/node /app/node_modules/thread-loader/dist/worker.js 20
root      281535 56.8  2.6 343732 101908 ?       Ssl  12:01   0:03 /usr/local/bin/node /app/node_modules/thread-loader/dist/worker.js 20
root      281541 53.8  2.7 345824 104636 ?       Ssl  12:01   0:03 /usr/local/bin/node /app/node_modules/thread-loader/dist/worker.js 20

Once the build completes, the situation is the same:

root      281386  0.5  1.5 309656 59676 ?        Ssl  12:01   0:00 node /opt/yarn-v1.22.15/bin/yarn.js build-and-start
root      281456  0.3  1.1 288832 42180 ?        Sl   12:01   0:00 /usr/local/bin/node /app/node_modules/.bin/npm-run-all --parallel build-watch start
root      281467  0.6  1.5 310192 59964 ?        Sl   12:01   0:00 /usr/local/bin/node /opt/yarn-v1.22.15/bin/yarn.js run build-watch
root      281468  0.5  1.5 310208 59948 ?        Sl   12:01   0:00 /usr/local/bin/node /opt/yarn-v1.22.15/bin/yarn.js run start
root      281489  0.8  1.4 305568 56388 ?        Sl   12:01   0:00 /usr/local/bin/node server
root      281490 56.7  5.8 445300 223408 ?       Sl   12:01   0:55 /usr/local/bin/node /app/node_modules/.bin/vue-cli-service build --watch --mode production

The compose file used to produce the above output:

Exactly the same as you suggest in the documentation. I'm time constrained, but a cursory look through your code (entrypoint scripts, etc) indicates there is no use of the provided UID/GID, and definitely the container executes as root:

# docker-compose exec dashy id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)

With a whole host of unwise groups (dialout can allow privileged IO to devices, video can allow the same to GPU devices, etc...).

Furthermore, searching for files owned by the UID/GID confirms this:

# docker-compose exec dashy find / -user 9999
/app/public/conf.yml

You should mark this as a security issue, as the documentation provides a false sense of security with the implicit assumption that privilege drop will work, where this is obviously not the case. The way the image is developed, it assumes and relies on running as root as far as the above tests show.

@ghost
Copy link
Author

ghost commented Nov 18, 2021

Missed an additional output:

# docker-compose exec dashy ps auxw
PID   USER     TIME  COMMAND
    1 root      0:00 node /opt/yarn-v1.22.15/bin/yarn.js build-and-start
   28 root      0:00 /usr/local/bin/node /app/node_modules/.bin/npm-run-all --p
   39 root      0:00 /usr/local/bin/node /opt/yarn-v1.22.15/bin/yarn.js run bui
   40 root      0:00 /usr/local/bin/node /opt/yarn-v1.22.15/bin/yarn.js run sta
   61 root      0:00 /usr/local/bin/node server
   62 root      0:55 /usr/local/bin/node /app/node_modules/.bin/vue-cli-service
  203 root      0:00 ps auxw

Some notes to clarify why this is a Dashy issue and not a deployment issue: you cannot rely on a non-root user operating the docker daemon socket as your assumed privilege level. In other words, if user Z within the 'docker' group was to run this container it would still not make any difference as he has effective privileges to communicate to the docker daemon, and this allows trivial root escalation in most configurations. Running docker root-less would mitigate this, but this still does not displace responsibility away from the container developer: the technically appropriate way to run a container as another user, beyond dropping privileges inside (which is better than nothing but still suboptimal) is to use the 'user' directive in the compose file so that the container is instantiated from the very beginning as a non-root user. For this to work, you need to fix the directory permissions and owner assignments and cleverly leverage DAC to allow whatever upgrades/writes must be done, without allowing anything beyond the necessary. The place to do this is the entrypoint of the container, or, optionally, an altered entrypoint and custom Dockerfile. If you can provide me with a list of sensible directories that need to be written to on runtime, I can probably test it for you sometime today.

Hopefully this clarifies the context and problem in depth.

FTR, most folks get it wrong as far as privilege drop goes with docker, and if you learn from most docker containers and how they are built you will find very few actually developed from the ground up to tolerate, let alone work well with, running with an unprivileged user in the 'user' directive. In some cases they also resort to using a nobody:root approach, where the container user still runs with root group privileges.

@ghost
Copy link
Author

ghost commented Nov 18, 2021

Had a moment to give this a go, this Dockerfile essentially fixes the problem but should not be used as-is:

FROM lissy93/dashy

RUN chown -v 0:9999 -R /app
RUN chmod -v -R g+w /app
RUN chown 9999:9999 /app/dist
RUN chown -v 0:9999 /tmp
RUN chmod -v go+rw /tmp
RUN mkdir /.cache
RUN chown 9999:9999 /.cache

Note that /.cache needn't be owned by the user, and so forth. You might need to very carefully consider if you want self-buildable objects at all. I highly suggest against this. If an user wants to upgrade he can do so from your signed image upstream or one in his own registry. The content of the image should be as static as possible, save for the config.

In any case, using user: 9999:9999 this will allow the image to run fully disengaged from the privileges of root or the user in the docker group that launched the container:

# docker-compose exec dashy id
uid=9999 gid=9999

@Lissy93 Lissy93 reopened this Nov 18, 2021
@Lissy93
Copy link
Owner

Lissy93 commented Nov 18, 2021

Thank you so much for all the details, it's super helpful :)
I'm pretty new to Docker, so your info was super helpful. I've reopened the issue, and will try and get a fix pushed out this weekend, and update the security docs referencing this.

@ghost
Copy link
Author

ghost commented Nov 18, 2021

Hello Lissy,

Yours is a hobby project and I (and our company) have a policy of reporting security bugs in cases like yours (if you were a big traded corp we wouldn't do it ;-) ) and helping whenever we can. I'm on a tight schedule at the moment but since I have personally used your project to save myself some work for internal dashboards that link to a plethora of services that I deploy for testing, I can try to help you.

OK, so:

  • Take the Dockerfile I suggested and essentially modify the upstream/original so that after you are done setting up all the directories and root-owned files (ex. the underlying OS image), you immediately launch USER 9999 or a random UID/GID.

  • Make sure you previously run all the relevant chmod/chown commands, and figure out what needs to be written. Try to avoid giving too many permissions where you don't need them. You want as much to be immutable as possible.

  • Modify the entrypoint to test if the user is passing the UID/GID in the env and also if the container is running with uid=9999 and effective gid=9999, and if it isn't, swap permissions over to the user specified and then continue with the real entrypoint.

You can take a look at how linuxserver containers do the privdrop and also how they span commands. Since I'm not very knowledgeable with JS frameworks, I'm not quite sure how much you need to be 'mutable' on runtime.

Let me know if this helps. I can review whatever you implement, but this should get it going. You can always just use a static GID/UID, but make sure you chmod/chown and take care of permissions and print errors if something is off. Permission issues are annoying to debug because they often show up in abstract errors and finding the origin requires grepping through a lot of source.

Also, this isn't your fault. The Docker documentation isn't exactly great for this and they are lacking a simple tutorial covering this. The marketing also doesn't help. Docker has been billed as a security "as a feature" thing until very recently, but the damage is done. It was meant to be used for development and reproducible testing, not containment or isolation. All the security features in Docker are patchwork.

@liss-bot
Copy link
Collaborator

This issue has gone 6 weeks without an update. To keep the ticket open, please indicate that it is still relevant in a comment below. Otherwise it will be closed in 5 working days.

@liss-bot liss-bot added the ⚰️ Stale [ISSUE] [PR] No activity for over 1 month label Dec 19, 2021
@Lissy93
Copy link
Owner

Lissy93 commented Dec 19, 2021

Still relevant. /keep-open
Got a bit stuck on the implementation, but still on the todo list.

@Lissy93 Lissy93 removed the ⚰️ Stale [ISSUE] [PR] No activity for over 1 month label Dec 19, 2021
@nallej
Copy link

nallej commented Jan 14, 2022

Did some tests and came to this docker-compose file that works:

version: '3.7'

networks:
backbone:
external: true

volumes:
public:
icons:

services:
dashy:
image: lissy93/dashy:latest
container_name: dashy
restart: unless-stopped
volumes:
- ./public/conf.yml:/app/public/conf.yml
- ./icons/png:/app/public/item-icons
environment:
- NODE_ENV=production
ports:
- 4000:80
networks:
- backbone
healthcheck:
test: ['CMD', 'node', '/app/services/healthcheck']
interval: 1m30s
retries: 3
start_period: 40s
timeout: 10s

@liss-bot

This comment has been minimized.

@nallej
Copy link

nallej commented Jan 14, 2022

Check it out: https://github.com/nallej/HomeStack/blob/main/Dashy/docker-compose.yml

Scripts to set up my HomeServers Docker containers easily. I use Proxmox 7.1 on 3 servers. Heimdall, Home-Assistant, pi-hole and nginx forming the base. For monitoring the vm's and nodes NetDat...

@Lissy93
Copy link
Owner

Lissy93 commented Jan 16, 2022

Awesome, thank you so much!! I am away from home this week, but will test and mege next week.
Really appreciate your help with this, I got stuck on this for ages.

@liss-bot

This comment was marked as off-topic.

@liss-bot liss-bot added the ⚰️ Stale [ISSUE] [PR] No activity for over 1 month label Feb 17, 2022
@Lissy93 Lissy93 added 📌 Keep Open [ISSUE][PR] Prevent auto-closing and removed ⚰️ Stale [ISSUE] [PR] No activity for over 1 month labels Feb 17, 2022
@Lissy93
Copy link
Owner

Lissy93 commented Feb 19, 2022

Thank you @Singebob 🙌 :)

@jawys
Copy link

jawys commented Jul 12, 2022

Hello @Lissy93 👋

As of today I cannot get the latest (v2.1.1) image running with non-root user, neither by using the --user flag (tried values node and 1000:1000) nor setting GID and UID in compose.

Am I missing something?

@murniox
Copy link

murniox commented Jun 20, 2023

I can confirm. This is the same for me with the following docker-compose file, which is adapted to run on an Umbrel node. I confirmed the user ID and group ID by running id -u and id -g. (1000:1000)

version: "3.8"
services:
  app_proxy:
    environment:
      APP_HOST: glimbox-dashy_server_1
      APP_PORT: 80

  server:
    image: lissy93/dashy:latest
    user: '1000:1000'
    environment:
      - NODE_ENV=production
    volumes:
      - ${APP_DATA_DIR}/data/config/conf.yml:/app/public/conf.yml
      - ${APP_DATA_DIR}/data/icons:/app/public/item-icons
    restart: on-failure

    healthcheck:
      test: ['CMD', 'node', '/app/services/healthcheck']
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s

@ghost
Copy link
Author

ghost commented Jun 21, 2023

I will suggest to look into using FROM ... AS ... constructs in the Dockerfile. Build the image on top of a working base, but make the one meant to be deployed immutable. This will also solve the problem with necessitating a bunch of permissions from the get go.

asterling8516 pushed a commit to asterling8516/dashy that referenced this issue Nov 23, 2023
Signed-off-by: Bjorn Lammers <walkxnl@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 Bug [ISSUE] Ticket describing something that isn't working 📌 Keep Open [ISSUE][PR] Prevent auto-closing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants