[bitnami/kafka] Cannot create directory '/bitnami/kafka/config': Permission denied

[sairaghavak]

Name and Version

bitnami/kafka:latest

What architecture are you using?

amd64

What steps will reproduce the bug?

I am using docker-compose.yml to spin up a kafka contiainer based on bitnami's kafka image without zookeeper

Here is the specific service excerpt from docker-compose

kafka-container:
    image: bitnami/kafka:latest
    ports:
      - "9092:9092"
    volumes:
      - type: volume
        source: kafka-data
        target: /bitnami/kafka
    environment:
      - KAFKA_KRAFT_CLUSTER_ID=abcde
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_BROKER_ID=1
      - KAFKA_CFG_AUTO.CREATE.TOPICS.ENABLE=false
      - KAFKA_CFG_LISTENERS=PLAINTEXT://kafka-container:9092
      - KAFKA_CFG_ADVERTISED_HOST_NAME=kafka-container
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka-container:9092

The docker contianer keeps restarting and when I checked the container logs it says

cannot create directory '/bitnami/kafka/config': Permission denied

Does the user inside the container has admin permissions, not sure, any pointers would help.

What is the expected behavior?

Container should be running smoothly

What do you see instead?

kafka 16:38:36.12 INFO  ==> ** Starting Kafka setup **
kafka 16:38:36.17 WARN  ==> KAFKA_CFG_LISTENERS must include a listener for CONTROLLER
kafka 16:38:36.17 WARN  ==> You set the environment variable ALLOW_PLAINTEXT_LISTENER=yes. For safety reasons, do not use this flag in a production environment.
kafka 16:38:36.18 INFO  ==> Initializing Kafka...
kafka 16:38:36.19 INFO  ==> No injected configuration files found, creating default config files
kafka 16:38:36.33 INFO  ==> Initializing KRaft...
kafka 16:38:36.34 INFO  ==> Formatting storage directories to add metadata...

> cannot create directory '/bitnami/kafka/config': Permission denied

Additional information

Looks like the user in the container doesn't have permission

[OneCricketeer]

Related #41393 (comment)

[javsalgar]

Hi,

It seems that the permissions of the volume are not enough for the non-root container. How did you create the volume? Is it a bind-mount?

[sairaghavak]

No, it's not a bind mount, it's a named volume
specified at the top level of docker-compose file like this

volumes:
  kafka-data:

I think you are referring to Bind mount From Bitnami's containers/bitnami/kafka/README.md

[OneCricketeer]

I've been able to use named volumes perfectly fine

I suggest not using latest tags, though, since issues with those images aren't repeatable as new commits are made to the repo

[sairaghavak]

This issue is resolved after using specific tag i.e., bitnami/kafka:3.4.1 instead of latest

[migruiz4]

Hi @sairaghavak,

We have released a new major version of the bitnami/kafka image including major changes, could you please confirm if your issue persists?

[wxywizard]

I'm having the same problem with this.
May I ask how you solved it?

[migruiz4]

Hi @wxywizard,

Could you please share the docker-compose you used in your deployment?

[wxywizard]

Hi @wxywizard,

Could you please share the docker-compose you used in your deployment?

sure, something like this

  kafka-broker:
    container_name: kafka-broker
    <<: *kafka-common
    volumes:
      - "/etc/localtime:/etc/localtime"
      - "./cluster-data/kafka-broker:/bitnami/kafka"
    environment:
      <<: *kafka-common-env
      KAFKA_CFG_NODE_ID: 4
      KAFKA_CFG_BROKER_ID: 4
      KAFKA_CFG_PROCESS_ROLES: broker
      KAFKA_CFG_LISTENERS: BROKER://:9095  # kafka监听地址
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://host.docker.internal:9095   
    ports:
      - "9095:9095"
    networks:
      kafka:
        ipv4_address: 172.22.6.33

[wxywizard]

Hi @wxywizard,
Could you please share the docker-compose you used in your deployment?

sure, something like this

  kafka-broker:
    container_name: kafka-broker
    <<: *kafka-common
    volumes:
      - "/etc/localtime:/etc/localtime"
      - "./cluster-data/kafka-broker:/bitnami/kafka"
    environment:
      <<: *kafka-common-env
      KAFKA_CFG_NODE_ID: 4
      KAFKA_CFG_BROKER_ID: 4
      KAFKA_CFG_PROCESS_ROLES: broker
      KAFKA_CFG_LISTENERS: BROKER://:9095  # kafka监听地址
      KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://host.docker.internal:9095   
    ports:
      - "9095:9095"
    networks:
      kafka:
        ipv4_address: 172.22.6.33

The host user I use is root

[migruiz4]

Hi @wxywizard,

I think in this case the issue is not related to the container but the permissions of the mounted volume.

Ownership and permissions of ./cluster-data/kafka-broker? Is it using an external file system such as NFS? Is the user owning the folder the same user running the container?

[wxywizard]

Hi @wxywizard,

I think in this case the issue is not related to the container but the permissions of the mounted volume.

Ownership and permissions of ./cluster-data/kafka-broker? Is it using an external file system such as NFS? Is the user owning the folder the same user running the container?

This folder is a new directory I created with the mkdir command under root user, do I need to give any more permissions?

[migruiz4]

There are several factors that could cause a container not to have write permission over a mounted directory.

In your case I would like to suggest to not create the volume manually and instead let the docker-compose up create it.

[wxywizard]

./cluster-data/kafka-broker

I resolved the issue for this directory by using the command chown -R 1001:1001 ./cluster-data/kafka-broker. Is this because it was manually created?

[carrodher]

Yes, if the directory from your host is mapped to the container using a volume, the owner of that directory should be the UID 1001

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[AntonOfTheWoods]

@carrodher , did this change/evolve at some point? I had to reinstall a node and am now getting it refusing to create the directory, where I had no issues when installing initially. I am using the bitnami helm chart, and installing using a local volume in k3s, if that makes a difference.

[carrodher]

There were several changes lately in the Kafka container image and Helm chart, please, create a new issue describing your specific use case

[AntonOfTheWoods]

Actually I think it may be k3s specific. The local volume directory didn't have g/u write permissions, and adding them allowed kube to create the necessary directory. Thanks.

[gaoyuliang123]

./cluster-data/kafka-broker

I resolved the issue for this directory by using the command chown -R 1001:1001 ./cluster-data/kafka-broker. Is this because it was manually created?

I did this and it still didn't solve the problem, what could be the possible reason?

[TNemoo]

when I tried to put output from 3 brokers in one directiry on HD, I had the same exception