Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NEW] Valkey docker image #65

Closed
iakat opened this issue Mar 28, 2024 · 36 comments
Closed

[NEW] Valkey docker image #65

iakat opened this issue Mar 28, 2024 · 36 comments
Assignees

Comments

@iakat
Copy link

iakat commented Mar 28, 2024

The problem/use-case that the feature addresses

Ability to deploy Valkey on a docker environment, or replace existing redis deployments

Description of the feature

Automated docker builds for each release and/or commit, pushed ghcr.io and/or docker hub's registry under the open source program for increased pull limits

@onedr0p
Copy link

onedr0p commented Mar 28, 2024

GHCR would be best, Dockerhub's rate limits and retentions are bad. Plus remember when they tried to pull one over on us by forcing Docker Teams to pay up or apply for their OSS program (which people had problems with applying to in the past) and if they didn't they would be deleted. Pepperidge Farm Remembers

@roshkhatri
Copy link
Member

Hi, I was looking into it and implementing a few workflows for automation and GHCR looks like a good option.

@roshkhatri roshkhatri self-assigned this Mar 28, 2024
@carrodher
Copy link

carrodher commented Mar 29, 2024

I'm wearing my Bitnami hat 🎩

We are closely following this project with the idea of adding a Valkey container and Helm chart to the Bitnami catalog so they're available for users. By default, images and charts are released into DockerHub (without rate limit restrictions) and other registries such as AWS Public Gallery.

Once the first release is cut in this project, we will evaluate the addition.

@Maia-Everett
Copy link

Eagerly awaiting this for an independent project so I can replace Redis with Valkey in my Docker Compose file!

@tobybellwood
Copy link

I would also recommend publishing to dockerhub (for discoverability) but applying for DSOS status to get rate limiting and retentions lifted - https://www.docker.com/community/open-source/application/

@rlfnb
Copy link

rlfnb commented Mar 31, 2024

Please consider podman/buildah as default (and true cloud first, addressing kubernetes/Openshift) and addressing rootless containers with a reasonable capability set (LinuxSE). quay.io would be nice for a.deployment as well.

@junior
Copy link

junior commented Apr 2, 2024

Vote to publish to ghcr and docker.io (with OSS program rate limit) with an official image from valkey. Bitnami and others can release Redis paring containers for easy switch, and still be supported by the community.

@roshkhatri
Copy link
Member

I am testing the new workflow to publish to docker and once we have a release we will have a workflow to publish on docker and ghcr.
This is the image from unstable I was able to publish: https://hub.docker.com/r/roshkhatri/valkey.
Once we know the platforms, operating systems we want a docket image for, we can add rest of the implementations.

@junior
Copy link

junior commented Apr 2, 2024

I am testing the new workflow to publish to docker and once we have a release we will have a workflow to publish on docker and ghcr. This is the image from unstable I was able to publish: https://hub.docker.com/r/roshkhatri/valkey. Once we know the platforms, operating systems we want a docket image for, we can add rest of the implementations.

Let us know if need help. Remember to make the images multi-platform (amd64 and arm64 at least), to make easier for everybody.

@roshkhatri
Copy link
Member

roshkhatri commented Apr 3, 2024

I have forked from https://github.com/docker-library/redis to https://github.com/roshkhatri/valkey-container.

Building the Images for officially supported architectures is one thing this project is missing, Which is picked up by the the official images build infrastructure. I am currently trying to add multi-arch support

The Images can be found here: https://hub.docker.com/r/roshkhatri/valkey

@roshkhatri
Copy link
Member

I've added support for linux/amd64, linux/arm64 and this would be here https://hub.docker.com/r/valkey/valkey

The github repo for Dockerfile source can be found https://github.com/valkey-io/valkey-container

@onedr0p
Copy link

onedr0p commented Apr 3, 2024

@roshkhatri Any love for GHCR? It should be pretty straight forward to add that now

@metal3d
Copy link

metal3d commented Apr 7, 2024

Here's my contribution. In my humble opinion, for reasons of versioning and distribution cleanliness, I'm leaning towards Fedora (minimal version).

My Dockerfile looks like this one and I'm using podman instead of Docker (but it's of course compatible)

FROM registry.fedoraproject.org/fedora-minimal:39 as builder
ARG VALKEY_VERSION=redis-7.2.4
RUN microdnf install -y make gcc glibc-devel git curl tar python3; \
    microdnf clean all;

RUN set -xe; \
    curl -L https://github.com/valkey-io/valkey/archive/refs/tags/$VALKEY_VERSION.tar.gz -o /tmp/valkey.tar.gz; \
    tar -xzf /tmp/valkey.tar.gz -C /tmp; \
    cd /tmp/valkey-$VALKEY_VERSION; \
    make;
    make install;


FROM registry.fedoraproject.org/fedora-minimal:39 as cli
COPY --from=builder /usr/local/bin/redis-cli /usr/local/bin/valkey-cli

FROM registry.fedoraproject.org/fedora-minimal:39 as server
COPY --from=builder /usr/local/bin/redis-server /usr/local/bin/valkey-server

I haven't yet planned to copy the configuration file and docker-entry.

I'm really wondering why the Dockerfile (which you retrieved, I'm not castigating you) is so heavy. And why use Debian "by default", when other distributions are clearly suitable.

My dockerfile uses multistage and seems much simpler... So yes, a couple of things are certainly missing, but wouldn't it be wiser to start from scratch, adapt, and do things right?

@metal3d
Copy link

metal3d commented Apr 7, 2024

Ho and for the unstable branch:

(Edited)

FROM registry.fedoraproject.org/fedora-minimal:39 as builder
ARG VALKEY_VERSION=unstable
RUN microdnf install -y make gcc glibc-devel git curl tar python3 which; \
    microdnf clean all;

RUN set -xe; \
    curl -L https://github.com/valkey-io/valkey/archive/refs/heads/unstable.tar.gz -o /tmp/valkey.tar.gz; \
    tar -xzf /tmp/valkey.tar.gz -C /tmp; \
    mv /tmp/valkey-$VALKEY_VERSION /tmp/valkey; \
    cd /tmp/valkey; \
    cd deps; \
    make jemalloc lua hdr_histogram fpconv; \
    cd ../src; \
    make; \
    make install;


FROM registry.fedoraproject.org/fedora-minimal:39 as cli
COPY --from=builder /usr/local/bin/valkey-cli /usr/local/bin/valkey-cli

FROM registry.fedoraproject.org/fedora-minimal:39 as server
COPY --from=builder /usr/local/bin/valkey-server /usr/local/bin/valkey-server
COPY --from=builder /usr/local/bin/valkey-check-aof /usr/local/bin/valkey-check-aof
COPY --from=builder /usr/local/bin/valkey-check-rdb /usr/local/bin/valkey-check-rdb
COPY --from=builder /tmp/valkey/valkey.conf /etc/valkey.conf
CMD ["/usr/local/bin/valkey-server", "/etc/valkey.conf"]

@tobybellwood
Copy link

And why use Debian "by default", when other distributions are clearly suitable?

My 2c (as large-scale consumers of images ourselves and a re-packager of upstream images for our downstream consumers) is that adhering to the established practice is generally a smoother and more consistent experience for downstream users, who rely on timely upstream security updates, consistent platforms to extend, and stable inclusions. Debian and Alpine security best practices, disclosures, and transparency are A++ (in my opinion) and make my life far easier.

A quick (albeit rudimentary) search of the official docker-library images shows over 100 based on Debian, ~90 on Alpine, 10 on Ubuntu, and none on Fedora, which would certainly lean towards them should adoption as an official image be sought (which it probably should 😉!).

The projects whose images we support that have moved away from Debian/Alpine ((we've crossed paths with AL2, Oracle Linux, and UBI), are the ones that cause us the most lost time, usually require refactoring, and may have unpredictable security releases or unclear remediation pathways.

@metal3d
Copy link

metal3d commented Apr 7, 2024

On the whole, I'm against this view. Debian poses many problems that require constant tinkering with images. The "httpd" image, for example, requires configuration files to be moved from a standard path (/etc/httpd/conf.f) to a path that makes no sense (/etc/apache2 + "site-available" etc...).

Alpine is problematic because of its "libmuscl", which often forces you to find very complex compilation routes.

I don't force anyone to use Fedora, or Red Hat (but that's the distribution our customers ask us for, and we get turned down if the image is based on Ubuntu or Debian).

Just because the majority of images have been going down a sorry path for years, doesn't mean we have to constantly follow in their footsteps. I'm proposing that we have a Fedora-based image to fill a need. I'm willing to maintain this image if need be, but in an official way.

@metal3d
Copy link

metal3d commented Apr 7, 2024

An important point: Fedora is much less prone to security problems: http://www.diva-portal.se/smash/get/diva2:1212525/FULLTEXT01.pdf (p. 17)

I had to carry out security and adaptability (and performance) studies on images based on Debian, then recompiled under the same terms on RH and Fedora.

Debian is widespread, but I guarantee it's not as reliable as people want to make out.

Still, I think, and I'm far from being the only one, that offering images outside ".deb based" is a good idea. For reasons of flexibility and respect for good practice (particularly with regard to the labeling of mounted volumes), it's potentially worth looking into the matter. In short, I repeat, I'm capable of following valkey's startup principles from the others images and adapting the image accordingly, but I'll only do so if it's "visible" to the community.

Doing it in my corner, on an independent repository, has no scope. (And I'm not looking for fame, I'm looking to help).

@vincentbernat
Copy link

vincentbernat commented Apr 8, 2024 via email

@metal3d
Copy link

metal3d commented Apr 8, 2024

The document explains that this is not the number of CVE which is the problem but the number of days to react that is smaller.

One more time, I only explain that it is important and useful for many consumers to have several possibilities.

I'm not here to fight against distributions, I'm here to defend other distributions. I only reacted to a statement about security.

And to conclude, it's obvious that a distribution with fewer (often useless) packages makes it less prone to security flaws. Which supports my point...

I disagree. I'm not attacking anyone.

(I edited my comment because I badly understood the answer you made. Sorry)

@vincentbernat
Copy link

The document says "most in 5 days", "average in 90 days" for both Debian and Fedora. It also says that Debian has a lower median (49 vs 56). All that makes little sense for a base image which only includes essential packages.

The result showed that the most common number of days for
a CVE to be fixed was approximately 5 days on Debian.
The result also showed that all CVE fixes for Debian has an average number of days from
the published date to when the fix was found in the changelog of 90.63 days. The median of
all CVE fixes for Debian was 49 and the standard deviation was 131.05.

The result showed that the most common number of days for
a CVE to be fixed was approximately 5 days on Fedora.
The result also showed that all CVE fixes for Fedora has an average number of days from
the published date to when the fix was found in the changelog of 90.59 days. The median of
all CVE fixes for Fedora was 56 and the standard deviation was 116.53.

@roshkhatri
Copy link
Member

roshkhatri commented Apr 10, 2024

Hi, How about we support all? If so, how much overhead will we have to maintain these distributions?
If you one want to contribute to make something better, you are very much welcome. We do need more expertise on docker and containers, as I am also learning new things and might need help. I would be glad to work with you all personally.
Please feel free to open new issues on https://github.com/valkey-io/valkey-container/issues so we can discuss more topics in the community about improvements and new stuff.

We do have a new 7.2.4-rc1 version out and I did also push images on docker-hub https://hub.docker.com/r/valkey/valkey
I have opened some new issues to discuss topics from this one. Please add more if you think of something.

Publishing images on GHCR: valkey-io/valkey-container#9
Adding Support for Fedora-based images: valkey-io/valkey-container#10

I think this issue was related to adding Valkey docker images, and we can close this issue, unless we get any response.

@Maia-Everett
Copy link

We do have a new 7.2.4-rc1 version out and I did also push images on docker-hub https://hub.docker.com/r/valkey/valkey
I have opened some new issues to discuss topics from this one. Please add more if you think of something.

Thanks! I've tested it and it seems to work fine as a drop-in replacement for Redis in Docker Compose.

@madolson
Copy link
Member

I think we can close this now. I see that there are now followup issues for other distribution containers. Help is definitely appreciated to get all of those setup. Please 👍 the other issues if you would like to see them but don't know how to implement it.

@hairmare
Copy link

@carrodher you mentioned potentially providing a helm chart

might i ask you to follow up here? https://github.com/orgs/valkey-io/discussions/340

@carrodher
Copy link

Bitnami just released the container images, see the images in DockerHub

and the source code in GitHub

Any feedback is more than welcome, you can report any issue or suggestion at https://github.com/bitnami/containers/issues/new/choose

Now it's time for the Helm chart! 🚀

@kamulos
Copy link

kamulos commented May 8, 2024

@carrodher thank you! Will there be also the equivalent to the redis-cluster image in the future?

@carrodher
Copy link

@carrodher thank you! Will there be also the equivalent of the redis-cluster image in the future?

In the short term, our focus is to develop and then improve the regular Valkey Helm chart, then we'll listen to the feedback from the community and users and based on that will define the following actions; but yes, it is a possibility at some point.

@carrodher
Copy link

At bitnami/charts#25643 you can see the PR implementing the Valkey Helm chart

@carrodher
Copy link

I'm excited to share that we've just released the Bitnami Valkey Helm chart, now available on Docker Hub at https://hub.docker.com/r/bitnamicharts/valkey, simplifying the deployment of Valkey in Kubernetes.

Built upon the same principles as our other Bitnami charts, Valkey follows the latest best practices and security industry standards. Learn more about our commitment to security here.

We welcome your feedback and contributions on GitHub. Whether it's reporting issues, suggesting improvements, or contributing code, your input is invaluable so we can, together, continue to enhance the product.

@nellyk
Copy link

nellyk commented Aug 8, 2024

Hi @roshkhatri ,

Just checking to see if you applied for the Docker.io OSS program . We are currently getting rate limited, and I wanted to check if this is something in the pipeline.

Aside from that, we are looking at other options.

@roshkhatri
Copy link
Member

Thank you for letting me know . It is some thing in pipeline. We would also look at other options like GHCR.

@roshkhatri
Copy link
Member

Just a followup, @nellyk We have already applied for the OSS program. And we are looking at two options, GHCR and ECR.

@nellyk
Copy link

nellyk commented Aug 8, 2024

Thanks for the update @roshkhatri please let us know once its been added to the OSS program

@roshkhatri
Copy link
Member

Sounds good!

@roshkhatri
Copy link
Member

roshkhatri commented Sep 4, 2024

The Good News is here, We are now added to the OSS program at DockerHub and Valkey now has a Sponsored OSS badge.
Now Valkey on DockerHub will have access to unlimited pulls and unlimited egress

CC: @nellyk , @madolson

@nellyk
Copy link

nellyk commented Sep 5, 2024

This is great thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests