feat: add vulkan images #22
Conversation
chat/vulkan/amd64/Containerfile
Outdated
| RUN dnf install -y python3-dnf-plugin-versionlock | ||
| RUN dnf install -y mesa-vulkan-drivers-23.3.3-1.el9.x86_64 | ||
| RUN dnf versionlock mesa-vulkan-drivers-23.3.3-1.el9.x86_64 | ||
| RUN dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm | ||
| RUN dnf install -y git cmake ninja-build gcc gcc-c++ | ||
| RUN dnf copr enable -y ligenix/enterprise-sandbox epel-9-x86_64 | ||
| RUN dnf install -y vulkan-headers vulkan-tools |
There was a problem hiding this comment.
as it looks like it's not a multi-stage build, you should mergre layers and cleanup .rpm files / rpm cache at the end to reduce the size of the image
or maybe better, use the default image and copy the custom rootfs so we've a scratch/smaller image (this is how ubi micro is done)
There was a problem hiding this comment.
I've been trying to copy what they did on ubi-micro but i cannot make it work with installroot. There is something else to set up but i was not able to find it.
I went to the good old way to check which folders to copy and i added the commands. If you know how to make the custom rootfs works, i'm all ears
It is primarily intended to be used on macOS with podman and krunkit but Sergio also said that it can be used on Linux with a VMM exposing a Venus-capable virtio-gpu. |
Did you check that performances with this images are the same compared to the benchmarks we did on MacOS/krunkit ? |
What do you mean? Didn't we use the image generated by the ai-lab-recipes team for the benchmark? I just copy/paste the containerfile so the result should be the same, no? |
Does not seem to me as the ai-lab-recipe one contains libkrun specific libraries: see https://github.com/containers/ai-lab-recipes/blob/624155c699b7c2abc3c5878fba21ad848b23af25/model_servers/llamacpp_python/vulkan/arm64/Containerfile#L6 |
.github/workflows/build-publish.yaml
Outdated
| build-image-name: "ai-lab-playground-chat" | ||
| archs: amd64, arm64 | ||
| - containerfile: "./chat/vulkan/amd64/Containerfile" | ||
| build-image-name: "ai-lab-playground-chat-vulkan-amd64" |
There was a problem hiding this comment.
The image name should not contain the arch so this should be handled through a manifest
There was a problem hiding this comment.
there are two versions of the same image. One for amd and one for arm. We should use the one we need based on the system
There was a problem hiding this comment.
Yes but we should select based on the arch not the name just like the base image (CPU)
There was a problem hiding this comment.
Removed the arch from the name. is that enough?
There was a problem hiding this comment.
Ops, updated the other file :face-palm:
|
.github/workflows/release.yaml
Outdated
| @@ -32,8 +32,13 @@ env: | |||
|
|
|||
| jobs: | |||
|
|
|||
| build: | |||
| tag: | |||
There was a problem hiding this comment.
Why do you need this extra job ?
There was a problem hiding this comment.
I thought that separating it in various jobs was more clean. Updated to merge all jobs into one as before
lstocchi
force-pushed
the
addVulkan
branch
3 times, most recently
from
e529d18 to
7d9e218
Compare
Signed-off-by: lstocchi <lstocchi@redhat.com>
Signed-off-by: lstocchi <lstocchi@redhat.com>
Signed-off-by: lstocchi <lstocchi@redhat.com>
Signed-off-by: lstocchi <lstocchi@redhat.com>
What does this PR do?
libkrun provider has been added to podman and soon we should have a way to start an inference server that is able to use gpu on a mac. This PR adds the creation of vulkan images so that, eventually, we can use them to start an inference server with gpu capabilities.
Screenshot / video of UI
N/A
What issues does this PR fix or reference?
N/A
How to test this PR?