[CI] Adding CPU docker pipeline #11261

zhouyuan · 2024-12-17T13:31:04Z

This patch adds the pipeline for CPU docker image. On each release(tag) it will build push the image to vllm/vllm-cpu
Fixes: #4771

github-actions · 2024-12-17T13:31:17Z

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

Add ready label to the PR
Enable auto-merge.

🚀

This patch adds the pipeline for CPU docker image Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

zhouyuan · 2024-12-18T01:02:04Z

.buildkite/release-pipeline.yaml

+    depends_on: ~
+    if: build.tag != null
+    agents:
+      queue: cpu_queue_postmerge


@khluu not sure if this cpu-queue-postmerge queue is right to build the publish docker image

We can use this .. but it should push to an ECR instead of Dockerhub. If the cadence is per release version, we can pull the release image manually from ECR then retag and push to Dockerhub cause it's safer that way (putting Dockerhub token on the CI machine has some risk)

@zhouyuan Can you add this command:
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7
then we can tag it like public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:$BUILDKITE_TAG then push

khluu · 2024-12-18T05:08:07Z

.buildkite/release-pipeline.yaml

+    depends_on: ~
+    if: build.tag != null
+    agents:
+      queue: cpu_queue_postmerge


We can use this .. but it should push to an ECR instead of Dockerhub. If the cadence is per release version, we can pull the release image manually from ECR then retag and push to Dockerhub cause it's safer that way (putting Dockerhub token on the CI machine has some risk)

khluu · 2024-12-18T05:08:59Z

.buildkite/release-pipeline.yaml

+    depends_on: ~
+    if: build.tag != null
+    agents:
+      queue: cpu_queue_postmerge


@zhouyuan Can you add this command:
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7
then we can tag it like public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:$BUILDKITE_TAG then push

khluu · 2024-12-18T05:09:53Z

.buildkite/release-pipeline.yaml

@@ -55,3 +55,18 @@ steps:
          password-env: DOCKERHUB_TOKEN
    env:
      DOCKER_BUILDKIT: "1"
+


Can we add a block step here, since it only needs to run on-demand (every release)? The syntax is similar to the Build release image step.

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

khluu · 2024-12-18T10:46:11Z

I think the PR is ready but the thing is I'm not sure if BUILDKITE_TAG works here. For example, @simon-mo tagged a commit earlier today to release and this build was triggered: https://buildkite.com/vllm/release/builds/2257 but it didn't register any tag (you can see in the Environment tab for each step).
Maybe we can just remove the if: build.tag ... line and have it use $RELEASE_VERSION as tag instead of $BUILDKITE_TAG. We can input $RELEASE_VERSION manually when triggering a new build on Release pipeline.

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

zhouyuan · 2024-12-19T01:36:55Z

@khluu
Hi Kevin, I have updated the code to use $RELEASE_VERSION. Could you please help to take a look? The CI failure seems not related with this patch(?)

I think the PR is ready but the thing is I'm not sure if BUILDKITE_TAG works here. For example, @simon-mo tagged a commit earlier today to release and this build was triggered: https://buildkite.com/vllm/release/builds/2257 but it didn't register any tag (you can see in the Environment tab for each step). Maybe we can just remove the if: build.tag ... line and have it use $RELEASE_VERSION as tag instead of $BUILDKITE_TAG. We can input $RELEASE_VERSION manually when triggering a new build on Release pipeline.

khluu · 2024-12-19T08:46:49Z

@khluu Hi Kevin, I have updated the code to use $RELEASE_VERSION. Could you please help to take a look? The CI failure seems not related with this patch(?)

I think the PR is ready but the thing is I'm not sure if BUILDKITE_TAG works here. For example, @simon-mo tagged a commit earlier today to release and this build was triggered: https://buildkite.com/vllm/release/builds/2257 but it didn't register any tag (you can see in the Environment tab for each step). Maybe we can just remove the if: build.tag ... line and have it use $RELEASE_VERSION as tag instead of $BUILDKITE_TAG. We can input $RELEASE_VERSION manually when triggering a new build on Release pipeline.

Yes let me test it on the Release pipeline then will merge!

DarkLight1337 · 2024-12-19T14:52:50Z

@khluu did you forget to approve this?

khluu · 2024-12-19T19:46:55Z

@khluu did you forget to approve this?

oops yes

zhouyuan · 2024-12-20T00:53:56Z

Hi @khluu @DarkLight1337 Thank you for the review and quick merge!

Thanks,
-yuan

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Signed-off-by: lucast2021 <lucast2021@headroyce.org>

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com>

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Signed-off-by: Bowen Wang <abmfy@icloud.com>

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com>

mergify bot added the ci/build label Dec 17, 2024

mgoin requested a review from simon-mo December 17, 2024 16:13

[CI] Adding CPU docker pipeline

2fad51d

This patch adds the pipeline for CPU docker image Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

zhouyuan force-pushed the wip_cpu_docker_pipeline branch from 8744af2 to 2fad51d Compare December 18, 2024 00:51

zhouyuan marked this pull request as ready for review December 18, 2024 00:53

simon-mo requested a review from khluu December 18, 2024 01:00

zhouyuan commented Dec 18, 2024

View reviewed changes

khluu reviewed Dec 18, 2024

View reviewed changes

zhouyuan and others added 2 commits December 18, 2024 18:21

adding block step for CPU docker build, modify to push to aws registry

3190285

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Update release-pipeline.yaml

015fab3

update to use RELEASE_VERSION as docker tag

5294be0

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

khluu added the ready ONLY add when PR is ready to merge/full CI is needed label Dec 19, 2024

khluu enabled auto-merge (squash) December 19, 2024 11:00

DarkLight1337 disabled auto-merge December 19, 2024 14:52

DarkLight1337 enabled auto-merge (squash) December 19, 2024 14:52

khluu approved these changes Dec 19, 2024

View reviewed changes

DarkLight1337 merged commit a985f7a into vllm-project:main Dec 19, 2024
44 of 46 checks passed

BKitor pushed a commit to BKitor/vllm that referenced this pull request Dec 30, 2024

[CI] Adding CPU docker pipeline (vllm-project#11261)

2e95b6d

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com>

joennlae pushed a commit to 44ai-labs/vllm that referenced this pull request Jan 19, 2025

[CI] Adding CPU docker pipeline (vllm-project#11261)

6d451d7

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com>

joennlae pushed a commit to 44ai-labs/vllm that referenced this pull request Jan 19, 2025

[CI] Adding CPU docker pipeline (vllm-project#11261)

ffff83f

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com>

abmfy pushed a commit to abmfy/vllm-flashinfer that referenced this pull request Jan 24, 2025

[CI] Adding CPU docker pipeline (vllm-project#11261)

44ddc09

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com>

This was referenced Jan 30, 2025

e2e CI Job kubernetes-sigs/gateway-api-inference-extension#259

Open

[Bugfix] Fix cpu image tagging #12615

Closed

VMinB12 mentioned this pull request Feb 3, 2025

[Feature]: Host CPU Docker image on Docker Hub #4771

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] Adding CPU docker pipeline #11261

[CI] Adding CPU docker pipeline #11261

zhouyuan commented Dec 17, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Dec 17, 2024

zhouyuan Dec 18, 2024

khluu Dec 18, 2024

khluu Dec 18, 2024

khluu Dec 18, 2024

khluu Dec 18, 2024

khluu Dec 18, 2024

khluu commented Dec 18, 2024

zhouyuan commented Dec 19, 2024

khluu commented Dec 19, 2024

DarkLight1337 commented Dec 19, 2024

khluu commented Dec 19, 2024

zhouyuan commented Dec 20, 2024

[CI] Adding CPU docker pipeline #11261

[CI] Adding CPU docker pipeline #11261

Conversation

zhouyuan commented Dec 17, 2024 • edited by github-actions bot Loading

github-actions bot commented Dec 17, 2024

zhouyuan Dec 18, 2024

Choose a reason for hiding this comment

khluu Dec 18, 2024

Choose a reason for hiding this comment

khluu Dec 18, 2024

Choose a reason for hiding this comment

khluu Dec 18, 2024

Choose a reason for hiding this comment

khluu Dec 18, 2024

Choose a reason for hiding this comment

khluu Dec 18, 2024

Choose a reason for hiding this comment

khluu commented Dec 18, 2024

zhouyuan commented Dec 19, 2024

khluu commented Dec 19, 2024

DarkLight1337 commented Dec 19, 2024

khluu commented Dec 19, 2024

zhouyuan commented Dec 20, 2024

zhouyuan commented Dec 17, 2024 •

edited by github-actions bot

Loading