Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add REPO_URL in Dockerfile to allow docker builds from contributor repos #3291

Merged
merged 4 commits into from
Aug 20, 2024

Conversation

mreso
Copy link
Collaborator

@mreso mreso commented Aug 20, 2024

Description

When an external contributor creates a PR his branch lives in his own serve fork but our CI tries to load the branch from pytorch/serve when building the docker images.
This PR introduces a new command line option for build_image.sh to set the repo url which we also set in the workflow file for the docker ci using github.repositoryUrl context variable.

#Risk analysis
External PR will still need to be acknowledged by a maintainer and PRs are already running outside docker, so there is no additional risk involved.

Fixes #(issue)

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A
./build_image.sh -t torchserve:1.0 -b feature/docker_url -repo https://github.com/mreso/serve
[+] Building 250.1s (28/28) FINISHED                                                                                                                                                                                                                                                                                                                       docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                 0.0s
 => => transferring dockerfile: 9.72kB                                                                                                                                                                                                                                                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                                                                                                      0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                                                                                                                                                                                                                0.5s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                                                                                                                                                                                                           0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                                                                                                                                                                                                                                      0.3s
 => [internal] load build context                                                                                                                                                                                                                                                                                                                                    0.6s
 => => transferring context: 2.40MB                                                                                                                                                                                                                                                                                                                                  0.6s
 => CACHED [production-image 1/9] FROM docker.io/library/ubuntu:20.04@sha256:fa17826afb526a9fc7250e0fbcbfd18d03fe7a54849472f86879d8bf562c629e                                                                                                                                                                                                                        0.0s
 => => resolve docker.io/library/ubuntu:20.04@sha256:fa17826afb526a9fc7250e0fbcbfd18d03fe7a54849472f86879d8bf562c629e                                                                                                                                                                                                                                                0.0s
 => [compile-image  2/11] RUN --mount=type=cache,id=apt-dev,target=/var/cache/apt     apt-get update &&     apt-get upgrade -y &&     apt-get install software-properties-common -y &&     add-apt-repository -y ppa:deadsnakes/ppa &&     apt remove python-pip  python3-pip &&     DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y      92.7s
 => [compile-image  3/11] RUN python3.9 -m venv /home/venv                                                                                                                                                                                                                                                                                                           2.8s
 => [compile-image  4/11] RUN python -m pip install -U pip setuptools                                                                                                                                                                                                                                                                                                3.3s
 => [compile-image  5/11] RUN export USE_CUDA=1                                                                                                                                                                                                                                                                                                                      0.3s
 => [compile-image  6/11] COPY ./ serve                                                                                                                                                                                                                                                                                                                             63.5s
 => [compile-image  7/11] RUN     if echo "false" | grep -q "false"; then         rm -rf serve;        git clone --recursive https://github.com/mreso/serve -b feature/docker_url;     fi                                                                                                                                                                           25.9s
 => [compile-image  8/11] WORKDIR serve                                                                                                                                                                                                                                                                                                                              0.0s
 => [compile-image  9/11] RUN cp docker/dockerd-entrypoint.sh /usr/local/bin/dockerd-entrypoint.sh                                                                                                                                                                                                                                                                   0.2s
 => [compile-image 10/11] RUN     if echo "ubuntu:20.04" | grep -q "cuda:"; then         if [ "" ]; then             python ./ts_scripts/install_dependencies.py --cuda ;        else             python ./ts_scripts/install_dependencies.py;        fi;     else         python ./ts_scripts/install_dependencies.py;     fi                                      41.5s
 => [compile-image 11/11] RUN     if echo "false" | grep -q "true"; then         python -m pip install -r requirements/developer.txt;        python ts_scripts/install_from_src.py;    elif echo "false" | grep -q "false"; then         python -m pip install --no-cache-dir torchserve torch-model-archiver torch-workflow-archiver;    else         python -m pi  1.6s
 => CACHED [production-image 2/9] RUN --mount=type=cache,target=/var/cache/apt     apt-get update &&     apt-get upgrade -y &&     apt-get install software-properties-common -y &&     add-apt-repository ppa:deadsnakes/ppa -y &&     apt remove python-pip  python3-pip &&     DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y     pyt  0.0s
 => CACHED [production-image 3/9] RUN useradd -m model-server     && mkdir -p /home/model-server/tmp                                                                                                                                                                                                                                                                 0.0s
 => [production-image 4/9] COPY --chown=model-server --from=compile-image /home/venv /home/venv                                                                                                                                                                                                                                                                      9.4s
 => [production-image 5/9] COPY --from=compile-image /usr/local/bin/dockerd-entrypoint.sh /usr/local/bin/dockerd-entrypoint.sh                                                                                                                                                                                                                                       0.0s
 => [production-image 6/9] RUN chmod +x /usr/local/bin/dockerd-entrypoint.sh     && chown -R model-server /home/model-server                                                                                                                                                                                                                                         0.2s
 => [production-image 7/9] COPY docker/config.properties /home/model-server/config.properties                                                                                                                                                                                                                                                                        0.0s
 => [production-image 8/9] RUN mkdir /home/model-server/model-store && chown -R model-server /home/model-server/model-store                                                                                                                                                                                                                                          0.3s
 => [production-image 9/9] WORKDIR /home/model-server                                                                                                                                                                                                                                                                                                                0.0s
 => exporting to image                                                                                                                                                                                                                                                                                                                                               4.3s
 => => exporting layers                                                                                                                                                                                                                                                                                                                                              4.2s
 => => writing image sha256:9fc95390ef86ecfe819c6aca0c8c9bf1a4ea69c21ca4af77abdf00667923a14d                                                                                                                                                                                                                                                                         0.0s
 => => naming to docker.io/library/torchserve:1.0

Checklist:

  • Did you have fun?
  • Have you made corresponding changes to the documentation?

@mreso mreso marked this pull request as ready for review August 20, 2024 03:59
@mreso mreso changed the title [WIP] Add REPO_URL in Dockerfile to allow docker builds from contributor repos Add REPO_URL in Dockerfile to allow docker builds from contributor repos Aug 20, 2024
Copy link
Collaborator

@agunapal agunapal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this.
LGTM.

Do you think this should be the full url or we can have just the username as the arg ex: mreso

@mreso
Copy link
Collaborator Author

mreso commented Aug 20, 2024

I think full url is good. That enables users to point to internal git repos too.

@mreso mreso added this pull request to the merge queue Aug 20, 2024
Merged via the queue into pytorch:master with commit aa9bb73 Aug 20, 2024
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants