diff --git a/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md b/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md index fa4ed86235cfe..4df2374cf4768 100644 --- a/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md +++ b/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md @@ -142,10 +142,8 @@ In pipeline execution modes where a Beam runner launches SDK workers in Docker c 1. Provide the container engine. Beam supports `local_docker`(requires local installation of Docker) and `cloud_build`(requires a GCP project with Cloud Build API enabled). --prebuild_sdk_container_engine= -2. To pass a base image for pre-building dependencies, provide `--sdk_container_image`. If not, Apache beam's base [image](https://hub.docker.com/search?q=apache%2Fbeam&type=image) would be used. - --sdk_container_image= -3. If using `local_docker` engine, provide a URL for the remote registry to which the image will be pushed by passing +2. If using `local_docker` engine, provide a URL for the remote registry to which the image will be pushed by passing --docker_registry_push_url= # Example: --docker_registry_push_url=/beam @@ -153,6 +151,11 @@ In pipeline execution modes where a Beam runner launches SDK workers in Docker c # tag is generated by Beam SDK. **NOTE:** `docker_registry_push_url` must be a remote registry. -> To use Docker, the `--sdk_container_image` should be compatible with Apache Beam Runner. Please follow the [instructions](https://beam.apache.org/documentation/runtime/environments/#building-and-pushing-custom-containers) on how to build a base container image compatible with Apache Beam. +> The pre-building feature requires the Apache Beam SDK for Python, version 2.25.0 or later. +The container images created during prebuilding will persist beyond the pipeline runtime. +Once your job is finished or stopped, you can remove the pre-built image from the container registry. + +>If your pipeline is using a custom container image, most likely you will not benefit from pre-building step as extra dependencies can be preinstalled in the custom image at build time. If you still would like to use pre-building with custom images, use Apache Beam SDK 2.38.0 or newer and + supply your custom image via `--sdk_container_image` pipeline option. **NOTE**: This feature is available only for the `Dataflow Runner v2`.