From 3c756c4f3de780eb2ccbb01cd73f9bbf0bc34a46 Mon Sep 17 00:00:00 2001 From: Dongjoon Hyun Date: Sun, 25 Feb 2024 23:21:49 -0800 Subject: [PATCH] [SPARK-47160][K8S] Update K8s `Dockerfile` to include `hive-jackson` directory if exists ### What changes were proposed in this pull request? This PR aims to update K8s `Dockerfile` to include `hive-jackson` jar directory if exists. ### Why are the changes needed? After SPARK-47152, we can have `hive-jackson` directory. ### Does this PR introduce _any_ user-facing change? No, this is used by Spark internal by default. ### How was this patch tested? Pass the CIs and manual check. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45251 from dongjoon-hyun/SPARK-47160. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun --- .../kubernetes/docker/src/main/dockerfiles/spark/Dockerfile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile b/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile index c9c68fdafc08a..b580e70553371 100644 --- a/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile +++ b/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile @@ -52,6 +52,8 @@ RUN set -ex && \ rm -rf /var/cache/apt/* && rm -rf /var/lib/apt/lists/* COPY jars /opt/spark/jars +# Copy hive-jackson directory if exists +COPY hive-jackso[n] /opt/spark/hive-jackson # Copy RELEASE file if exists COPY RELEAS[E] /opt/spark/RELEASE COPY bin /opt/spark/bin