-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add hive 4.0 image #218
Add hive 4.0 image #218
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
version: '2.0' | ||
services: | ||
hiveserver2: | ||
hostname: hiveserver2 | ||
image: testing/hive4.0-hive:latest$ARCH | ||
environment: | ||
- SERVICE_NAME=hiveserver2 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
FROM apache/hive:4.0.0 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. does the base image include some arch-specific code? if not we can make this multi-arch by inlining the base image. I can take a look at this as follow-up. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hmmm, the hive image is actually multi-arch already There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nothing speaks against it but I am not sure on what major/minor releases do we test. |
||
|
||
# TODO replace with aws sdk v2 by following https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/aws_sdk_upgrade.html | ||
ARG AWS_JAVA_SDK_BUNDLE_VERSION=1.12.367 | ||
mayankvadariya marked this conversation as resolved.
Show resolved
Hide resolved
|
||
ARG HADOOP_AWS_VERSION=3.3.6 | ||
mayankvadariya marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
USER root | ||
RUN apt-get -y update | ||
RUN apt install curl -y | ||
|
||
mayankvadariya marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# Install AWS SDK so we can access S3; the version must match the hadoop-aws* jars which are part of SPARK distribution | ||
RUN mkdir -p /opt/hive/auxlib && \ | ||
curl -fLsS -o /opt/hive/auxlib/aws-java-sdk-bundle-$AWS_JAVA_SDK_BUNDLE_VERSION.jar https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/$AWS_JAVA_SDK_BUNDLE_VERSION/aws-java-sdk-bundle-$AWS_JAVA_SDK_BUNDLE_VERSION.jar && \ | ||
curl -fLsS -o /opt/hive/auxlib/hadoop-aws-$HADOOP_AWS_VERSION.jar https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/$HADOOP_AWS_VERSION/hadoop-aws-$HADOOP_AWS_VERSION.jar |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't aded
linux/arm64
platform as test image had been complaining about different archhttps://github.com/trinodb/docker-images/actions/runs/12242033317/job/34148454814?pr=218#step:5:837
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add a code comment I guess
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
on ci this is expected, what happens when you build the image locally? if it builds then you should allow all platforms here - CI will use emulation to build the arm version of the image but then devs who have arm machines will get arm images instead of having to emulate amd64 images.
on ci the node running the build is not an arm runner hence the warning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ya I realized about the runner spec. However, its just not about warning but image test is failing with arm. Please advise if there is a way to run only amd64 test and not arm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
feel free to merge without arm image.
The fix isn't to skip tests on arm but rather to find what is failing. The logs here are almost useless, I'll take a look at this and submit follow up PR.