-
Notifications
You must be signed in to change notification settings - Fork 443
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Enable arm64 architecture support for katib images and fix grpc healt…
…h probe multiarch error. (#897) Change-Id: I5ddee7e8fbe96b8e0a025e3f182b4a5192c45597 Signed-off-by: Henry Wang <henry.wang@arm.com>
- Loading branch information
1 parent
d9bb39e
commit c18bab6
Showing
24 changed files
with
256 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
28 changes: 28 additions & 0 deletions
28
cmd/metricscollector/v1alpha3/tfevent-metricscollector/Dockerfile.aarch64
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
FROM ubuntu:18.04 | ||
|
||
RUN apt-get update \ | ||
&& apt-get -y install software-properties-common \ | ||
autoconf \ | ||
automake \ | ||
build-essential \ | ||
cmake \ | ||
pkg-config \ | ||
wget \ | ||
python-pip \ | ||
libhdf5-dev \ | ||
libhdf5-serial-dev \ | ||
hdf5-tools\ | ||
&& apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
RUN wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.11.0/tensorflow-1.11.0-cp27-none-linux_aarch64.whl \ | ||
&& pip install tensorflow-1.11.0-cp27-none-linux_aarch64.whl \ | ||
&& rm tensorflow-1.11.0-cp27-none-linux_aarch64.whl \ | ||
&& rm -rf .cache | ||
|
||
RUN pip install rfc3339 grpcio googleapis-common-protos jupyter | ||
ADD . /usr/src/app/github.com/kubeflow/katib | ||
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/metricscollector/v1alpha3/tfevent-metricscollector/ | ||
RUN pip install --no-cache-dir -r requirements.txt | ||
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/tfevent-metricscollector/:/usr/src/app/github.com/kubeflow/katib/pkg/metricscollector/v1alpha3/common/ | ||
ENTRYPOINT ["python", "main.py"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
FROM ubuntu:18.04 | ||
|
||
#install bazel | ||
RUN apt-get update \ | ||
&& apt-get install -y software-properties-common \ | ||
build-essential \ | ||
curl \ | ||
openjdk-8-jdk \ | ||
openjdk-8-jre-headless \ | ||
pkg-config \ | ||
zip \ | ||
g++ \ | ||
wget \ | ||
git \ | ||
zlib1g-dev \ | ||
unzip \ | ||
python3.6 \ | ||
python3-pip \ | ||
python3.6-dev \ | ||
python3-setuptools \ | ||
libhdf5-serial-dev \ | ||
libcurl3-dev \ | ||
libfreetype6-dev \ | ||
libzmq3-dev \ | ||
rsync \ | ||
&& apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
#set default python as python3 | ||
RUN ln -s /usr/bin/python3.6 /usr/bin/python && ln -s /usr/bin/pip3 /usr/bin/pip | ||
|
||
ENV BAZEL_VERSION=0.18.1 | ||
|
||
RUN mkdir -p /bazel \ | ||
&& cd /bazel \ | ||
&& curl -fSsL -O https://github.com/bazelbuild/bazel/releases/download/$BAZEL_VERSION/bazel-$BAZEL_VERSION-dist.zip \ | ||
&& unzip bazel-$BAZEL_VERSION-dist.zip \ | ||
&& ./compile.sh \ | ||
&& cp output/bazel /usr/local/bin \ | ||
&& rm -rf /bazel | ||
|
||
#build and install tensorflow | ||
RUN pip install -U cython six 'numpy==1.16.4' wheel mock 'future>=0.17.1' \ | ||
&& pip install keras_applications==1.0.6 --no-deps \ | ||
&& pip install keras_preprocessing==1.0.5 --no-deps \ | ||
&& pip install tensorflow_estimator --no-deps | ||
|
||
ENV TF_ROOT=/tensorflow | ||
ENV PYTHON_BIN_PATH=/usr/bin/python3 | ||
ENV PYTHON_LIB_PATH="$($PYTHON_BIN_PATH -c 'import site; print(site.getsitepackages()[0])')" | ||
ENV PYTHONPATH=${TF_ROOT}/lib | ||
ENV PYTHON_ARG=${TF_ROOT}/lib | ||
ENV TF_NEED_GCP=0 | ||
ENV TF_NEED_CUDA=0 | ||
ENV TF_NEED_HDFS=0 | ||
ENV TF_NEED_OPENCL=0 | ||
ENV TF_NEED_JEMALLOC=0 | ||
ENV TF_ENABLE_XLA=0 | ||
ENV TF_NEED_VERBS=0 | ||
ENV TF_NEED_MKL=0 | ||
ENV TF_DOWNLOAD_MKL=0 | ||
ENV TF_NEED_AWS=0 | ||
ENV TF_NEED_MPI=0 | ||
ENV TF_NEED_GDR=0 | ||
ENV TF_NEED_S3=0 | ||
ENV TF_NEED_OPENCL_SYCL=0 | ||
ENV TF_SET_ANDROID_WORKSPACE=0 | ||
ENV TF_NEED_COMPUTECPP=0 | ||
ENV CC_OPT_FLAGS="-march=native" | ||
ENV TF_SET_ANDROID_WORKSPACE=0 | ||
ENV TF_NEED_KAFKA=0 | ||
ENV TF_NEED_TENSORRT=0 | ||
ENV TF_NEED_AWS=0 | ||
|
||
RUN git clone https://github.com/tensorflow/tensorflow.git \ | ||
&& cd tensorflow \ | ||
&& git checkout v1.12.0 \ | ||
&& sed -i 's/ "-mfpu=neon",//' /tensorflow/tensorflow/contrib/lite/kernels/internal/BUILD \ | ||
&& sed -i '0,/conditions:default": \[\],/s//conditions:default": glob(\["aws-cpp-sdk-core\/source\/platform\/linux-shared\/*\.cpp",\]),/' /tensorflow/third_party/aws.BUILD \ | ||
&& ./configure \ | ||
&& bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package \ | ||
&& bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg \ | ||
&& pip install --upgrade --force-reinstall /tmp/tensorflow_pkg/tensorflow*.whl\ | ||
&& rm -rf /tensorflow \ | ||
&& rm -r /tmp/tensorflow_pkg/tensorflow*.whl | ||
|
||
ADD . /usr/src/app/github.com/kubeflow/katib | ||
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/suggestion/nasrl/v1alpha2 | ||
RUN pip install --no-cache-dir -r requirements.txt | ||
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha2/python | ||
|
||
ENTRYPOINT ["python", "-u", "main.py"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
FROM golang:alpine AS build-env | ||
# The GOPATH in the image is /go. | ||
ADD . /go/src/github.com/kubeflow/katib | ||
RUN if [ "$(uname -m)" = "ppc64le" ] || [ "$(uname -m)" = "aarch64" ]; then \ | ||
apk --update add git gcc musl-dev && \ | ||
go get github.com/grpc-ecosystem/grpc-health-probe && \ | ||
mv $GOPATH/bin/grpc-health-probe /bin/grpc_health_probe && \ | ||
chmod +x /bin/grpc_health_probe; \ | ||
else \ | ||
GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \ | ||
wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && \ | ||
chmod +x /bin/grpc_health_probe; \ | ||
fi | ||
|
||
FROM python:3.7-slim-buster | ||
|
||
RUN apt-get update \ | ||
&& apt-get -y install software-properties-common \ | ||
autoconf \ | ||
automake \ | ||
build-essential \ | ||
cmake \ | ||
libtool \ | ||
pkg-config \ | ||
wget \ | ||
gfortran \ | ||
libopenblas-dev \ | ||
liblapack-dev \ | ||
libhdf5-dev \ | ||
libhdf5-serial-dev \ | ||
hdf5-tools \ | ||
&& apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
RUN pip install cython numpy | ||
|
||
RUN wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.14.0-buster/tensorflow-1.14.0-cp37-none-linux_aarch64.whl \ | ||
&& pip install tensorflow-1.14.0-cp37-none-linux_aarch64.whl \ | ||
&& rm tensorflow-1.14.0-cp37-none-linux_aarch64.whl \ | ||
&& rm -rf .cache | ||
|
||
RUN pip install 'grpcio==1.23.0' 'protobuf==3.9.1' 'googleapis-common-protos==1.6.0' | ||
|
||
COPY --from=build-env /bin/grpc_health_probe /bin/ | ||
|
||
ADD . /usr/src/app/github.com/kubeflow/katib | ||
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/suggestion/nasrl/v1alpha3 | ||
|
||
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/health/python | ||
|
||
ENTRYPOINT ["python", "main.py"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
FROM ubuntu:18.04 | ||
|
||
RUN apt-get update \ | ||
&& apt-get -y install software-properties-common \ | ||
autoconf \ | ||
automake \ | ||
build-essential \ | ||
cmake \ | ||
pkg-config \ | ||
wget \ | ||
python-pip \ | ||
libhdf5-dev \ | ||
libhdf5-serial-dev \ | ||
hdf5-tools\ | ||
&& apt-get clean \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
RUN wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v1.11.0/tensorflow-1.11.0-cp27-none-linux_aarch64.whl \ | ||
&& pip install tensorflow-1.11.0-cp27-none-linux_aarch64.whl \ | ||
&& rm tensorflow-1.11.0-cp27-none-linux_aarch64.whl \ | ||
&& rm -rf .cache | ||
|
||
RUN pip install rfc3339 grpcio googleapis-common-protos jupyter | ||
ADD . /usr/src/app/github.com/kubeflow/katib | ||
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/tfevent-metricscollector/v1alpha2 | ||
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha2/python:/usr/src/app/github.com/kubeflow/katib/pkg/util/v1alpha2/tfevent-metricscollector | ||
|
||
CMD ["bash","-c","jupyter notebook --ip 0.0.0.0 --no-browser --allow-root"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.