-
Notifications
You must be signed in to change notification settings - Fork 56
Building Couchbase
linuxonz edited this page Jan 6, 2025
·
52 revisions
The instructions provided below specify the steps to build Couchbase version 7.6.2 on Linux on IBM Z for following distributions:
- RHEL (8.8, 8.10, 9.2, 9.4)
- SLES (15 SP5, 15 SP6)
- Ubuntu (20.04, 22.04)
- When following the steps below please use a standard permission user unless otherwise specified.
- A directory
/<source_root>/will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it. - You can increase the sudo timeout by adding
Defaults:USER timestamp_timeout=TIME(replace USER and TIME) at the end of sudo file (sudo visudo).
If you want to build Couchbase using manual steps, go to STEP 2.
Use the following commands to build Couchbase using the build script. Please make sure you have wget installed.
wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Couchbase/7.6.2/build_couchbase.sh
# Build couchbase
bash build_couchbase.sh #[Provide -t option for executing build with tests]If the build completes successfully, go to STEP 5. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.
- defining deps and source root
export SOURCE_ROOT=<source root>
source "/etc/os-release"
export DISTRO=$ID$VERSION_ID
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Couchbase/7.6.2/patch"
export CACHE_DIRECTORY=~/.cbdepscache
mkdir $SOURCE_ROOT/patch
mkdir -p $CACHE_DIRECTORY- RHEL (8.8, 8.10)
sudo subscription-manager repos --enable codeready-builder-for-rhel-8-s390x-rpms
sudo subscription-manager repos --enable rhel-8-for-s390x-appstream-rpms
sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
sudo yum install -y gcc-toolset-11 gcc-toolset-11-gcc gcc-toolset-11-gcc-c++ gcc-toolset-11-libatomic-devel gcc-toolset-11-libstdc++-devel
sudo ln -sf /opt/rh/gcc-toolset-11/root/usr/bin/gcc /usr/local/bin/gcc
sudo ln -sf /opt/rh/gcc-toolset-11/root/usr/bin/g++ /usr/local/bin/g++
sudo yum install -y atk-devel autoconf automake binutils-devel bison bzip2 ccache \
cmake cups-devel flex git gnome-keyring libcurl-devel langpacks-en glibc-all-langpacks \
libev-devel libtool make ncurses-devel ninja-build openssl-devel perl openssl-perl libpsl-devel \
python2 python38 python38-devel python38-pip python3-httplib2 tar texinfo unzip wget which xz xz-devel \
glib2-devel clang diffutils procps asciidoctor
sudo ln -sf /usr/bin/python3 /usr/bin/python
sudo ln -sf /opt/rh/gcc-toolset-11/root/usr/bin/as /usr/bin/as
pip3 install httplib2 --user
export CC=/opt/rh/gcc-toolset-11/root/usr/bin/gcc
export CXX=/opt/rh/gcc-toolset-11/root/usr/bin/g++
export PATH=/opt/rh/gcc-toolset-11/root/usr/bin:$PATH
export LANG=en_US.UTF-8- RHEL (9.2, 9.4)
sudo subscription-manager repos --enable codeready-builder-for-rhel-9-s390x-rpms
sudo subscription-manager repos --enable rhel-9-for-s390x-appstream-rpms
sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
sudo yum install -y gcc gcc-c++ atk-devel autoconf automake binutils-devel bison bzip2 perl \
cmake cups-devel flex git gnome-keyring libcurl-devel langpacks-en glibc-all-langpacks ccache \
libtool make ncurses-devel ninja-build openssl-devel openssl-perl perl-core libpsl-devel \
python3 python3-devel tar texinfo unzip wget which xz xz-devel glib2-devel clang compat-openssl11 python3-httplib2 asciidoctor
sudo ln -sf /usr/bin/python3 /usr/bin/python
sudo ln -sf /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc
sudo ldconfig- SLES 15 (SP5, SP6)
sudo zypper install -y asciidoc autoconf automake cmake curl flex \
gcc gcc-c++ gcc10-c++ gcc10 git-core glib2 glib2-devel \
libopenssl-devel libncurses6 xz glibc-locale bzip2 ccache\
libtirpc-devel libtool libxml2-tools libxslt-tools binutils-devel\
make makedepend ncurses-devel ninja patch pkg-config libpsl-devel\
python311 python-xml python3-httplib2 re2c ruby sqlite3 tar \
unixODBC wget which xinetd xmlto python311-pip python311-devel
sudo ln -sf /usr/bin/python3.11 /usr/bin/python3
sudo ln -sf /usr/bin/python3 /usr/bin/python
pip3 install pyparsing
pip3 install httplib2 --user
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-10 10
sudo ln -sf /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export LANG=en_US.UTF-8- Ubuntu (20.04)
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive TZ=America/Toronto apt-get install -y \
autoconf automake autotools-dev binutils-dev bison ccache cmake curl flex \
git libssl-dev ncurses-dev ninja-build python3 locales locales-all libpsl-dev \
python3-httplib2 python3-six pkg-config re2c texinfo tzdata unzip wget \
g++-10 gcc-10 gcc-10-multilib g++-10-multilib libglib2.0-dev libtool
sudo ln -sf /usr/bin/python3 /usr/bin/python
export LANG=en_US.UTF-8
export CC=gcc
export CXX=g++
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10- Ubuntu (22.04)
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive TZ=America/Toronto apt-get install -y \
autoconf automake autotools-dev binutils-dev bison ccache cmake curl flex \
git libssl-dev ncurses-dev ninja-build python3 locales locales-all libpsl-dev \
python3-httplib2 python3-six pkg-config re2c texinfo tzdata unzip wget \
g++-11 gcc-11 gcc-11-multilib g++-11-multilib libglib2.0-dev libtool
sudo ln -sf /usr/bin/python3 /usr/bin/python
export LANG=en_US.UTF-8
export CC=gcc
export CXX=g++
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 11- Install Go 1.22.2
cd $SOURCE_ROOT
wget https://go.dev/dl/go1.22.2.linux-s390x.tar.gz
chmod ugo+r go1.22.2.linux-s390x.tar.gz
sudo tar -C /usr/local -xzf go1.22.2.linux-s390x.tar.gz
export PATH=/usr/local/go/bin:$PATH
export GOPATH=$(go env GOPATH)
go version- Install CMake 3.27.4
cd $SOURCE_ROOT
rm -rdf cmake-3.27.4
wget https://cmake.org/files/v3.27/cmake-3.27.4.tar.gz
tar -xzf cmake-3.27.4.tar.gz
cd cmake-3.27.4
./bootstrap --prefix=/usr
make -j$(nproc)
sudo make install
hash -r- Install gflags 2.2.2
cd $SOURCE_ROOT
version=2.2.2
wget https://github.com/gflags/gflags/archive/refs/tags/v$version.tar.gz
tar xzf v$version.tar.gz
cd gflags-$version
mkdir build && cd build
cmake -D BUILD_SHARED_LIBS=ON ..
make -j$(nproc)
sudo make install- Packaging benchmark v1.6.2
NAME_BENCHMARK=benchmark
VERSION_BENCHMARK=v1.6.2
BUILD_BENCHMARK=cb2
PACKAGE_BENCHMARK=$NAME_BENCHMARK-linux-s390x-$VERSION_BENCHMARK-$BUILD_BENCHMARK
cd $SOURCE_ROOT
git clone -b $VERSION_BENCHMARK --depth 1 https://github.com/couchbasedeps/benchmark.git
cd benchmark
cmake -D CMAKE_INSTALL_PREFIX=$(pwd)/_build -D CMAKE_BUILD_TYPE=RelWithDebInfo -D CMAKE_INSTALL_LIBDIR=lib -D CMAKE_CXX_STANDARD=17 -D CMAKE_CXX_STANDARD_REQUIRED=ON -D BUILD_SHARED_LIBS=OFF -D BENCHMARK_ENABLE_TESTING=OFF -D BENCHMARK_ENABLE_GTEST_TESTS=OFF -D BENCHMARK_ENABLE_INSTALL=ON -D BENCHMARK_DOWNLOAD_DEPENDENCIES=OFF
cmake --build . --target install
cmake -E remove_directory ./_build/lib/pkgconfig
cd _build
# Packaging build files
tar -czvf $PACKAGE_BENCHMARK.tgz *
MD5_BENCHMARK=($(md5sum $PACKAGE_BENCHMARK.tgz))
echo $MD5_BENCHMARK > $PACKAGE_BENCHMARK.md5
# Copy the package to destination
cp $PACKAGE_BENCHMARK.* $CACHE_DIRECTORY- Packaging boost 1.82.0
NAME_BOOST=boost
VERSION_BOOST=1.82.0
BUILD_BOOST=cb1
PACKAGE_BOOST=$NAME_BOOST-linux-s390x-$VERSION_BOOST-$BUILD_BOOST
sudo ldconfig /usr/local/lib64 /usr/local/lib
cd $SOURCE_ROOT
TOOLSET=gcc
URL=https://github.com/boostorg/boost/releases/download/boost-$VERSION_BOOST/boost-$VERSION_BOOST.tar.gz
curl -sSL $URL | tar xzf -
cd boost-1.82.0
./bootstrap.sh --with-libraries=context,chrono,date_time,filesystem,program_options,regex,system,thread --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib
options=(toolset=$TOOLSET variant=release link=static runtime-link=shared threading=multi --without-python address-model=64 cflags=-fno-omit-frame-pointer cxxflags=-fno-omit-frame-pointer cxxflags=-fPIC cxxflags=-std=c++17)
./b2 -j 4 ${options[@]} install
cd _build
# Packaging build files
tar -czvf $PACKAGE_BOOST.tgz *
MD5_BOOST=($(md5sum $PACKAGE_BOOST.tgz))
echo $MD5_BOOST > $PACKAGE_BOOST.md5
# Copy the package to destination
cp $PACKAGE_BOOST.* $CACHE_DIRECTORY- Packaging cbpy 3.11.8
NAME_CBPY=cbpy
VERSION_CBPY=3.11.8
BUILD_CBPY=2
PACKAGE_CBPY=$NAME_CBPY-linux-s390x-$VERSION_CBPY-$BUILD_CBPY
cd $SOURCE_ROOT
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source $HOME/.cargo/env
wget https://repo.anaconda.com/miniconda/Miniconda3-py311_23.5.0-3-Linux-s390x.sh
chmod a+x Miniconda3-py311_23.5.0-3-Linux-s390x.sh
./Miniconda3-py311_23.5.0-3-Linux-s390x.sh -b -p ./cbpy
cd cbpy
./bin/pip3 install msgpack-python
./bin/pip3 install natsort
./bin/pip3 install pem
./bin/pip3 install pycryptodome
./bin/pip3 install python-snappy
./bin/pip3 install requests
./bin/pip3 install cryptography==38.0.4
# Packaging build files
tar -czvf $PACKAGE_CBPY.tgz bin lib share ssl
MD5_CBPY=($(md5sum $PACKAGE_CBPY.tgz))
echo $MD5_CBPY > $PACKAGE_CBPY.md5
# Copy the package to destination
cp $PACKAGE_CBPY.* $CACHE_DIRECTORY- Packaging curl 8.6.0
NAME_CURL=curl
VERSION_CURL=8.6.0
BUILD_CURL=1
PACKAGE_CURL=$NAME_CURL-linux-s390x-$VERSION_CURL-$BUILD_CURL
cd $SOURCE_ROOT
sudo ldconfig /usr/local/lib64 /usr/local/lib
git clone -b curl-8_6_0 --depth 1 https://github.com/curl/curl.git
cd curl
cmake -G "Unix Makefiles" .
autoreconf -fi
./configure --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib --without-ssl
make -j$(nproc)
make install
hash -r
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY bin lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
echo "$VERSION_CURL-$BUILD_CURL" > VERSION.txt
# Packaging build files
tar -czvf $PACKAGE_CURL.tgz *
MD5_CURL=($(md5sum $PACKAGE_CURL.tgz))
echo $MD5_CURL > $PACKAGE_CURL.md5
# Copy the package to destination
cp $PACKAGE_CURL.* $CACHE_DIRECTORY- Packaging double-conversion 3.0.0
NAME_DCONVERTION=double-conversion
VERSION_DCONVERTION=3.0.0
BUILD_DCONVERTION=cb6
PACKAGE_DCONVERTION=$NAME_DCONVERTION-linux-s390x-$VERSION_DCONVERTION-$BUILD_DCONVERTION
cd $SOURCE_ROOT
git clone -b v$VERSION_DCONVERTION --depth 1 https://github.com/google/double-conversion.git
cd double-conversion
cmake -D BUILD_SHARED_LIBS=OFF -D BUILD_TESTING=OFF -D CMAKE_INSTALL_PREFIX=$(pwd)/_build -D CMAKE_INSTALL_LIBDIR=lib .
make -j$(nproc)
make install
cmake -E remove_directory $(pwd)/_build/lib/cmake
cd _build
# Packaging build files
tar -czvf $PACKAGE_DCONVERTION.tgz *
MD5_DCONVERTION=($(md5sum $PACKAGE_DCONVERTION.tgz))
echo $MD5_DCONVERTION > $PACKAGE_DCONVERTION.md5
# Copy the package to destination
cp $PACKAGE_DCONVERTION.* $CACHE_DIRECTORY- Packaging erlang 25.3
NAME_ERLANG=erlang
VERSION_ERLANG=25.3
BUILD_ERLANG=11
PACKAGE_ERLANG=$NAME_ERLANG-linux-s390x-$VERSION_ERLANG-$BUILD_ERLANG
cd $SOURCE_ROOT
git clone -b couchbase-$VERSION_ERLANG --depth 1 https://github.com/couchbasedeps/erlang.git
cd erlang
./configure --prefix=$(pwd)/_build \
--libdir=$(pwd)/_build/lib \
--enable-smp-support \
--disable-hipe \
--disable-fp-exceptions \
--without-javac \
--enable-m64-build \
CFLAGS="-fno-strict-aliasing -O3 -ggdb3"
make -j$(nproc)
make install
hash -r
cd _build
echo $VERSION_ERLANG > VERSION.txt
cat <<EOF > CMakeLists.txt
FILE (COPY bin lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EXECUTE_PROCESS (
COMMAND "\${CMAKE_INSTALL_PREFIX}/lib/erlang/Install"
-minimal "\${CMAKE_INSTALL_PREFIX}/lib/erlang"
)
EOF
# Packaging build files
tar -czvf $PACKAGE_ERLANG.tgz *
MD5_ERLANG=($(md5sum $PACKAGE_ERLANG.tgz))
echo $MD5_ERLANG > $PACKAGE_ERLANG.md5
# Copy the package to destination
cp $PACKAGE_ERLANG.* $CACHE_DIRECTORY- Packaging flatbuffers 1.10.0
NAME_FBUF=flatbuffers
VERSION_FBUF=v1.10.0
BUILD_FBUF=cb7
PACKAGE_FBUF=$NAME_FBUF-linux-s390x-$VERSION_FBUF-$BUILD_FBUF
cd $SOURCE_ROOT
git clone -b $VERSION_FBUF --depth 1 https://github.com/google/flatbuffers
cd flatbuffers
wget $PATCH_URL/flatbuffers.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/flatbuffers.diff
curl https://github.com/google/flatbuffers/commit/2e865f4d4e67a9b628c137aab7da8140dd9339a4.patch | git apply -
cmake -G "Unix Makefiles" -D CMAKE_INSTALL_PREFIX=$(pwd)/_build -D CMAKE_BUILD_TYPE=Release -D FLATBUFFERS_BUILD_TESTS=OFF -D CMAKE_INSTALL_LIBDIR=lib
make -j $(nproc)
make install
hash -r
cd _build
touch CMakeLists.txt
# Packaging build files
tar -czvf $PACKAGE_FBUF.tgz *
MD5_FBUF=($(md5sum $PACKAGE_FBUF.tgz))
echo $MD5_FBUF > $PACKAGE_FBUF.md5
# Copy the package to destination
cp $PACKAGE_FBUF.* $CACHE_DIRECTORY- Packaging fmt 8.1.1
NAME_FMT=fmt
VERSION_FMT=8.1.1
BUILD_FMT=cb4
PACKAGE_FMT=$NAME_FMT-linux-s390x-$VERSION_FMT-$BUILD_FMT
cd $SOURCE_ROOT
git clone -b $VERSION_FMT --depth 1 https://github.com/fmtlib/fmt.git
cd fmt
cmake -D CMAKE_INSTALL_PREFIX=$(pwd)/_build -D CMAKE_BUILD_TYPE=RelWithDebInfo -D CMAKE_CXX_VISIBILITY_PRESET=hidden -D CMAKE_POSITION_INDEPENDENT_CODE=ON -D CMAKE_INSTALL_LIBDIR=lib .
make -j$(nproc)
make install
cd _build
# Packaging build files
tar -czvf $PACKAGE_FMT.tgz *
MD5_FMT=($(md5sum $PACKAGE_FMT.tgz))
echo $MD5_FMT > $PACKAGE_FMT.md5
# Copy the package to destination
cp $PACKAGE_FMT.* $CACHE_DIRECTORY- Packaging snappy 1.1.10
NAME_SNAPPY=snappy
VERSION_SNAPPY=1.1.10
BUILD_SNAPPY=cb2
PACKAGE_SNAPPY=$NAME_SNAPPY-linux-s390x-$VERSION_SNAPPY-$BUILD_SNAPPY
cd $SOURCE_ROOT
git clone -b $VERSION_SNAPPY --depth 1 https://github.com/google/snappy.git
cd snappy
wget --no-check-certificate https://raw.githubusercontent.com/couchbase/tlm/7.6.2/deps/packages/snappy/snappy.patch
git apply snappy.patch
cmake -D CMAKE_INSTALL_PREFIX=$(pwd)/_build \
-D SNAPPY_BUILD_TESTS=OFF \
-D BUILD_SHARED_LIBS=ON \
-D CMAKE_INSTALL_LIBDIR=lib \
-D SNAPPY_BUILD_BENCHMARKS=OFF .
make -j$(nproc)
make install
hash -r
cd _build
cat <<EOF > CMakeLists.txt
file(GLOB snappy_libs lib/*snappy*)
file(COPY \${snappy_libs} DESTINATION "\${CMAKE_INSTALL_PREFIX}/lib")
EOF
# Packaging build files
tar -czvf $PACKAGE_SNAPPY.tgz *
MD5_SNAPPY=($(md5sum $PACKAGE_SNAPPY.tgz))
echo $MD5_SNAPPY > $PACKAGE_SNAPPY.md5
# Copy the package to destination
cp $PACKAGE_SNAPPY.* $CACHE_DIRECTORY- Packaging OpenSSL 3.1.4
NAME_OSSL=openssl
VERSION_OSSL=3.1.4
BUILD_OSSL=1
PACKAGE_OSSL=$NAME_OSSL-linux-s390x-$VERSION_OSSL-$BUILD_OSSL
cd $SOURCE_ROOT
wget https://github.com/openssl/openssl/releases/download/openssl-$VERSION_OSSL/openssl-$VERSION_OSSL.tar.gz
tar -xzf openssl-$VERSION_OSSL.tar.gz
cd openssl-$VERSION_OSSL
./config --prefix=$(pwd)/_build --openssldir=$(pwd)/_build/etc/openssl --libdir=$(pwd)/_build/lib
make -j$(nproc)
make install
hash -r
cd _build
echo "$VERSION_OSSL-$BUILD_OSSL" > VERSION.txt
cat <<EOF > CMakeLists.txt
FILE (COPY bin DESTINATION "\${CMAKE_INSTALL_PREFIX}")
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
FILE (COPY etc DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
# Packaging build files
tar -czvf $PACKAGE_OSSL.tgz *
MD5_OSSL=($(md5sum $PACKAGE_OSSL.tgz))
echo $MD5_OSSL > $PACKAGE_OSSL.md5
# Copy the package to destination
cp $PACKAGE_OSSL.* $CACHE_DIRECTORY- Packaging libevent 2.1.11
NAME_LIBEVENT=libevent
VERSION_LIBEVENT=2.1.11
BUILD_LIBEVENT=cb12
PACKAGE_LIBEVENT=$NAME_LIBEVENT-linux-s390x-$VERSION_LIBEVENT-$BUILD_LIBEVENT
OPENSSL_BUILD=$SOURCE_ROOT/openssl-3.1.4/_build
cd $SOURCE_ROOT
git clone -b release-$VERSION_LIBEVENT-stable --depth 1 https://github.com/libevent/libevent.git
cd libevent
cmake -D CMAKE_INSTALL_PREFIX=`pwd`/_build \
-D CMAKE_INSTALL_LIBDIR=lib \
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D EVENT__DISABLE_BENCHMARK=ON \
-D EVENT__DISABLE_REGRESS=ON \
-D EVENT__DISABLE_SAMPLES=ON \
-D OPENSSL_ROOT_DIR=$OPENSSL_BUILD .
make -j$(nproc)
make install
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION \${CMAKE_INSTALL_PREFIX})
EOF
# Packaging build files
tar -czvf $PACKAGE_LIBEVENT.tgz *
MD5_LIBEVENT=($(md5sum $PACKAGE_LIBEVENT.tgz))
echo $MD5_LIBEVENT > $PACKAGE_LIBEVENT.md5
# Copy the package to destination
cp $PACKAGE_LIBEVENT.* $CACHE_DIRECTORY- Packaging glog 0.4.0
NAME_GLOG=glog
VERSION_GLOG=v0.4.0
BUILD_GLOG=cb3
PACKAGE_GLOG=$NAME_GLOG-linux-s390x-$VERSION_GLOG-$BUILD_GLOG
cd $SOURCE_ROOT
git clone -b $VERSION_GLOG --depth 1 https://github.com/google/glog.git
cd glog
cmake -S . -B build -G "Unix Makefiles" -D BUILD_SHARED_LIBS=OFF -D CMAKE_INSTALL_PREFIX=$(pwd)/_build -D CMAKE_BUILD_TYPE=RelWithDebInfo -D CMAKE_INSTALL_LIBDIR=lib
cmake --build build --target install
cd _build
# Packaging build files
tar -czvf $PACKAGE_GLOG.tgz *
MD5_GLOG=($(md5sum $PACKAGE_GLOG.tgz))
echo $MD5_GLOG > $PACKAGE_GLOG.md5
# Copy the package to destination
cp $PACKAGE_GLOG.* $CACHE_DIRECTORY- Packaging zstd 1.5.0
NAME_ZSTD=zstd-cpp
VERSION_ZSTD=1.5.0
BUILD_ZSTD=4
PACKAGE_ZSTD=$NAME_ZSTD-linux-s390x-$VERSION_ZSTD-$BUILD_ZSTD
cd $SOURCE_ROOT
wget https://github.com/facebook/zstd/archive/refs/tags/v$VERSION_ZSTD.tar.gz
tar -xzvf v$VERSION_ZSTD.tar.gz
cd zstd-$VERSION_ZSTD
make DESTDIR=$(pwd)/_build install
cd _build
cp -r ./usr/local/* . && rm -drf usr bin share
echo "${VERSION_ZSTD}-4" >VERSION.txt
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EOF
# Packaging build files
tar -czvf $PACKAGE_ZSTD.tgz *
MD5_ZSTD=($(md5sum $PACKAGE_ZSTD.tgz))
echo $MD5_ZSTD > $PACKAGE_ZSTD.md5
# Copy the package to destination
cp $PACKAGE_ZSTD.* $CACHE_DIRECTORY- Packaging Jemalloc 5.3.0
NAME_JEMALLOC=jemalloc
VERSION_JEMALLOC=5.3.0
BUILD_JEMALLOC=13
PACKAGE_JEMALLOC=$NAME_JEMALLOC-linux-s390x-$VERSION_JEMALLOC-$BUILD_JEMALLOC
cd $SOURCE_ROOT
git clone -b $VERSION_JEMALLOC --depth 1 https://github.com/couchbasedeps/jemalloc.git
cd jemalloc
configure_args="--prefix=$(pwd)/_build \
--with-jemalloc-prefix=je_ \
--disable-cache-oblivious \
--disable-zone-allocator \
--disable-initial-exec-tls \
--disable-cxx \
--enable-prof \
--libdir=$(pwd)/_build/lib"
./autogen.sh ${configure_args}
make -j8 build_lib_shared
make -j8 check
make install_lib_shared install_include install_bin
configure_args="--prefix=$(pwd)/_build \
--with-jemalloc-prefix=je_ \
--enable-debug \
--disable-cache-oblivious \
--disable-zone-allocator \
--disable-initial-exec-tls \
--disable-cxx \
--enable-prof \
--with-install-suffix=d \
--libdir=$(pwd)/_build/lib"
CFLAGS=-Og ./autogen.sh ${configure_args}
make -j8 build_lib_shared
make -j8 check
make install_lib_shared
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY bin/jeprof DESTINATION "\${CMAKE_INSTALL_PREFIX}/bin")
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
mkdir -p cmake && cd cmake
wget https://raw.githubusercontent.com/couchbase/build-tools/master/cbdeps/jemalloc/package/cmake/JemallocConfig.cmake
cd $SOURCE_ROOT/jemalloc/_build
# Packaging build files
tar -czvf $PACKAGE_JEMALLOC.tgz *
MD5_JEMALLOC=($(md5sum $PACKAGE_JEMALLOC.tgz))
echo $MD5_JEMALLOC > $PACKAGE_JEMALLOC.md5
# Copy the package to destination
cp $PACKAGE_JEMALLOC.* $CACHE_DIRECTORY- Packaging folly v2022.05.23.00-couchbase
NAME_FOLLY=folly
VERSION_FOLLY=v2022.05.23.00-couchbase
BUILD_FOLLY=cb15
PACKAGE_FOLLY=$NAME_FOLLY-linux-s390x-$VERSION_FOLLY-$BUILD_FOLLY
JEMALLOC_BUILD=$SOURCE_ROOT/jemalloc/_build
FMT_BUILD=$SOURCE_ROOT/fmt/_build
GLOG_BUILD=$SOURCE_ROOT/glog/_build
BOOST_BUILD=$SOURCE_ROOT/boost-1.82.0/_build
DOUBLE_CONVERSION_BUILD=$SOURCE_ROOT/double-conversion/_build
LIBEVENT_BUILD=$SOURCE_ROOT/libevent/_build
OPENSSL_BUILD=$SOURCE_ROOT/openssl-3.1.4/_build
SNAPPY_BUILD=$SOURCE_ROOT/snappy/_build
ZSTD_BUILD=$SOURCE_ROOT/zstd-1.5.0/_build
LIBRARIES="$JEMALLOC_BUILD/lib;$FMT_BUILD/lib;$GLOG_BUILD/lib;$BOOST_BUILD/lib;$DOUBLE_CONVERSION_BUILD/lib;$LIBEVENT_BUILD/lib;$OPENSSL_BUILD/lib;$SNAPPY_BUILD/lib;$ZSTD_BUILD/lib;"
INCLUDES="$JEMALLOC_BUILD/include;$FMT_BUILD/include;$GLOG_BUILD/include;$BOOST_BUILD/include;$DOUBLE_CONVERSION_BUILD/include;$LIBEVENT_BUILD/include;$OPENSSL_BUILD/include;$SNAPPY_BUILD/include;$ZSTD_BUILD/include"
cd $SOURCE_ROOT
git clone -b $VERSION_FOLLY --depth 1 https://github.com/couchbasedeps/folly.git
cd folly
wget $PATCH_URL/folly.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/folly.diff
sudo ldconfig /usr/local/lib64 /usr/local/lib
cmake \
-D CMAKE_INCLUDE_PATH=$INCLUDES \
-D CMAKE_LIBRARY_PATH=$LIBRARIES \
-D CMAKE_CXX_FLAGS="-fPIC -fvisibility=hidden" \
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D CMAKE_INSTALL_PREFIX=$(pwd)/_build \
-D BUILD_SHARED_LIBS:STRING=OFF \
-D Boost_INCLUDE_DIR=$BOOST_BUILD/include \
-D Boost_ADDITIONAL_VERSIONS=1.82 \
-D Boost_USE_STATIC_LIBS=ON \
-D Boost_NO_SYSTEM_PATHS=ON \
-D Boost_NO_BOOST_CMAKE=ON \
-D BOOST_ROOT=$BOOST_BUILD \
-D CMAKE_PREFIX_PATH=$FMT_BUILD \
-D CMAKE_DISABLE_FIND_PACKAGE_ZLIB=TRUE .
make -j$(nproc)
make install
cd _build
echo "${VERSION_FOLLY}-${BUILD_FOLLY}" >VERSION.txt
# Packaging build files
tar -czvf $PACKAGE_FOLLY.tgz *
MD5_FOLLY=($(md5sum $PACKAGE_FOLLY.tgz))
echo $MD5_FOLLY > $PACKAGE_FOLLY.md5
# Copy the package to destination
cp $PACKAGE_FOLLY.* $CACHE_DIRECTORY- Packaging googletest 1.14.0
NAME_GTEST=googletest
VERSION_GTEST=1.14.0
BUILD_GTEST=cb1
PACKAGE_GTEST=$NAME_GTEST-linux-s390x-$VERSION_GTEST-$BUILD_GTEST
cd $SOURCE_ROOT
git clone -b v$VERSION_GTEST --depth 1 https://github.com/google/googletest.git
cd googletest
cmake -D CMAKE_INSTALL_PREFIX=$(pwd)/_build \
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D CMAKE_INSTALL_LIBDIR=lib \
-D CMAKE_CXX_STANDARD=17 \
-D CMAKE_CXX_STANDARD_REQUIRED=ON \
-D BUILD_SHARED_LIBS=OFF \
-D gtest_force_shared_crt=ON .
make -j$(nproc)
make install
cd _build
rm -drf ./lib/pkgconfig
# Packaging build files
tar -czvf $PACKAGE_GTEST.tgz *
MD5_GTEST=($(md5sum $PACKAGE_GTEST.tgz))
echo $MD5_GTEST > $PACKAGE_GTEST.md5
# Copy the package to destination
cp $PACKAGE_GTEST.* $CACHE_DIRECTORY- Packaging zlib 1.2.13
NAME_ZLIB=zlib
VERSION_ZLIB=1.2.13
BUILD_ZLIB=2
PACKAGE_ZLIB=$NAME_ZLIB-linux-s390x-$VERSION_ZLIB-$BUILD_ZLIB
cd $SOURCE_ROOT
git clone -b v$VERSION_ZLIB --depth 1 https://github.com/madler/zlib.git
cd zlib
./configure --prefix=$(pwd)/_build --64
make -j$(nproc)
make install
cd _build
echo "${VERSION_ZLIB}-2" >VERSION.txt
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
# Packaging build files
tar -czvf $PACKAGE_ZLIB.tgz *
MD5_ZLIB=($(md5sum $PACKAGE_ZLIB.tgz))
echo $MD5_ZLIB > $PACKAGE_ZLIB.md5
# Copy the package to destination
cp $PACKAGE_ZLIB.* $CACHE_DIRECTORY- Packaging GRPC 1.59.3
NAME_GRPC=grpc
VERSION_GRPC=1.59.3
BUILD_GRPC=cb1
PACKAGE_GRPC=$NAME_GRPC-linux-s390x-$VERSION_GRPC-$BUILD_GRPC
cd $SOURCE_ROOT
git clone -b v$VERSION_GRPC https://github.com/grpc/grpc
cd grpc/
git submodule update --init --recursive
if [ $ID$VERSION_ID == "ubuntu22.04" ] || [ $ID$VERSION_ID == "rhel9.2" ] || [ $ID$VERSION_ID == "rhel9.4" ]; then
sed -i '59i #undef SIGSTKSZ' third_party/abseil-cpp/absl/debugging/failure_signal_handler.cc
sed -i '60i #define SIGSTKSZ 16384' third_party/abseil-cpp/absl/debugging/failure_signal_handler.cc
fi
if [ $ID$VERSION_ID == "rhel9.2" ] || [ $ID$VERSION_ID == "rhel9.4" ]; then
sed -i '20i #include <limits>' third_party/abseil-cpp/absl/flags/usage_config.cc
fi
BUILD_DIR=$(pwd)/_build
# Point to zlib build file.
ZLIB_BUILD_DIR=$SOURCE_ROOT/zlib/_build
# Point to OpenSSL build file.
OPENSSL_BUILD_DIR=$SOURCE_ROOT/openssl-3.1.4/_build
(
cd third_party/abseil-cpp
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="$BUILD_DIR" .
make -j$(nproc) install
)
(
cd third_party/protobuf
CXXFLAGS="-fcommon" cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="$BUILD_DIR" \
-Dprotobuf_BUILD_TESTS=OFF \
-Dprotobuf_ABSL_PROVIDER=package \
-DCMAKE_PREFIX_PATH="$BUILD_DIR" \
-D CMAKE_PREFIX_PATH="$ZLIB_BUILD_DIR" .
make -j$(nproc) install
)
(
cd third_party/cares/cares
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="$BUILD_DIR" \
-DCARES_STATIC=ON -DCARES_STATIC_PIC=ON -DCARES_SHARED=OFF .
make -j$(nproc) install
)
(
cd third_party/re2
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INSTALL_PREFIX="$BUILD_DIR" .
make -j$(nproc) install
)
cmake -D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D CMAKE_INSTALL_PREFIX=$BUILD_DIR \
-D CMAKE_PREFIX_PATH="$ZLIB_BUILD_DIR;$OPENSSL_BUILD_DIR;$BUILD_DIR" \
-DgRPC_INSTALL=ON \
-DgRPC_BUILD_TESTS=OFF \
-DgRPC_ABSL_PROVIDER=package \
-DgRPC_PROTOBUF_PROVIDER=package \
-DgRPC_ZLIB_PROVIDER=package \
-DgRPC_CARES_PROVIDER=package \
-DgRPC_RE2_PROVIDER=package \
-DgRPC_SSL_PROVIDER=package \
-D CMAKE_INSTALL_LIBDIR=lib .
make -j$(nproc) install
cd _build
# Packaging build files
tar -czvf $PACKAGE_GRPC.tgz *
MD5_GRPC=($(md5sum $PACKAGE_GRPC.tgz))
echo $MD5_GRPC > $PACKAGE_GRPC.md5
# Copy the package to destination
cp $PACKAGE_GRPC.* $CACHE_DIRECTORY- Packaging Jemalloc_noprefix 5.2.1
NAME_JEMALLOC=jemalloc_noprefix
VERSION_JEMALLOC=5.2.1
BUILD_JEMALLOC=11
PACKAGE_JEMALLOC=$NAME_JEMALLOC-linux-s390x-$VERSION_JEMALLOC-$BUILD_JEMALLOC
cd $SOURCE_ROOT
git clone -b $VERSION_JEMALLOC --depth 1 https://github.com/couchbasedeps/jemalloc.git jemalloc_noprefix
cd jemalloc_noprefix
configure_args="--prefix=$(pwd)/_build \
--with-jemalloc-prefix= \
--with-install-suffix=_noprefix \
--enable-prof \
--libdir=$(pwd)/_build/lib"
./autogen.sh ${configure_args}
make -j8 build_lib_shared
make -j8 check
make install_lib_shared install_include install_bin
configure_args="--prefix=$(pwd)/_build \
--with-jemalloc-prefix= \
--enable-debug \
--with-install-suffix=_noprefix \
--enable-prof \
--with-install-suffix=_noprefixd \
--libdir=$(pwd)/_build/lib"
CFLAGS=-Og ./autogen.sh ${configure_args}
make -j8 build_lib_shared
make -j8 check
make install_lib_shared
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY bin/jeprof DESTINATION "\${CMAKE_INSTALL_PREFIX}/bin")
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
mkdir -p cmake && cd cmake
wget https://raw.githubusercontent.com/couchbase/build-tools/master/cbdeps/jemalloc_noprefix/package/cmake/Jemalloc_NoprefixConfig.cmake
cd $SOURCE_ROOT/jemalloc_noprefix/_build
# Packaging build files
tar -czvf $PACKAGE_JEMALLOC.tgz *
MD5_JEMALLOC=($(md5sum $PACKAGE_JEMALLOC.tgz))
echo $MD5_JEMALLOC > $PACKAGE_JEMALLOC.md5
# Copy the package to destination
cp $PACKAGE_JEMALLOC.* $CACHE_DIRECTORY- Packaging JSON 3.11.2
NAME_JSON=json
VERSION_JSON=3.11.2
BUILD_JSON=cb1
PACKAGE_JSON=$NAME_JSON-linux-s390x-$VERSION_JSON-$BUILD_JSON
cd $SOURCE_ROOT
git clone -b v$VERSION_JSON --depth 1 https://github.com/nlohmann/json
cd $SOURCE_ROOT/json
cmake -D CMAKE_INSTALL_PREFIX=$(pwd)/_build -D CMAKE_BUILD_TYPE=RelWithDebInfo -D JSON_BuildTests=OFF -D JSON_Install=ON -D JSON_MultipleHeaders=ON -D JSON_SystemInclude=ON -D CMAKE_INSTALL_LIBDIR=lib
cmake --build . --target install
cd _build
# Packaging build files
tar -czvf $PACKAGE_JSON.tgz *
MD5_JSON=($(md5sum $PACKAGE_JSON.tgz))
echo $MD5_JSON > $PACKAGE_JSON.md5
# Copy the package to destination
cp $PACKAGE_JSON.* $CACHE_DIRECTORY- Packaging liburing 0.6
NAME_LIBURING=liburing
VERSION_LIBURING=0.6
BUILD_LIBURING=3
PACKAGE_LIBURING=$NAME_LIBURING-linux-s390x-$VERSION_LIBURING-$BUILD_LIBURING
cd $SOURCE_ROOT
git clone https://github.com/axboe/liburing.git
cd liburing
git checkout liburing-$VERSION_LIBURING
if [[ $DISTRO != rhel8* ]]; then
git cherry-pick 8aac320ae8445c5434ab3be1761414a5247e5d42
fi
sed -i 's/|\sMAP_HUGE_2MB\s//g' ./test/io_uring_register.c
./configure --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib
make -j$(nproc)
make install
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EOF
# Packaging build files
tar -czvf $PACKAGE_LIBURING.tgz *
MD5_LIBURING=($(md5sum $PACKAGE_LIBURING.tgz))
echo $MD5_LIBURING > $PACKAGE_LIBURING.md5
# Copy the package to destination
cp $PACKAGE_LIBURING.* $CACHE_DIRECTORY- Packaging libuv 1.20.3
NAME_LIBUV=libuv
VERSION_LIBUV=1.20.3
BUILD_LIBUV=23
PACKAGE_LIBUV=$NAME_LIBUV-linux-s390x-$VERSION_LIBUV-$BUILD_LIBUV
cd $SOURCE_ROOT
git clone -b v$VERSION_LIBUV --depth 1 https://github.com/couchbasedeps/libuv
cd libuv
./autogen.sh
./configure --disable-silent-rules --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib
make -j$(nproc)
make install
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EOF
# Packaging build files
tar -czvf $PACKAGE_LIBUV.tgz *
MD5_LIBUV=($(md5sum $PACKAGE_LIBUV.tgz))
echo $MD5_LIBUV > $PACKAGE_LIBUV.md5
# Copy the package to destination
cp $PACKAGE_LIBUV.* $CACHE_DIRECTORY- Packaging lz4 1.9.2
NAME_LZ4=lz4
VERSION_LZ4=1.9.2
BUILD_LZ4=cb5
PACKAGE_LZ4=$NAME_LZ4-linux-s390x-$VERSION_LZ4-$BUILD_LZ4
cd $SOURCE_ROOT
wget https://github.com/lz4/lz4/archive/refs/tags/v$VERSION_LZ4.tar.gz
tar -xzf v$VERSION_LZ4.tar.gz
cd lz4-$VERSION_LZ4
make -j$(nproc)
make DESTDIR=$(pwd)/_build install
hash -r
cd _build
cp -r ./usr/local/* . && rm -drf usr
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EOF
# Packaging build files
tar -czvf $PACKAGE_LZ4.tgz *
MD5_LZ4=($(md5sum $PACKAGE_LZ4.tgz))
echo $MD5_LZ4 > $PACKAGE_LZ4.md5
# Copy the package to destination
cp $PACKAGE_LZ4.* $CACHE_DIRECTORY- Packaging numactl 2.0.14
NAME_NUMCTL=numactl
VERSION_NUMCTL=2.0.11
BUILD_NUMCTL=cb4
PACKAGE_NUMCTL=$NAME_NUMCTL-linux-s390x-$VERSION_NUMCTL-$BUILD_NUMCTL
cd $SOURCE_ROOT
# v2.0.11 doesn't work on some distros
git clone -b v2.0.14 --depth 1 https://github.com/numactl/numactl.git
cd numactl
./autogen.sh
./configure --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib
make -j$(nproc)
make install
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
rm -dfr share bin
# Packaging build files
tar -czvf $PACKAGE_NUMCTL.tgz *
MD5_NUMCTL=($(md5sum $PACKAGE_NUMCTL.tgz))
echo $MD5_NUMCTL > $PACKAGE_NUMCTL.md5
# Copy the package to destination
cp $PACKAGE_NUMCTL.* $CACHE_DIRECTORY- Packaging pcre 8.44
NAME_PCRE=pcre
VERSION_PCRE=8.44
BUILD_PCRE=cb3
PACKAGE_PCRE=$NAME_PCRE-linux-s390x-$VERSION_PCRE-$BUILD_PCRE
cd $SOURCE_ROOT
wget https://sourceforge.net/projects/pcre/files/pcre/$VERSION_PCRE/pcre-$VERSION_PCRE.tar.gz
tar -xzf pcre-$VERSION_PCRE.tar.gz
cd pcre-$VERSION_PCRE
./configure --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib
make -j$(nproc)
make install
cd _build
cat <<EOF > CMakeLists.txt
FILE (COPY bin lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
# Packaging build files
tar -czvf $PACKAGE_PCRE.tgz *
MD5_PCRE=($(md5sum $PACKAGE_PCRE.tgz))
echo $MD5_PCRE > $PACKAGE_PCRE.md5
# Copy the package to destination
cp $PACKAGE_PCRE.* $CACHE_DIRECTORY- Packaging prometheus 2.45.0
cd $SOURCE_ROOT
NAME_PROM=prometheus
VERSION_PROM=2.45.0
BUILD_PROM=6
PACKAGE_PROM=$NAME_PROM-linux-s390x-$VERSION_PROM-$BUILD_PROM
cd $SOURCE_ROOT
wget https://nodejs.org/dist/v20.15.0/node-v20.15.0-linux-s390x.tar.xz
tar xf node-v20.15.0-linux-s390x.tar.xz
export PATH=$SOURCE_ROOT/node-v20.15.0-linux-s390x/bin:$PATH
npm install -g yarn
mkdir -p $GOPATH/src/github.com
cd $GOPATH/src/github.com
git clone -b v$VERSION_PROM --depth 1 https://github.com/couchbasedeps/prometheus.git
cd prometheus
make build
mkdir -p _build/bin && cp prometheus _build/bin && cd _build
echo "${VERSION_PROM}-${BUILD_PROM}" >VERSION.txt
cat <<EOF > CMakeLists.txt
FILE (COPY bin DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EOF
# Packaging build files
tar -czvf $PACKAGE_PROM.tgz *
MD5_PROM=($(md5sum $PACKAGE_PROM.tgz))
echo $MD5_PROM > $PACKAGE_PROM.md5
# Copy the package to destination
cp $PACKAGE_PROM.* $CACHE_DIRECTORY- Packaging prometheus-cpp v1.2.1
NAME_PROMCPP=prometheus-cpp
VERSION_PROMCPP=v1.2.1-couchbase
BUILD_PROMCPP=cb1
PACKAGE_PROMCPP=$NAME_PROMCPP-linux-s390x-$VERSION_PROMCPP-$BUILD_PROMCPP
# Point to zlib build file.
ZLIB_BUILD_DIR=$SOURCE_ROOT/zlib/_build
cd $SOURCE_ROOT
git clone -b $VERSION_PROMCPP --depth 1 https://github.com/couchbasedeps/prometheus-cpp.git
cd prometheus-cpp
git submodule init
git submodule update
mkdir _build
cd _build
cmake -DBUILD_SHARED_LIBS=OFF \
-D ENABLE_PUSH=OFF \
-D CMAKE_POSITION_INDEPENDENT_CODE=ON \
-D CMAKE_INSTALL_PREFIX=$(pwd)/_build \
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D ZLIB_ROOT=$ZLIB_BUILD_DIR \
-D CMAKE_INSTALL_LIBDIR=lib ..
make -j$(nproc)
make install
cd _build
# Packaging build files
tar -czvf $PACKAGE_PROMCPP.tgz *
MD5_PROMCPP=($(md5sum $PACKAGE_PROMCPP.tgz))
echo $MD5_PROMCPP > $PACKAGE_PROMCPP.md5
# Copy the package to destination
cp $PACKAGE_PROMCPP.* $CACHE_DIRECTORY- Packaging protoc-gen-go 1.2.5
NAME_PROTOC=protoc-gen-go
VERSION_PROTOC=1.2.5
BUILD_PROTOC=7
PACKAGE_PROTOC=$NAME_PROTOC-linux-s390x-$VERSION_PROTOC-$BUILD_PROTOC
mkdir -p $SOURCE_ROOT/protoc-gen-go/bin && cd $SOURCE_ROOT/protoc-gen-go
go install github.com/golang/protobuf/protoc-gen-go@v1.2.0
cp $GOPATH/bin/protoc-gen-go ./bin
# Packaging build files
tar -czvf $PACKAGE_PROTOC.tgz *
MD5_PROTOC=($(md5sum $PACKAGE_PROTOC.tgz))
echo $MD5_PROTOC > $PACKAGE_PROTOC.md5
# Copy the package to destination
cp $PACKAGE_PROTOC.* $CACHE_DIRECTORY- Packaging spdlog v1.10.0
NAME_SPDLOG=spdlog
VERSION_SPDLOG=v1.10.0
BUILD_SPDLOG=cb6
PACKAGE_SPDLOG=$NAME_SPDLOG-linux-s390x-$VERSION_SPDLOG-$BUILD_SPDLOG
cd $SOURCE_ROOT
# Point to fmt build file.
FMT_BUILD_DIR=$SOURCE_ROOT/fmt/_build
git clone -b $VERSION_SPDLOG --depth 1 https://github.com/gabime/spdlog.git
cd spdlog
wget https://raw.githubusercontent.com/couchbase/tlm/7.6.2/deps/packages/spdlog/custom_level_names.patch
git apply custom_level_names.patch
mkdir build && cd build
cmake \
-D CMAKE_CXX_VISIBILITY_PRESET=hidden \
-D CMAKE_POSITION_INDEPENDENT_CODE=ON \
-D SPDLOG_BUILD_EXAMPLE=OFF \
-D SPDLOG_FMT_EXTERNAL=ON \
-D CMAKE_PREFIX_PATH=$FMT_BUILD_DIR \
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D CMAKE_INSTALL_PREFIX=$(pwd)/_build \
-D CMAKE_INSTALL_LIBDIR=lib ..
make -j$(nproc)
make install
cd _build
# Packaging build files
tar -czvf $PACKAGE_SPDLOG.tgz *
MD5_SPDLOG=($(md5sum $PACKAGE_SPDLOG.tgz))
echo $MD5_SPDLOG > $PACKAGE_SPDLOG.md5
# Copy the package to destination
cp $PACKAGE_SPDLOG.* $CACHE_DIRECTORY- Packaging libsodium v1.0.18
NAME_LIBSODIUM=libsodium
VERSION_LIBSODIUM=1.0.18
BUILD_LIBSODIUM=5
PACKAGE_LIBSODIUM=$NAME_LIBSODIUM-linux-s390x-$VERSION_LIBSODIUM-$BUILD_LIBSODIUM
cd $SOURCE_ROOT
git clone -b $VERSION_LIBSODIUM --depth 1 https://github.com/jedisct1/libsodium.git
cd libsodium
./autogen.sh
./configure --prefix=$(pwd)/_build --libdir=$(pwd)/_build/lib
make -j$(nproc)
make install
cd _build
cat <<EOF >CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
EOF
# Packaging build files
tar -czvf $PACKAGE_LIBSODIUM.tgz *
MD5_LIBSODIUM=($(md5sum $PACKAGE_LIBSODIUM.tgz))
echo $MD5_LIBSODIUM > $PACKAGE_LIBSODIUM.md5
# Copy the package to destination
cp $PACKAGE_LIBSODIUM.* $CACHE_DIRECTORY- Packaging OpenBLAS v0.3.25 #faiss requires this dependency
NAME_OPENBLAS=OpenBLAS
VERSION_OPENBLAS=0.3.25
BUILD_OPENBLAS=1
PACKAGE_OPENBLAS=$NAME_OPENBLAS-linux-s390x-$VERSION_OPENBLAS-$BUILD_OPENBLAS
cd $SOURCE_ROOT
git clone -b v$VERSION_OPENBLAS --depth 1 https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS
make -j$(nproc)
make PREFIX=$(pwd)/_build install
cd _build
# Packaging build files
tar -czvf $PACKAGE_OPENBLAS.tgz *
MD5_OPENBLAS=($(md5sum $PACKAGE_OPENBLAS.tgz))
echo $MD5_OPENBLAS >$PACKAGE_OPENBLAS.md5
# Copy the package to destination
cp $PACKAGE_OPENBLAS.* $CACHE_DIRECTORY- Packaging faiss v1.7.4
NAME_FAISS=faiss
VERSION_FAISS=1.7.4
BUILD_FAISS=17
PACKAGE_FAISS=$NAME_FAISS-linux-s390x-$VERSION_FAISS-$BUILD_FAISS
cd $SOURCE_ROOT
git clone -b v$VERSION_FAISS --depth 1 https://github.com/facebookresearch/faiss.git
cd faiss
cmake \
-DFAISS_ENABLE_GPU=OFF \
-DMKL_LIBRARIES=$SOURCE_ROOT/OpenBLAS/_build/lib \
-DFAISS_ENABLE_PYTHON=OFF \
-DBUILD_TESTING=OFF\
-DCMAKE_INSTALL_PREFIX=$(pwd)/_build \
-DCMAKE_INSTALL_LIBDIR=$(pwd)/_build/lib
make -j$(nproc) install
cd _build
cat <<EOF >CMakeLists.txt
FILE (COPY lib DESTINATION "\${CMAKE_INSTALL_PREFIX}")
SET_PROPERTY (GLOBAL APPEND PROPERTY CBDEPS_PREFIX_PATH "\${CMAKE_CURRENT_SOURCE_DIR}")
EOF
# Packaging build files
tar -czvf $PACKAGE_FAISS.tgz *
MD5_FAISS=($(md5sum $PACKAGE_FAISS.tgz))
echo $MD5_FAISS >$PACKAGE_FAISS.md5
# Copy the package to destination
cp $PACKAGE_FAISS.* $CACHE_DIRECTORY- Packaging simdutf v3.2.14
NAME_SIMDUTF=simdutf
VERSION_SIMDUTF=3.2.14
BUILD_SIMDUTF=cb1
PACKAGE_SIMDUTF=$NAME_SIMDUTF-linux-s390x-$VERSION_SIMDUTF-$BUILD_SIMDUTF
cd $SOURCE_ROOT
git clone -b v$VERSION_SIMDUTF --depth 1 https://github.com/simdutf/simdutf.git
cd simdutf
cmake\
-D CMAKE_BUILD_TYPE=RelWithDebInfo \
-D CMAKE_CXX_STANDARD=17 \
-D CMAKE_CXX_STANDARD_REQUIRED=ON \
-D BUILD_SHARED_LIBS=OFF \
-D CMAKE_INSTALL_PREFIX=$(pwd)/_build \
-D CMAKE_INSTALL_LIBDIR=lib
make -j$(nproc)
make PREFIX=$(pwd)/_build install
cd _build
# Packaging build files
tar -czvf $PACKAGE_SIMDUTF.tgz *
MD5_SIMDUTF=($(md5sum $PACKAGE_SIMDUTF.tgz))
echo $MD5_SIMDUTF >$PACKAGE_SIMDUTF.md5
# Copy the package to destination
cp $PACKAGE_SIMDUTF.* $CACHE_DIRECTORY- Packaging v8 12.1.285.26
NAME_V8=v8
VERSION_V8=12.1.285.26
BUILD_V8=1
PACKAGE_V8=$NAME_V8-linux-s390x-$VERSION_V8-$BUILD_V8
cd $SOURCE_ROOT
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
cd $SOURCE_ROOT/depot_tools
git checkout 081bca8cb31b7e96e663806b2493bce10dbb42f0
export PATH=$PATH:$SOURCE_ROOT/depot_tools/
export VPYTHON_BYPASS="manually managed python not supported by chrome operations"
export DEPOT_TOOLS_UPDATE=0
cd $SOURCE_ROOT
git clone https://gn.googlesource.com/gn
cd gn
git checkout 415b3b1
sed -i -e 's/-Wl,--icf=all//g' ./build/gen.py
python3 build/gen.py
ninja -C out
export PATH=$SOURCE_ROOT/gn/out:$PATH
cd $SOURCE_ROOT
sudo ldconfig /usr/local/lib64 /usr/local/lib
cat > .gclient <<EOF
solutions = [
{
"url": "https://chromium.googlesource.com/v8/v8.git@$VERSION_V8",
"managed": False,
"name": "v8",
"deps_file": "DEPS",
},
];
EOF
gclient sync
cd v8
wget $PATCH_URL/v8.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/v8.diff
sed -i 's/-Wl,-z,relro/-Wl,-z,relro,-lstdc++/' ./build/config/compiler/BUILD.gn
V8_ARGS='is_component_build=true target_cpu="s390x" v8_target_cpu="s390x" use_goma=false goma_dir="None" v8_enable_backtrace=true treat_warnings_as_errors=false is_clang=false use_custom_libcxx_for_host=false use_custom_libcxx=false v8_use_external_startup_data=false use_sysroot=false use_gold=false linux_use_bundled_binutils=false v8_enable_pointer_compression=false'
# build release
gn gen out/s390x.release --args="$V8_ARGS is_debug=false"
LD_LIBRARY_PATH=$SOURCE_ROOT/v8/out/s390x.release ninja -C $SOURCE_ROOT/v8/out/s390x.release -j$(nproc)
gn gen out/s390x.debug --args="$V8_ARGS is_debug=true"
LD_LIBRARY_PATH=$SOURCE_ROOT/v8/out/s390x.debug ninja -C $SOURCE_ROOT/v8/out/s390x.debug -j$(nproc)
INSTALL_DIR=$(pwd)/_build
mkdir -p \
$INSTALL_DIR/lib/Release \
$INSTALL_DIR/lib/Debug \
$INSTALL_DIR/include/libplatform \
$INSTALL_DIR/include/cppgc \
$INSTALL_DIR/include/unicode
(
cd out/s390x.release
cp -avi libv8*.* $INSTALL_DIR/lib/Release
cp -avi libchrome*.* $INSTALL_DIR/lib/Release
cp -avi libcppgc*.* $INSTALL_DIR/lib/Release
cp -avi libicu*.* $INSTALL_DIR/lib/Release
cp -avi icu*.* $INSTALL_DIR/lib/Release
cp -avi libthird_party*.* $INSTALL_DIR/lib/Release
rm -f $INSTALL_DIR/lib/Release/*.TOC
rm -f $INSTALL_DIR/lib/Release/*for_testing*
rm -f $INSTALL_DIR/lib/Release/*debug_helper*
)
(
cd include
cp -avi v8*.h $INSTALL_DIR/include
cp -avi libplatform/[a-z]*.h $INSTALL_DIR/include/libplatform
cp -avi cppgc/* $INSTALL_DIR/include/cppgc
)
(
cd third_party/icu/source/common/unicode
cp -avi *.h $INSTALL_DIR/include/unicode
)
(
cd third_party/icu/source/io/unicode
cp -avi *.h $INSTALL_DIR/include/unicode
)
(
cd third_party/icu/source/i18n/unicode
cp -avi *.h $INSTALL_DIR/include/unicode
)
(
cd third_party/icu/source/extra/uconv/unicode
cp -avi *.h $INSTALL_DIR/include/unicode
)
(
cd out/s390x.debug
cp -avi libv8*.* $INSTALL_DIR/lib/Debug
cp -avi libchrome*.* $INSTALL_DIR/lib/Debug
cp -avi libcppgc*.* $INSTALL_DIR/lib/Debug
cp -avi libicu*.* $INSTALL_DIR/lib/Debug
cp -avi icu*.* $INSTALL_DIR/lib/Debug
cp -avi libthird_party*.* $INSTALL_DIR/lib/Debug
rm -f $INSTALL_DIR/lib/Debug/*.TOC
rm -f $INSTALL_DIR/lib/Debug/*for_testing*
rm -f $INSTALL_DIR/lib/Debug/*debug_helper*
)
cd _build
cat <<EOF > CMakeLists.txt
FILE(MAKE_DIRECTORY \${CMAKE_INSTALL_PREFIX}/bin)
# Determine which directory to copy libs from
IF (CMAKE_BUILD_TYPE STREQUAL "Debug" AND
IS_DIRECTORY "\${CMAKE_CURRENT_SOURCE_DIR}/lib/Debug")
SET (LIB_DIR Debug)
ELSE()
SET (LIB_DIR Release)
ENDIF ()
FILE (COPY lib/\${LIB_DIR}/
DESTINATION "\${CMAKE_INSTALL_PREFIX}/lib"
PATTERN .dat EXCLUDE)
FILE (COPY lib/\${LIB_DIR}/icudtb.dat DESTINATION "\${CMAKE_INSTALL_PREFIX}/bin")
EOF
# Packaging build files
tar -czvf $PACKAGE_V8.tgz *
MD5_V8=($(md5sum $PACKAGE_V8.tgz))
echo $MD5_V8 > $PACKAGE_V8.md5
# Copy the package to destination
cp $PACKAGE_V8.* $CACHE_DIRECTORYDownload the Repo tool and ensure that it has execute permissions
cd $SOURCE_ROOT
curl https://storage.googleapis.com/git-repo-downloads/repo > repo
chmod a+x repo
sudo mv repo /usr/binexport PATH=$SOURCE_ROOT/openssl-3.1.4/_build/bin:$PATH
export LD_LIBRARY_PATH=$SOURCE_ROOT/openssl-3.1.4/_build/lib:$LD_LIBRARY_PATH
export OPENSSL_ROOT_DIR=$SOURCE_ROOT/openssl-3.1.4/_build
export OPENSSL_CONF=$SOURCE_ROOT/openssl-3.1.4/_build/etc/openssl-
Download the ssh tool (only for ubuntu22.04, rhel9.2, sles15.6 to avoid openssl mismatch error)
wget --no-check-certificate https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-9.4p1.tar.gz tar -xzf openssh-9.4p1.tar.gz cd openssh-9.4p1 ./configure --prefix=/usr/local --with-ssl-dir=$OPENSSL_ROOT_DIR make -j$(nproc) set +e sudo useradd -r -d /var/empty -s /sbin/nologin sshd set -e sudo make install ssh -V
-
Clone Couchbase
Set up Git user
git config --global user.email "your@email.addr" git config --global user.name "your_name"
Clone the Couchbase 7.6.2 release using the the manifest file via Repo tool with the init and sync commands
export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt #for ubuntu export SSL_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt #for rhel export SSL_CERT_FILE=/etc/ssl/ca-bundle.pem #for sles mkdir -p $SOURCE_ROOT/couchbase cd $SOURCE_ROOT/couchbase repo init -u https://github.com/couchbase/manifest -m released/couchbase-server/7.6.2.xml repo sync sudo git config --system --add safe.directory '*'
-
Patch Files
cd $SOURCE_ROOT/couchbase/tlm wget "${PATCH_URL}"/tlm.diff git apply tlm.diff cd $SOURCE_ROOT/couchbase/kv_engine wget "${PATCH_URL}"/kv_engine.diff git apply kv_engine.diff cd $SOURCE_ROOT/couchbase/platform wget "${PATCH_URL}"/platform.diff git apply platform.diff # add s390x to supported arches by replacing aarch64 sed -i '272s/aarch64/s390x/g' $SOURCE_ROOT/couchbase/forestdb/src/arch.h
cd $SOURCE_ROOT/couchbase/
sudo ldconfig /usr/local/lib64 /usr/local/lib
sudo cp tlm/CMakeLists.txt CMakeLists.txt
LD_LIBRARY_PATH=$SOURCE_ROOT/couchbase/install/lib:$LD_LIBRARY_PATH CC=$CC CXX=$CXX ./Build.sh everythingcd $SOURCE_ROOT/couchbase/build
sudo ctest --timeout 1000Notes:
- The following tests failed on both s390x and x86_64:
platform_unit_tests
ep-engine_ep_unit_tests.WarmupDiskTest
ep-engine_ep_unit_tests.MutationLogTest
ep_testsuite.value_eviction.comp_passive
ep_testsuite.value_eviction.comp_active
ep_testsuite.full_eviction.comp_passive
ep_testsuite.full_eviction.comp_active
ep_testsuite_basic.value_eviction.comp_passive
ep_testsuite_basic.value_eviction.comp_active
ep_testsuite_basic.full_eviction.comp_passive
ep_testsuite_basic.full_eviction.comp_active
memcached_testapp.ep.TransportProtocols/AuditTest
memcached_testapp.ep-full_eviction.TransportProtocols/AuditTest
cluster_tests
-
ep-engine_ep_unit_tests.BloomFilterTestfailed as there were less false positives on s390x than the expected number on x86_64. -
If you install couchbase in a non default path,
ns_testwould fail. -
The
couchdb-couch_set_viewtest suites are failing on both x86_64 and s390x architectures with the same error message. This problem is caused by a specific commit - https://github.com/couchbase/couchdb/commit/4658499c2d639384a7264971e32d852cb770e71a. For additional details, please refer to the corresponding JIRA issue: - https://issues.couchbase.com/browse/MB-57682.
- Use the below command to start Couchbase Server
sudo $SOURCE_ROOT/couchbase/install/bin/couchbase-server --startYou can view the Couchbase UI at http://<hostname>:8091. Follow this tutorial to get started with couchbase. Please note that you can not load sample buckets because the cbimport tool is not supported on s390x. Check this link for more details.
The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.