Skip to content

Building Couchbase

Jump to bottom
aborkar-ibm edited this page Dec 5, 2022 · 52 revisions

Building Couchbase

The instructions provided below specify the steps to build Couchbase version 7.0.2 on Linux on IBM Z for following distributions:

  • Ubuntu (18.04, 20.04)
  • RHEL (7.8, 7.9, 8.4)
  • SLES (12 SP5, 15 SP3)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it

Step 1: Building and Installing Couchbase

1.1) Build using script

If you want to build Couchbase using manual steps, go to STEP 1.2.

Use the following commands to build Couchbase using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Couchbase/7.0.2/build_couchbase.sh

# Build couchbase
bash build_couchbase.sh #[Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 3. In case of error, check logs for more details or go to STEP 1.2 to follow manual build steps.

1.2) Install dependencies

export SOURCE_ROOT=/<source_root>/
export PATCH_URL="https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Couchbase/7.0.2/patch"
mkdir $SOURCE_ROOT/patch
  • Ubuntu 18.04
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive TZ=America/Toronto apt-get install -y \
  autoconf automake autotools-dev \
  binutils-dev bison ccache clang cmake flex g++ gcc gcc-multilib g++-multilib \
  git gnome-keyring libatk1.0-dev libcups2-dev libdouble-conversion-dev libev-dev \
  libevent-dev libgconf2-dev libglib2.0-dev libgoogle-glog-dev libgflags-dev \
  libgtk-3-dev libiberty-dev liblz4-dev liblz4-tool liblzma-dev libnss3-dev \
  libpango1.0-dev libsnappy-dev libssl-dev libtool libuv1 libuv1-dev locales locales-all make \
  ncurses-dev ninja-build python python3 python3-httplib2 python3-six \
  pkg-config re2c texinfo tzdata unzip wget zlib1g-dev

sudo ln -sf /usr/bin/python2 /usr/bin/python

export LANG=en_US.UTF-8
  • Ubuntu 20.04
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive TZ=America/Toronto apt-get install -y \
  autoconf automake autotools-dev \
  binutils-dev bison ccache clang-12 cmake curl flex g++-10 gcc-10 gcc-10-multilib g++-10-multilib \
  git gnome-keyring libatk1.0-dev libcups2-dev libdouble-conversion-dev libev-dev \
  libevent-dev libgconf2-dev libglib2.0-dev libgoogle-glog-dev libgflags-dev \
  libgtk-3-dev libiberty-dev liblz4-dev liblz4-tool liblzma-dev libnss3-dev \
  libpango1.0-dev libsnappy-dev libssl-dev libtool libuv1 libuv1-dev locales locales-all make \
  ncurses-dev ninja-build python python3 python3-httplib2 python3-six \
  pkg-config re2c texinfo tzdata unzip wget zlib1g-dev

sudo ln -sf /usr/bin/python2 /usr/bin/python

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 9
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-12 12
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-12 12

export LANG=en_US.UTF-8
  • RHEL (7.8, 7.9)
sudo yum install -y atk-devel autoconf automake binutils-devel bison bzip2 \
  ca-certificates cmake cups-devel flex gcc gcc-c++ git gnome-keyring \
  libcurl-devel libtool make ncurses-devel \
  python2 python3 python3-devel tar texinfo \
  unzip wget which xmlto xz xz-devel zlib-devel

sudo pip3 install six httplib2

sudo ln -sf /usr/bin/python2 /usr/bin/python

export LANG=en_US.UTF-8

export LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib/:/usr/lib64:/usr/lib/:$LD_LIBRARY_PATH
  • RHEL 8.4
sudo yum install -y atk-devel autoconf automake binutils-devel bison bzip2 \
  clang cmake cups-devel flex gcc gcc-c++ git gnome-keyring \
  libcurl-devel libev-devel libevent-devel libuv libuv-devel \
  libtool lz4-devel make ncurses-devel ninja-build numactl-devel openssl-devel openssl-perl\
  pcre-devel python2 python3 python3-devel python3-httplib2 snappy-devel tar texinfo \
  unzip wget which xz xz-devel zlib-devel

sudo ln -sf /usr/bin/python2 /usr/bin/python

export LANG=en_US.UTF-8
  • SLES 12 SP5
sudo zypper install -y asciidoc autoconf automake binutils-devel bison cmake \
  cups-libs flex gcc gcc-c++ git-core glib2-devel libatk-1_0-0 \
  libgconfmm-2_6-1 libgtk-3-0 liblzma5 libncurses6 \
  libopenssl1_1 libopenssl-1_1-devel libpango-1_0-0 \
  libtirpc-devel libtool libxml2-tools libxslt-tools libz1 make makedepend \
  ncurses-devel ninja openssl-1_1 patch pkg-config python re2c ruby \
  snappy-devel sqlite3 tar texinfo unixODBC wget which xinetd xmlto zlib-devel xz glibc-locale

export LANG=en_US.UTF-8

export LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib/:/usr/lib64:/usr/lib/:$LD_LIBRARY_PATH

Install Python 3.8+ using script:

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Python3/3.8.6/build_python3.sh
bash build_python3.sh -y
# use system openssl
cd $SOURCE_ROOT/openssl-1.1.1h
sudo make uninstall

pip3 install httplib2 --upgrade
pip3 install six
  • SLES 15 SP3

Basesystem Module and Development Tools Module are required on SLES 15 SP3. For details on how to install these modules, please see SUSE documentation.

sudo sudo zypper install -y asciidoc autoconf automake clang12 cmake curl flex \
  gcc gcc-c++ gcc10-c++-10.3.0+git1587 gcc10-10.3.0+git1587 git-core glib2 glib2-devel glibc-locale go1.15 \
  libcurl-devel libevent-devel libopenssl-devel libncurses6 \
  libsnappy1 libtirpc-devel libtool libuv1 libuv-devel libxml2-tools libxslt-tools \
  liblz4-1 libz1 liblz4-devel make makedepend \
  ncurses-devel ninja patch pkg-config \
  python python-xml python3-httplib2 re2c ruby snappy-devel sqlite3 tar \
  unixODBC wget which xinetd xmlto zlib-devel xz python3-pip

pip3 install pyparsing

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 10
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-10 10
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-12 12 \
--slave /usr/bin/clang++ clang++ /usr/bin/clang++-12
sudo ln -sf /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc

export LANG=en_US.UTF-8
  • Install Go 1.15.5 (For Ubuntu, RHEL and SLES 12 SP5 only) 
cd $SOURCE_ROOT
wget https://golang.org/dl/go1.15.5.linux-s390x.tar.gz
chmod ugo+r go1.15.5.linux-s390x.tar.gz
sudo tar -C /usr/local -xzf go1.15.5.linux-s390x.tar.gz
export PATH=/usr/local/go/bin:$PATH
sudo ln -sf /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc
go version
  • Install Git 2.10.1 (For RHEL 7.x only) 
cd $SOURCE_ROOT
sudo yum install -y gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel
wget https://github.com/git/git/archive/v2.10.1.tar.gz -O git.tar.gz
tar -zxf git.tar.gz
cd git-2.10.1
make configure
./configure --prefix=/usr/local
make -j$(nproc)
sudo make install
  • Install GCC 10.2.0 (For Ubuntu 18.04, RHEL and SLES 12 SP5 only) 
cd $SOURCE_ROOT
ver=10.2.0
wget https://ftp.gnu.org/gnu/gcc/gcc-${ver}/gcc-${ver}.tar.gz
tar xzf gcc-${ver}.tar.gz
cd gcc-${ver}
./contrib/download_prerequisites
mkdir build-gcc
cd build-gcc
../configure --enable-languages=c,c++ --disable-multilib
make -j$(nproc)
sudo make install
sudo ldconfig /usr/local/lib64 /usr/local/lib

For Ubuntu 18.04:

sudo update-alternatives --install /usr/bin/gcc gcc /usr/local/bin/gcc 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/local/bin/g++ 10
sudo ln -sf /usr/local/lib64/libstdc++.so.6 /usr/lib/s390x-linux-gnu/libstdc++.so.6
export CC=/usr/local/bin/s390x-ibm-linux-gnu-gcc
export CXX=/usr/local/bin/s390x-ibm-linux-gnu-g++

For RHEL 7.x:

sudo mv /usr/bin/gcc /usr/bin/gcc-4.8.5
sudo mv /usr/bin/g++ /usr/bin/g++-4.8.5
sudo mv /usr/bin/c++ /usr/bin/c++-4.8.5
sudo update-alternatives --install /usr/bin/cc cc /usr/local/bin/gcc 40
sudo update-alternatives --install /usr/bin/gcc gcc /usr/local/bin/gcc 40
sudo update-alternatives --install /usr/bin/g++ g++ /usr/local/bin/g++ 40
sudo update-alternatives --install /usr/bin/c++ c++ /usr/local/bin/c++ 40
export CC=/usr/local/bin/gcc
export CXX=/usr/local/bin/g++

For RHEL 8.x:

sudo mv /usr/bin/gcc /usr/bin/gcc-8.5.0
sudo mv /usr/bin/g++ /usr/bin/g++-8.5.0
sudo mv /usr/bin/c++ /usr/bin/c++-8.5.0
sudo mv /usr/bin/cc /usr/bin/cc-8.5.0
sudo update-alternatives --install /usr/bin/cc cc /usr/local/bin/gcc 40
sudo update-alternatives --install /usr/bin/gcc gcc /usr/local/bin/gcc 40
sudo update-alternatives --install /usr/bin/g++ g++ /usr/local/bin/g++ 40
sudo update-alternatives --install /usr/bin/c++ c++ /usr/local/bin/c++ 40
sudo ln -sf /usr/bin/gcc /usr/bin/s390x-linux-gnu-gcc
export CC=/usr/local/bin/s390x-ibm-linux-gnu-gcc
export CXX=/usr/local/bin/s390x-ibm-linux-gnu-g++

For SLES 12 SP5:

sudo update-alternatives --install /usr/bin/cc cc /usr/local/bin/gcc 40
sudo update-alternatives --install /usr/bin/gcc gcc /usr/local/bin/gcc 40
sudo update-alternatives --install /usr/bin/g++ g++ /usr/local/bin/g++ 40
sudo update-alternatives --install /usr/bin/c++ c++ /usr/local/bin/c++ 40
export CC=/usr/local/bin/s390x-ibm-linux-gnu-gcc
export CXX=/usr/local/bin/s390x-ibm-linux-gnu-g++
  • Install CMake 3.16.0 (For Ubuntu 18.04, RHEL 7.x and SLES 12 SP5 only) 
cd $SOURCE_ROOT
wget https://cmake.org/files/v3.16/cmake-3.16.0.tar.gz
tar -xzf cmake-3.16.0.tar.gz
cd cmake-3.16.0
./bootstrap --prefix=/usr
make -j$(nproc)
sudo make install
hash -r
  • Install cURL 7.66.0 (For Ubuntu, RHEL 7.x and SLES 12 SP5 only) 
cd $SOURCE_ROOT
sudo ldconfig /usr/local/lib64 /usr/local/lib
git clone https://github.com/curl/curl.git
cd curl
git checkout curl-7_66_0
cmake -G "Unix Makefiles" .
make -j $(nproc)
sudo make install
hash -r
  • Install OpenSSL 1.1.1k (For Ubuntu and RHEL 7.x only) 
cd $SOURCE_ROOT
wget https://www.openssl.org/source/openssl-1.1.1k.tar.gz
tar -xzf openssl-1.1.1k.tar.gz
cd openssl-1.1.1k
./config --prefix=/usr/local --openssldir=/usr/local
make -j$(nproc)
sudo make install
sudo rm -rf /usr/local/certs
sudo ln -sf /etc/ssl/certs /usr/local/certs
sudo ldconfig /usr/local/lib64 /usr/local/lib
hash -r
  • Install LZ4, Snappy, Libevent and Libuv (For RHEL 7.x and SLES 12 SP 5 only) 
cd $SOURCE_ROOT
wget https://github.com/lz4/lz4/archive/v1.8.1.2.tar.gz
tar -xzf v1.8.1.2.tar.gz
cd lz4-1.8.1.2
make
sudo make install
hash -r

cd $SOURCE_ROOT
git clone https://github.com/google/snappy.git
cd snappy
git checkout 1.1.5
mkdir build
cd build && cmake ../ && make
sudo make install
hash -r

cd $SOURCE_ROOT
wget https://github.com/libevent/libevent/releases/download/release-2.1.8-stable/libevent-2.1.8-stable.tar.gz
tar -xzf libevent-2.1.8-stable.tar.gz
cd libevent-2.1.8-stable
./configure
make
sudo make install

cd $SOURCE_ROOT
git clone https://github.com/couchbasedeps/libuv
cd libuv
git checkout v1.20.3
./autogen.sh
./configure --disable-silent-rules --prefix=/usr/local
make
sudo make install
  • Install Jemalloc 5.2.1
cd $SOURCE_ROOT
git clone https://github.com/couchbasedeps/jemalloc.git
cd jemalloc && git checkout 5.2.1
autoconf configure.ac > configure
chmod u+x configure
CPPFLAGS=-I/usr/local/include ./configure --prefix=/usr/local \
--with-jemalloc-prefix=je_ --disable-cache-oblivious --disable-zone-allocator \
--enable-prof --disable-initial-exec-tls
make build_lib_shared
sudo make install_lib_shared install_include
sudo cp $SOURCE_ROOT/jemalloc/bin/jeprof /usr/local/bin/jeprof
sudo chmod a+x /usr/local/bin/jeprof

Note for SLES: ls /usr/lib64/libjemalloc.so.2 should show No such file or directory. If /usr/lib64/libjemalloc.so.2 exists, then perform the following:

mv /usr/lib64/libjemalloc.so.2 ~/badlibjemalloc
  • Install Ninja 1.8.2 (For RHEL 7.x only) 
git clone https://github.com/ninja-build/ninja
cd ninja
git checkout v1.8.2
./configure.py --bootstrap
sudo cp ninja /usr/bin
  • Install V8 8.3
cd $SOURCE_ROOT
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
cd $SOURCE_ROOT/depot_tools
git checkout a0382d39be0d7bf0f0766633185f20dcdd32a459
export PATH=$PATH:$SOURCE_ROOT/depot_tools/
export VPYTHON_BYPASS="manually managed python not supported by chrome operations"
export DEPOT_TOOLS_UPDATE=0

cd $SOURCE_ROOT
git clone https://gn.googlesource.com/gn
cd gn
git checkout 8948350
sed -i -e 's/-Wl,--icf=all//g' ./build/gen.py
sed -i -e 's/-lpthread/-pthread/g' ./build/gen.py
sed -i -e 's/clang++/gcc/g' ./build/gen.py # For SLES 12 SP 5 only
python build/gen.py
ninja -C out
export PATH=$SOURCE_ROOT/gn/out:$PATH

cd $SOURCE_ROOT
sudo ldconfig /usr/local/lib64 /usr/local/lib

cat > .gclient <<EOF
solutions = [
  {
    "url": "https://chromium.googlesource.com/v8/v8.git@8.3.110.9",
    "managed": False,
    "name": "v8",
    "deps_file": "DEPS",
  },
];
EOF
gclient sync

cd v8
wget $PATCH_URL/v8.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/v8.diff
mkdir out/s390x.release
gn gen out/s390x.release --args='is_component_build=true target_cpu="s390x" v8_target_cpu="s390x" use_goma=false goma_dir="None" v8_enable_backtrace=true treat_warnings_as_errors=false is_clang=false use_custom_libcxx_for_host=false use_custom_libcxx=false v8_use_external_startup_data=false is_debug=false'
ninja -C $SOURCE_ROOT/v8/out/s390x.release -j$(nproc)
cd out/s390x.release
sudo cp libv8*.so /usr/local/lib
sudo cp libchrome*.so /usr/local/lib
sudo cp libcppgc*.so /usr/local/lib
sudo cp libicu*.so /usr/local/lib
sudo cp icu*.* /usr/local/lib
cd $SOURCE_ROOT/v8/include
sudo mkdir -p /usr/local/include/libplatform /usr/local/include/cppgc /usr/local/include/unicode
sudo cp v8*.h /usr/local/include
sudo cp libplatform/*.h /usr/local/include/libplatform
sudo cp cppgc/[a-z]*.h /usr/local/include/cppgc
cd $SOURCE_ROOT/v8
sudo cp ./third_party/icu/source/common/unicode/*.h /usr/local/include/unicode
sudo cp ./third_party/icu/source/io/unicode/*.h /usr/local/include/unicode
sudo cp ./third_party/icu/source/i18n/unicode/*.h /usr/local/include/unicode
sudo cp ./third_party/icu/source/extra/uconv/unicode/*.h /usr/local/include/unicode

For Ubuntu 18.04 only:

cd /usr/local/lib
sudo ln -s libicui18n.so libicui18n.so.65
sudo ln -s libicuuc.so libicuuc.so.65
  • Install Boost 1.74.0
sudo ldconfig /usr/local/lib64 /usr/local/lib
cd $SOURCE_ROOT
TOOLSET=gcc
CENV=(PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH LD_RUN_PATH=$LD_RUN_PATH CC=$CC CXX=$CXX)
URL=https://boostorg.jfrog.io/artifactory/main/release/1.74.0/source/boost_1_74_0.tar.gz
curl -sSL $URL | tar xzf -
cd boost_1_74_0
sed -i 's/array\.hpp/array_wrapper.hpp/g' boost/numeric/ublas/matrix.hpp
sed -i 's/array\.hpp/array_wrapper.hpp/g' boost/numeric/ublas/storage.hpp
./bootstrap.sh --with-libraries=context,chrono,date_time,filesystem,program_options,regex,system,thread --with-icu=/usr/local/lib
options=( toolset=$TOOLSET variant=release link=shared runtime-link=shared threading=multi --without-python )
sudo ${CENV[@]} ./b2 -j 4 ${options[@]} install
  • Install double-conversion, glog and gflags (For RHEL and SLES only) 
cd $SOURCE_ROOT
git clone https://github.com/google/double-conversion.git
cd double-conversion
git checkout v3.0.0
cmake -D BUILD_SHARED_LIBS=OFF -D BUILD_TESTING=OFF .
make
sudo make install

cd $SOURCE_ROOT
git clone https://github.com/google/glog.git
cd glog
git checkout v0.4.0
cmake -S . -B build -G "Unix Makefiles" -D BUILD_SHARED_LIBS=OFF
cmake --build build
sudo cmake --build build --target install

cd $SOURCE_ROOT
version=2.2.2
wget https://github.com/gflags/gflags/archive/refs/tags/v$version.tar.gz
tar xzf v$version.tar.gz
cd gflags-$version
mkdir build && cd build
cmake ..
make
sudo make install
  • Install fmt 6.2.1 and Folly v2020.08.24.00
cd $SOURCE_ROOT
git clone https://github.com/fmtlib/fmt.git
cd fmt
git checkout 6.2.1
mkdir build
cd build
cmake ..
make -j$(nproc)
sudo make install

sudo ldconfig /usr/local/lib64 /usr/local/lib
cd $SOURCE_ROOT
git clone https://github.com/facebook/folly
cd folly
git checkout v2020.08.24.00
curl https://github.com/facebook/folly/commit/eedb340bd5fff6a4a44006ff641cb3ecc2b293fb.patch | git apply -
wget $PATCH_URL/folly.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/folly.diff
mkdir _build
cd _build/
cmake -DCMAKE_INCLUDE_PATH=/usr/local/include -DCMAKE_LIBRARY_PATH=/usr/local/lib .. -DBUILD_SHARED_LIBS=OFF
make -j$(nproc)
sudo make install
  • Install Erlang cheshirecat
cd $SOURCE_ROOT
git clone https://github.com/couchbasedeps/erlang.git
cd erlang && git checkout couchbase-cheshirecat
./otp_build autoconf
touch lib/debugger/SKIP lib/megaco/SKIP lib/observer/SKIP lib/wx/SKIP lib/et/SKIP
./configure --prefix=/usr/local --enable-smp-support --disable-hipe --disable-fp-exceptions --without-javac CFLAGS="-fno-strict-aliasing -O3 -ggdb3"
make -j $(nproc)
sudo make install
hash -r
  • Install Flatbuffers 1.10.0
cd $SOURCE_ROOT
git clone https://github.com/google/flatbuffers
cd flatbuffers && git checkout v1.10.0
wget $PATCH_URL/flatbuffers.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/flatbuffers.diff
curl https://github.com/google/flatbuffers/commit/2e865f4d4e67a9b628c137aab7da8140dd9339a4.patch | git apply -
cmake -G "Unix Makefiles"
make -j $(nproc)
sudo make install
sudo cp flatc flathash flattests flatsamplebinary flatsampletext /usr/local/bin
hash -r
  • Install JSON hpp
cd $SOURCE_ROOT
git clone https://github.com/nlohmann/json
cd $SOURCE_ROOT/json
git checkout v3.5.0
cd $SOURCE_ROOT/json/include/nlohmann
sudo mkdir /usr/local/include/nlohmann
sudo cp -r * /usr/local/include/nlohmann
  • Install prometheus-cpp 0.10.0
cd $SOURCE_ROOT
git clone https://github.com/jupp0r/prometheus-cpp.git
cd prometheus-cpp
git checkout v0.10.0
git submodule init
git submodule update
mkdir _build
cd _build
cmake .. -DBUILD_SHARED_LIBS=OFF \
-DENABLE_PUSH=OFF \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
make -j$(nproc)
sudo make install
  • Install prometheus
cd $SOURCE_ROOT
export GOPATH=$(go env GOPATH)
wget https://nodejs.org/dist/v16.14.2/node-v16.14.2-linux-s390x.tar.xz
tar xf node-v16.14.2-linux-s390x.tar.xz
export PATH=$SOURCE_ROOT/node-v16.14.2-linux-s390x/bin:$PATH
npm install -g yarn
mkdir -p $GOPATH/src/github.com
cd $GOPATH/src/github.com
git clone https://github.com/couchbasedeps/prometheus.git
cd prometheus
git checkout couchbase-v2.22
make build
sudo cp prometheus /usr/local/bin/prometheus
  • Install numactl (For Ubuntu, RHEL 7.x and SLES only) 
cd $SOURCE_ROOT
git clone https://github.com/numactl/numactl.git
cd numactl

# For RHEL 7.x only:
git checkout v2.0.14

# For all other distros:
git checkout v2.0.11
curl https://github.com/numactl/numactl/commit/25691a084a2012a339395ade567dbae814e237e9.patch | git apply -

./autogen.sh
./configure
make
sudo make install
  • Install PCRE 8.43
cd $SOURCE_ROOT
wget https://sourceforge.net/projects/pcre/files/pcre/8.43/pcre-8.43.tar.gz
tar -xzf pcre-8.43.tar.gz
cd pcre-8.43
./configure --prefix=/usr/local
make
sudo make install
  • Install protoc 3.11.2
cd $SOURCE_ROOT
export GO111MODULE=on
go get github.com/golang/protobuf/protoc-gen-go@v1.2.0
sudo cp $HOME/go/bin/protoc-gen-go /usr/local/bin
git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
git checkout v3.11.2
git submodule update --init --recursive
./autogen.sh
./configure --prefix=/usr/local
make -j$(nproc)
sudo make install
sudo cp src/.libs/protoc /usr/local/bin
  • Install GRPC 1.33.2
cd $SOURCE_ROOT
git clone -b v1.33.2 https://github.com/grpc/grpc
cd grpc/
git submodule update --init
mkdir -p cmake/build
cd cmake/build
mv ../../third_party/boringssl-with-bazel ../../third_party/boringssl-with-bazel_ORIG
cd ../../third_party/
git clone https://github.com/linux-on-ibm-z/boringssl.git
cd boringssl
git checkout patch-s390x-Aug2019
cd ..
mv boringssl boringssl-with-bazel
cd ../cmake/build/
cmake -DgRPC_INSTALL=ON \
  -DgRPC_BUILD_TESTS=OFF \
  -DCMAKE_INSTALL_PREFIX=/usr/local \
  ../..
make -j$(nproc)
sudo make install
cd $SOURCE_ROOT/grpc
mkdir -p third_party/abseil-cpp/cmake/build
cd third_party/abseil-cpp/cmake/build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local \
  -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE \
  ../..
make -j$(nproc)
sudo make install
  • Install cbpy
cd $SOURCE_ROOT
wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.10.3-Linux-s390x.sh
chmod a+x Miniconda3-py38_4.10.3-Linux-s390x.sh
sudo ./Miniconda3-py38_4.10.3-Linux-s390x.sh -b -p /usr/local/lib/python/runtime
  • Install crc32
cd $SOURCE_ROOT
git clone https://github.com/linux-on-ibm-z/crc32-s390x.git
cd crc32-s390x
wget $PATCH_URL/crc32-s390x.diff -P $SOURCE_ROOT/patch
git apply $SOURCE_ROOT/patch/crc32-s390x.diff
make
sudo cp crc32-s390x.h /usr/local/include/
sudo cp libcrc32_s390x.a /usr/local/lib/
  • Install RocksDB v5.18.3
cd $SOURCE_ROOT
git clone https://github.com/facebook/rocksdb.git
cd rocksdb/
git checkout v5.18.3
CXXFLAGS='-Wno-error=deprecated-copy -Wno-error=pessimizing-move -Wno-error=redundant-move' make -j$(nproc) shared_lib
sudo make install-shared INSTALL_PATH=/usr/local
  • Install re2c 3.0 (For RHEL and SLES 12 SP5 only) 
cd $SOURCE_ROOT
wget https://github.com/skvadrik/re2c/releases/download/3.0/re2c-3.0.tar.xz
tar xf re2c-3.0.tar.xz
cd re2c-3.0/
./configure
make
sudo make install
sudo cp /usr/local/bin/re2c /usr/bin/re2c
  • Install Clang 12 (For Ubuntu 18.04, RHEL and SLES 12 SP5 only) 
cd $SOURCE_ROOT
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout llvmorg-12.0.0
mkdir build && cd build
cmake -DLLVM_ENABLE_PROJECTS=clang -G "Unix Makefiles" ../llvm
make -j$(nproc)
sudo make install
clang -v # The selected GCC installation should be gcc 10.2.0 that was built in previous step

1.3) Download the Repo tool

Download the Repo tool and ensure that it has execute permissions

cd $SOURCE_ROOT
curl https://storage.googleapis.com/git-repo-downloads/repo > repo
chmod a+x repo
sudo mv repo /usr/bin

1.4) Setup Couchbase source

  • Clone Couchbase

    Set up Git user

    git config --global user.email "your@email.addr"
git config --global user.name  "your_name"

    Clone the Couchbase 7.0.2 release using the the manifest file via Repo tool with the init and sync commands

    mkdir -p $SOURCE_ROOT/couchbase
cd $SOURCE_ROOT/couchbase
repo init -u https://github.com/couchbase/manifest -m released/couchbase-server/7.0.2.xml
repo sync

    Note: repo init command might fail on first run. Please re-run the command to initialize the repo.

  • Patch Files

    Download patches from https://github.com/linux-on-ibm-z/scripts/tree/master/Couchbase/7.0.2/patch and apply them as follows:

    cd $SOURCE_ROOT/couchbase/tlm
wget "${PATCH_URL}"/tlm-ub18.diff
git apply tlm-ub18.diff # For Ubuntu 18.04
wget "${PATCH_URL}"/tlm-ub20.diff
git apply tlm-ub20.diff # For Ubuntu 20.04
wget "${PATCH_URL}"/tlm-rhel7.diff
git apply tlm-rhel7.diff # For RHEL 7.x
wget "${PATCH_URL}"/tlm-rhel8.diff
git apply tlm-rhel8.diff # For RHEL 8.x
wget "${PATCH_URL}"/tlm-sles.diff
git apply tlm-sles.diff # For SLES
cd $SOURCE_ROOT/couchbase/couchdb
wget "${PATCH_URL}"/couchdb.diff
git apply couchdb.diff
cd $SOURCE_ROOT/couchbase/couchstore
wget "${PATCH_URL}"/couchstore.diff
git apply couchstore.diff
cd $SOURCE_ROOT/couchbase/forestdb
wget "${PATCH_URL}"/forestdb.diff
git apply forestdb.diff
cd $SOURCE_ROOT/couchbase/kv_engine
wget "${PATCH_URL}"/kv_engine.diff
git apply kv_engine.diff
cd $SOURCE_ROOT/couchbase/platform
wget "${PATCH_URL}"/platform.diff
git apply platform.diff

1.5) Build Couchbase

cd $SOURCE_ROOT/couchbase/
sudo ldconfig /usr/local/lib64 /usr/local/lib

# For Ubuntu 20.04 and SLES 15 SP3:
sudo make -j$(nproc) EXTRA_CMAKE_OPTIONS='-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++'

# For all other distros:
sudo make -j$(nproc) EXTRA_CMAKE_OPTIONS='-DCMAKE_C_COMPILER=/usr/local/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/bin/clang++'

1.6) Install libraries and binaries

cd $SOURCE_ROOT
wget https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Couchbase/7.0.2/installDeps.sh
source installDeps.sh

Step 2: Run test cases (Optional)

cd $SOURCE_ROOT/couchbase/build/
sudo ctest --timeout 1000

Note:

  • The following tests failed on both s390x and x86_64:

    platform-dirutils-test
memcached-audit-test
ep-engine_ep_unit_tests.WarmupDiskTest
ep-engine_ep_unit_tests.MutationLogTest
ep_testsuite.value_eviction.comp_passive
ep_testsuite.value_eviction.comp_active
ep_testsuite.full_eviction.comp_passive
ep_testsuite.full_eviction.comp_active
ep_testsuite_basic.value_eviction.comp_passive
ep_testsuite_basic.value_eviction.comp_active
ep_testsuite_basic.full_eviction.comp_passive
ep_testsuite_basic.full_eviction.comp_active

  • ep-engine_ep_unit_tests.BloomFilterTest failed as there were less false positives on s390x than the expected number on x86_64.

  • ns_test failure is intermittent and may pass on multiple runs. To run a test case individually, sudo ctest -R <test name>.

  • IPv6 related failures can be resolved by enabling IPv6 on SLES.

  • SSL related failures are due to RHEL 8's stronger default cryptographic policies. The tests will pass with the LEGACY level policies.

Step 3: Start the server

  • Use the below command to start Couchbase Server

    sudo $SOURCE_ROOT/couchbase/install/bin/couchbase-server -- -noinput &

    You can view the Couchbase UI at http://hostname:8091

References:

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally