From 97e0c972178177011ee928407719b2e002fa116f Mon Sep 17 00:00:00 2001 From: Zach Kimberg Date: Tue, 11 Dec 2018 15:23:13 -0800 Subject: [PATCH] [MXNET-1225] Always use config.mk in make install instructions (#13364) * Always use config.mk in make install instructions * Specify Cuda 0 for ubuntu with mkldnn * Scala install doc avoid build_from_source Minor doc fixes * Fix build_from_source CMake usage * CPP Install Instruction with CMake * Use cmake out of source build --- docs/install/build_from_source.md | 41 +++++++++++++++++++++---------- docs/install/c_plus_plus.md | 3 ++- docs/install/java_setup.md | 4 ++- docs/install/osx_setup.md | 9 ++++++- docs/install/scala_setup.md | 4 ++- docs/install/ubuntu_setup.md | 21 +++++++++++++--- 6 files changed, 61 insertions(+), 21 deletions(-) diff --git a/docs/install/build_from_source.md b/docs/install/build_from_source.md index e41b1d0f1804..e807fb44b599 100644 --- a/docs/install/build_from_source.md +++ b/docs/install/build_from_source.md @@ -2,6 +2,7 @@ This document explains how to build MXNet from source code. +**For Java/Scala/Clojure, please follow [this guide instead](./scala_setup.md)** ## Overview @@ -27,7 +28,6 @@ MXNet's newest and most popular API is Gluon. Gluon is built into the Python bin - [Python (includes Gluon)](../api/python/index.html) - [C++](../api/c++/index.html) - [Clojure](../api/clojure/index.html) - - Java (coming soon) - [Julia](../api/julia/index.html) - [Perl](../api/perl/index.html) - [R](../api/r/index.html) @@ -35,6 +35,7 @@ MXNet's newest and most popular API is Gluon. Gluon is built into the Python bin - [Java](../api/java/index.html)
+ ## Build Instructions by Operating System Detailed instructions are provided per operating system. Each of these guides also covers how to install the specific [Language Bindings](#installing-mxnet-language-bindings) you require. @@ -160,7 +161,7 @@ More information on turning these features on or off are found in the following ## Build Configurations There is a configuration file for make, -[`make/config.mk`](https://github.com/apache/incubator-mxnet/blob/master/make/config.mk), that contains all the compilation options. You can edit it and then run `make` or `cmake`. `cmake` is recommended for building MXNet (and is required to build with MKLDNN), however you may use `make` instead. +[`make/config.mk`](https://github.com/apache/incubator-mxnet/blob/master/make/config.mk), that contains all the compilation options. You can edit it and then run `make` or `cmake`. `cmake` is recommended for building MXNet (and is required to build with MKLDNN), however you may use `make` instead. For building with Java/Scala/Clojure, only `make` is supported.
@@ -203,18 +204,18 @@ It is recommended to set environment variable NCCL_LAUNCH_MODE to PARALLEL when ### Build MXNet with C++ -* To enable C++ package, just add `USE_CPP_PACKAGE=1` when you run `make` or `cmake`. +* To enable C++ package, just add `USE_CPP_PACKAGE=1` when you run `make` or `cmake` (see examples).
### Usage Examples -* `-j` runs multiple jobs against multi-core CPUs. - For example, you can specify using all cores on Linux as follows: ```bash -cmake -j$(nproc) +mkdir build && cd build +cmake -GNinja . +ninja -v ``` @@ -222,28 +223,36 @@ cmake -j$(nproc) * Build MXNet with `cmake` and install with MKL DNN, GPU, and OpenCV support: ```bash -cmake -j USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 USE_MKLDNN=1 +mkdir build && cd build +cmake -DUSE_CUDA=1 -DUSE_CUDA_PATH=/usr/local/cuda -DUSE_CUDNN=1 -DUSE_MKLDNN=1 -GNinja . +ninja -v ``` #### Recommended for Systems with NVIDIA GPUs * Build with both OpenBLAS, GPU, and OpenCV support: ```bash -cmake -j BLAS=open USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 +mkdir build && cd build +cmake -DBLAS=open -DUSE_CUDA=1 -DUSE_CUDA_PATH=/usr/local/cuda -DUSE_CUDNN=1 -GNinja . +ninja -v ``` #### Recommended for Systems with Intel CPUs * Build MXNet with `cmake` and install with MKL DNN, and OpenCV support: ```bash -cmake -j USE_CUDA=0 USE_MKLDNN=1 +mkdir build && cd build +cmake -DUSE_CUDA=0 -DUSE_MKLDNN=1 -GNinja . +ninja -v ``` #### Recommended for Systems with non-Intel CPUs * Build MXNet with `cmake` and install with OpenBLAS and OpenCV support: ```bash -cmake -j USE_CUDA=0 BLAS=open +mkdir build && cd build +cmake -DUSE_CUDA=0 -DBLAS=open -GNinja . +ninja -v ``` #### Other Examples @@ -251,20 +260,26 @@ cmake -j USE_CUDA=0 BLAS=open * Build without using OpenCV: ```bash -cmake USE_OPENCV=0 +mkdir build && cd build +cmake -DUSE_OPENCV=0 -GNinja . +ninja -v ``` * Build on **macOS** with the default BLAS library (Apple Accelerate) and Clang installed with `xcode` (OPENMP is disabled because it is not supported by the Apple version of Clang): ```bash -cmake -j BLAS=apple USE_OPENCV=0 USE_OPENMP=0 +mkdir build && cd build +cmake -DBLAS=apple -DUSE_OPENCV=0 -DUSE_OPENMP=0 -GNinja . +ninja -v ``` * To use OpenMP on **macOS** you need to install the Clang compiler, `llvm` (the one provided by Apple does not support OpenMP): ```bash brew install llvm -cmake -j BLAS=apple USE_OPENMP=1 +mkdir build && cd build +cmake -DBLAS=apple -DUSE_OPENMP=1 -GNinja . +ninja -v ```
diff --git a/docs/install/c_plus_plus.md b/docs/install/c_plus_plus.md index 6078877c27c8..6ad67e2803db 100644 --- a/docs/install/c_plus_plus.md +++ b/docs/install/c_plus_plus.md @@ -6,7 +6,8 @@ To enable C++ package, just add `USE_CPP_PACKAGE=1` in the [build from source](b For example to build MXNet with GPU support and the C++ package, OpenCV, and OpenBLAS, from the project root you would run: ```bash -make -j USE_CPP_PACKAGE=1 USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1 +cmake -DUSE_CUDA=1 -DUSE_CUDA_PATH=/usr/local/cuda -DUSE_CUDNN=1 -DUSE_MKLDNN=1 -DUSE_CPP_PACKAGE=1 -GNinja . +ninja -v ``` You may also want to add the MXNet shared library to your `LD_LIBRARY_PATH`: diff --git a/docs/install/java_setup.md b/docs/install/java_setup.md index 34b0967c421e..0075e9205880 100644 --- a/docs/install/java_setup.md +++ b/docs/install/java_setup.md @@ -89,11 +89,13 @@ The official Java Packages will be released with the release of MXNet 1.4 and wi The previously mentioned setup with Maven is recommended. Otherwise, the following instructions for macOS and Ubuntu are provided for reference only: +**If you have already built mxnet from source using `cmake`, run `make clean` and then follow the appropriate guide below*** + | OS | Step 1 | Step 2 | |---|---|---| |macOS | [Shared Library for macOS](../install/osx_setup.html#build-the-shared-library) | [Scala Package for macOS](http://mxnet.incubator.apache.org/install/osx_setup.html#install-the-mxnet-package-for-scala) | | Ubuntu | [Shared Library for Ubuntu](../install/ubuntu_setup.html#installing-mxnet-on-ubuntu) | [Scala Package for Ubuntu](http://mxnet.incubator.apache.org/install/ubuntu_setup.html#install-the-mxnet-package-for-scala) | -| Windows | [Shared Library for Windows](../install/windows_setup.html#build-the-shared-library) | Call for Contribution | +| Windows | | Call for Contribution | #### Build Java from an Existing MXNet Installation diff --git a/docs/install/osx_setup.md b/docs/install/osx_setup.md index 4e9293efce93..a2b59fe03618 100644 --- a/docs/install/osx_setup.md +++ b/docs/install/osx_setup.md @@ -96,7 +96,14 @@ The file called ```osx.mk``` has the configuration required for building MXNet o To build with MKLDNN ```bash -LIBRARY_PATH=$(brew --prefix llvm)/lib/ make -j $(sysctl -n hw.ncpu) CC=$(brew --prefix llvm)/bin/clang++ CXX=$(brew --prefix llvm)/bin/clang++ USE_OPENCV=1 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple USE_PROFILER=1 +echo "CC=$(brew --prefix llvm)/bin/clang++" >> ./config.mk +echo "CXX=$(brew --prefix llvm)/bin/clang++" >> ./config.mk +echo "USE_OPENCV=1" >> ./config.mk +echo "USE_OPENMP=1" >> ./config.mk +echo "USE_MKLDNN=1" >> ./config.mk +echo "USE_BLAS=apple" >> ./config.mk +echo "USE_PROFILER=1" >> ./config.mk +LIBRARY_PATH=$(brew --prefix llvm)/lib/ make -j $(sysctl -n hw.ncpu) ``` If building with ```GPU``` support, add the following configuration to config.mk and build: diff --git a/docs/install/scala_setup.md b/docs/install/scala_setup.md index 0dadd8bca400..98e752b21dd5 100644 --- a/docs/install/scala_setup.md +++ b/docs/install/scala_setup.md @@ -79,11 +79,13 @@ https://mvnrepository.com/artifact/org.apache.mxnet The previously mentioned setup with Maven is recommended. Otherwise, the following instructions for macOS, Ubuntu, and Windows are provided for reference only: +**If you have already built mxnet from source using `cmake`, run `make clean` and then follow the appropriate guide below*** + | OS | Step 1 | Step 2 | |---|---|---| |macOS | [Shared Library for macOS](http://mxnet.incubator.apache.org/install/osx_setup.html#build-the-shared-library) | [Scala Package for macOS](http://mxnet.incubator.apache.org/install/osx_setup.html#install-the-mxnet-package-for-scala) | | Ubuntu | [Shared Library for Ubuntu](http://mxnet.incubator.apache.org/install/ubuntu_setup.html#installing-mxnet-on-ubuntu) | [Scala Package for Ubuntu](http://mxnet.incubator.apache.org/install/ubuntu_setup.html#install-the-mxnet-package-for-scala) | -| Windows | [Shared Library for Windows](http://mxnet.incubator.apache.org/install/windows_setup.html#build-the-shared-library) | Call for Contribution | +| Windows | | Call for Contribution | #### Build Scala from an Existing MXNet Installation diff --git a/docs/install/ubuntu_setup.md b/docs/install/ubuntu_setup.md index bd1b441d5556..bf964182b50a 100644 --- a/docs/install/ubuntu_setup.md +++ b/docs/install/ubuntu_setup.md @@ -153,7 +153,9 @@ If building on CPU and using OpenBLAS: ```bash git clone --recursive https://github.com/apache/incubator-mxnet.git cd incubator-mxnet - make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas + echo "USE_OPENCV = 1" >> ./config.mk + echo "USE_BLAS = openblas" >> ./config.mk + make -j $(nproc) ``` If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md)): @@ -161,7 +163,11 @@ If building on CPU and using MKL and MKL-DNN (make sure MKL is installed accordi ```bash git clone --recursive https://github.com/apache/incubator-mxnet.git cd incubator-mxnet - make -j $(nproc) USE_OPENCV=1 USE_BLAS=mkl USE_MKLDNN=1 + echo "USE_OPENCV = 1" >> ./config.mk + echo "USE_BLAS = openblas" >> ./config.mk + echo "USE_CUDA = 0" >> ./config.mk + echo "USE_MKLDNN = 1" >> ./config.mk + make -j $(nproc) ``` If building on GPU and you want OpenCV and OpenBLAS (make sure you have installed the [CUDA dependencies first](#cuda-dependencies)): @@ -169,7 +175,12 @@ If building on GPU and you want OpenCV and OpenBLAS (make sure you have installe ```bash git clone --recursive https://github.com/apache/incubator-mxnet.git cd incubator-mxnet - make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 + echo "USE_OPENCV = 1" >> ./config.mk + echo "USE_BLAS = openblas" >> ./config.mk + echo "USE_CUDA = 1" >> ./config.mk + echo "USE_CUDA_PATH = /usr/local/cuda" >> ./config.mk + echo "USE_CUDNN = 1" >> ./config.mk + make -j $(nproc) ``` *Note* - USE_OPENCV and USE_BLAS are make file flags to set compilation options to use OpenCV and BLAS library. You can explore and use more compilation options in `make/config.mk` and also review common [usage examples](build_from_source.html#usage-examples). @@ -339,7 +350,9 @@ $ sudo apt-get install -y libopencv-dev ```bash $ git clone --recursive https://github.com/apache/incubator-mxnet $ cd incubator-mxnet -$ make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas +$ echo "USE_OPENCV = 1" >> ./config.mk +$ echo "USE_BLAS = openblas" >> ./config.mk +$ make -j $(nproc) ``` *Note* - USE_OPENCV and USE_BLAS are make file flags to set compilation options to use OpenCV and BLAS library. You can explore and use more compilation options in `make/config.mk`.