Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-1225] Always use config.mk in make install instructions #13364

Merged
merged 6 commits into from
Dec 11, 2018
Merged

[MXNET-1225] Always use config.mk in make install instructions #13364

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2f2f24e
Always use config.mk in make install instructions
zachgk Nov 21, 2018
d2970bb
Specify Cuda 0 for ubuntu with mkldnn
zachgk Nov 28, 2018
a332c3c
Scala install doc avoid build_from_source
zachgk Dec 5, 2018
f050321
Fix build_from_source CMake usage
zachgk Dec 6, 2018
e5ffd32
CPP Install Instruction with CMake
zachgk Dec 7, 2018
1923114
Use cmake out of source build
zachgk Dec 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions docs/install/build_from_source.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

This document explains how to build MXNet from source code.

**For Java/Scala/Clojure, please follow [this guide instead](./scala_setup.md)**

## Overview

Expand All @@ -27,14 +28,14 @@ MXNet's newest and most popular API is Gluon. Gluon is built into the Python bin
- [Python (includes Gluon)](../api/python/index.html)
- [C++](../api/c++/index.html)
- [Clojure](../api/clojure/index.html)
- Java (coming soon)
- [Julia](../api/julia/index.html)
- [Perl](../api/perl/index.html)
- [R](../api/r/index.html)
- [Scala](../api/scala/index.html)
- [Java](../api/java/index.html)

<hr>

## Build Instructions by Operating System

Detailed instructions are provided per operating system. Each of these guides also covers how to install the specific [Language Bindings](#installing-mxnet-language-bindings) you require.
Expand Down Expand Up @@ -160,7 +161,7 @@ More information on turning these features on or off are found in the following
## Build Configurations

There is a configuration file for make,
[`make/config.mk`](https://github.com/apache/incubator-mxnet/blob/master/make/config.mk), that contains all the compilation options. You can edit it and then run `make` or `cmake`. `cmake` is recommended for building MXNet (and is required to build with MKLDNN), however you may use `make` instead.
[`make/config.mk`](https://github.com/apache/incubator-mxnet/blob/master/make/config.mk), that contains all the compilation options. You can edit it and then run `make` or `cmake`. `cmake` is recommended for building MXNet (and is required to build with MKLDNN), however you may use `make` instead. For building with Java/Scala/Clojure, only `make` is supported.

<hr>

Expand Down Expand Up @@ -203,68 +204,74 @@ It is recommended to set environment variable NCCL_LAUNCH_MODE to PARALLEL when

### Build MXNet with C++

* To enable C++ package, just add `USE_CPP_PACKAGE=1` when you run `make` or `cmake`.
* To enable C++ package, just add `USE_CPP_PACKAGE=1` when you run `make` or `cmake` (see examples).

<hr>

### Usage Examples

* `-j` runs multiple jobs against multi-core CPUs.

For example, you can specify using all cores on Linux as follows:

```bash
cmake -j$(nproc)
cmake -GNinja .
ninja -v
zachgk marked this conversation as resolved.
Show resolved Hide resolved
```


#### Recommended for Systems with NVIDIA GPUs and Intel CPUs
* Build MXNet with `cmake` and install with MKL DNN, GPU, and OpenCV support:

```bash
cmake -j USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 USE_MKLDNN=1
cmake -DUSE_CUDA=1 -DUSE_CUDA_PATH=/usr/local/cuda -DUSE_CUDNN=1 -DUSE_MKLDNN=1 -GNinja .
ninja -v
```

#### Recommended for Systems with NVIDIA GPUs
* Build with both OpenBLAS, GPU, and OpenCV support:

```bash
cmake -j BLAS=open USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1
cmake -DBLAS=open -DUSE_CUDA=1 -DUSE_CUDA_PATH=/usr/local/cuda -DUSE_CUDNN=1 -GNinja .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does ninja do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a build system (https://ninja-build.org/). I believe CMake generates the ninja configuration files and then ninja performs the actual build.

ninja -v
```

#### Recommended for Systems with Intel CPUs
* Build MXNet with `cmake` and install with MKL DNN, and OpenCV support:

```bash
cmake -j USE_CUDA=0 USE_MKLDNN=1
cmake -DUSE_CUDA=0 -DUSE_MKLDNN=1 -GNinja .
ninja -v
```

#### Recommended for Systems with non-Intel CPUs
* Build MXNet with `cmake` and install with OpenBLAS and OpenCV support:

```bash
cmake -j USE_CUDA=0 BLAS=open
cmake -DUSE_CUDA=0 -DBLAS=open -GNinja .
ninja -v
```

#### Other Examples

* Build without using OpenCV:

```bash
cmake USE_OPENCV=0
cmake -DUSE_OPENCV=0 -GNinja .
ninja -v
```

* Build on **macOS** with the default BLAS library (Apple Accelerate) and Clang installed with `xcode` (OPENMP is disabled because it is not supported by the Apple version of Clang):

```bash
cmake -j BLAS=apple USE_OPENCV=0 USE_OPENMP=0
cmake -DBLAS=apple -DUSE_OPENCV=0 -DUSE_OPENMP=0 -GNinja .
ninja -v
```

* To use OpenMP on **macOS** you need to install the Clang compiler, `llvm` (the one provided by Apple does not support OpenMP):

```bash
brew install llvm
cmake -j BLAS=apple USE_OPENMP=1
cmake -DBLAS=apple -DUSE_OPENMP=1 -GNinja .
ninja -v
```

<hr>
Expand Down
3 changes: 2 additions & 1 deletion docs/install/c_plus_plus.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ To enable C++ package, just add `USE_CPP_PACKAGE=1` in the [build from source](b
For example to build MXNet with GPU support and the C++ package, OpenCV, and OpenBLAS, from the project root you would run:

```bash
make -j USE_CPP_PACKAGE=1 USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1
cmake -DUSE_CUDA=1 -DUSE_CUDA_PATH=/usr/local/cuda -DUSE_CUDNN=1 -DUSE_MKLDNN=1 -DUSE_CPP_PACKAGE=1 -GNinja .
ninja -v
```

You may also want to add the MXNet shared library to your `LD_LIBRARY_PATH`:
Expand Down
4 changes: 3 additions & 1 deletion docs/install/java_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,11 +89,13 @@ The official Java Packages will be released with the release of MXNet 1.4 and wi

The previously mentioned setup with Maven is recommended. Otherwise, the following instructions for macOS and Ubuntu are provided for reference only:

**If you have already built mxnet from source using `cmake`, run `make clean` and then follow the appropriate guide below***

| OS | Step 1 | Step 2 |
|---|---|---|
|macOS | [Shared Library for macOS](../install/osx_setup.html#build-the-shared-library) | [Scala Package for macOS](http://mxnet.incubator.apache.org/install/osx_setup.html#install-the-mxnet-package-for-scala) |
| Ubuntu | [Shared Library for Ubuntu](../install/ubuntu_setup.html#installing-mxnet-on-ubuntu) | [Scala Package for Ubuntu](http://mxnet.incubator.apache.org/install/ubuntu_setup.html#install-the-mxnet-package-for-scala) |
| Windows | [Shared Library for Windows](../install/windows_setup.html#build-the-shared-library) | <a class="github-button" href="https://github.com/apache/incubator-mxnet/issues/10549" data-size="large" data-show-count="true" aria-label="Issue apache/incubator-mxnet on GitHub">Call for Contribution</a> |
| Windows | <a class="github-button" href="https://github.com/apache/incubator-mxnet/issues/10549" data-size="large" data-show-count="true" aria-label="Issue apache/incubator-mxnet on GitHub"> | <a class="github-button" href="https://github.com/apache/incubator-mxnet/issues/10549" data-size="large" data-show-count="true" aria-label="Issue apache/incubator-mxnet on GitHub">Call for Contribution</a> |


#### Build Java from an Existing MXNet Installation
Expand Down
9 changes: 8 additions & 1 deletion docs/install/osx_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,14 @@ The file called ```osx.mk``` has the configuration required for building MXNet o
To build with MKLDNN

```bash
LIBRARY_PATH=$(brew --prefix llvm)/lib/ make -j $(sysctl -n hw.ncpu) CC=$(brew --prefix llvm)/bin/clang++ CXX=$(brew --prefix llvm)/bin/clang++ USE_OPENCV=1 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple USE_PROFILER=1
echo "CC=$(brew --prefix llvm)/bin/clang++" >> ./config.mk
echo "CXX=$(brew --prefix llvm)/bin/clang++" >> ./config.mk
echo "USE_OPENCV=1" >> ./config.mk
echo "USE_OPENMP=1" >> ./config.mk
echo "USE_MKLDNN=1" >> ./config.mk
echo "USE_BLAS=apple" >> ./config.mk
echo "USE_PROFILER=1" >> ./config.mk
LIBRARY_PATH=$(brew --prefix llvm)/lib/ make -j $(sysctl -n hw.ncpu)
```

If building with ```GPU``` support, add the following configuration to config.mk and build:
Expand Down
4 changes: 3 additions & 1 deletion docs/install/scala_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,13 @@ https://mvnrepository.com/artifact/org.apache.mxnet

The previously mentioned setup with Maven is recommended. Otherwise, the following instructions for macOS, Ubuntu, and Windows are provided for reference only:

**If you have already built mxnet from source using `cmake`, run `make clean` and then follow the appropriate guide below***

| OS | Step 1 | Step 2 |
|---|---|---|
|macOS | [Shared Library for macOS](http://mxnet.incubator.apache.org/install/osx_setup.html#build-the-shared-library) | [Scala Package for macOS](http://mxnet.incubator.apache.org/install/osx_setup.html#install-the-mxnet-package-for-scala) |
| Ubuntu | [Shared Library for Ubuntu](http://mxnet.incubator.apache.org/install/ubuntu_setup.html#installing-mxnet-on-ubuntu) | [Scala Package for Ubuntu](http://mxnet.incubator.apache.org/install/ubuntu_setup.html#install-the-mxnet-package-for-scala) |
| Windows | [Shared Library for Windows](http://mxnet.incubator.apache.org/install/windows_setup.html#build-the-shared-library) | <a class="github-button" href="https://github.com/apache/incubator-mxnet/issues/10549" data-size="large" data-show-count="true" aria-label="Issue apache/incubator-mxnet on GitHub">Call for Contribution</a> |
| Windows | <a class="github-button" href="https://github.com/apache/incubator-mxnet/issues/10549" data-size="large" data-show-count="true" aria-label="Issue apache/incubator-mxnet on GitHub"> | <a class="github-button" href="https://github.com/apache/incubator-mxnet/issues/10549" data-size="large" data-show-count="true" aria-label="Issue apache/incubator-mxnet on GitHub">Call for Contribution</a> |


#### Build Scala from an Existing MXNet Installation
Expand Down
21 changes: 17 additions & 4 deletions docs/install/ubuntu_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,23 +165,34 @@ If building on CPU and using OpenBLAS:
```bash
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
make -j $(nproc)
zachgk marked this conversation as resolved.
Show resolved Hide resolved
```

If building on CPU and using MKL and MKL-DNN (make sure MKL is installed according to [Math Library Selection](build_from_source.html#math-library-selection) and [MKL-DNN README](https://github.com/apache/incubator-mxnet/blob/master/MKLDNN_README.md)):

```bash
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
make -j $(nproc) USE_OPENCV=1 USE_BLAS=mkl USE_MKLDNN=1
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
echo "USE_CUDA = 0" >> ./config.mk
echo "USE_MKLDNN = 1" >> ./config.mk
make -j $(nproc)
```

If building on GPU and you want OpenCV and OpenBLAS (make sure you have installed the [CUDA dependencies first](#cuda-dependencies)):

```bash
git clone --recursive https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1
echo "USE_OPENCV = 1" >> ./config.mk
echo "USE_BLAS = openblas" >> ./config.mk
echo "USE_CUDA = 1" >> ./config.mk
echo "USE_CUDA_PATH = /usr/local/cuda" >> ./config.mk
echo "USE_CUDNN = 1" >> ./config.mk
make -j $(nproc)
```

*Note* - USE_OPENCV and USE_BLAS are make file flags to set compilation options to use OpenCV and BLAS library. You can explore and use more compilation options in `make/config.mk` and also review common [usage examples](build_from_source.html#usage-examples).
Expand Down Expand Up @@ -351,7 +362,9 @@ $ sudo apt-get install -y libopencv-dev
```bash
$ git clone --recursive https://github.com/apache/incubator-mxnet
$ cd incubator-mxnet
$ make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas
$ echo "USE_OPENCV = 1" >> ./config.mk
$ echo "USE_BLAS = openblas" >> ./config.mk
$ make -j $(nproc)
```

*Note* - USE_OPENCV and USE_BLAS are make file flags to set compilation options to use OpenCV and BLAS library. You can explore and use more compilation options in `make/config.mk`.
Expand Down