Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Add PAI environment setup steps in DOC #2011

Merged
merged 42 commits into from
Feb 8, 2020
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
704b50e
Merge pull request #200 from microsoft/master
SparkSnail Aug 6, 2019
5b0034e
Merge pull request #204 from microsoft/master
SparkSnail Aug 20, 2019
8fe2588
Merge pull request #205 from microsoft/master
SparkSnail Aug 30, 2019
9fae194
Merge pull request #206 from microsoft/master
SparkSnail Sep 16, 2019
c785655
Merge pull request #207 from microsoft/master
SparkSnail Oct 21, 2019
2f5272c
Merge pull request #208 from microsoft/master
SparkSnail Oct 24, 2019
1892bc2
Merge pull request #209 from microsoft/master
SparkSnail Oct 28, 2019
7c1ab11
Merge pull request #210 from microsoft/master
SparkSnail Oct 28, 2019
8c203f3
Merge pull request #211 from microsoft/master
SparkSnail Oct 31, 2019
d7a62f6
check pylint for nni_cmd
SparkSnail Oct 31, 2019
e259d10
fix id error
SparkSnail Oct 31, 2019
4997295
Merge pull request #212 from microsoft/master
SparkSnail Nov 3, 2019
c037a7c
Merge pull request #213 from microsoft/master
SparkSnail Nov 10, 2019
7620e7c
Merge pull request #214 from microsoft/master
SparkSnail Nov 14, 2019
d16dbe9
Merge pull request #215 from microsoft/master
SparkSnail Nov 19, 2019
9ce751d
Merge pull request #216 from microsoft/master
SparkSnail Nov 21, 2019
a0846f2
Merge pull request #217 from microsoft/master
SparkSnail Nov 22, 2019
cd3a912
Merge pull request #218 from microsoft/master
SparkSnail Nov 27, 2019
32efaa3
Merge pull request #219 from microsoft/master
SparkSnail Dec 10, 2019
543239c
Merge pull request #220 from microsoft/master
SparkSnail Dec 12, 2019
36e6e35
Merge pull request #221 from microsoft/master
SparkSnail Dec 19, 2019
f9ee589
Merge pull request #222 from microsoft/master
SparkSnail Dec 24, 2019
b9a7a95
Merge pull request #223 from microsoft/master
SparkSnail Dec 25, 2019
1a5c017
Merge pull request #224 from microsoft/master
SparkSnail Jan 6, 2020
392460a
Merge pull request #225 from microsoft/master
SparkSnail Jan 8, 2020
9bafa4c
Merge pull request #226 from microsoft/master
SparkSnail Jan 8, 2020
c23b807
Merge pull request #227 from microsoft/master
SparkSnail Jan 10, 2020
4132f62
Merge pull request #228 from microsoft/master
SparkSnail Jan 10, 2020
4c7ae00
add foreground mode
SparkSnail Jan 14, 2020
9b5acf3
remove pass
SparkSnail Jan 14, 2020
a4159da
fix windows pipe
SparkSnail Jan 14, 2020
fb0af3b
fix pylint
SparkSnail Jan 14, 2020
536a9cb
fix pylint
SparkSnail Jan 14, 2020
4f66d0c
Merge pull request #229 from microsoft/master
SparkSnail Feb 1, 2020
129c4a5
Merge pull request #230 from microsoft/master
SparkSnail Feb 4, 2020
954a1b1
fix comments
Feb 6, 2020
26a000b
fix eslint
Feb 6, 2020
8a94c33
Merge branch 'master' of https://github.com/SparkSnail/nni into dev-n…
Feb 6, 2020
2f644f0
add doc for PAI
Feb 7, 2020
3fe117f
Merge pull request #231 from microsoft/master
SparkSnail Feb 7, 2020
25a6394
Merge branch 'master' of https://github.com/SparkSnail/nni into dev-f…
Feb 7, 2020
65e0c27
fix comments
Feb 7, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion docs/en_US/TrainingService/PaiMode.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,33 @@
NNI supports running an experiment on [OpenPAI](https://github.com/Microsoft/pai) (aka pai), called pai mode. Before starting to use NNI pai mode, you should have an account to access an [OpenPAI](https://github.com/Microsoft/pai) cluster. See [here](https://github.com/Microsoft/pai#how-to-deploy) if you don't have any OpenPAI account and want to deploy an OpenPAI cluster. In pai mode, your trial program will run in pai's container created by Docker.

## Setup environment
Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
Step 1. Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to point out that the setup steps is for paiyarn or paiK8s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this doc is only for paik8s, do not need to point out again. paiYarn has another doc paiYarnMode.md.

Step 2. Get PAI token.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest adding an extra blank line between steps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

Click `My profile` button in the top-right side of PAI's webprotal.
![](../../img/pai_token_button.jpg)
Find the token management region, copy one of the token as your account token.
![](../../img/pai_token_profile.jpg)
Step 3. Mount NFS storage to local machine.
Click `Submit job` button in PAI's webportal.
![](../../img/pai_job_submission_page.jpg)
Find the data management region in job submission page.
![](../../img/pai_data_management_page.jpg)
The `DEFAULT_STORAGE`field is the path to be mounted in PAI's container when a job is started. The `Preview container paths` is the NFS host and path that PAI provided, you need to mount the corresponding host and path to your local machine first, then NNI could use the PAI's NFS storage.
For example, use the following command:
```
sudo mount nfs://gcr-openpai-infra02:/pai/data /local/mnt
```
Then the `/data` folder in container will be mounted to `/local/mnt` folder in your local machine.
You could use the following configuration in your NNI's config file:
```
nniManagerNFSMountPath: /local/mnt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra indent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

containerNFSMountPath: /data
```
Step 4. Get PAI's storage plugin name.
Contact PAI's admin, and get the PAI's storage plugin name for NFS storage. The default storage name is `teamwise_storage`, the configuration in NNI's config file is in following value:
```
paiStoragePlugin: teamwise_storage
```

## Run an experiment
Use `examples/trials/mnist-annotation` as an example. The NNI config YAML file's content is like:
Expand Down
Binary file added docs/img/pai_data_management_page.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/pai_job_submission_page.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/pai_token_button.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/pai_token_profile.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.