From 853af91b769b68e588b3b4faf32a0e7b31c690f8 Mon Sep 17 00:00:00 2001 From: Kaushik Bokka Date: Tue, 2 Mar 2021 14:22:29 +0530 Subject: [PATCH 1/2] add note for ddp spawn being default --- docs/source/advanced/multi_gpu.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/source/advanced/multi_gpu.rst b/docs/source/advanced/multi_gpu.rst index 439e62eb65ff3..41bd71b16c376 100644 --- a/docs/source/advanced/multi_gpu.rst +++ b/docs/source/advanced/multi_gpu.rst @@ -362,6 +362,9 @@ Distributed Data Parallel Spawn ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `ddp_spawn` is exactly like `ddp` except that it uses .spawn to start the training processes. +.. note:: + If you request multiple GPUs or nodes without setting a mode, DDP Spawn will be the default. + .. warning:: It is STRONGLY recommended to use `DDP` for speed and performance. .. code-block:: python From c6838923c8c9337119591768f2df29eae1927595 Mon Sep 17 00:00:00 2001 From: Kaushik Bokka Date: Sat, 20 Mar 2021 01:25:38 +0530 Subject: [PATCH 2/2] Update Docs --- docs/source/advanced/multi_gpu.rst | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/source/advanced/multi_gpu.rst b/docs/source/advanced/multi_gpu.rst index 41bd71b16c376..22a14bcf73be4 100644 --- a/docs/source/advanced/multi_gpu.rst +++ b/docs/source/advanced/multi_gpu.rst @@ -267,7 +267,7 @@ Lightning allows multiple ways of training - TPUs (``tpu_cores=8|x``) (tpu or TPU pod) .. note:: - If you request multiple GPUs or nodes without setting a mode, DDP will be automatically used. + If you request multiple GPUs or nodes without setting a mode, DDP Spawn will be automatically used. For a deeper understanding of what Lightning is doing, feel free to read this `guide `_. @@ -362,9 +362,6 @@ Distributed Data Parallel Spawn ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `ddp_spawn` is exactly like `ddp` except that it uses .spawn to start the training processes. -.. note:: - If you request multiple GPUs or nodes without setting a mode, DDP Spawn will be the default. - .. warning:: It is STRONGLY recommended to use `DDP` for speed and performance. .. code-block:: python