Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docstrings for backward methods #13886

Merged
merged 22 commits into from
Aug 3, 2022
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/pytorch_lightning/plugins/precision/apex_amp.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ def backward(
model: the model to be optimized
closure_loss: the loss value obtained from the closure
optimizer: current optimizer being used. ``None`` if using manual optimization
optimizer_idx: the index of the current optimizer. ``None`` if using manual optimization
"""
opt = optimizer or model.trainer.optimizers
with amp.scale_loss(closure_loss, opt) as closure_loss:
Expand Down
10 changes: 10 additions & 0 deletions src/pytorch_lightning/plugins/precision/deepspeed.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,16 @@ def backward(
*args: Any,
**kwargs: Any,
) -> None:
"""Performs back-propagatin using DeepSpeed's engine.

Args:
model: the model to be optimized
closure_loss: the loss tensor
optimizer: ignored for DeepSpeed
optimizer_idx: ignored for DeepSpeed
*args: additional positional arguments for the :meth:`deepspeed.DeepSpeedEngine.backward` call
*kwargs: additional keyword arguments for the :meth:`deepspeed.DeepSpeedEngine.backward` call
awaelchli marked this conversation as resolved.
Show resolved Hide resolved
"""
if is_overridden("backward", model):
warning_cache.warn(
"You have overridden the `LightningModule.backward` hook but it will be ignored since DeepSpeed handles"
Expand Down
3 changes: 3 additions & 0 deletions src/pytorch_lightning/plugins/precision/precision_plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ def backward(
model: the model to be optimized
closure_loss: the loss value obtained from the closure
optimizer: current optimizer being used. ``None`` if using manual optimization
awaelchli marked this conversation as resolved.
Show resolved Hide resolved
*args: Positional arguments intended for the actual function that performs the backward, like
:meth:`~torch.Tensor.backward`.
*kwargs: Keyword arguments for the same purpose as `*args`.
awaelchli marked this conversation as resolved.
Show resolved Hide resolved
"""
# do backward pass
if model is not None and isinstance(model, pl.LightningModule):
Expand Down
5 changes: 5 additions & 0 deletions src/pytorch_lightning/strategies/strategy.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,11 @@ def backward(

Args:
closure_loss: a tensor holding the loss value to backpropagate
optimizer: An optional optimizer that gets passed down to the precision plugin's backward
optimizer_idx: An optional optimizer index that gets passed down to the precision plugin's backward
*args: Positional arguments that get passed down to the precision plugin's backward, intended as arguments
for the actual function that performs the backward, like :meth:`~torch.Tensor.backward`.
*kwargs: Keyword arguments for the same purpose as `*args`.
awaelchli marked this conversation as resolved.
Show resolved Hide resolved
"""
self.pre_backward(closure_loss)
assert self.lightning_module is not None
Expand Down