From 03b8905dae3f34746a1473fec37e15f3fd5531c3 Mon Sep 17 00:00:00 2001 From: Johnson Sun Date: Sun, 4 Jun 2023 04:19:04 +0800 Subject: [PATCH] Fix log-softmax unused issue Fixes: #800 --- beginner_source/transformer_tutorial.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/beginner_source/transformer_tutorial.py b/beginner_source/transformer_tutorial.py index cce52eefdb..2148af83a2 100644 --- a/beginner_source/transformer_tutorial.py +++ b/beginner_source/transformer_tutorial.py @@ -41,7 +41,10 @@ # the earlier positions in the sequence. For the language modeling task, any # tokens on the future positions should be masked. To produce a probability # distribution over output words, the output of the ``nn.TransformerEncoder`` -# model is passed through a linear layer followed by a log-softmax function. +# model is passed through a linear layer to output unnormalized logits. +# The log-softmax function isn't applied here due to the later use of +# `CrossEntropyLoss `__, +# which requires the inputs to be unnormalized logits. # import math