the LeakyReLU layer after dense1

[Walstruzz]

Hi, thanks for your reimplementation. It is so elegant.

But I found that you have a LeakyReLU layer after dense1, which does not exist in the tensorflow implementation.

StyleGAN_PyTorch/networks_stylegan.py

Line 670 in 92b7a3d

x = F.leaky_relu(self.dense1(x), 0.2, inplace=True)

https://github.com/NVlabs/stylegan/blob/66813a32aac5045fcde72751522a0c0ba963f6f2/training/networks_stylegan.py#L617

I think that adding activation after the last fully connected layer is not a good idea.

[tomguluson92]

Thanks for pointing that problem, do you found that useful if removing this?