Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NCE loss using positional params - swapped #9

Open
RadekSadowski opened this issue Dec 3, 2019 · 0 comments
Open

NCE loss using positional params - swapped #9

RadekSadowski opened this issue Dec 3, 2019 · 0 comments

Comments

@RadekSadowski
Copy link

I have used the code to run in Python notebook. Tensorflow 1.12.

I love this piece of implementation - straightforward step-by-step logic of CNN. It is so much more transparent and easy to adjust than today's Keras models.

One thing that caused an issue was loss function definition. I have got this error of type mismatch int32 vs float32. I found out that input and output params in tf.nn.nce_loss were swapped compared to 1.12 documentation:
https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/nn/nce_loss

I wonder if your application worked because of change in tf api or maybe NCE loss is invariant in terms of input-output data? I could see people reported issue with exaggerate loss values - maybe it was related? Anyway, I have swapped them back and the code is now running.

So I suggest you adjust your code to use named parameters. Maybe it will save someone hustle in the future :)

self.loss = tf.reduce_mean(tf.nn.nce_loss(weights=softmax_w, 
                                                  biases=softmax_b, 
                                                  labels=self.y, 
                                                  inputs=h,
                                                  num_sampled=conf.num_sampled, 
                                                  num_classes=conf.vocab_size))

BR, Radek

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant