fairseq/examples/roberta/README.pretraining.md #1073
Comments
|
Can you please share the logs? |
|
Closing due to inactivity. Please open a new issue if you are still having problems. |
facebook-github-bot
pushed a commit
that referenced
this issue
Mar 8, 2020
…nt (#1073) Summary: [This commit](dd1298e) made it so that duplicate entries in a dictionary are ignored. Unfortunately the Camembert model depends on overwriting `<unk>`, `<s>` and `</s>`. The proposed solution here is to allow the dictionary to have entries like: ``` <unk> 999 #fairseq:overwrite <s> 999 #fairseq:overwrite </s> 999 #fairseq:overwrite , 999 ▁de 999 . 999 (...) ``` These will preserve the old overwriting behavior. Thus we can release a new `camembert.v0.tar.gz` with a dictionary like above and it works. Pull Request resolved: fairinternal/fairseq-py#1073 Reviewed By: kahne Differential Revision: D20284569 Pulled By: myleott fbshipit-source-id: bf78fbff13c94bf8a6485cbdda62305ddc30c056
louismartin
pushed a commit
to louismartin/fairseq
that referenced
this issue
Mar 24, 2020
…nt (facebookresearch#1073) Summary: [This commit](facebookresearch@dd1298e) made it so that duplicate entries in a dictionary are ignored. Unfortunately the Camembert model depends on overwriting `<unk>`, `<s>` and `</s>`. The proposed solution here is to allow the dictionary to have entries like: ``` <unk> 999 #fairseq:overwrite <s> 999 #fairseq:overwrite </s> 999 #fairseq:overwrite , 999 ▁de 999 . 999 (...) ``` These will preserve the old overwriting behavior. Thus we can release a new `camembert.v0.tar.gz` with a dictionary like above and it works. Pull Request resolved: fairinternal/fairseq-py#1073 Reviewed By: kahne Differential Revision: D20284569 Pulled By: myleott fbshipit-source-id: bf78fbff13c94bf8a6485cbdda62305ddc30c056
moussaKam
pushed a commit
to moussaKam/language-adaptive-pretraining
that referenced
this issue
Sep 29, 2020
…nt (facebookresearch#1073) Summary: [This commit](facebookresearch@dd1298e) made it so that duplicate entries in a dictionary are ignored. Unfortunately the Camembert model depends on overwriting `<unk>`, `<s>` and `</s>`. The proposed solution here is to allow the dictionary to have entries like: ``` <unk> 999 #fairseq:overwrite <s> 999 #fairseq:overwrite </s> 999 #fairseq:overwrite , 999 ▁de 999 . 999 (...) ``` These will preserve the old overwriting behavior. Thus we can release a new `camembert.v0.tar.gz` with a dictionary like above and it works. Pull Request resolved: fairinternal/fairseq-py#1073 Reviewed By: kahne Differential Revision: D20284569 Pulled By: myleott fbshipit-source-id: bf78fbff13c94bf8a6485cbdda62305ddc30c056
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I followed this tutorial for pretraining the Roberta on a 16GPUs machine, I found it didn't output a step result, like a loss, ppl etc. However, the GPUs and CPUs utilization both are 100%. Is there any solutions?
The text was updated successfully, but these errors were encountered: