Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added ESRGAN Example #47

Merged
merged 21 commits into from
Sep 7, 2021
Merged

Added ESRGAN Example #47

merged 21 commits into from
Sep 7, 2021

Conversation

nivedwho
Copy link
Contributor

Added python script files for training and evaluating the ESRGAN model.

@google-cla google-cla bot added the cla: yes to pull into main branch, I think? label Aug 18, 2021
joel-shor
joel-shor suggested changes Aug 25, 2021
View reviewed changes
Copy link
Contributor

@joel-shor joel-shor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure variable names follow the style guide: https://google.github.io/styleguide/pyguide.html#3162-naming-conventions

tensorflow_gan/examples/esrgan/README.md Show resolved Hide resolved
tensorflow_gan/examples/esrgan/data_provider.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/data_provider.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/data_provider.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/data_provider_test.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/train_lib.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/train_test.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/utils.py Show resolved Hide resolved
tensorflow_gan/examples/esrgan/utils.py Outdated Show resolved Hide resolved
tensorflow_gan/examples/esrgan/utils_test.py Outdated Show resolved Hide resolved
@nivedwho
Update Link
042af79
@nivedwho
Update data_provider.py
99affbe 
Variable name changes, added 'name' argument for `get_div2k_data()`, used `AUTOTUNE` for parallel calls.
@nivedwho
Update data_provider_test.py
cbfbc2d 
Variable name changes, removed unnecessary imports,  moved `assertIsInstance()`
@nivedwho
Update eval_lib.py
339896f 
Variable name changes and fixed spacing
@nivedwho
Update eval_test.py
efb0250 
Variable name changes, removed unnecessary imports
@nivedwho
Update losses.py
a5425f4 
Added a TODO comment to remove the two loss functions once TF-GAN losses are updated.
@nivedwho
Update losses_test.py
71f58ff 
Removed unnecessary imports
@nivedwho
Update networks.py
f96701e 
Variable name changes and removed extra imports
@nivedwho
Update networks_test.py
6f33f60 
Variable name changes and removed unnecessary imports.
@nivedwho
Update train_lib.py
cd2b712 
Variable name changes, removed `list.pop()`, fixed imports and added default arguments for `_get_optimizer()`
@nivedwho
Update train_test.py
f803538 
Removed unnecessary imports
@nivedwho
Update utils.py
391920d 
Added explanation for `preprocess_input()`
@nivedwho
Update utils_test.py
bdd33c6 
Removed unnecessary imports
@joel-shor
Copy link
Contributor

A few things:

  1. Class names need to be capitalized, variables need to be lowercase. You made classnames lowercase, and they should be capitalized (hparams vs HParams)

  2. Docstrings shouldn't start with spaces

  3. Every file needs to have a docstring.

  4. Docstrings need to end with periods.

  5. Function docstrings need to start with a single line, ending with a period.

@joel-shor joel-shor self-requested a review August 28, 2021 19:28
joel-shor
joel-shor suggested changes Aug 28, 2021
View reviewed changes
Copy link
Contributor

@joel-shor joel-shor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see PR-level comments

joel-shor
joel-shor suggested changes Aug 29, 2021
View reviewed changes
Copy link
Contributor

@joel-shor joel-shor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a number of lint issues. You can ignore the "invalid import order" errors, but please fix the rest:

CheckLint/0: Warning (//depot/google3/third_party/py/tensorflow_gan/METADATA:113:3)
CheckLint found errors.
Missing call to super() for method setUp [g-missing-super-call]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks_test.py:13-16:3
Invalid import order.
Diff:
import collections

  • import networks
    import tensorflow as tf
  • import networks
    Copy and pastable ordering:
    import collections

import networks
import tensorflow as tf
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks_test.py:5:1
Multiple import [g-multiple-import]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train.py:3:1
Invalid import order.
Diff:
from absl import flags, logging, app

  • import data_provider
    import train_lib
  • import data_provider
    Copy and pastable ordering:
    from absl import flags, logging, app
    import data_provider
    import train_lib
    [g-bad-import-order]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train.py:5:1
    One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:1:1
    Use "import tensorflow as tf" directly instead. [g-direct-tensorflow-import]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:9:1
    Missing function or method docstring [missing-function-docstring]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:12-24:1
    Redefining built-in 'input' [redefined-builtin]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:12:17
    Redefining built-in 'input' [redefined-builtin]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:27:19
    Missing function or method docstring [missing-function-docstring]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:38-56:1
    Redefining built-in 'input' [redefined-builtin]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:38:17
    Redefining built-in 'input' [redefined-builtin]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:59:10
    One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:77:1
    Args section is missing, does not match arg list or is badly indented.

Expected: hparams, num_filters, out_channels

Found: num_filters, out_channels, use_bias

Missing: hparams

Unexpected: use_bias
[g-doc-args]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:76:1
Args section is missing, does not match arg list or is badly indented.

Expected: hparams, num_filters, training

Found: num_filters

Missing: hparams, training

Unexpected:
[g-doc-args]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:142:1
Unused argument 'training' [unused-argument]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/networks.py:142:52
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval_test.py:1:1
Missing call to super() for method setUp [g-missing-super-call]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval_test.py:17-29:3
Invalid import order.
Diff:
import collections

  • import tensorflow as tf
    import eval_lib
    import networks
  • import tensorflow as tf
    Copy and pastable ordering:
    import collections

import eval_lib
import networks
import tensorflow as tf
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval_test.py:6:1
There should not be a space before a one-line docstring summary [g-space-before-docstring-summary]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils_test.py:28:1
Extraneous space in one-line docstring [g-short-docstring-space]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils_test.py:51:1
Invalid import order.
Diff:
import collections

  • import networks
    import tensorflow as tf
    import utils
  • import networks
    Copy and pastable ordering:
    import collections

import networks
import tensorflow as tf
import utils
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils_test.py:6:1
Use "import tensorflow as tf" directly instead. [g-direct-tensorflow-import]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/data_provider.py:5:1
'lambda' may be too long [g-long-lambda]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/data_provider.py:13-14:47
Unused argument 'args' [unused-argument]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/data_provider.py:45:1
Invalid TODO format. Should be # TODO(username) or # TODO(b/buganizer_id). [g-bad-todo]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:157:7
No space after start of comment [g-no-space-after-comment]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:233:3
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:21:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:47:1
Use % formatting in logging functions and pass the % parameters as arguments [logging-format-interpolation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:81-82:20
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:96:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:137:1
Args section is missing, does not match arg list or is badly indented.

Expected: image_lr, image_hr, step

Found: image_lr, image_hr

Missing: step

Unexpected:
[g-doc-args]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:136:1
Variable name "loss_RaG" doesn't conform to '^[a-z][a-z0-9_]*$' pattern [invalid-name]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:158:7
Unused argument 'step' [unused-argument]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:136:38
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:180:1
Use % formatting in logging functions and pass the % parameters as arguments [logging-format-interpolation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:210-212:11
Invalid import order.
Diff:

  • import collections
    import os
  • import collections
  • from absl import logging
  • import losses
  • import networks
    import tensorflow as tf
  • from absl import logging
  • import networks
  • import losses
    import utils
    Copy and pastable ordering:
    import collections
    import os

from absl import logging
import losses
import networks
import tensorflow as tf
import utils
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_lib.py:4:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval_lib.py:15:1
Use % formatting in logging functions and pass the % parameters as arguments [logging-format-interpolation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval_lib.py:56-57:16
Invalid import order.
Diff:
import collections

  • from absl import logging
    import tensorflow as tf
  • from absl import logging
    import utils
    Copy and pastable ordering:
    import collections

from absl import logging
import tensorflow as tf
import utils
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval_lib.py:5:1
Missing Returns section in docstring, or wrong choice of Returns/Yields [g-doc-return-or-yield]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils.py:12:1
Args section is missing, does not match arg list or is badly indented.

Expected: image

Found:

Missing: image

Unexpected:
[g-doc-args]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils.py:11:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils.py:33:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils.py:71:1
Invalid import order.
Diff:
import os

  • from absl import logging
  • import numpy as np
  • import PIL
    import tensorflow as tf
    import tensorflow_gan as tfgan
  • from absl import logging
  • import PIL
  • import numpy as np
    Copy and pastable ordering:
    import os

from absl import logging
import numpy as np
import PIL
import tensorflow as tf
import tensorflow_gan as tfgan
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils.py:6:1
Unused logging imported from absl [unused-import]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/utils.py:6:1
Invalid import order.
Diff:
import os
from absl.testing import absltest

  • import losses
    import tensorflow as tf
  • import losses
    Copy and pastable ordering:
    import os

from absl.testing import absltest
import losses
import tensorflow as tf
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses_test.py:7:1
Unused import os [unused-import]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses_test.py:3:1
Invalid import order.
Diff:
import collections
from absl.testing import absltest

  • import data_provider
    import tensorflow as tf
  • import data_provider
    Copy and pastable ordering:
    import collections

from absl.testing import absltest
import data_provider
import tensorflow as tf
[g-bad-import-order]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/data_provider_test.py:6:1
Line too long (83/80) [line-too-long]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:20:1
Line too long (83/80) [line-too-long]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:54:1
Invalid TODO format. Should be # TODO(username) or # TODO(b/buganizer_id). [g-bad-todo]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:20:1
Invalid TODO format. Should be # TODO(username) or # TODO(b/buganizer_id). [g-bad-todo]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:54:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:22:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:56:1
One-line docstring summary should end with ".", ".)", "?", or "!" [g-short-docstring-punctuation]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:89:1
Missing Returns section in docstring, or wrong choice of Returns/Yields [g-doc-return-or-yield]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:89:1
Args section is missing, does not match arg list or is badly indented.

Expected: weight, input_shape

Found: weights, input_shape

Missing: weight

Unexpected: weights
[g-doc-args]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/losses.py:88:1
Missing call to super() for method setUp [g-missing-super-call]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_test.py:12-26:3
Unused import collections [unused-import]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/train_test.py:3:1
Multiple import [g-multiple-import]
//depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval.py:3:1
Invalid import order.
Diff:
from absl import flags, logging, app

  • import data_provider
  • import eval_lib
    import tensorflow as tf
  • import eval_lib
  • import data_provider
    Copy and pastable ordering:
    from absl import flags, logging, app
    import data_provider
    import eval_lib
    import tensorflow as tf
    [g-bad-import-order]
    //depot/google3/third_party/py/tensorflow_gan/examples/esrgan/eval.py:5:1

@copybara-service copybara-service bot merged commit 1291afb into tensorflow:master Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joel-shor joel-shor Awaiting requested review from joel-shor

Assignees
No one assigned
Labels
cla: yes to pull into main branch, I think?
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nivedwho @joel-shor