[Example] Add Pytorch Geometric Example #4568

tchaton · 2020-11-07T14:54:09Z

What does this PR do?

This PR adds Pytorch Geometric Examples with Lightning.

Fixes # (issue)

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?
Did you verify new and existing tests pass locally with your changes?
If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In in short, see following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified; Bugfixes should be including in bug-fix release milestones (m.f.X) and features should be included in (m.X.b) releases.

Did you have fun?

Make sure you had fun coding 🙃

pep8speaks · 2020-11-07T14:54:13Z

Hello @tchaton! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-11-18 16:50:21 UTC

tchaton · 2020-11-07T15:18:11Z

codecov · 2020-11-07T20:43:05Z

Codecov Report

Merging #4568 (1477eb0) into master (e7134a9) will decrease coverage by 0%.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #4568   +/-   ##
======================================
- Coverage      93%     93%   -0%     
======================================
  Files         117     117           
  Lines        8949    8949           
======================================
- Hits         8319    8316    -3     
- Misses        630     633    +3

pl_examples/pytorch_ecosystem/pytorch_geometric/README.md

pl_examples/pytorch_ecosystem/pytorch_geometric/poetry.lock

…ing/pytorch-lightning into ecosystem_examples

SeanNaren

I'm hoping that lightning logo will make its way into the trainer, would be neat :)

Borda

minor formating

pl_examples/pytorch_ecosystem/pytorch_geometric/README.md

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

teddykoker

LGTM, cool example

Borda

can we pls add these: @tchaton

needed packages to requirements/examples.txt
test run, even GPU only, how large is the dataset?

Borda · 2020-11-18T19:48:14Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+    from torch_geometric.data import NeighborSampler
+    from lightning import lightning_logo, nice_print
+except Exception:
+    HAS_PYTORCH_GEOMETRIC = False


Suggested change

HAS_PYTORCH_GEOMETRIC = False

PYTORCH_GEOMETRIC_AVALAIBLE = False

to be the same as everywhere

Borda · 2020-11-18T19:50:54Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+
+    def prepare_data(self):
+        path = osp.join(
+            osp.dirname(osp.realpath(__file__)), "..", "..", "data", self.NAME


Let's move this path to the module top as constant, bc if you move this file/package one level up it does not work...
or see and use similar as in tests.__init__ var PACKAGE_ROOT

Borda · 2020-11-18T19:51:33Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+        self.dataset = Planetoid(path, self.NAME, transform=self._transform)
+        self.data = self.dataset[0]
+
+    def create_neighbor_sampler(self, batch_size=2, stage=None):


shall we ad types or just create an issue for adding later?
it would be useful to add docs here as it seems to be very specific step

Borda · 2020-11-18T19:52:54Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+        and create a NamedTuple Object.
+        """
+
+        usual_keys = ["x", "edge_index", "edge_attr", "batch"]


can we rather get names from the TensorBatch definition?

Borda · 2020-11-18T19:54:13Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+        Batch: TensorBatch = namedtuple("Batch", usual_keys)
+        return (
+            Batch(
+                self.data.x[batch[1]],
+                [e.edge_index for e in batch[2]],
+                None,
+                None,
+            ),
+            self.data.y[batch[1]],
+        )


I am sorry, just not sure about this notation, not sure what it really does...

Borda · 2020-11-18T19:56:18Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+
+        # utils from Lightning to save __init__ arguments
+        self.save_hyperparameters()
+        hparams = self.hparams


any special reason? :]

Borda · 2020-11-18T19:57:55Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+        return F.log_softmax(self.lin2(x), -1)
+
+    def step(self, batch, batch_nb):
+        typed_batch, targets = self.gather_data_and_convert_to_namedtuple(batch, batch_nb)


Suggested change

typed_batch, targets = self.gather_data_and_convert_to_namedtuple(batch, batch_nb)

typed_batch, targets = self._gather_data_and_convert_to_namedtuple(batch, batch_nb)

is this good as public API?

update: I see later that it assigned from outside, then it is a but confusion, can we make a placeholder in init as

self.gather_data_and_convert_to_namedtuple = ...

Borda · 2020-11-18T19:58:43Z

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

+        parser.add_argument("--groups", type=int, default=16)
+        parser.add_argument("--dropout", type=float, default=0.8)
+        parser.add_argument("--cached", type=int, default=0)
+        parser.add_argument("--jit", default=True)


Suggested change

parser.add_argument("--jit", default=True)

parser.add_argument("--jit", type=bool, default=True)

Borda · 2020-11-18T20:01:44Z

pl_examples/pytorch_ecosystem/pytorch_geometric/lightning.py

@@ -0,0 +1,31 @@
+def nice_print(msg, last=False):


Let's move this to PL_examples root?

Borda · 2020-11-18T20:02:16Z

pl_examples/pytorch_ecosystem/pytorch_geometric/pyproject.toml

+name = "lightning-geometric"
+version = "0.1.0"
+description = "TorchScripted Pytorch Geometric Examples with Pytorch Lightning"
+authors = ["Thomas Chaton <thomas.ai@grid.com>"]


still the mail here...

Suggested change

authors = ["Thomas Chaton <thomas.ai@grid.com>"]

authors = ["Thomas Chaton <thomas@pytorchlightning.ai>"]

* add example for Pytorch Geometric * remove hydra * add docstring * remove description * rename folder * update script to not break test * remove .lock * add Pytorch Geometric to doc * add docstring at the begining * add comments * Update pl_examples/pytorch_ecosystem/pytorch_geometric/README.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * Update pl_examples/pytorch_ecosystem/pytorch_geometric/README.md Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * Update pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> * add toml Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>

Borda · 2020-12-11T11:38:06Z

some fixes come in #5085

add example for Pytorch Geometric

115cc62

tchaton added the example label Nov 7, 2020

tchaton added this to the 1.1 milestone Nov 7, 2020

tchaton requested review from ananyahjha93, awaelchli, Borda, justusschock, nateraw, SeanNaren, teddykoker and williamFalcon as code owners November 7, 2020 14:54

tchaton self-assigned this Nov 7, 2020

tchaton added 4 commits November 7, 2020 14:55

remove hydra

cd595f4

add docstring

d2500ce

remove description

3f68072

rename folder

2c054c0

tchaton added 4 commits November 7, 2020 15:27

update script to not break test

dd1093a

Merge branch 'master' into ecosystem_examples

ccbc95e

remove .lock

6145bd9

Merge branch 'master' into ecosystem_examples

c0ca670

Borda reviewed Nov 8, 2020

View reviewed changes

pl_examples/pytorch_ecosystem/pytorch_geometric/README.md Show resolved Hide resolved

pl_examples/pytorch_ecosystem/pytorch_geometric/poetry.lock Outdated Show resolved Hide resolved

tchaton added 5 commits November 9, 2020 08:01

Merge branch 'master' into ecosystem_examples

ed8b9e0

Merge branch 'ecosystem_examples' of https://github.com/PyTorchLightn…

c5b8658

…ing/pytorch-lightning into ecosystem_examples

add Pytorch Geometric to doc

e7747da

add docstring at the begining

3f0df39

add comments

291764c

Merge branch 'master' into ecosystem_examples

8a5623e

SeanNaren self-requested a review November 9, 2020 17:22

SeanNaren approved these changes Nov 9, 2020

View reviewed changes

tchaton added 9 commits November 9, 2020 17:23

Merge branch 'master' into ecosystem_examples

83c9deb

Merge branch 'master' into ecosystem_examples

9e58b36

Merge branch 'master' into ecosystem_examples

8fd078e

Merge branch 'master' into ecosystem_examples

5947123

Merge branch 'master' into ecosystem_examples

ea06b28

Merge branch 'master' into ecosystem_examples

75cd4c4

Merge branch 'master' into ecosystem_examples

e653101

Merge branch 'master' into ecosystem_examples

4a0a502

Merge branch 'master' into ecosystem_examples

104e096

Borda reviewed Nov 13, 2020

View reviewed changes

pl_examples/pytorch_ecosystem/pytorch_geometric/README.md Outdated Show resolved Hide resolved

pl_examples/pytorch_ecosystem/pytorch_geometric/README.md Outdated Show resolved Hide resolved

pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py Outdated Show resolved Hide resolved

tchaton and others added 8 commits November 14, 2020 09:34

Merge branch 'master' into ecosystem_examples

3de0cef

Update pl_examples/pytorch_ecosystem/pytorch_geometric/README.md

5739c3d

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Update pl_examples/pytorch_ecosystem/pytorch_geometric/README.md

a0cded4

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Update pl_examples/pytorch_ecosystem/pytorch_geometric/cora_dna.py

e990667

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Merge branch 'master' into ecosystem_examples

860367d

Merge branch 'master' into ecosystem_examples

e0679e7

add toml

3ab225b

Merge branch 'master' into ecosystem_examples

1477eb0

ananyahjha93 approved these changes Nov 18, 2020

View reviewed changes

teddykoker approved these changes Nov 18, 2020

View reviewed changes

tchaton merged commit b7601e9 into master Nov 18, 2020

tchaton deleted the ecosystem_examples branch November 18, 2020 20:04

Borda reviewed Nov 18, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Example] Add Pytorch Geometric Example #4568

[Example] Add Pytorch Geometric Example #4568

tchaton commented Nov 7, 2020 •

edited

Loading

pep8speaks commented Nov 7, 2020 •

edited

Loading

tchaton commented Nov 7, 2020

codecov bot commented Nov 7, 2020 •

edited

Loading

SeanNaren left a comment

Borda left a comment

teddykoker left a comment

Borda left a comment •

edited

Loading

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda Nov 18, 2020

Borda commented Dec 11, 2020

	HAS_PYTORCH_GEOMETRIC = False
	PYTORCH_GEOMETRIC_AVALAIBLE = False

	typed_batch, targets = self.gather_data_and_convert_to_namedtuple(batch, batch_nb)
	typed_batch, targets = self._gather_data_and_convert_to_namedtuple(batch, batch_nb)

	parser.add_argument("--jit", default=True)
	parser.add_argument("--jit", type=bool, default=True)

	authors = ["Thomas Chaton <thomas.ai@grid.com>"]
	authors = ["Thomas Chaton <thomas@pytorchlightning.ai>"]

[Example] Add Pytorch Geometric Example #4568

[Example] Add Pytorch Geometric Example #4568

Conversation

tchaton commented Nov 7, 2020 • edited Loading

What does this PR do?

Before submitting

PR review

Did you have fun?

pep8speaks commented Nov 7, 2020 • edited Loading

Comment last updated at 2020-11-18 16:50:21 UTC

tchaton commented Nov 7, 2020

codecov bot commented Nov 7, 2020 • edited Loading

Codecov Report

SeanNaren left a comment

Choose a reason for hiding this comment

Borda left a comment

Choose a reason for hiding this comment

teddykoker left a comment

Choose a reason for hiding this comment

Borda left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Borda commented Dec 11, 2020

tchaton commented Nov 7, 2020 •

edited

Loading

pep8speaks commented Nov 7, 2020 •

edited

Loading

codecov bot commented Nov 7, 2020 •

edited

Loading

Borda left a comment •

edited

Loading