-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Benchmark] Add inference test for current benchmarking and record performance #4892
[Benchmark] Add inference test for current benchmarking and record performance #4892
Conversation
Codecov Report
@@ Coverage Diff @@
## master #4892 +/- ##
==========================================
- Coverage 84.65% 82.82% -1.84%
==========================================
Files 330 330
Lines 17908 17924 +16
==========================================
- Hits 15160 14845 -315
- Misses 2748 3079 +331
Continue to review full report at Codecov.
|
334cec0
to
719d6b3
Compare
f88bc93
to
5eb0491
Compare
for more information, see https://pre-commit.ci
Hi @rusty1s , I retest the failure of |
Will have a look! |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! This looks good. Mostly left comments regarding styling and re-usability of code blocks. Let me know what you think!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! Looks great. I added some nitpicky comments :). Some styling issues can be fixed by installing the pre-commit hooks: pre-commit install
.
14cc047
to
9bd310a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! I think this looks great. My only concern is that I would like to keep PyG examples lightweight - for PNA
, we could think about adding examples to benchmark/kernel
, for benchmarking to_hetero_mag.py
we could implement a separate script in benchmark/
(okay to be done in a follow-up PR).
Hi @rusty1s , thank you for your review. I will remove |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! LGTM.
@yanbing-j Currently, |
Hi @rusty1s , It seems no |
Yes, this sounds good. |
@rusty1s Could you please help merge this to master? |
Currently, PyG benchmark only contains training test. It is not convenient for us to optimize starting from inference. In this PR, we add inference test for citation benchmark and to_hetero_mag, and record end-to-end time in one epoch.