Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update usage of PyTorch's custom op API #2193

Merged
merged 1 commit into from
Sep 28, 2023

Conversation

zou3519
Copy link
Contributor

@zou3519 zou3519 commented Aug 11, 2023

Description

Hi, I maintain the custom ops story in PyTorch. This PR updates the the usage of PyTorch's private custom op API to a newer API. This API is still private but closer to what we want it to be.

Type of change

API migration.

@github-actions github-actions bot added component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes labels Aug 11, 2023
Copy link
Collaborator

@gs-olive gs-olive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good - thanks for this update. Added a few comments/questions. Additionally, please run the Python linting on this PR. You can do so by running the following from the root directory of the repository:

pip install -r requirements-dev.txt
pre-commit install

Subsequent commits will be automatically linted.

This PR is currently blocked pending a Torch nightly upgrade on the repository.

@gs-olive
Copy link
Collaborator

gs-olive commented Sep 1, 2023

Hi @zou3519 - we recently updated the stack to use the latest nightly distribution of PyTorch. Could you run linting and rebase to the latest main? Thank you

@zou3519
Copy link
Contributor Author

zou3519 commented Sep 5, 2023

Will do!

@gs-olive
Copy link
Collaborator

Fixes #2328

Copy link
Collaborator

@gs-olive gs-olive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Just pending the Python linting fixes mentioned in the comments below.

@custom_op(
qualname="tensorrt::einsum",
manual_schema="(str equation, Tensor[] tensors) -> Tensor",
library.custom_op(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove one space above this call; add one more space below line 15

manual_schema="(Tensor x, int[1] kernel_size, int[1] stride, int[1] padding, int[1] dilation, bool ceil_mode) -> Tensor",
library.custom_op(
"tensorrt::maxpool1d",
"(Tensor x, int[1] kernel_size, int[1] stride, int[1] padding, int[1] dilation, bool ceil_mode) -> Tensor"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comma at end of this line; add another space below line 26

@zou3519
Copy link
Contributor Author

zou3519 commented Sep 22, 2023

Thanks; I'm not actually sure how to run the linter (do I need to undo the commit and then redo it? for the pre-commit hook to pick it up?)

@gs-olive
Copy link
Collaborator

Yes, you would need to somehow change those files for the pre-commit hook to track it. Any small change to the file (like an additional newline at the end) will cause the whole file to lint, which also works.

@zou3519
Copy link
Contributor Author

zou3519 commented Sep 22, 2023

Yes, you would need to somehow change those files for the pre-commit hook to track it. Any small change to the file (like an additional newline at the end) will cause the whole file to lint, which also works.

Perfect, that did the trick (newline at the end)

@gs-olive
Copy link
Collaborator

Great! I think py/torch_tensorrt/dynamo/lowering/substitutions/maxpool1d.py might need it as well, then will merge.

Hi, I maintain the custom ops story in PyTorch. This PR updates the the
usage of PyTorch's private custom op API to a newer API. This API is still
private but closer to what we want it to be.

Test Plan:
- wait for CI
@zou3519
Copy link
Contributor Author

zou3519 commented Sep 25, 2023

@gs-olive I think the linting passed, but I'm not sure the test failures are significant. Thoughts?

@gs-olive
Copy link
Collaborator

@zou3519 - I reran the tests and will merge once they pass - I don't believe the failures are related to the content of this PR

Copy link
Collaborator

@gs-olive gs-olive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Now passing CI

@gs-olive gs-olive merged commit bece720 into pytorch:main Sep 28, 2023
22 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🐛 [Bug] Encountered bug when using Torch-TensorRT
3 participants