-
Notifications
You must be signed in to change notification settings - Fork 22.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ONNX] Support ONNXFakeContext with op_level_debug #105874
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/105874
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit f76d61c: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
ghstack-source-id: 0c0e27f87b8a70ae171d5642d755b1637b6c7ba9 Pull Request resolved: #105874
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just minor nits
list_int_shape.append(symbolic_shapes.hint_int(dim)) | ||
else: | ||
raise ValueError( | ||
f"Unexpected SymInt found in shape. SymInt: {dim}; " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this unexpected, or is it a case we can expect but cannot handle? Just to make sure for clarity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice catch, overlooked this one. Please reveal keyword "unbacked", will easily lead to many helpful resources such as this https://pytorch-dev-podcast.simplecast.com/episodes/unbacked-symints-oyqu0_P6
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Previous to the PR, op_level_debug doesn't support OnnxFakeConext because it relies on real tensor in args to do shape type inference propagation in fx graph to get static shapes helping simulating the op args/kwargs. However, OnnxFakeContext will fakify the args/kwargs at the very begining, so the op_level_debug can't have the static_shapes to utilize. This PR uses SymInt API: `has_hint` and `hint_int` to fully replace the functionality of shape type inference propagation. The static shapes are obtained through SymInt. Therefore, the pass `ShapeInferenceWithFakeTensor` is eliminated. Also moved the args/kwargs processing into op_validation to live under the rule `op_level_debug`. [ghstack-poisoned]
ghstack-source-id: 19a82b4636e3400e93aa498108549b3be7a6c864 Pull Request resolved: #105874
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Stack from ghstack (oldest at bottom):
Previous to the PR, op_level_debug doesn't support OnnxFakeConext because it relies on real tensor in args to do shape type inference propagation in fx graph to get static shapes helping simulating the op args/kwargs. However, OnnxFakeContext will fakify the args/kwargs at the very begining, so the op_level_debug can't have the static_shapes to utilize.
This PR uses SymInt API:
has_hint
andhint_int
to fully replace the functionality of shape type inference propagation. The static shapes are obtained through SymInt. Therefore, the passShapeInferenceWithFakeTensor
is eliminated.Also moved the args/kwargs processing into op_validation to live under the rule
op_level_debug
.