[ONNX] Fix export of images for KeypointRCNN #2272
Conversation
|
@neginraoof please review |
| x = misc_nn_ops.interpolate( | ||
| x, scale_factor=float(self.up_scale), mode="bilinear", align_corners=False | ||
| return torch.nn.functional.interpolate( | ||
| x, scale_factor=float(self.up_scale), mode="bilinear", align_corners=False, recompute_scale_factor=False |
There was a problem hiding this comment.
Isn't up_scale a integer (equals to 2)? As such, is there a difference between recompute_scale_factor=True and False? Or is it because the output_size in this case gets registered as a constant?
There was a problem hiding this comment.
With recompute_scale_factor, when input has a 0 dim, the inferred scale_factor will be 0. Although this is expected, ONNX spec does not allow for Resize scale to be <= 0.
Just a note that I think the default value for recompute_scale_factor is changing to False soon.
There was a problem hiding this comment.
With recompute_scale_factor, when input has a 0 dim, the inferred scale_factor will be 0.
This to me looks like a bug while inferring the scale_factor, as the batch dimension shouldn't affect the other sizes of the tensor (which are still valid).
Note that we use tensors of shape [0, C, H ,W], so it is fully-possible to infer the correct scale factor I believe
There was a problem hiding this comment.
I looked more into this. Looks like this is an error in inferring scale_factor in ONNX Runtime.
Upsample_bicubic2d is exported to ONNX and running in ONNX Runtime if you have an input like: torch.randn(1, 2, 4, 6, requires_grad=True)
ONNX graph also looks ok.
But the same mode fails for an input like torch.randn(0, 2, 4, 6, requires_grad=True)
The reason is that ONNX Runtime is inferring scale for across all dimensions:
for (size_t i = 0, end = input_dims.size(); i < end; ++i) {
scales[i] = static_cast<float>(output_dims[i]) / static_cast<float>(input_dims[i]);
}
For a 4D input, output_dims is inferred output shape (e.g. [n, c, hscale, wscale]) and input_dims is actual input shape (e.g. [n, c, h, w]).
This is a bug in such edge cases where n = 0.
We will create an issue for ORT to address this.
But we still need Ksenija's update to switch to nn.functional.interpolate.
And here is PR I sent out to switch the default recompute_scale_factor to False:
pytorch/pytorch#39453
|
@fmassa can this PR be merged? Thanks! |
|
@KsenijaS I've made a comment on the top, not sure if you agree with it? |
|
@fmassa Thanks a lot! |
Add support in KeypointRCNN for:
exporting for image w/ detection -> running on image w/ no detection
exporting for image w/ no detection -> running on image w/ detection