[Frontend][PaddlePaddle] Add autopad for conv/pool #9295

jiangjiajun · 2021-10-15T06:38:13Z

This will solve the problem with dynamic input shape while padding==SAME in conv2d/pool2d

also include pad3d and squeeze, this two operators will be used for padding tensor.
And the _autopad function refers to ONNX frontend @mbrookhart

Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

merge to newest code

upate

jiangjiajun · 2021-10-20T02:27:09Z

@areusch @comaniac @AndrewZhaoLuo @mbrookhart Hi, Could you help to review this pull request, all the tests have passed

AndrewZhaoLuo · 2021-10-20T21:35:49Z

I'll take a look today.

AndrewZhaoLuo

Some comments, overall LGTM though I need to maybe read the spec a bit more closely.

AndrewZhaoLuo · 2021-10-22T06:29:48Z

python/tvm/relay/frontend/paddlepaddle.py

@@ -43,18 +44,56 @@
 __all__ = ["from_paddle"]


-def _get_pad_size(in_size, dilated_kernel_size, stride_size):
-    """Calculate the paddings size for Conv/Pool in SAME padding mode."""
+def _autopad(


Can you just use

tvm/python/tvm/relay/frontend/onnx.py

Lines 412 to 472 in d0c6ca5

def autopad(

data,

strides,

kernel_shape,

dilations,

ndim,

pad_type="constant",

deconv=False,

mode="SAME_UPPER",

pad_value=0.0,

):

"""

Perform autopadding with dynamic input shapes

"""

# get attributes as constants

strides = _op.const(np.array(strides), dtype="int64")

dilated_kernel_shape = _op.const(

np.array(

[(kernel - 1) * dilation + 1 for kernel, dilation in zip(kernel_shape, dilations)]

),

dtype="int64",

)

# get input shape

shape = _op.strided_slice(shape_of(data, dtype="int64"), [2], [ndim])

# set up integer constants

zero = _op.const(0, dtype="int64")

one = _op.const(1, dtype="int64")

two = _op.const(2, dtype="int64")

# Calculate total padding

mod = _op.mod(shape, strides)

left = _op.maximum(dilated_kernel_shape - strides, zero)

right = _op.maximum(dilated_kernel_shape - mod, zero)

total_pad = _op.where(_op.equal(mod, zero), left, right)

if deconv:

total_pad = _op.const(np.array(kernel_shape), dtype="int64") - one - total_pad

# split total padding into before and after

pad_before = _op.floor_divide(total_pad, two)

pad_after = total_pad - pad_before

# combine

if "LOWER" in mode:

pad = _op.concatenate(

[_op.reshape(pad_after, [-1, 1]), _op.reshape(pad_before, [-1, 1])], axis=1

)

else:

pad = _op.concatenate(

[_op.reshape(pad_before, [-1, 1]), _op.reshape(pad_after, [-1, 1])], axis=1

)

# pad N and C with zeros

pad = _op.concatenate([_op.const(np.zeros([2, 2], dtype="int64"), dtype="int64"), pad], axis=0)

if isinstance(pad_value, (float, int)):

pad_value = _op.const(pad_value)

return _op.nn.pad(data, fold_constant(pad), pad_value, pad_type)

?

Refactor _autopad in the onnx.py file to tvm/python/tvm/relay/frontend/common.py

Good advice, I think this function also works for tensorflow and tflite to solve dynamic shape problem.

shape_of and autopad are both removed to common.py

AndrewZhaoLuo · 2021-10-22T06:30:53Z

python/tvm/relay/frontend/paddlepaddle.py

-            pad_h = _get_pad_size(in_h, (k_h - 1) * dilations[0] + 1, strides[0])
-            pad_w = _get_pad_size(in_w, (k_w - 1) * dilations[1] + 1, strides[1])
-        paddings = [pad_h[0], pad_w[0], pad_h[1], pad_w[1]]
+        dilations = [1, 1]


do we mean to override the dilations?

This is a history issue for Paddle framework, while padding==SAME, it will force dliations = 1
Here is the implementation code https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/conv_op.h#L113

To avoid confusion, I put a comment on this line of code to explain this problem.

…/common.py

Refactor autopad in the onnx.py and paddlepaddle.py to relay/frontend…

AndrewZhaoLuo

LGTM

* Add autopad for conv/pool * add autopad for conv/pool * fix pylint warning * add some annotations * add som annotations * add som annotations * Refactor autopad in the onnx.py and paddlepaddle.py to relay/frontend/common.py * add comment for conv2d Co-authored-by: heliqi <1101791222@qq.com>

jiangjiajun and others added 9 commits August 3, 2021 17:14

Merge pull request #2 from apache/main

5b39c79

merge to newest code

Merge pull request #8 from apache/main

74cc942

upate

Merge branch 'apache:main' into main

e0420bd

Merge branch 'apache:main' into main

f181b0a

Merge branch 'apache:main' into main

800b187

Merge branch 'apache:main' into pr002

a0a33fb

Add autopad for conv/pool

73a21b0

add autopad for conv/pool

bf3b0a9

add autopad for conv/pool

30a9922

jiangjiajun requested review from areusch, comaniac, Huyuwei, jroesch, junrushao, jwfromm, kazum, mbrookhart, merrymercy, siju-samuel, srkreddy1238, tqchen and yzhliu as code owners October 15, 2021 06:38

jiangjiajun closed this Oct 15, 2021

jiangjiajun reopened this Oct 16, 2021

jiangjiajun added 4 commits October 16, 2021 08:25

fix pylint warning

7c2afce

add some annotations

cd8296a

add som annotations

4ebcdeb

add som annotations

ab3278a

AndrewZhaoLuo reviewed Oct 22, 2021

View reviewed changes

heliqi and others added 3 commits October 22, 2021 17:24

Refactor autopad in the onnx.py and paddlepaddle.py to relay/frontend…

fee1886

…/common.py

Merge pull request #67 from heliqi/pr002

9c442a9

Refactor autopad in the onnx.py and paddlepaddle.py to relay/frontend…

add comment for conv2d

211022b

AndrewZhaoLuo approved these changes Oct 22, 2021

View reviewed changes

masahi approved these changes Oct 22, 2021

View reviewed changes

masahi merged commit 4fb6fa5 into apache:main Oct 22, 2021

This was referenced Oct 28, 2021

[RFC][Tracking][RFC-0019] Add PaddlePaddle Frontend #8751

Closed

Apache TVM v0.8 Release Note Candidate #9416

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Frontend][PaddlePaddle] Add autopad for conv/pool #9295

[Frontend][PaddlePaddle] Add autopad for conv/pool #9295

jiangjiajun commented Oct 15, 2021 •

edited

Loading

jiangjiajun commented Oct 20, 2021

AndrewZhaoLuo commented Oct 20, 2021

AndrewZhaoLuo left a comment

AndrewZhaoLuo Oct 22, 2021

jiangjiajun Oct 22, 2021

AndrewZhaoLuo Oct 22, 2021

jiangjiajun Oct 22, 2021 •

edited

Loading

jiangjiajun Oct 22, 2021

AndrewZhaoLuo left a comment

	def autopad(
	data,
	strides,
	kernel_shape,
	dilations,
	ndim,
	pad_type="constant",
	deconv=False,
	mode="SAME_UPPER",
	pad_value=0.0,
	):
	"""
	Perform autopadding with dynamic input shapes
	"""
	# get attributes as constants
	strides = _op.const(np.array(strides), dtype="int64")
	dilated_kernel_shape = _op.const(
	np.array(
	[(kernel - 1) * dilation + 1 for kernel, dilation in zip(kernel_shape, dilations)]
	),
	dtype="int64",
	)
	# get input shape
	shape = _op.strided_slice(shape_of(data, dtype="int64"), [2], [ndim])

	# set up integer constants
	zero = _op.const(0, dtype="int64")
	one = _op.const(1, dtype="int64")
	two = _op.const(2, dtype="int64")

	# Calculate total padding
	mod = _op.mod(shape, strides)

	left = _op.maximum(dilated_kernel_shape - strides, zero)
	right = _op.maximum(dilated_kernel_shape - mod, zero)

	total_pad = _op.where(_op.equal(mod, zero), left, right)
	if deconv:
	total_pad = _op.const(np.array(kernel_shape), dtype="int64") - one - total_pad

	# split total padding into before and after
	pad_before = _op.floor_divide(total_pad, two)
	pad_after = total_pad - pad_before

	# combine
	if "LOWER" in mode:
	pad = _op.concatenate(
	[_op.reshape(pad_after, [-1, 1]), _op.reshape(pad_before, [-1, 1])], axis=1
	)
	else:
	pad = _op.concatenate(
	[_op.reshape(pad_before, [-1, 1]), _op.reshape(pad_after, [-1, 1])], axis=1
	)

	# pad N and C with zeros
	pad = _op.concatenate([_op.const(np.zeros([2, 2], dtype="int64"), dtype="int64"), pad], axis=0)

	if isinstance(pad_value, (float, int)):
	pad_value = _op.const(pad_value)

	return _op.nn.pad(data, fold_constant(pad), pad_value, pad_type)

[Frontend][PaddlePaddle] Add autopad for conv/pool #9295

[Frontend][PaddlePaddle] Add autopad for conv/pool #9295

Conversation

jiangjiajun commented Oct 15, 2021 • edited Loading

jiangjiajun commented Oct 20, 2021

AndrewZhaoLuo commented Oct 20, 2021

AndrewZhaoLuo left a comment

Choose a reason for hiding this comment

AndrewZhaoLuo Oct 22, 2021

Choose a reason for hiding this comment

jiangjiajun Oct 22, 2021

Choose a reason for hiding this comment

AndrewZhaoLuo Oct 22, 2021

Choose a reason for hiding this comment

jiangjiajun Oct 22, 2021 • edited Loading

Choose a reason for hiding this comment

jiangjiajun Oct 22, 2021

Choose a reason for hiding this comment

AndrewZhaoLuo left a comment

Choose a reason for hiding this comment

jiangjiajun commented Oct 15, 2021 •

edited

Loading

jiangjiajun Oct 22, 2021 •

edited

Loading