[TOPI] C++ topi operators #312

nicolasvasilache · 2017-08-11T10:33:32Z

Summary:
This diff implements C++ topi contributions for:

relu with parametrix threshold
pad with generic padBefore / padAfter specification
matmult with transposes
conv2d_nchw, conv2d_hwcn with runtime constant padding and strides
depthwise_conv2d_nchw with runtime constant padding and strides
group_conv2d_ngchw with runtime constant padding and strides
broadcast_to a broadcastable shape
broadcast_bop where bop is an usual binary op (+ - * / %)

Convolution padding is implemented using the pad operation.
To avoid extra memory consumption, it is generally recommended to inline the padding with the autoinliner.
Unfortunately in its current form the elemwise checks are too restrictive to allow inlining.
So this diff also proposes an extension to LHS injective (i.e. no reduction axis in the current IR design)

Test Plan:
Tested in C++ testsuite in a separate repository, I am looking for suggestions to quickly spin up some tests for tvm.

Reviewers: tqchen

Subscribers:

Tasks:

Tags:

Blame Revision:

Summary: This diff implements C++ topi contributions for: - relu with parametrix threshold - pad with generic padBefore / padAfter specification - matmult with transposes - conv2d_nchw, conv2d_hwcn with runtime constant padding and strides - depthwise_conv2d_nchw with runtime constant padding and strides - group_conv2d_ngchw with runtime constant padding and strides - broadcast_to a broadcastable shape - broadcast_bop where bop is an usual binary op (+ - * / %) Convolution padding is implemented using the pad operation. To avoid extra memory consumption, it is generally recommended to inline the padding with the autoinliner. Unfortunately in its current form the elemwise checks are too restrictive to allow inlining. So this diff also proposes an extension to LHS injective (i.e. no reduction axis in the current IR design) Test Plan: Tested in C++ testsuite in a separate repository, I am looking for suggestions to quickly spin up some tests for tvm. Reviewers: tqchen Subscribers: Tasks: Tags: Blame Revision:

tqchen · 2017-08-11T15:34:46Z

It is indeed helpful to introduce an AutoInlineInjective pass. Now, there are three types of of "injective like relatiosn"

Strictly ewise
Broadcast relation
General injective

It is helpful to still keep these three categories, since they might leads to different ways of scheduling. For example, sometimes it is helpful to introduce a caching stage for general injective operator to make the memory layout access more friendly

tqchen · 2017-08-11T15:37:05Z

In light of that, maybe it is helpful to introduce an AutoInlineInjective pass, this makes the intention explicit through function name, as opposed to the additional boolean argument, which user need to lookup through the document

tqchen · 2017-08-11T15:47:28Z

topi/include/topi/detail/broadcast.h

+  return ivars;
+}
+
+typedef std::function<tvm::Expr(tvm::Expr, tvm::Expr)> BinaryExpr;


C++11 style ```using BinaryExpr =````

see also next comment, this can be safely deleted

tqchen · 2017-08-11T15:50:12Z

topi/include/topi/nn.h

+namespace topi {
+
+template <typename T>
+tvm::Expr map(const tvm::Array<tvm::Expr>& exprs, T op) {


for user facing function, use doxygen style comment

tqchen · 2017-08-11T16:07:13Z

topi/include/topi/broadcast.h

+
+namespace topi {
+
+inline tvm::Tensor broadcast_to(const tvm::Array<tvm::Expr>& outputShape,


to be consistent with numpy, use broadcast_to(array, shape)

tqchen · 2017-08-11T16:08:41Z

topi/include/topi/detail/broadcast.h

+  for (int i = 0; i < ovars.size(); ++i) {
+    bool found = false;
+    for (int j = 0; j < myVars.size(); ++j) {
+      if (tvm::ir::Equal(allVars[i], myVars[j])) {


use allVars[i].same_as, will be faster than deep compare

tqchen · 2017-08-11T16:09:04Z

topi/include/topi/detail/broadcast.h

+  return bh;
+}
+
+inline tvm::Array<tvm::Expr> inputShapeFromBroadcast(


should be more approperiate as InputIndexFromBroadcast?

tqchen · 2017-08-11T16:10:50Z

topi/include/topi/detail/broadcast.h

+
+typedef std::function<tvm::Expr(tvm::Expr, tvm::Expr)> BinaryExpr;
+
+inline tvm::Tensor withBroadcast(BinaryExpr op,


templatize op will make compiler much easier to inline it, if we want good performance without the explicit overhead of creating std::function.

template<FBinaryExpr> function(FBinaryExpr op, ...)

tqchen · 2017-08-11T16:11:55Z

topi/include/topi/detail/broadcast.h

+    // Only inject 0 here if we have not yet reached the dimension of I
+    // (i.e. this must be a 1)
+    if (!found && (ovars.size() - i) <= expectedDims) {
+      ivars.push_back(tvm::Expr(0));


safter as make_zero(myVars[0].type()), in case type of index differ from int32

tqchen · 2017-08-11T16:14:58Z

topi/include/topi/nn.h

+}
+
+template <typename T>
+inline tvm::Tensor relu(const tvm::Tensor& x, T threshold) {


default threshold to static_cast(0)

nicolasvasilache · 2017-08-13T17:13:22Z

Thanks for the review! This should address the requested changes as well as style and lint issues.
Note that I didn't implement the broadcast check for inlining as I don't have a test case for it yet and would rather see this land first.

tqchen · 2017-08-13T22:55:43Z

Thanks! this is merged. Please have follow-up PRs to add doxygen comment to the user facing functions

Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com> Co-authored-by: Junru Shao <junrushao1994@gmail.com> Co-authored-by: Tianqi Chen <tqchen@users.noreply.github.com> Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com> Co-authored-by: Hongyi Jin <3231950289@qq.com> Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com> Co-authored-by: Junru Shao <junrushao1994@gmail.com> Co-authored-by: Tianqi Chen <tqchen@users.noreply.github.com> Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com> Co-authored-by: Hongyi Jin <3231950289@qq.com> Co-authored-by: Wuwei Lin <wuwei@apache.org> Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com> Co-authored-by: Junru Shao <junrushao1994@gmail.com> Co-authored-by: Tianqi Chen <tqchen@users.noreply.github.com> Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com> Co-authored-by: Hongyi Jin <3231950289@qq.com> Co-authored-by: Wuwei Lin <wuwei@apache.org>

tqchen requested changes Aug 11, 2017

View reviewed changes

Nicolas Vasilache and others added 2 commits August 13, 2017 14:28

Review + Lint + GSG C++

d087b42

Merge branch 'master' into pr/topi_cpp

e94d792

tqchen approved these changes Aug 13, 2017

View reviewed changes

tqchen merged commit f08de2b into apache:master Aug 13, 2017

tqchen changed the title ~~[WIP] C++ topi contributions~~ [TOPI] C++ topi operators Aug 13, 2017

Hzfengsy added a commit to Hzfengsy/tvm that referenced this pull request Mar 1, 2021

[TensorIR] introduce Block and BlockRealize (apache#312)

476d0c3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TOPI] C++ topi operators #312

[TOPI] C++ topi operators #312

nicolasvasilache commented Aug 11, 2017

tqchen commented Aug 11, 2017 •

edited

Loading

tqchen commented Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

tqchen Aug 11, 2017

nicolasvasilache commented Aug 13, 2017 •

edited

Loading

tqchen commented Aug 13, 2017


		namespace topi {

		inline tvm::Tensor broadcast_to(const tvm::Array<tvm::Expr>& outputShape,


		typedef std::function<tvm::Expr(tvm::Expr, tvm::Expr)> BinaryExpr;

		inline tvm::Tensor withBroadcast(BinaryExpr op,

[TOPI] C++ topi operators #312

[TOPI] C++ topi operators #312

Conversation

nicolasvasilache commented Aug 11, 2017

tqchen commented Aug 11, 2017 • edited Loading

tqchen commented Aug 11, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolasvasilache commented Aug 13, 2017 • edited Loading

tqchen commented Aug 13, 2017

tqchen commented Aug 11, 2017 •

edited

Loading

nicolasvasilache commented Aug 13, 2017 •

edited

Loading