Visualization of Relay IR #8668

kueitang · 2021-08-05T08:08:49Z

Hi,
This utility is designed for users who are prone to snap-shot the relay IR graph on terminal.
The AST structure format covers basic relay function/expressions such as Function, Call, Let, Var, GlobalVar, If, Tuple, Constant currently.

For example:
A Relay IRModule:

x = relay.var("x")
y = relay.const(1)
z = relay.add(x, y)
z = relay.multiply(x, z)
mod = tvm.ir.IRModule().from_expr(z)

with:

viz_res = retv.ASTVisualization()
mod = viz_res(mod)
res = viz_res.get_output()

will be showed as:

== The AST view of the IRModule is ==
@main([Var(x)])
   `--(call)
      |--multiply
      |--x
      `--(call)
         |--add
         |--x
         `--1

It is welcome to have your thoughts on the utility.

Anyways, thanks. :)

Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

leandron

Hi @kueitang, this seems interesting as a way to see Relay.

I think this feature could be integrated with @chiwwang's ongoing work at #8448. Have you seen that PR?

I suggest you two work together to define the correct APIs needed, so that we have one way to implement visualizers for Relay using a common foundation. What do you think?

kueitang · 2021-08-05T11:33:14Z

Thanks @leandron!
Yes, I have seen #8448. It uses an interface (Plottor) to seperate frontend and backend.
It is believed that the design uses only node and edge information to plot a graph.
As for this PR, there are more information derived from depth that I need to record while traversing.
It seems a little hard to unify the current implementation to that interface, or it needs to be refactored.

chiwwang · 2021-08-06T12:38:35Z

Yes I consider graphs just as nodes and edges. So the Plotter interface is very simple.
I think depth, or "rank" can be obtained by a topological sort, but indeed maybe we should consider interfaces on different abstraction levels, for easier implementing other kinds of visualizers.
I only have a rough thoughts that we should allow customized visitors in a different interface.
Need some time to chew it over.

-Add a AST dump pass -It provides a snap shot to the relay IR graph

kueitang · 2021-08-10T02:07:13Z

hi @tqchen,
would you like to give it any thoughts? Thanks a lot! :)

comaniac · 2021-08-10T06:27:22Z

If the high-level design or the interface needs more feedback, it would be better and more efficient to file a formal RFC at the same time.

https://github.com/apache/tvm-rfcs

leandron · 2021-08-10T13:13:11Z

If the high-level design or the interface needs more feedback, it would be better and more efficient to file a formal RFC at the same time.

https://github.com/apache/tvm-rfcs

Agree with @comaniac. With an RFC, perhaps we could also find alignment to have this and other features e.g. #8448, to be built on top of the same interface, and also other parts consuming this (I can imagine tvmc as a consumer) to also be able to integrate better with a single and organized interface.

chiwwang · 2021-08-21T18:16:08Z

Hi @kueitang
Maybe we can re-use this RFC, https://discuss.tvm.apache.org/t/rfc-visualizing-relay-program-as-graph/4825
There are some valuable thoughts there :)

Back to interfaces of Relay visualization, how do you think about a backend-specific render-callback? Specifically, in the constructor of RelayVisualizer in #8448 :

class RelayVisualizer:
    """Relay IR Visualizer"""

    def __init__(
        self, relay_mod, relay_param=None, plotter_be=PlotterBackend.BOKEH, render_cb=_dft_render_cb
    ):

I try to port this PR with the interface in the following commit in the file _terminal.py.
chiwwang@4d1e14a
(I also make the callback coupled with the backend....but this seems not a good idea. A common render-callback can save efforts of implementing a new backend.)

The output looks like:

`@main([Var(x)])
  `--Call multiply
    |--Var(Input)
    `--Call add
      |--Var(Input)
      `--Constshape: (), dtype: int32`

I don't know if I should consider multiply/add as an argument of Call, so I put it in Call string.
This might be fixed by modifying tmv.ir.Op branch in the callback.

So, currently interfaces of relay-viz in my mind is:

class Plotter and class Graph in plotter.py (one plotter can have lots of graphs)
If needed, a backend-specific render-callback can overwrite the default one, with the prototype

def render_cb(graph, node_to_id, relay_param)
    """
    Parameters
    ----------
    graph : class plotter.Graph
    node_to_id : Dict[relay.expr, int]
    relay_param : Dict[string, NDarray]
    """

Additionally, the dict node_to_id ensure its order of keys is in the post-order of a relay GlobalVar.
How do you think this kind of integration?
Thanks :)

* BOKEH BACKEND * Scalable Text for Node type * Add arrow-head and TODO * Use pydot. Remove networkx and pygraphviz dependency * Add interactive legend and show information based on zoom level * Support multiple GlobalVar. One global var, one graph * test reserved post order. try terminal viz by kueitang

…ntegrated two backends * Add a RenderCallback class for reusable nodes type processing

kueitang · 2021-08-31T07:01:47Z

Users can visualize relay by a single entry point, RelayVisualizer. They could choose different renderers by options(backend in RelayVisualizer constructor)
For those who want to implement new renderer, it is expected to implement Graph, Plotter (in plotter.py). We provide default RenderCallback (render_callbak.py) for different relay ops to trigger Graph interface.
Specific method for a certain op could be overrided if needed, like TermRenderCallback.

@chiwwang @leandron

python/tvm/contrib/relay_viz/README.md

python/tvm/contrib/relay_viz/__init__.py

chiwwang · 2021-10-25T12:01:50Z

Hi @areusch, @PhilippvK, @jroesch, @leandron,
I finish an implementation, with tests, docs, and a tutorial.
You might want to take a look. :)
In short, RelayVisualizer class serve as a single entry point for different renderer.
All it does is using interfaces defined in plotter.py and node_edge_gen.py.
A tutorial show a simple use-case and some possible customization.
Thanks!

huajsj · 2021-12-02T01:55:33Z

Thanks @chiwwang.

huajsj · 2021-12-01T23:36:01Z

python/tvm/contrib/relay_viz/__init__.py

+
+
+def get_plotter_and_generator(backend):
+    """Specify the Plottor and its NodeEdgeGenerator"""


need "Parameters" and "return".

huajsj · 2021-12-02T01:18:58Z

python/tvm/contrib/relay_viz/node_edge_gen.py

+
+
+class VizNode:
+    """Node carry information used by `plotter.Graph` interface."""


need parameters docstring

huajsj · 2021-12-02T01:21:39Z

python/tvm/contrib/relay_viz/node_edge_gen.py

+
+class VizEdge:
+    """Edges for `plotter.Graph` interface."""
+


huajsj · 2021-12-02T01:46:46Z

python/tvm/contrib/relay_viz/node_edge_gen.py

+    def _function_node(
+        self,
+        node: relay.Expr,
+        _: Dict[str, tvm.runtime.NDArray],  # relay_param


remove relay_param

chiwwang · 2021-12-06T15:34:43Z

Thanks @huajsj for reviewing!

huajsj · 2021-12-15T19:46:33Z

@kueitang @chiwwang, do you think it is possible to split this PR into couple small patches? a smaller patch may help to accelerate the process and more review friendly.

chiwwang · 2022-01-17T05:16:00Z

Sure, I will figure out how to split this PR to smaller patches.
I will raise new PRs and then close this one.
Thanks for suggestions @huajsj !

This PR follows apache#8668, with splitting out interfaces class and terminal ast-dump implementation. This visualizer is aimed for quick look-then-fix, so the interface is simple. Despite that, customization is still possbile through implementing interfaces defined in `interface.py` or overriding existent implementations inside a renderer module, like `terminal.py`. A tutorial is also provided in this PR. A graphviz renderer will also be contributed after this PR.

chiwwang · 2022-01-27T15:18:04Z

A follow-up PR: #10085
@kueitang please help to close this thread. Thanks!

AndrewZhaoLuo · 2022-02-01T18:47:38Z

Closing in favor of #10085

* RelayViz interface and terminal ast-dump. This PR follows #8668, with splitting out interfaces class and terminal ast-dump implementation. This visualizer is aimed for quick look-then-fix, so the interface is simple. Despite that, customization is still possbile through implementing interfaces defined in `interface.py` or overriding existent implementations inside a renderer module, like `terminal.py`. A tutorial is also provided in this PR. A graphviz renderer will also be contributed after this PR. * lint and typo

* RelayViz interface and terminal ast-dump. This PR follows apache#8668, with splitting out interfaces class and terminal ast-dump implementation. This visualizer is aimed for quick look-then-fix, so the interface is simple. Despite that, customization is still possbile through implementing interfaces defined in `interface.py` or overriding existent implementations inside a renderer module, like `terminal.py`. A tutorial is also provided in this PR. A graphviz renderer will also be contributed after this PR. * lint and typo

kueitang requested review from areusch, comaniac, jroesch, junrushao, merrymercy, tqchen, yzhliu and zhiics as code owners August 5, 2021 08:08

leandron reviewed Aug 5, 2021

View reviewed changes

kueitang force-pushed the ast-dump branch 3 times, most recently from b0bfea6 to fb24fd3 Compare August 5, 2021 13:18

kueitang force-pushed the ast-dump branch from fb24fd3 to daca3ca Compare August 6, 2021 17:44

[Visualization Relay IR on terminal]

af7361b

-Add a AST dump pass -It provides a snap shot to the relay IR graph

kueitang force-pushed the ast-dump branch from daca3ca to af7361b Compare August 6, 2021 17:49

chiwwang mentioned this pull request Aug 21, 2021

[Draft PR] Relay IR visualizer. #8448

Closed

kueitang and others added 4 commits August 31, 2021 02:38

Merge branch 'apache:main' into ast-dump

b44aa6e

Change the design on relay Vizualizer

fcb993b

Integrated Version of relay Vizualizer * Add the TERMINAL backend * I…

54bb86c

…ntegrated two backends * Add a RenderCallback class for reusable nodes type processing

chiwwang reviewed Sep 4, 2021

View reviewed changes

python/tvm/contrib/relay_viz/README.md Outdated Show resolved Hide resolved

chiwwang reviewed Sep 4, 2021

View reviewed changes

python/tvm/contrib/relay_viz/__init__.py Outdated Show resolved Hide resolved

chiwwang reviewed Sep 4, 2021

View reviewed changes

python/tvm/contrib/relay_viz/__init__.py Outdated Show resolved Hide resolved

add testing for default parser. (TODO: tutorial)

f59a3fe

chiwwang force-pushed the ast-dump branch from b17b4a2 to f59a3fe Compare October 17, 2021 14:33

Merge remote-tracking branch 'upstream/main' into ast-dump

fb2cf61

kueitang requested a review from a team as a code owner October 25, 2021 11:49

chiwwang force-pushed the ast-dump branch from c57fe4d to 39dae20 Compare October 25, 2021 11:54

tutorial and doc

9955d7c

chiwwang force-pushed the ast-dump branch from 39dae20 to 9955d7c Compare October 25, 2021 13:00

huajsj reviewed Dec 2, 2021

View reviewed changes

add doc-string according to feedback

6fcd64c

kueitang requested a review from icemelon as a code owner December 6, 2021 15:31

chiwwang added 2 commits December 6, 2021 15:36

Merge remote-tracking branch 'upstream/main' into ast-dump

b3f6d59

fix lint...?

bcca757

kueitang requested review from kparzysz-quic, masahi, vinx13 and ZihengJiang as code owners December 6, 2021 15:56

jroesch added status: need review needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it labels Jan 19, 2022

chiwwang mentioned this pull request Jan 27, 2022

RelayViz interface and terminal ast-dump #10085

Merged

AndrewZhaoLuo closed this Feb 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Visualization of Relay IR #8668

Visualization of Relay IR #8668

kueitang commented Aug 5, 2021 •

edited

Loading

leandron left a comment

kueitang commented Aug 5, 2021

chiwwang commented Aug 6, 2021

kueitang commented Aug 10, 2021

comaniac commented Aug 10, 2021

leandron commented Aug 10, 2021 •

edited

Loading

chiwwang commented Aug 21, 2021

kueitang commented Aug 31, 2021

chiwwang commented Oct 25, 2021 •

edited

Loading

huajsj commented Dec 2, 2021

huajsj Dec 1, 2021

chiwwang Dec 6, 2021

huajsj Dec 2, 2021

chiwwang Dec 6, 2021

huajsj Dec 2, 2021

chiwwang Dec 6, 2021

huajsj Dec 2, 2021

chiwwang Dec 6, 2021

chiwwang commented Dec 6, 2021

huajsj commented Dec 15, 2021

chiwwang commented Jan 17, 2022

chiwwang commented Jan 27, 2022

AndrewZhaoLuo commented Feb 1, 2022



		def get_plotter_and_generator(backend):
		"""Specify the Plottor and its NodeEdgeGenerator"""



		class VizNode:
		"""Node carry information used by `plotter.Graph` interface."""

Visualization of Relay IR #8668

Visualization of Relay IR #8668

Conversation

kueitang commented Aug 5, 2021 • edited Loading

leandron left a comment

Choose a reason for hiding this comment

kueitang commented Aug 5, 2021

chiwwang commented Aug 6, 2021

kueitang commented Aug 10, 2021

comaniac commented Aug 10, 2021

leandron commented Aug 10, 2021 • edited Loading

chiwwang commented Aug 21, 2021

kueitang commented Aug 31, 2021

chiwwang commented Oct 25, 2021 • edited Loading

huajsj commented Dec 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chiwwang commented Dec 6, 2021

huajsj commented Dec 15, 2021

chiwwang commented Jan 17, 2022

chiwwang commented Jan 27, 2022

AndrewZhaoLuo commented Feb 1, 2022

kueitang commented Aug 5, 2021 •

edited

Loading

leandron commented Aug 10, 2021 •

edited

Loading

chiwwang commented Oct 25, 2021 •

edited

Loading