[docs] microTVM model training tutorial with Colab support #10921

guberti · 2022-04-06T21:05:49Z

This PR adds a new tutorial to the microTVM how_to gallery, showing how a MobileNet V1 model can be trained with transfer learning and modified to fit on embedded devices (in this case, the Arduino Nano 33 BLE). This tutorial also addresses #10706 by showing what it would take to allow tutorials to be opened using Google Colab. See the below screenshot, or view the HTML page at the link below:

Alternatively, to make it easier I've also copied the output HTML file, and hosted it at the following link:

https://guberti.github.io/tvm-docs-proof-of-concept/

guberti · 2022-04-15T06:57:33Z

How does "Open in Colab" work?

Google Colab has a feature where using a URL, .ipynb files can be loaded from GitHub links. It just so happens that TVM's doc files are already held at https://github.com/apache/tvm-site/tree/asf-site, so we only need to point Google Colab to that. Since this tutorial is not yet merged, all the "Open in Colab" button is is a link to the following address:

https://colab.research.google.com/github/guberti/tvm-site/blob/asf-site/docs/_downloads/a7c7ea4b5017ae70db1f51dd8e6dcd82/micro_train.ipynb

What had to be changed to get this working?

In theory, nothing about Sphinx Gallery would have to be changed to make this work. However, in practice the generation of .ipynb files is extremely buggy, so the output Python notebooks do not work anyway. I made two pull requests to add features to Sphinx Gallery to make this possible:

These have all been merged, but a new version of sphinx-gallery has not yet been released yet. Until then, I had to set pip to install "git+https://github.com/sphinx-gallery/sphinx-gallery.git". Updating this version will also fix #11117.

I also needed some new data files and graphics for this tutorial. Those can be found at:

Add graphics for microTVM model training tutorial tlc-pack/web-data#15

Currently, the URLs in the tutorial point to my fork of the web-data repo, but once tlc-pack/web-data#15 is merged those can be updated to just point to standard web-data.

I did a similar thing for tvm-site, where this tutorial currently points to my fork. This is just so the tutorial works before being merged - once this PR lands, the URL can be updated to the regular tvm-site repo (the asf-site branch will be auto updated by @tvm-bot).

Lastly, #11164 was needed so that microTVM template projects could be used in the CI.

driazati · 2022-04-15T17:53:51Z

This is fantastic, I’ll try it out today! Colab will be great to have in the docs. If the sphinx-gallery PRs don’t get merged in a timely manner we can apply your patches and build it from source in the Docker images so we don’t need to wait

@guberti

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

@guberti

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

@guberti

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

@guberti

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

areusch · 2022-04-29T21:46:09Z

retriggered now that #11164 has landed

areusch · 2022-05-02T18:31:28Z

@guberti could you try pushing an empty commit? i think retriggering somehow didn't pick up the right Jenkinsfile changes.

areusch

thanks @guberti, few questions

cc @jwfromm @junrushao1994 @tqchen @Mousius in case you would like to have a look over this

gallery/how_to/work_with_microtvm/micro_train.py

areusch · 2022-05-02T21:53:16Z

tests/scripts/ci.py

@@ -267,7 +267,7 @@ def docs(
            "tlcpack-sphinx-addon==0.2.1",
            "synr==0.5.0",
            "image==1.5.33",
-            "sphinx-gallery==0.4.0",
+            "git+https://github.com/guberti/sphinx-gallery.git@ipynb-include-bash",


should we update docker/install scripts too if we're going to go this route? also, is this based off 0.4.0? may need to backport or verify it won't break anything to update.

No, this is based on the latest sphinx-gallery version. You're right that this could potentially cause problems, but I haven't ran into any. We should definitely verify nothing breaks, and if there is an issue we can discuss backporting.

guberti · 2022-05-04T19:45:52Z

thanks @guberti, few questions

Thanks for taking a look! Addressed your comments and merged main into this branch to fix the tests.

areusch

Ahh sorry I didn't notice we weren't pinning. Aside from that I think we can merge.

apps/microtvm/pyproject.toml

areusch · 2022-05-27T15:39:15Z

tests/scripts/task_python_docs.sh

@@ -84,6 +84,7 @@ IGNORED_WARNINGS=(
    'autotvm:Cannot find config for target=llvm -keys=cpu -link-params=0'
    'autotvm:One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.'
    'autotvm:Cannot find config for target=cuda -keys=cuda,gpu'
+    'absl:For model inputs containing unsupported operations'


Can you link me to an example log line?

The full error is:

WARNING:absl:For model inputs containing unsupported operations which cannot be quantized, the `inference_input_type` attribute will default to the original type.

This warning also occurs in the official TensorFlow tutorial - https://www.tensorflow.org/lite/performance/post_training_integer_quant.

Changes from code review Use official sphinx-gallery repo Correctly specify version Import pyserial only when necessary

Try to avoid throwing warning Fix linting, try verbosity filter Try adding to ignore file Remove fix attempts

Include full git hashes

guberti changed the title ~~Model training tutorial for microTVM~~ [WIP] Proof of concept for Google Colab compatible tutorials Apr 15, 2022

guberti mentioned this pull request Apr 15, 2022

[Docs] Add Google Colab support to tutorials #10706

Closed

driazati added a commit to driazati/tvm that referenced this pull request Apr 15, 2022

[ci] Use custom sphinx-gallery in ci_gpu

2e06431

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

driazati mentioned this pull request Apr 15, 2022

[ci] Use custom sphinx-gallery in ci_gpu #11023

Closed

driazati added a commit to driazati/tvm that referenced this pull request Apr 15, 2022

[ci] Use custom sphinx-gallery in ci_gpu

3ebcd37

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

driazati added a commit to driazati/tvm that referenced this pull request Apr 15, 2022

[ci] Use custom sphinx-gallery in ci_gpu

5283913

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

driazati added a commit to driazati/tvm that referenced this pull request Apr 15, 2022

[ci] Use custom sphinx-gallery in ci_gpu

2846a3b

To enable apache#10921 this builds sphinx-gallery using @guberti's changes

guberti force-pushed the micro-train-tutorial branch from 53d13fd to f501842 Compare April 24, 2022 05:42

guberti changed the title ~~[WIP] Proof of concept for Google Colab compatible tutorials~~ [docs] Google Colab compatible microTVM model training tutorial Apr 28, 2022

guberti mentioned this pull request May 2, 2022

Add graphics for microTVM model training tutorial tlc-pack/web-data#15

Merged

guberti marked this pull request as ready for review May 2, 2022 20:59

guberti changed the title ~~[docs] Google Colab compatible microTVM model training tutorial~~ [docs] microTVM model training tutorial with Colab support May 2, 2022

areusch reviewed May 2, 2022

View reviewed changes

guberti force-pushed the micro-train-tutorial branch from fb50053 to ff377e9 Compare May 20, 2022 18:44

guberti mentioned this pull request May 24, 2022

[CI Problem] CI has logging errors even when successful #11117

Closed

guberti force-pushed the micro-train-tutorial branch 3 times, most recently from b525482 to 339d646 Compare May 26, 2022 21:42

areusch reviewed May 27, 2022

View reviewed changes

guberti force-pushed the micro-train-tutorial branch 2 times, most recently from 2599fb7 to 14ece0b Compare June 3, 2022 04:29

guberti added 4 commits June 3, 2022 00:30

First draft of micro train tutorial

b4267c3

unit test code

4566b57

Fix obvious formatting issues

6dd8f77

Linting

eb66af9

guberti added 8 commits June 3, 2022 00:30

Proof of concept showing that "Open in Colab" is possible

d248f9a

Make test Python script more readable

586d79a

Fix formatting

8c56eb3

Ready for review

42355f0

Import pyserial only when needed

95676f2

Changes from code review Use official sphinx-gallery repo Correctly specify version Import pyserial only when necessary

Add warning to ignored list

bde134a

Try to avoid throwing warning Fix linting, try verbosity filter Try adding to ignore file Remove fix attempts

Grammar fixes

b994a0f

Address code review comments

14e5750

Include full git hashes

guberti force-pushed the micro-train-tutorial branch from 14ece0b to 14e5750 Compare June 3, 2022 04:30

guberti added 2 commits June 3, 2022 10:56

Rerun tests

5095a5c

Rerun again

ed96b7e

areusch approved these changes Jun 3, 2022

View reviewed changes

areusch merged commit f05ebde into apache:main Jun 3, 2022

mehrdadh mentioned this pull request Jun 8, 2022

[microTVM] Update pyproject to python3.7 #11634

Merged

guberti mentioned this pull request Jun 15, 2022

Fix index link text color for current sphinx-gallery version tlc-pack/tlcpack-sphinx-addon#7

Merged

driazati mentioned this pull request Jul 14, 2022

TVM v0.9.0.rc0 Release Candidate Notes #12102

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[docs] microTVM model training tutorial with Colab support #10921

[docs] microTVM model training tutorial with Colab support #10921

guberti commented Apr 6, 2022 •

edited

Loading

guberti commented Apr 15, 2022 •

edited

Loading

driazati commented Apr 15, 2022 •

edited

Loading

areusch commented Apr 29, 2022

areusch commented May 2, 2022

areusch left a comment

areusch May 2, 2022

guberti May 4, 2022

guberti commented May 4, 2022

areusch left a comment

areusch May 27, 2022

guberti May 27, 2022 •

edited

Loading

[docs] microTVM model training tutorial with Colab support #10921

[docs] microTVM model training tutorial with Colab support #10921

Conversation

guberti commented Apr 6, 2022 • edited Loading

guberti commented Apr 15, 2022 • edited Loading

How does "Open in Colab" work?

What had to be changed to get this working?

driazati commented Apr 15, 2022 • edited Loading

areusch commented Apr 29, 2022

areusch commented May 2, 2022

areusch left a comment

Choose a reason for hiding this comment

areusch May 2, 2022

Choose a reason for hiding this comment

guberti May 4, 2022

Choose a reason for hiding this comment

guberti commented May 4, 2022

areusch left a comment

Choose a reason for hiding this comment

areusch May 27, 2022

Choose a reason for hiding this comment

guberti May 27, 2022 • edited Loading

Choose a reason for hiding this comment

guberti commented Apr 6, 2022 •

edited

Loading

guberti commented Apr 15, 2022 •

edited

Loading

driazati commented Apr 15, 2022 •

edited

Loading

guberti May 27, 2022 •

edited

Loading