allow libbacktrace to be used when cross compiling the runtime #7917

mherkazandjian · 2021-04-24T02:21:04Z

This PR fixes issue #7916

to build for aarch64 the cmake variable -DCMAKE_HOST=aarch64-linux-gnu should be passed that
is in turn propagated to the autotools (configure step) for libbacktrace.

cmake .. \
    -DCMAKE_SYSTEM_NAME=Linux \
    -DCMAKE_SYSTEM_VERSION=1 \
    -DCMAKE_C_COMPILER=/usr/bin/aarch64-linux-gnu-gcc-8 \
    -DCMAKE_CXX_COMPILER=/usr/bin/aarch64-linux-gnu-g++-8 \
    -DCMAKE_FIND_ROOT_PATH=/usr/aarch64-linux-gnu \
    -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \
    -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY \
    -DCMAKE_HOST=aarch64-linux-gnu

I tested this to build the runtime for aarch64 and riscv64, the runtime .so was produced in both
cases with no errors displayed during the build.

…tools

tqchen · 2021-04-24T12:52:00Z

@areusch @tkonolige @leandron please help to review this PR

junrushao · 2021-04-24T17:47:30Z

CC @tkonolige @xiebaiyuan The issue is possibly related to #7705.

leandron

Thanks for this PR. As the default value for libbacktrace is AUTO, this patch enables cross-compilation of the source in the original cloning state - great!

On a broader view, it would be awesome to also update the documentation such as https://tvm.apache.org/docs/install/from_source.html, with some guidance for cross-compilation as there are non-obvious steps to be taken, such as the need for -DCMAKE_HOST=... to be populated, for example.

What do you think?

cmake/libs/Libbacktrace.cmake

leandron · 2021-04-26T09:50:03Z

On another note, i think an end to end example on inference on a remote machine would be great.
i could probably contribute that end to end example if it does not exist, does it?

Yes, that would be really useful, specially if it uses TVMC rather than low level APIs, IMHO.

cmake/libs/Libbacktrace.cmake

tkonolige

These changes look good to me.

tkonolige · 2021-04-26T15:37:01Z

btw, I tried these same changes when debugging #7705 but they did not fix it.

areusch

thanks @mherkazandjian for the PR! just one question, I agree with @leandron that it would be great to provide some docs.

cmake/libs/Libbacktrace.cmake

leandron

A few suggestions on the tutorial wording, feel free to decide whether you want to use it. @hogepodge recently did some work on tutorials and might be interested as well.

Apart from the suggestions, LGTM.

docs/deploy/index.rst

docs/install/from_source.rst

mherkazandjian · 2021-04-28T09:55:10Z

@leandron tnx for the feedback. actually the CI build is failing, i think the in the CI we (I) should modify the cmake flags so that
libbacktrace now builds (it is failing in the CI). I will do it now.

leandron

I happy to approve this PR. Did you check @tkonolige's comment about building with debug flags?

mherkazandjian · 2021-04-28T10:46:24Z

I happy to approve this PR. Did you check @tkonolige's comment about building with debug flags?

great. thank you.
i looked into the comment of @tkonolige but i thought it is not strictly related to this issue in the sense that
this PR would not affect that issue negatively. I will look into it closely.

i have a question though, in the CI it looks like the arm build is failing
https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-7917/7/pipeline/#step-163-log-112
should i try to fix it? please advise.

tkonolige · 2021-04-28T15:44:04Z

@mherkazandjian I'm sorry it took me so long to remember this, but I don't think this PR will work on macOS. Here is what I had in my version of this:

# On MacOS, the default C compiler (/usr/bin/cc) is actually a small script that dispatches to a
# compiler the default SDK (usually /Library/Developer/CommandLineTools/usr/bin/ or
# /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/). CMake
# automatically detects what is being dispatched and uses it instead along with all the flags it
# needs. CMake makes this second compiler avaliable through the CMAKE_C_COMPILER variable, but it
# does not make the necessary flags available. This leads to configuration errors in libbacktrace
# because it can't find system libraries. Our solution is to detect if CMAKE_C_COMPILER lives in
# /Library or /Applications and switch to the default compiler instead.
if(CMAKE_SYSTEM_NAME MATCHES "Darwin" AND (CMAKE_C_COMPILER MATCHES "^/Library"
  OR CMAKE_C_COMPILER MATCHES "^/Applications"))
  set(c_compiler "/usr/bin/cc")
else()
  set(c_compiler "${CMAKE_C_COMPILER}")
endif()

Note that this solution is not quite correct. You probably want to set the sdk path to whatever the user set it to.

mherkazandjian · 2021-04-28T16:31:15Z

btw, I tried these same changes when debugging #7705 but they did not fix it.

hi @tkonolige

@mherkazandjian I'm sorry it took me so long to remember this, but I don't think this PR will work on macOS. Here is what I had in my version of this:

# On MacOS, the default C compiler (/usr/bin/cc) is actually a small script that dispatches to a
# compiler the default SDK (usually /Library/Developer/CommandLineTools/usr/bin/ or
# /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/). CMake
# automatically detects what is being dispatched and uses it instead along with all the flags it
# needs. CMake makes this second compiler avaliable through the CMAKE_C_COMPILER variable, but it
# does not make the necessary flags available. This leads to configuration errors in libbacktrace
# because it can't find system libraries. Our solution is to detect if CMAKE_C_COMPILER lives in
# /Library or /Applications and switch to the default compiler instead.
if(CMAKE_SYSTEM_NAME MATCHES "Darwin" AND (CMAKE_C_COMPILER MATCHES "^/Library"
  OR CMAKE_C_COMPILER MATCHES "^/Applications"))
  set(c_compiler "/usr/bin/cc")
else()
  set(c_compiler "${CMAKE_C_COMPILER}")
endif()

Note that this solution is not quite correct. You probably want to set the sdk path to whatever the user set it to.

ok tnx for the update.
Actually the CI is failing for all the builds (see below) not just for arm as i was thinking.

i think we need to pass -DMACHINE_NAME to cmake in the CI build scripts too.

tkonolige · 2021-04-28T16:45:08Z

@mherkazandjian We should not require MACHINE_NAME be passed in as part of the build. The cmake script should check if it is set and then add the correct configure flags.

mherkazandjian · 2021-04-29T00:47:50Z

@mherkazandjian We should not require MACHINE_NAME be passed in as part of the build. The cmake script should check if it is set and then add the correct configure flags.

@tkonolige i fixed the issue with the CI, the CPPFLAGS=${CMAKE_CXX_FLAGS} was the cause, I removed it since
g++ is not used/needed for libbacktrace (it is written in C).

the macOS build seems to be passing

following up on your comment #7917 (comment)
you mean including the snippet below in Libbacktrace.cmake (although ) fixes the issue for macOS? what did you mean
by "although not correct?"
i do not have access to a mac.

if(CMAKE_SYSTEM_NAME MATCHES "Darwin" AND (CMAKE_C_COMPILER MATCHES "^/Library"
  OR CMAKE_C_COMPILER MATCHES "^/Applications"))
  set(c_compiler "/usr/bin/cc")
else()
  set(c_compiler "${CMAKE_C_COMPILER}")
endif()

mherkazandjian · 2021-04-29T00:50:46Z

@tqchen it looks like the docker daemon did not start in the CI instance
https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-7917/9/pipeline/259
how can one re-trigger the build? it looks like an issue with the CI machine/vm itself.
( i can just push a dummy commit to re-trigger...)

areusch · 2021-04-29T03:08:03Z

@mherkazandjian i think the CI is having some trouble now, feel free to push an empty commit to retriever

hogepodge · 2021-04-30T18:02:15Z

We put images into the https://github.com/tlc-pack/web-data repository.

…ng approval

… branch of web-data

mherkazandjian · 2021-05-03T07:53:40Z

@hogepodge did you get a chance to look at the changes? i think this PR is good to go (the CI tests pass too)

tkonolige

You need a couple of changes to make the macOS workaround work.

cmake/libs/Libbacktrace.cmake

docs/deploy/index.rst

tqchen · 2021-05-11T16:21:02Z

gentle ping @mherkazandjian please follow up on the review comments and let us merge soon !

mherkazandjian · 2021-05-12T10:36:35Z

gentle ping @mherkazandjian please follow up on the review comments and let us merge soon !

Hi @tqchen
sorry got distracted with some other work. I'll wrap this up before the end of the week.

mherkazandjian · 2021-05-22T19:25:43Z

@tqchen i finished all the requested changes :)

tqchen · 2021-05-24T13:09:43Z

@areusch @leandron @tkonolige @hogepodge please take another look and https://tvm.apache.org/docs/contribute/code_review.html?highlight=approve#approve-and-request-changes-explicitly

hogepodge

Throughout this document, there's a mix of using "we" and "you" to describe the process of building TVM. This should be reconciled, preferably to use the 3rd person. That may be a larger task for another PR. A number of other comments to help clarify language and usage.

hogepodge · 2021-05-24T15:51:08Z

cmake/libs/Libbacktrace.cmake

@@ -14,14 +14,39 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
+
+# On MacOS, the default C compiler (/usr/bin/cc) is actually a small script that dispatches to a
+# compiler the default SDK (usually /Library/Developer/CommandLineTools/usr/bin/ or


On MacOS, the default C compiler (/usr/bin/cc) is script that dispatches to the compiler
provided by the default SDK (usually /Library/Developer/CommandLineTools/usr/bin/ or

hogepodge · 2021-05-24T15:52:41Z

cmake/libs/Libbacktrace.cmake

+# needs. CMake makes this second compiler avaliable through the CMAKE_C_COMPILER variable, but it
+# does not make the necessary flags available. This leads to configuration errors in libbacktrace
+# because it can't find system libraries. Our solution is to detect if CMAKE_C_COMPILER lives in
+# /Library or /Applications and switch to the default compiler instead.


Our solution is to detect if CMAKE_C_COMPILER is installed in

hogepodge · 2021-05-24T15:54:48Z

docs/deploy/index.rst

 - TVM runtime, which runs on the target devices.

-In order to integrate the compiled module, we **do not** need to build entire TVM on the target device. You only need to build the TVM compiler stack on your desktop and use that to cross-compile modules that are deployed on the target device.
+In order to integrate the compiled module, we **do not** need to build entire
+TVM on the target device. You only need to build the TVM compiler stack on your 


we do not need to build the entire TVM project on the target device. Rather, we only need to build the TVM compiler stack on a host device and use that to cross-compile modules that can be deployed to a target device.

tqchen · 2021-06-01T01:02:02Z

Thank you @mherkazandjian ! Thanks @tkonolige @hogepodge @areusch @leandron for reviewing

…e#7917)

modify the libbacktrace cmake file to pass the toolchain info to auto…

45a744e

…tools

tqchen assigned areusch Apr 24, 2021

tqchen added the status: need review label Apr 24, 2021

junrushao linked an issue Apr 24, 2021 that may be closed by this pull request

libbacktrace does not inherit the CMAKE_C_COMPILER and CMAKE_CXX_COMPILER values of the top level project #7916

Closed

leandron reviewed Apr 26, 2021

View reviewed changes

cmake/libs/Libbacktrace.cmake Outdated Show resolved Hide resolved

leandron reviewed Apr 26, 2021

View reviewed changes

cmake/libs/Libbacktrace.cmake Outdated Show resolved Hide resolved

tkonolige approved these changes Apr 26, 2021

View reviewed changes

areusch reviewed Apr 26, 2021

View reviewed changes

cmake/libs/Libbacktrace.cmake Show resolved Hide resolved

mherkazandjian added 3 commits April 27, 2021 11:34

replace CMAKE_C_FLAGS by CMAKE_CXX_FLAGS for the value of CPPFLAGS

6abee49

udpate the documention for cross compiling the runtime library

77c18e6

Merge branch 'main' into mher/libbacktrace-cmake-fix

8066ce6

leandron reviewed Apr 27, 2021

View reviewed changes

mherkazandjian added 2 commits April 28, 2021 11:45

fix typo in the documentation

c257e97

use uppercase for raspberry pi

369e010

fix the wording of some sentences in the documentation

e7ba943

leandron reviewed Apr 28, 2021

View reviewed changes

mherkazandjian added 2 commits April 28, 2021 23:53

remove the CXX flags from list of flags propagated to libbacktrace

ed25260

Merge branch 'main' into mher/libbacktrace-cmake-fix

4932f47

mherkazandjian mentioned this pull request Apr 30, 2021

add diagram for cross compiling model and runtime for target tlc-pack/web-data#5

Merged

mherkazandjian added 3 commits April 30, 2021 22:29

move the image to the web content repo - private branch for now pendi…

fd358ec

…ng approval

replace the private repo branch figure link with the link in the main…

9f8db83

… branch of web-data

a33669d

tkonolige requested changes May 3, 2021

View reviewed changes

cmake/libs/Libbacktrace.cmake Outdated Show resolved Hide resolved

cmake/libs/Libbacktrace.cmake Outdated Show resolved Hide resolved

cmake/libs/Libbacktrace.cmake Show resolved Hide resolved

hogepodge reviewed May 3, 2021

View reviewed changes

docs/deploy/index.rst Outdated Show resolved Hide resolved

tqchen added the status: need update need update based on feedbacks label May 11, 2021

mherkazandjian added 4 commits May 22, 2021 15:49

Merge branch 'main' into mher/libbacktrace-cmake-fix

a7221c8

add comment on the procedure used by MacOS to find the right toolchain

b1f79a8

use the define c_compiler variable in the cmake file for libbacktrace

c5585fe

remove duplicate sentence in the deploy/integration doc

0a9f3ae

tqchen removed the status: need update need update based on feedbacks label May 24, 2021

hogepodge suggested changes May 24, 2021

View reviewed changes

tkonolige approved these changes May 24, 2021

View reviewed changes

tqchen approved these changes Jun 1, 2021

View reviewed changes

tqchen merged commit 06a466c into apache:main Jun 1, 2021

tqchen added status: accepted and removed status: need review labels Jun 1, 2021

mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Jun 3, 2021

allow libbacktrace to be used when cross compiling the runtime (apach…

a70629c

…e#7917)

mherkazandjian deleted the mher/libbacktrace-cmake-fix branch June 10, 2021 21:26

trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Jun 17, 2021

allow libbacktrace to be used when cross compiling the runtime (apach…

40ec81e

…e#7917)

trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Jun 17, 2021

allow libbacktrace to be used when cross compiling the runtime (apach…

9c593a1

…e#7917)

tqchen mentioned this pull request Jul 21, 2021

build tvm failed with libbacktrace.a : ld: symbol(s) not found for architecture x86_64 #7705

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

allow libbacktrace to be used when cross compiling the runtime #7917

allow libbacktrace to be used when cross compiling the runtime #7917

mherkazandjian commented Apr 24, 2021

tqchen commented Apr 24, 2021

junrushao commented Apr 24, 2021

leandron left a comment

leandron commented Apr 26, 2021

tkonolige left a comment

tkonolige commented Apr 26, 2021

areusch left a comment

leandron left a comment

mherkazandjian commented Apr 28, 2021

leandron left a comment

mherkazandjian commented Apr 28, 2021

tkonolige commented Apr 28, 2021

mherkazandjian commented Apr 28, 2021

tkonolige commented Apr 28, 2021

mherkazandjian commented Apr 29, 2021

mherkazandjian commented Apr 29, 2021 •

edited

Loading

areusch commented Apr 29, 2021

hogepodge commented Apr 30, 2021

mherkazandjian commented May 3, 2021

tkonolige left a comment

tqchen commented May 11, 2021

mherkazandjian commented May 12, 2021

mherkazandjian commented May 22, 2021

tqchen commented May 24, 2021

hogepodge left a comment

hogepodge May 24, 2021

hogepodge May 24, 2021

hogepodge May 24, 2021

tqchen commented Jun 1, 2021

allow libbacktrace to be used when cross compiling the runtime #7917

allow libbacktrace to be used when cross compiling the runtime #7917

Conversation

mherkazandjian commented Apr 24, 2021

tqchen commented Apr 24, 2021

junrushao commented Apr 24, 2021

leandron left a comment

Choose a reason for hiding this comment

leandron commented Apr 26, 2021

tkonolige left a comment

Choose a reason for hiding this comment

tkonolige commented Apr 26, 2021

areusch left a comment

Choose a reason for hiding this comment

leandron left a comment

Choose a reason for hiding this comment

mherkazandjian commented Apr 28, 2021

leandron left a comment

Choose a reason for hiding this comment

mherkazandjian commented Apr 28, 2021

tkonolige commented Apr 28, 2021

mherkazandjian commented Apr 28, 2021

tkonolige commented Apr 28, 2021

mherkazandjian commented Apr 29, 2021

mherkazandjian commented Apr 29, 2021 • edited Loading

areusch commented Apr 29, 2021

hogepodge commented Apr 30, 2021

mherkazandjian commented May 3, 2021

tkonolige left a comment

Choose a reason for hiding this comment

tqchen commented May 11, 2021

mherkazandjian commented May 12, 2021

mherkazandjian commented May 22, 2021

tqchen commented May 24, 2021

hogepodge left a comment

Choose a reason for hiding this comment

hogepodge May 24, 2021

Choose a reason for hiding this comment

hogepodge May 24, 2021

Choose a reason for hiding this comment

hogepodge May 24, 2021

Choose a reason for hiding this comment

tqchen commented Jun 1, 2021

mherkazandjian commented Apr 29, 2021 •

edited

Loading