Issue building on MacOS

[Alex-Mann]

Hi there,

I understand there that the C++ compiler that comes with Xcode is not supported for building verible... but are there any more details for how sub out the compiler and build on MacOS? Currently I'm getting the following error:

❯ bazel build -c opt //...
INFO: Analyzed 1195 targets (5 packages loaded, 1752 targets configured).
INFO: Found 1195 targets...
ERROR: /Users/alexandermann/Work/verible/common/text/BUILD:183:1: C++ compilation of rule '//common/text:tree_utils' failed (Exit 1) wrapped_clang failed: error executing command external/local_config_cc/wrapped_clang '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 28 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from common/text/tree_utils.cc:15:
./common/text/tree_utils.h:115:55: error: unknown type name 'nullptr_t'; did you mean 'std::nullptr_t'?
const SyntaxTreeNode* CheckOptionalSymbolAsNode(const nullptr_t& symbol, E) {
                                                      ^~~~~~~~~
                                                      std::nullptr_t
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__nullptr:56:31: note: 'std::nullptr_t' declared here
    typedef decltype(nullptr) nullptr_t;
                              ^
In file included from common/text/tree_utils.cc:15:
./common/text/tree_utils.h:130:55: error: unknown type name 'nullptr_t'; did you mean 'std::nullptr_t'?
const SyntaxTreeLeaf* CheckOptionalSymbolAsLeaf(const nullptr_t& symbol, E) {
                                                      ^~~~~~~~~
                                                      std::nullptr_t
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__nullptr:56:31: note: 'std::nullptr_t' declared here
    typedef decltype(nullptr) nullptr_t;
                              ^
2 errors generated.
INFO: Elapsed time: 10.413s, Critical Path: 2.66s
INFO: 10 processes: 10 darwin-sandbox.
FAILED: Build did NOT complete successfully

Any tips or help is appreciated.

[fangism]

Hi, we would like to support clang builds, and this is general help-wanted area. There are a few known issues at this time.

For your particular issue, it looks like tree_utils.h references nullptr_t without #include <cstddef>, which is the origin header, according to https://en.cppreference.com/w/cpp/types/nullptr_t .

Can you try adding #include <cstddef> near the top of the header to see if that makes further progress?

[fangism]

I've also observed in our codebase, we don't consistently qualify std::nullptr_t, but we probably should. (And directly addresses the diagnostic suggested by clang in the error you showed.)

[mithro]

If someone gets verible building and working on MacOS X, then we can set up Travis (or Kokoro) to make sure it continues to work and also publish prebuilt binaries in the same way we do Linux executables.

[Alex-Mann]

Alright, it looks like it made some progress. My Cpp fu is not great though so I'm a little blind... error is as follows (I am also using zsh as my shell, which is the default for MacOS now):

Error

❯ bazel build -c opt //...
INFO: Analyzed 1195 targets (2 packages loaded, 836 targets configured).
INFO: Found 1195 targets...
INFO: From Executing genrule //common/util:version_header [for host]:
date: illegal time format
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
ERROR: /Users/alexandermann/Work/verible/bazel/BUILD:20:1: error executing shell command: '/bin/bash -c function cleanup_function() {
local ecode=$?
if [ $ecode -eq 0 ]; then
cleanup_on_success
else
cleanup_on_failure
fi
}
set -e
function cleanup_on_success() {
printf "rules_foreign_cc: ...' failed (Exit 1) bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox

rules_foreign_cc: Build failed!
rules_foreign_cc: Keeping temp build directory  and dependencies directory  for debug.
rules_foreign_cc: Please note that the directories inside a sandbox are still cleaned unless you specify '--sandbox_debug' Bazel command line flag.

rules_foreign_cc: Printing build logs:

_____ BEGIN BUILD LOGS _____
Bazel external C/C++ Rules #0.0.8. Building library 'm4'
Environment:______________DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
TMPDIR=/var/folders/6r/9xry38yd1479gfjvj1y15r3c0000gn/T/
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk
EXT_BUILD_ROOT=/private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible
XCODE_VERSION_OVERRIDE=11.4.0.11E146
INSTALLDIR=/private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/bazel-out/darwin-opt/bin/bazel/m4
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x52
PATH=/private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible:/usr/gnu/bin:/usr/local/bin:/bin:/usr/bin:.
BUILD_TMPDIR=/var/folders/6r/9xry38yd1479gfjvj1y15r3c0000gn/T/tmp.aH5KCMjE
PWD=/private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible
EXT_BUILD_DEPS=/var/folders/6r/9xry38yd1479gfjvj1y15r3c0000gn/T/tmp.vPSIilyK
SHLVL=2
BUILD_LOG=bazel-out/darwin-opt/bin/bazel/m4/logs/Configure.log
BUILD_SCRIPT=bazel-out/darwin-opt/bin/bazel/m4/logs/Configure_script.sh
APPLE_SDK_PLATFORM=MacOSX
APPLE_SDK_VERSION_OVERRIDE=10.15
_=/usr/bin/env
__________________________PKG_CONFIG_PATH=
configure: WARNING: unrecognized options: --disable-nls
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/m4/build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for gcc... /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/wrapped_clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/wrapped_clang accepts -g... yes
checking for /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/wrapped_clang option to accept ISO C89... none needed
checking whether /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/wrapped_clang understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/wrapped_clang... gcc3
checking how to run the C preprocessor... /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/wrapped_clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking whether _XOPEN_SOURCE should be defined... no
checking for Minix Amsterdam compiler... no
checking the archiver (/private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/libtool) interface... unknown
configure: error: could not determine /private/var/tmp/_bazel_alexandermann/2810196003ad00dcc269ad8b02549ff0/sandbox/darwin-sandbox/75/execroot/com_google_verible/external/local_config_cc/libtool interface

_____ END BUILD LOGS _____
rules_foreign_cc: Build script location: bazel-out/darwin-opt/bin/bazel/m4/logs/Configure_script.sh
rules_foreign_cc: Build log location: bazel-out/darwin-opt/bin/bazel/m4/logs/Configure.log

INFO: Elapsed time: 7.517s, Critical Path: 6.38s
INFO: 40 processes: 40 darwin-sandbox.
FAILED: Build did NOT complete successfully

[fangism]

The general ticket for OS X build support is #64 .

The build issue with m4 is a known one that we haven't gotten past yet.

[fangism]

@Alex-Mann Nevertheless, we would like to accept your fix of nullptr_t (and give you author credit), care to submit just that change as a PR?

[hzeller]

the date call for creating the version header is date +"%Y-%m-%dT%H:%M:%SZ" outputting something like 2020-04-08T23:30:51Z.

I wonder if maybe the default date implementatino on a Mac (or the zsh?) does not allow for all strformatting characters or has trouble with non-format characters such as T or Z in the format string ? @Alex-Mann or @fangism who have access to a Mac, can you play around with the date format to see what the culprit is ?

[fangism]

Darwin (Mac OS X) gets date from the BSD subsystem commands.

Man page:
https://www.freebsd.org/cgi/man.cgi?date

[fangism]

Darwin:

$ date +"%Y-%m-%dT%H:%M:%SZ"
2020-04-08T23:49:37Z

Linux:

$ date +"%Y-%m-%dT%H:%M:%SZ"
2020-04-08T23:50:10Z

looks similar

[hzeller]

I just compiled the BSD date ( https://github.com/freebsd/freebsd/tree/master/bin/date ), and it also outputs things as expected

$ ./date +"%Y-%m-%dT%H:%M:%SZ"
2020-04-08T23:50:31Z

I wonder if maybe the git commands that extract a date, mmh.

So some more investigation manually calling common/util/create_version_header.sh would probably be needed.

[hzeller]

Fixed the date issue in #263
Is the next thing left to make m4 work ?

[renau]

I use m4/flex/bison with bazel (liveHD project) in linux/OSX using these rules.

https://github.com/jmillikin/rules_flex
https://github.com/jmillikin/rules_bison

As an example using the flex/bision/m4 when compiling yosys with bazel (LiveHD) (yosys uses flex/bison).
https://github.com/masc-ucsc/livehd/blob/e086c905d2216e87f15c6f8a9e4cedd873250811/external/BUILD.yosys#L52

[fangism]

More notes: Here are the rules that the Kythe project uses for locally-installed flex/bison:

https://cs.opensource.google/kythe/kythe/+/master:tools/build_rules/lexyacc/BUILD
https://cs.opensource.google/kythe/kythe/+/master:tools/build_rules/lexyacc/lexyacc.bzl

This pushes the problem of dependencies like m4 onto the user's package management system.

[hzeller]

Looks like I can compile it with clang (on Linux) by manually setting the necessary link option for clang. However, I don't know enough bazel-fu to make this toolchain-dependent (i.e. only setting this when clang is in use)

patch -p1 <<EOF
--- a/bazel/BUILD
+++ b/bazel/BUILD
@@ -24,6 +24,7 @@ configure_make(
     ],
     configure_options = [
         "--disable-nls",
+        "LDFLAGS='-rtlib=compiler-rt'",
     ],
     lib_source = "@m4//:all",
 )
EOF

Then, compiling and testing works with clang:

CC=clang CXX=clang++ bazel test //...

@fangism can you test if this will make it work on MacOS ?

[rachitnigam]

@hzeller I attempted to build on Mac with your patch. I get the same error message that @Alex-Mann got:

...
configure: error: could not determine /private/var/tmp/_bazel_rachit/7df8cf3decab2486ed2413d61d6c5bdb/sandbox/darwin-sandbox/1099/execroot/com_google_verible/external/local_config_cc/libtool interface

_____ END BUILD LOGS _____
rules_foreign_cc: Build script location: bazel-out/darwin-opt/bin/bazel/m4/logs/Configure_script.sh
rules_foreign_cc: Build log location: bazel-out/darwin-opt/bin/bazel/m4/logs/Configure.log

Target //:install failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /Users/rachit/git/verible/verilog/tools/lint/BUILD:215:10 error executing shell command: '/bin/bash -c function cleanup_function() {
local ecode=$?
if [ $ecode -eq 0 ]; then
cleanup_on_success
else
cleanup_on_failure
fi
}
set -e
function cleanup_on_success() {
printf "rules_foreign_cc: ...' failed (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 7.129s, Critical Path: 4.90s
INFO: 57 processes: 16 internal, 41 darwin-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

I assume that this is a problem with building m4.

[hzeller]

mmh, let's drill down to m4 specifically and see what we get. Can you try the following (after applying the patch) @rachitnigam ?

CC=clang CXX=clang++ bazel build bazel:m4

[rachitnigam]

I get:

configure: error: could not determine /private/var/tmp/_bazel_rachit/7df8cf3decab2486ed2413d61d6c5bdb/sandbox/darwin-sandbox/1/execroot/com_google_verible/external/local_config_cc/libtool interface

[hzeller]

This looks like you might need to install libtool and possiblye autoconf.
On MacOS, this is probably brew install libtool ?

[rachitnigam]

I already have them installed (I found this github issue when trying to fix this) :

⇒  libtool -V
Apple Inc. version cctools-949.0.1

⇒  autoconf -V
autoconf (GNU Autoconf) 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+/Autoconf: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>, <http://gnu.org/licenses/exceptions.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[rachitnigam]

It's worth noting that brew install libtool defines the glibtool alias (since it conflict's with Apple's libtool):

⇒  glibtool --version
glibtool (GNU libtool) 2.4.6
Written by Gordon Matzigkeit, 1996

I also tried running the bazel command with alias libtool = glibtool but no luck.

[hzeller]

So it looks like that the local_config_cc has trouble dealing with the libtool somehow. Hard to tell what is going on remotely; did you see anything interesting in the logs ? For instance, it would be good if bazel left behind the config.log generated by the configure script, but not sure if it does.

[rachitnigam]

I don't see config.log unfortunately. Is there a way to force bazel to generate/save that file?

[hzeller]

Alright, after a little bit of work, it should now be possible to compile verible on MacOS.

There is one external dependency on a somewhat recent flex and bison, so before compiling you need to have these installed and set the path to their location:

brew install flex bison
export PATH=$(brew --prefix flex)/bin:$(brew --prefix bison)/bin:$PATH

Then

bazel test ...
bazel build ...

should work.

There is one caveat though: the install procedure directly from bazel is not working yet as the install program used on BSD is slightly different than on GNU systems; This is tracked in google/bazel_rules_install#31

[hzeller]

One of the next steps: Creating a fresh binary automatically with each release #758

[johanvdhaegen]

GNU install is included in the coreutils homebrew package. $(brew --prefix coreutils)/libexec/gnubin is the path for the core utils with the g prefix in the name.

This is a proof-of-concept homebrew formula. The following works:

brew install --HEAD verible
brew test verible
verible-verilog-format -helpfull

There are still some unresolved issues:

• The FileUtil test fails if bazel test is included in the install part of the formula (CreateDir and StatusErrorReporting test cases)
• For some reason, bazel decides to rebuild between bazel build, bazel test, bazel run:
  INFO: Build options --cxxopt, --define, and --host_cxxopt have changed, discarding analysis cache.

A stable version and a URL with git hash is needed to brew bottles. This then needs to be updated on each commit to produce new bottles.

[hzeller]

Oh, you are working on a brew packet ? It looks like you might enjoy tackling #758 @johanvdhaegen ?

I think by default, bazel builds with -c fastbuild while in the install target it uses -c opt; this could be the reason why it re-builds for the install run as the flags change.

Do you have a log of the failure with the FileUtil ? If it is a system incompatibilty issue, maybe we should switch to use #include <filesystem> which is a C++17-ism, but I guess in 2021 we could go that route and assume all compilers can deal with it.

I could also imagine that it is some sandboxing issue, that we want to write somewhere where the sandbox does not allow to. Though it would be a bit weird as bazel already has a pretty tight sandbox around running tests.

With regard to the stable version requirements, we create a tag on each commit and create a release from there ( https://github.com/google/verible/releases ). So if we can hook to the brew build, that would be awesome (A discussion we should have on #758 ).

[hzeller]

Update:

brew install flex bison

Is not needed anymore for building verible on the Mac. So it should build straight with bazel build ...

[johanvdhaegen]

Yes, I verified that flex and bison are not needed anymore.

[johanvdhaegen]

The test errors I have been seeing are from permission errors on /tmp: on MacOS, gtest::TempDir is hard coded to /tmp and ignores the TEST_TMPDIR environment variable, which bazel sets to a private directory according to the bazel test encyclopdia.

https://github.com/google/googletest/blob/master/googletest/src/gtest.cc#L6723

I run homebrew under an account different from my regular user account, and test files left behind in /tmp by one user account cause permission errors for the other user account.

[hzeller]

I'll send a patch to gtest upstream.

[poetaman]

@hzeller I get build issues on M1 mac, have created a ticket: chipsalliance/homebrew-verible#10

[hzeller]

@reportaman did you also try to compile directly without the brew package ? The brew package is still work-in-progress, so extracting the error from a direct compile might help find the issue.

[poetaman]

@hzeller I had, and I just did it again. Here's the result:

❯ bazel run -c opt :install -- ~/.bin
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
ERROR: /Users/reportaman/Downloads/verible/verilog/tools/kythe/BUILD:206:10: Target '//verilog/tools/kythe:verible-verilog-kythe-extractor' depends on toolchain '@local_config_cc//:cc-compiler-darwin_arm64', which cannot be found: no such target '@local_config_cc//:cc-compiler-darwin_arm64': target 'cc-compiler-darwin_arm64' not declared in package '' defined by /private/var/tmp/_bazel_reportaman/33a4a5dc8fe326811efbef181e76bf63/external/local_config_cc/BUILD'
ERROR: Analysis of target '//:install' failed; build aborted: Analysis failed
INFO: Elapsed time: 12.350s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (52 packages loaded, 78 targets configured)
FAILED: Build did NOT complete successfully (52 packages loaded, 78 targets configured)

[hzeller]

That error message sounds like it didn't find a compiler (cc-compiler-darwin_arm64), which might be a toplevel thing to be configured for bazel. Does bazel work for other projects ?

[poetaman]

@hzeller Am using bazel for the very first time, not sure what would be the correct option. Came across this: bazelbuild/bazel#11628, though its not clear what needs to be done.

[rkshthrmsh]

Hi, I was wondering if it's now possible to install verible / build on MacOS?

[hzeller]

Verible builds for a while now on MacOs.

There is a homebrew build and also the releases contain macOS binary builds.

Only thing that is not working is that the install target would work. But I can't do anything about it as I don't have a mac, so waiting for someone with a mac to provide a pull request. Till then, use the .github/bin/simple-install.sh script to install on a mac.