Support for Windows

[corco]

I have been looking to add support for Windows but hit a few brick walls. I'm looking for advice on the best way to do that for the project, assuming supporting Windows is desired of course!

I looked at the current setup using cc_configure_make. cc_configure_make does not support Windows and besides, I don't believe m4/flex/bison will compile anyways.

I also looked rules_m4/rules_flex/rules_bison. m4/bison compiles (bison has a small issue at running) but flex does not compile on Windows and it doesn't look like it will anytime soon. However, the three repos works on OSX, so if you're interested I can do a pull request to replace cc_configure_make with these repos.

Finally, to support Windows I see three ways forward:

1. Commit flex/bison generated files in the repository, removing the requirements on flex/bison. We could keep the bazel rules to generate the files and a test, for CI, to make sure the committed files match the sources.
2. Compile for Windows on a LInux machine with MinGW. m4/flex/bison would be compiled with the host compiler, and verible on MinGW. It would not be possible to build on a Windows machine though.
3. Look to use winflexbison on Windows.

[fangism]

Support for building on Windows is welcome, but I have zero experience developing on Windows myself. (We've been having trouble getting m4 to build on Mac OS X.)

1. is not a bad idea, and is compatible with automake-style dist_ files. Perhaps this could be automated on branch that generates source distributions. @mithro Does this seem practical?

[fangism]

FYI, In #370 , @muhdmud [1] was able to follow instructions at https://docs.microsoft.com/en-us/windows/wsl/install-win10 before bazel build instructions to successfully build Verible.

[1] #370 (comment)

[corco]

WSL allows to run Linux application on Windows, but it is not the same as a native Windows application. For starters, you need to have WSL installed on your Windows machine and I don't think you can invoke WSL application from Windows command line and vice-versa. I don't use Windows so I may be wrong there.

I managed to build a native Windows verible by using generated files from flex/bison instead of building them. I wanted to keep the file generation as a bazel test that would run only on Linux.

However, I struggled with bazel as there doesn't seem to be an easy way to exclude rule on some platform, which make the "bazel test //..." fail on Windows. There is a bazel open issue that could lead to a fix once resolved. There may be another way to fix that, but it would need a bazel expert...

[mithro]

There might be some help looking at #260 (comment) ?

[suzizecat]

Hello there.
As I was considering integrating some verible support in TerosHDL (in particular with verible indexer), one semi-blocking point is that Verible can not be used with Windows (As teros is multiplatform).
Is there an update on this issue status ?

[hzeller]

We can use help here @suzizecat . None of the main contributors to Verible has experience on Windows.

It sounds like a lot of issues on other platforms (Mac and Windows) is due to bison or flex not compiling cleanly there, but limited reports show that it is possible to use an existing version of flex/bison on the machines in question, then the rest would compile. Maybe this is a good way to explore.

The Kythe subsystem we're linking to apparently is already going that path, so maybe this might be something we can extend to Verible ?

If you have access to a Windows machine with a setup to build stuff, help would be most welcome.

[corco]

@hzeller Another solution is to commit the outputs of flex/bison in the repository and add a github action to check that they are in sync. Considering how few commits these files have, I would think this approach is acceptable.

However, kythe does not work on Windows. As it is a dependency, verible won't be able to compile on Windows until they are able as well.

[hzeller]

The generated code by bison/flex differs per platform. E.g. <unistd.h> is included on Unix-like platforms, but probably something different in Windows, @corco ? (though it looks like that be switched off in flex; will have a look at that).

Also the generated code is not entirely self-contained: it needs to have an include (at least flex does, not sure about bison) that is coming with the particular version of the code generator including that library.
So I suspect that pre-generated code will open a large can of worms but I am happy to be convinced otherwise.

[hzeller]

Looks like the dependency on the header can be reduced by making a pure C lexer.

[suzizecat]

I will have a look at some time. However, as I will need to install a full toolchain to experiment, I might not be able to do that in the near future.

Just to mention it to you guys, did you knew about win_flexbison which seems to be a port of flex/bison to Windows (which seems to be, AFAIU, the current issue in making verible on Windows)

[corco]

I tried to hack something with win_flexbison Today and got pretty far.

There are a lot of issues remaining in order to build, but they seem to come from using a different compiler (with different set of warnings/errors) more than from being on Windows. I do get different failing point depending on if I use msvc or clang-cl.

There is at least 1 file completely incompatible with Windows: common/util/file_util.cc. It should be re-written to not use dirent.h (comments suggest std::filesystem...).

I will continue looking down that route next week, but it seems promising and could potentially make the kythe targets available on Windows as well.

[hzeller]

Don't worry too much about Kythe @corco : we're in the process of reducing the dependency and not pull in full kythe but maybe only the protocol buffers; @ivan444 is working on this. That means that Kythe would not need to be compiled as part of the build process.

So this makes the following two topics higher priority if you want to work on these:

If you find warnings/errors with other compilers, it might be worthwhile looking into these and send pull requests to address them. Having different sets of compilers looking at code and generating warnings to address helps improve the code quality (not always, but it is good to look at warnings that uncover actual potential issues).
(I use the --keep_going option so that bazel does not stop with the first encounter of an error; easier to see more instances of the same problem in one go).

For std::filesystem, we need to transition to C++17 ( #765 ). I don't think it is far away, but some work needs to be done (originally, we wanted to make sure to be compatible with C++11, but in the year 2021 this is less and less of a concern).
So this might also be something to work on.

[suzizecat]

I'm not sure but I think this is the right place to ask, will there be a Windows artefact as there are for ubuntu and centos in CI release job ?

[corco]

Yes, I plan to look into the release process and add Windows artefacts now that it compiles.

[qarlosalberto]

It would be really nice to have the Window artifact

[hzeller]

The releases now also have Windows executables: https://github.com/chipsalliance/verible/releases
Please try and see if things work for you.