Port to MSVC with CMake script #567

harrysummer · 2019-04-02T22:15:47Z

This PR aims to make it possible to build SILE on Windows. The building script is written in CMake. Dependencies are fetched and built in the building process. The PR would resolve #410 and workaround #82 (by avoiding using MinGW).

Steps towards merging:

Fix any CI build errors, especially making sure the PR doesn't break Linux build. This also includes the temporary Windows CI from here.
Merge libtexpdf from Port to MSVC with CMake script libtexpdf#6 .
Change .gitmodules to point libtexpdf to master branch in https://github.com/simoncozens/libtexpdf .
Create an Azure Pipeline CI build by duplicating this one, to build upon every new PR and every new commit in master. (maybe @simoncozens need to do this?)
Replace the build badge in README.md.
Merge this PR.

Note:

The dependencies are fetched with CMake ExternalProject commands which prefers CMake project for best integration. However, not all dependencies come with a CMake script for building. To workaround this, we are either using some unofficial git repositories for those projects, or applying a patch with CMake script and minimal neccessary changes. To summarize,
- Official package / mirror without modification:
  - zlib: https://github.com/madler/zlib
  - libpng: https://github.com/glennrp/libpng
  - libexpat: https://github.com/libexpat/libexpat
  - freetype2: https://github.com/aseprite/freetype2
  - harfbuzz: https://github.com/harfbuzz/harfbuzz
  - perl: http://strawberryperl.com/download/ (perl is already provided by Azure CI, but just want to add it explicitly for people who want to build SILE without installing it)
  - openssl: https://github.com/openssl/openssl (use perl script for building)
  - luarocks: https://github.com/luarocks/luarocks (need to configure a VS commandline before building it)
- Official packge with modification
  - fontconfig: https://github.com/freedesktop/fontconfig (version-specific dirty patch applied)
  - LuaJIT: https://github.com/LuaJIT/LuaJIT (only CMakeLists.txt added)
- Unofficial package but seems well-maintained
  - libiconv: https://github.com/win-iconv/win-iconv/ (Windows-port)
  - icu: https://github.com/hunter-packages/icu
  - srlua: https://github.com/LuaDist/srlua (we added a patch to set SILE_PATH before running SILE)
- Unofficial package and out-dated / abandoned
  - ragel: https://github.com/tamaskenez/ragel-cmake (used for generating source code of harfbuzz, so it is not a big issue here)
The current Azure Pipeline build is using VS2017 compiler because VS2019 hosts don't have up-to-date CMake as of writing. We can upgrade to VS2019 after https://github.com/Microsoft/azure-pipelines-image-generation/issues/754 is solved.

coveralls · 2019-04-02T22:27:53Z

Coverage remained the same at 92.392% when pulling 31291a5 on harrysummer:win32 into f4e72e8 on simoncozens:master.

alerque · 2019-04-03T06:00:20Z

Hey @harrysummer I'm really glad to see this contribution coming through. I haven't used Windows myself in over a decade now, but it will make a difference to the ecosystem and open this up to more use cases.

Can I suggest we do a bit of rebase work with this branch? I understand it's been a work in progress, but it's hard to even review and will certainly make a mess in the history if merged as is. For example there are nearly a dozen bumps to the submodule version. This will make any future work with git bisect trying to track down issues a nightmare. One submodule bump commit early on in the branch should be fine, all the rest can be made fixup commits of that one.

I'd be happy to help with this. In fact since this is a PR I can even do some of that work real quick myself if it's okay with you that I muck around in this branch.

harrysummer · 2019-04-03T06:18:23Z

@alerque Do you mean squash-and-merge? I think GitHub has that feature when you are to merge a PR if want a clean history. I'm also fine if you want me to do the squash rebase if that makes code review easier.

alerque · 2019-04-03T06:27:26Z

No I don't think this should all get squashed into one commit at merge time, having some granular history to follow how it came together is a good thing. I just think the history needs a bit of cleanup using rebase (which yes, can squash related commits into one, etc.). The idea isn't to flatten the whole thing, just to clean up the artifacts a bit.

harrysummer · 2019-04-03T06:37:06Z

OK. I think I can handle with reorganizing this PR. If you think it's good, I will put the changes into 4 commits:

Modification of the original code for Win32 compatibility.
CMakeLists.txt and libtexpdf bump
Patches of dependencies
CI build script

For libtexpdf, there will two commits:

Modificatoin of the code.
CMakeLists.txt

alerque · 2019-04-03T06:53:58Z

That would certainly work. I was thinking even less drastic than that. For example I just did a rebase on your branch and stuck it on my fork that cleans of the commit messages, squashes a couple of redundant things, etc. If you like I can push that branch in place of the one this PR is based on as a starting point.

Some further changes will probably be necessary. In particular the submodule commits might need revisiting before merging, but...

…to corresponding win32 function. Change to guarding macro from __MINGW32__ to WIN32.

harrysummer · 2019-04-03T07:21:00Z

Commit clean done.

simoncozens · 2019-04-03T09:13:58Z

I will have a look at this soon.

One question: we have half an appveyor CI build on Windows set up already, but obviously I never got it working. Would it be easier to work with that rather than Azure Pipeline? (I know very little about either.)

alerque · 2019-04-03T09:36:31Z

I was going to mention, if this PR sticks to Azure it should also add a commit that nukes the Appveyor bits.

harrysummer · 2019-04-03T12:04:49Z

Would it be easier to work with that rather than Azure Pipeline? (I know very little about either.)

I never used Appveyor before but it looks to me Appveyor is queuing a long time before it starts to build. On the other hand, Azure Pipeline is way faster and you could have 10 parallel building jobs for free. I think this is currently the main benefit to use Azure Pipeline. In the future, I am positive that Azure Pipeline will get even better integration with Microsoft's GitHub.

It is just a few clicks to set all things up. I am here to help if you need anything. One thing that could possibly be missing is setting up pull request as a triggering condition. We can add that later after CI is setup.

simoncozens · 2019-04-03T13:19:17Z

OK, I have signed up for Azure pipeline and associated a Github trigger with this repository. What I need from you is the "azure-pipelines.yml" file which defines your pipeline (I can't view the details of your pipeline at the URL you mentioned above). You can send me that by mail or attach it here and I'll set it up on my account.

simoncozens · 2019-04-03T13:26:28Z

Oh, duh, I just found it in your commits. Never mind. Let me try this.

harrysummer · 2019-04-03T13:58:23Z

/AzurePipelines run

azure-pipelines · 2019-04-03T14:02:16Z

Command 'run

EDIT:' is not supported by Azure Pipelines.

Supported commands
     help:
          Get descriptions, examples and documentation about supported commands
          Example: help "command_name"
     run:
          Run all pipelines or a specific pipeline for this repository using a comment. Use
          this command by itself to trigger all related pipelines, or specify a pipeline
          to run.
          Example: "run" or "run pipeline_name"

See additional documentation.

simoncozens · 2019-04-03T14:30:45Z

I've merged most of this into devel, and added the azure pipeline YAML file into master. Let's see what that does.

harrysummer · 2019-04-03T14:42:23Z

Great! I also updated the commit (removed my temporary build script and updated build badge) so that the Azure Pipeline CI is building on this PR too.

harrysummer · 2019-04-03T14:57:21Z

Looks like the devel branch build failed because cmake/fonts_conf.cmake is missing.

* fontconfig: Add generated headers and sources (generated from MinGW). Add Win32 compatible layer. Add CMake script. * ICU: Fix bugs in CMake script. * lua: Add CMake script (based on torch/luajit-rocks). * srlua: Change the main entry to set SILE_PATH environment variable which is needed by sile.lua.

harrysummer · 2019-04-03T19:55:11Z

Clean the commits again. I think it's ready to merge after the CI build passes.

simoncozens · 2019-04-04T07:52:00Z

Excellent, thank you so much!

normanlorrain · 2019-12-29T06:13:24Z

(Found this issue from the link in the README.md)

What is the status of the build on Windows? I've tried to run it with the latest VisualStudio, which includes CMake. Here's my output. I'd be happy to help with this project, I just need a bit of guidance to get started...

build_output.txt

harrysummer mentioned this pull request Apr 2, 2019

Port to MSVC with CMake script sile-typesetter/libtexpdf#6

Merged

Add win32 compatibility layer by defining strcasecmp and strncasecmp …

289ecc5

…to corresponding win32 function. Change to guarding macro from __MINGW32__ to WIN32.

harrysummer force-pushed the win32 branch from f870472 to 26a090f Compare April 3, 2019 07:19

harrysummer added 3 commits April 3, 2019 12:49

Add CMake build script. bump libtexpdf.

2e05f17

Add Azure Pipelines CI badge in README.md

31291a5

harrysummer force-pushed the win32 branch from 3d3d5dc to 31291a5 Compare April 3, 2019 19:52

simoncozens merged commit 9060599 into sile-typesetter:master Apr 4, 2019

alerque mentioned this pull request Apr 24, 2019

Drop appveyor #589

Merged

alerque added this to the v0.10.0 milestone Aug 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port to MSVC with CMake script #567

Port to MSVC with CMake script #567

harrysummer commented Apr 2, 2019 •

edited by simoncozens

Loading

coveralls commented Apr 2, 2019 •

edited

Loading

alerque commented Apr 3, 2019

harrysummer commented Apr 3, 2019

alerque commented Apr 3, 2019

harrysummer commented Apr 3, 2019

alerque commented Apr 3, 2019 •

edited

Loading

harrysummer commented Apr 3, 2019

simoncozens commented Apr 3, 2019

alerque commented Apr 3, 2019

harrysummer commented Apr 3, 2019 •

edited

Loading

simoncozens commented Apr 3, 2019

simoncozens commented Apr 3, 2019

harrysummer commented Apr 3, 2019 •

edited

Loading

azure-pipelines bot commented Apr 3, 2019

simoncozens commented Apr 3, 2019

harrysummer commented Apr 3, 2019

harrysummer commented Apr 3, 2019

harrysummer commented Apr 3, 2019

simoncozens commented Apr 4, 2019

normanlorrain commented Dec 29, 2019

Port to MSVC with CMake script #567

Port to MSVC with CMake script #567

Conversation

harrysummer commented Apr 2, 2019 • edited by simoncozens Loading

coveralls commented Apr 2, 2019 • edited Loading

alerque commented Apr 3, 2019

harrysummer commented Apr 3, 2019

alerque commented Apr 3, 2019

harrysummer commented Apr 3, 2019

alerque commented Apr 3, 2019 • edited Loading

harrysummer commented Apr 3, 2019

simoncozens commented Apr 3, 2019

alerque commented Apr 3, 2019

harrysummer commented Apr 3, 2019 • edited Loading

simoncozens commented Apr 3, 2019

simoncozens commented Apr 3, 2019

harrysummer commented Apr 3, 2019 • edited Loading

azure-pipelines bot commented Apr 3, 2019

simoncozens commented Apr 3, 2019

harrysummer commented Apr 3, 2019

harrysummer commented Apr 3, 2019

harrysummer commented Apr 3, 2019

simoncozens commented Apr 4, 2019

normanlorrain commented Dec 29, 2019

harrysummer commented Apr 2, 2019 •

edited by simoncozens

Loading

coveralls commented Apr 2, 2019 •

edited

Loading

alerque commented Apr 3, 2019 •

edited

Loading

harrysummer commented Apr 3, 2019 •

edited

Loading

harrysummer commented Apr 3, 2019 •

edited

Loading