Unable to compile v6.0.0 on windows

[steve-chavez]

I wasn't able to compile latest version on windows. This seems related to the GHC upgrade we had recently.

• Error for GHC 8.4.4: https://ci.appveyor.com/project/postgrest/postgrest/builds/25457530
• Error for GHC 8.6.5: https://ci.appveyor.com/project/steve-chavez/postgrest/builds/25506383

These errors are on AppVeyor, but I also hit the same errors when trying to compile on a Windows Server 2019 in an ec2 instance.

Any help would be greatly appreciated. Trying to build from source on Windows and reporting here would help.

[steve-chavez]

For the GHC 8.6.5 error, compiling actually gets suck and times out on AppVeyor. On Windows Server it never finished(tried it like for 2 hours or smth, maybe it could take longer?).

From the error, it seems it could be related to GHC optimization, an option could be to try compiling with -O0(see optimization) and see if it finishes.

[steve-chavez]

Additional help to build on Windows can be found on https://github.com/PostgREST/postgrest/blob/master/appveyor.yml.

[robx]

As one data-point, I just tried an appveyor build myself with GHC 8.6.5, and it got stuck at the exact same spot as yours, so I suspect it's not simply a timeout.

[robx]

Adding -O0 to ghc-options doesn't seem to make a difference (but unsure this really had an effect)

[steve-chavez]

@robx It gets stuck when building text-printer, I was thinking the next thing to try could be to build it isolated from postgrest.

I actually opened an issue mvv/text-printer#1, but maybe our failure could be unrelated.

[robx]

@steve-chavez you could reproduce the build failure on windows server, right? Perhaps you could weigh on on the ghc bug report at https://gitlab.haskell.org/ghc/ghc/issues/16917? In particular, it would be good to know how the branches windows-reduced and windows-buffer-error of my reduced text-printer fork behave there.

[steve-chavez]

@robx Yes, also happened on Windows Server(no timeout). I've deleted the ec2 instance but I'll see about creating a new one and checking your branches. One thing though, with text-printer-0.5.0.1 the compiling now succeeds?

[robx]

@steve-chavez Yes 0.5.0.1 appears to compile fine. Though note that the jose 0.8 update also causes trouble one Windows: frasertweedale/hs-jose#83

[steve-chavez]

@robx Both hang.

windows-reduced hangs with:

src\Integral.hs:106:1: warning:
    Forall'd constraint `Ord ▒' is not bound in RULE lhs
      Orig bndrs: [▒, p, $dOrd_a4Cz, $dIntegral_a4CA, $dPrinter_a4CB]
      Orig lhs: let {
                  $dPrinter_a4CI :: Printer p
                  [LclId]
                  $dPrinter_a4CI = $dPrinter_a4CB } in
                let {
                  $dIntegral_a4CH :: Integral ▒
                  [LclId]
                  $dIntegral_a4CH = $dIntegral_a4CA } in
                let {
                  $dPositionalSystem_a4CF :: PositionalSystem Binary
                  [LclId]
                  $dPositionalSystem_a4CF = Integral.$fPositionalSystemBinary } in
                let {
                  $dReal_a5aX :: Real ▒
                  [LclId]
                  $dReal_a5aX = GHC.Real.$p1Integral @ ▒ $dIntegral_a4CA } in
                let {
                  $dOrd_a5aZ :: Ord ▒
                  [LclId]
                  $dOrd_a5aZ = GHC.Real.$p2Real @ ▒ $dReal_a5aX } in
                let {
                  $dOrd_a4CG :: Ord ▒
                  [LclId]
                  $dOrd_a4CG = $dOrd_a5aZ } in
                number'
                  @ Binary
                  @ ▒
                  @ p
                  $dPositionalSystem_a4CF
                  $dOrd_a4CG
                  $dIntegral_a4CH
                  $dPrinter_a4CI
      optimised lhs: number'
                       @ Binary
                       @ ▒
                       @ p
                       $dPositionalSystem_a4CF
                       $dOrd_a4CG
                       $dIntegral_a4CA
                       $dPrinter_a4CB

For windows-buffer-error:

Building library for text-printer-0.5..
[1 of 1] Compiling Integral

src\Integral.hs:79:1: warning:
    Forall'd constraint `Ord ▒' is not bound in RULE lhs
      Orig bndrs: [▒, p, $dOrd_a4ze, $dIntegral_a4zf, $dPrinter_a4zg]
      Orig lhs: let {
                  $dPrinter_a4zn :: Printer p
                  [LclId]
                  $dPrinter_a4zn = $dPrinter_a4zg } in
                let {
                  $dIntegral_a4zm :: Integral ▒
                  [LclId]
                  $dIntegral_a4zm = $dIntegral_a4zf } in
                let {
                  $dPositionalSystem_a4zk :: PositionalSystem Binary
                  [LclId]
                  $dPositionalSystem_a4zk = Integral.$fPositionalSystemBinary } in
                let {
                  $dReal_a4X9 :: Real ▒
                  [LclId]
                  $dReal_a4X9 = GHC.Real.$p1Integral @ ▒ $dIntegral_a4zf } in
                let {
                  $dOrd_a4Xb :: Ord ▒
                  [LclId]
                  $dOrd_a4Xb = GHC.Real.$p2Real @ ▒ $dReal_a4X9 } in
                let {
                  $dOrd_a4zl :: Ord ▒
                  [LclId]
                  $dOrd_a4zl = $dOrd_a4Xb } in
                number'
                  @ Binary
                  @ ▒
                  @ p
                  $dPositionalSystem_a4zk
                  $dOrd_a4zl
                  $dIntegral_a4zm
                  $dPrinter_a4zn
      optimised lhs: number'
                       @ Binary
                       @ ▒
                       @ p
                       $dPositionalSystem_a4zk
                       $dOrd_a4zl
                       $dIntegral_a4zf
                       $dPrinter_a4zg
   |

[steve-chavez]

@robx I can give you access to the windows server to help you with this, you could send me an email(mine in profile, pgp key at http://pgp.mit.edu/) and I'll share you the credentials.

[robx]

@steve-chavez thanks for testing! I'll skip digging further, but thanks for the offer of access to the machine

[steve-chavez]

So, if text-printer-0.5.0.1 compiles now, we could pin jose by git commit and try building on windows. I'll give it a shot.

[steve-chavez]

Now I get:

[10 of 14] Compiling PostgREST.QueryBuilder
ghc.exe: getMBlocks: VirtualAlloc MEM_COMMIT failed: The paging file is too small for this operation to complete.
Completed 113 action(s).

This is on a t2.medium, with 4gb RAM.

[steve-chavez]

I get the same error even with stack build -j1.

No clue how to solve that. I remember getting a similar memory error when trying to compile on a t2.micro(1gb) and what I did was to upgrade to t2.medium.

Check it out:

[robx]

I found this seaching for info on page file size and appveyor: https://github.com/input-output-hk/cardano-sl/blob/master/appveyor.yml

[steve-chavez]

After trying some more times the build finished succesfully, with this patch over the v6.0.0 tag.

diff --git a/stack.yaml b/stack.yaml
index 8ded0bc..32b9979 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -1,18 +1,17 @@
 # stack.yaml is used for circle-ci tests. Profiling build fails on circleci
 # with GHC 8.6, so we build with 8.4 for now.
 
-resolver: lts-12.26
+resolver: lts-13.24
 extra-deps:
 - Ranged-sets-0.4.0
 - configurator-pg-0.1.0.3
-- http-types-0.12.3
-- hasql-1.4
-- hasql-pool-0.5.1
-- hasql-transaction-0.7.2
-- text-builder-0.6.5.1
-- deferred-folds-0.9.10.1
-- primitive-0.6.4.0
+- text-printer-0.5.0.1
+- hasql-1.4@sha256:fcb1b0046c1e888b6c4cad53c972d23318dbc6ababa9ceb7d9cfd3b546732bcd
+- hasql-transaction-0.7.2@sha256:d6d8ceb0b32be75686fe31c4b5bc15c569a71023fc60394893508ea733e8714b
+- jose-0.7.0.0
+
 ghc-options:
   postgrest: -O2 -Werror -Wall -fwarn-identities -fno-warn-redundant-constraints
 nix:
   packages: [pcre, pkgconfig, postgresql, zlib]

Already uploaded the binary to the releases page.

[steve-chavez]

We should enable appveyor builds for commits on master and PRs, to ensure new dependencies don't break windows releases.

Otherwise is more pain to fix the problem after the fact. Thanks a lot for your help on this @robx.

[robx]

Yay! You're welcome.

[steve-chavez]

I actually tried lts-13.28 locally(linux/NixOS) and I get the same issue as in the Windows EC2, trying to compile QueryBuilder eats up all my RAM and I couldn't get it to finish.

So, we should stick to GHC 8.4.4 for now. Problem is that ghc 8.4.4 still fails on Windows.

[steve-chavez]

Perhaps GHC 8.4.4 could succeed if we just pin a newer network version in stack.yaml haskell/network#313.

[Mistuke]

You could try using a lower optimization setting. -O2 is rarely needed and will cause much higher memory usage as those optimizations are quite aggressive. Does it work with -O1? Otherwise if you have a reduced testcase it would be good to report this upstream as it's a regression.

[robx]

Just to note that I commented on this rough issue here: https://gitlab.haskell.org/ghc/ghc/issues/16017#note_213721