Build NIF C code also on tests #244

jj1bdx · 2015-05-14T04:13:17Z

This patch ensures building the NIF code underc_src
even on running make tests
When c_src/Makefile exists:
- Split the Makefile invocation dependency into app-c_src
- Invoke app-c_src also in test-build as well as app
When c_src/Makefile does not exist:
- Add $(C_SRC_ENV) $(C_SRC_OUTPUT) dependencies to test-build

* This patch ensures building the NIF code under`c_src` even on running `make tests` * When `c_src/Makefile` exists: * Split the Makefile invocation dependency into `app-c_src` * Invoke `app-c_src` also in `test-build` as well as `app` * When `c_src/Makefile` does not exist: * Add `$(C_SRC_ENV) $(C_SRC_OUTPUT)` dependencies to `test-build`

essen · 2015-05-14T05:51:02Z

Looks good, will test after breakfast.

essen · 2015-05-14T15:55:21Z

Manual testing works, now confirming that it works with all packages in the index.

essen · 2015-05-14T16:40:37Z

All good, merged! Thanks!

essen · 2015-05-14T16:41:26Z

By the way @jj1bdx you may want to try downloading the erlang.mk repository and running 'make check' to see if there are issues on FreeBSD. It will take more than half an hour to compile everything though. :-)

jj1bdx · 2015-05-15T01:04:17Z

Thx for merge :)

On FreeBSD, any dependency to make means running BSD make, so hardcoding make will not work. The GNU make on FreeBSD is gmake.

jj1bdx · 2015-05-15T01:27:51Z

I've tested 'make check' with aliasing make with gmake, but some incompatibilities still remain. I think I need more rewriting.

kenji@minimax[1095]% make
Setting up app.
gmake[1]: Entering directory '/home/kenji/src/erlang.mk'
cat packages.v2.tsv | awk 'BEGIN { FS = "\t" }; { print $1 "\t" $3 "\t" $5 "\t" $6 }' > packages.v1.tsv
cp packages.v1.tsv packages.v1.txt
awk 'FNR==1 && NR!=1{print ""}1' core/core.mk core/deps.mk core/erlc.mk core/docs.mk core/test.mk plugins/asciidoc.mk plugins/bootstrap.mk plugins/c_src.mk plugins/ci.mk plugins/ct.mk plugins/dialyzer.mk plugins/edoc.mk plugins/elvis.mk plugins/erlydtl.mk plugins/escript.mk plugins/eunit.mk plugins/relx.mk plugins/shell.mk plugins/triq.mk plugins/xref.mk plugins/cover.mk > erlang.mk
gmake[1]: Leaving directory '/home/kenji/src/erlang.mk'
gmake[1]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
gmake[1]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
app: Testing the 'app' target.
Checking the modules line in the generated .app file.
gmake[1]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
grep: app1/ebin/app1.app: No such file or directory
Makefile:49: recipe for target 'app' failed
gmake: *** [app] Error 1
kenji@minimax[1096]%  ERLC   m.erl
 APP    app1.app.src
gmake[1]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'

jj1bdx · 2015-05-15T03:44:20Z

Note: the make check worked OK on GNU Make 3.81 on OS X. The FreeBSD Port GNU Make devel/gmake is GNU Make 4.1:

GNU Make 4.1
Built for amd64-portbld-freebsd10.1
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

jj1bdx · 2015-05-15T03:45:27Z

And FreeBSD gmake is -w option enabled as default. So make -w will produce the same result for the Entering directory... and Leaving directory... messages.

essen · 2015-05-15T05:40:45Z

I have Make 4.1 here also. I will fix a few 'make' into '$(MAKE)' and report back.

jj1bdx · 2015-05-15T06:48:26Z

@essen Great. I can't figure out why the current directories between GNU Make 3.81 and 4.1 after Checking the modules line in the generated .app file. happens differ with each other.

essen · 2015-05-15T07:21:30Z

That should do it: a294038. You shouldn't need the alias now.

Not sure what you mean about current directories, can you give more details?

jj1bdx · 2015-05-15T07:38:57Z

Thx. gmake check at the toplevel directory worked. > a294038

The error message still remains the same, however. The entering/leaving of the directories show different values in the results of GNU Make 3.81 on OS X and GNU Make 4.1 on FreeBSD. Note that the error messages are shown after the control returns to the shell on 4.1. Also see the grep error message.

On GNU Make 3.81 on OS X (10.10.3 with XCode 6.3) with make -w check:

make: Entering directory `/Users/kenji/src/erlang.mk'
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C test
make[1]: Entering directory `/Users/kenji/src/erlang.mk/test'
Setting up app.
make[2]: Entering directory `/Users/kenji/src/erlang.mk'
cat packages.v2.tsv | awk 'BEGIN { FS = "\t" }; { print $1 "\t" $3 "\t" $5 "\t" $6 }' > packages.v1.tsv
cp packages.v1.tsv packages.v1.txt
awk 'FNR==1 && NR!=1{print ""}1' core/core.mk core/deps.mk core/erlc.mk core/docs.mk core/test.mk plugins/asciidoc.mk plugins/bootstrap.mk plugins/c_src.mk plugins/ci.mk plugins/ct.mk plugins/dialyzer.mk plugins/edoc.mk plugins/elvis.mk plugins/erlydtl.mk plugins/escript.mk plugins/eunit.mk plugins/relx.mk plugins/shell.mk plugins/triq.mk plugins/xref.mk plugins/cover.mk > erlang.mk
make[2]: Leaving directory `/Users/kenji/src/erlang.mk'
make[2]: Entering directory `/Users/kenji/src/erlang.mk/test/app1'
make[2]: Leaving directory `/Users/kenji/src/erlang.mk/test/app1'
app: Testing the 'app' target.
Checking the modules line in the generated .app file.
Checking that '/Applications/Xcode.app/Contents/Developer/usr/bin/make clean-app' deletes ebin.
Checking that '/Applications/Xcode.app/Contents/Developer/usr/bin/make app' returns non-zero on compile errors.
Test 'app' passed.
[continuing]

On GNU Make 4.1 on FreeBSD:

kenji@minimax[1002]% gmake check
gmake -C test
gmake[1]: Entering directory '/home/kenji/src/erlang.mk/test'
Setting up app.
gmake[2]: Entering directory '/home/kenji/src/erlang.mk'
cat packages.v2.tsv | awk 'BEGIN { FS = "\t" }; { print $1 "\t" $3 "\t" $5 "\t" $6 }' > packages.v1.tsv
cp packages.v1.tsv packages.v1.txt
awk 'FNR==1 && NR!=1{print ""}1' core/core.mk core/deps.mk core/erlc.mk core/docs.mk core/test.mk plugins/asciidoc.mk plugins/bootstrap.mk plugins/c_src.mk plugins/ci.mk plugins/ct.mk plugins/dialyzer.mk plugins/edoc.mk plugins/elvis.mk plugins/erlydtl.mk plugins/escript.mk plugins/eunit.mk plugins/relx.mk plugins/shell.mk plugins/triq.mk plugins/xref.mk plugins/cover.mk > erlang.mk
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk'
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
app: Testing the 'app' target.
Checking the modules line in the generated .app file.
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
grep: app1/ebin/app1.app: No such file or directory
Makefile:49: recipe for target 'app' failed
gmake[1]: *** [app] Error 1
gmake[1]: Leaving directory '/home/kenji/src/erlang.mk/test'
Makefile:30: recipe for target 'check' failed
gmake: *** [check] Error 2
kenji@minimax[1003]%  ERLC   m.erl
 APP    app1.app.src
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'

essen · 2015-05-15T07:44:47Z

Does it do the same if you do this instead? Parallel should be disabled but you never know...

gmake check -j 1

If not then I guess I will set me up some FreeBSD this week-end.

jj1bdx · 2015-05-15T07:47:56Z

gmake check -j 1 didn't work. Result (virtually the same):

I also suspect some interpretation change of .NOTPARALLEL: pseudo target, but I cannot figure it out yet.

kenji@minimax[1006]% gmake check -j 1
gmake -C test
gmake[1]: Entering directory '/home/kenji/src/erlang.mk/test'
Setting up app.
gmake[2]: Entering directory '/home/kenji/src/erlang.mk'
cat packages.v2.tsv | awk 'BEGIN { FS = "\t" }; { print $1 "\t" $3 "\t" $5 "\t" $6 }' > packages.v1.tsv
cp packages.v1.tsv packages.v1.txt
awk 'FNR==1 && NR!=1{print ""}1' core/core.mk core/deps.mk core/erlc.mk core/docs.mk core/test.mk plugins/asciidoc.mk plugins/bootstrap.mk plugins/c_src.mk plugins/ci.mk plugins/ct.mk plugins/dialyzer.mk plugins/edoc.mk plugins/elvis.mk plugins/erlydtl.mk plugins/escript.mk plugins/eunit.mk plugins/relx.mk plugins/shell.mk plugins/triq.mk plugins/xref.mk plugins/cover.mk > erlang.mk
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk'
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
app: Testing the 'app' target.
Checking the modules line in the generated .app file.
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
grep: app1/ebin/app1.app: No such file or directory
Makefile:49: recipe for target 'app' failed
gmake[1]: *** [app] Error 1
gmake[1]: Leaving directory '/home/kenji/src/erlang.mk/test'
Makefile:30: recipe for target 'check' failed
gmake: *** [check] Error 2
kenji@minimax[1007]%  ERLC   m.erl
 APP    app1.app.src
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'

essen · 2015-05-15T08:01:02Z

Yep no clue.

jj1bdx · 2015-05-15T08:21:07Z

On FreeBSD with GNU Make 4.1, gmake V=2 check worked, or better say, proceeded after Checking the modules line in the generated .app file., so I see some race condition in the test/Makefile when V=0 or V=1.

jj1bdx · 2015-05-15T08:26:10Z

% result of `gmake V=2 check`
gmake -C test
gmake[1]: Entering directory '/home/kenji/src/erlang.mk/test'
 TEST   clean
== Setting up app.
 TEST   app1
 TEST   app1
gmake[2]: Entering directory '/home/kenji/src/erlang.mk'
cat packages.v2.tsv | awk 'BEGIN { FS = "\t" }; { print $1 "\t" $3 "\t" $5 "\t" $6 }' > packages.v1.tsv
cp packages.v1.tsv packages.v1.txt
awk 'FNR==1 && NR!=1{print ""}1' core/core.mk core/deps.mk core/erlc.mk core/docs.mk core/test.mk plugins/asciidoc.mk plugins/bootstrap.mk plugins/c_src.mk plugins/ci.mk plugins/ct.mk plugins/dialyzer.mk plugins/edoc.mk plugins/elvis.mk plugins/erlydtl.mk plugins/escript.mk plugins/eunit.mk plugins/relx.mk plugins/shell.mk plugins/triq.mk plugins/xref.mk plugins/cover.mk > erlang.mk
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk'
 TEST   app1
 TEST   app1
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
 TEST   app1
== app: Testing the 'app' target.
 TEST   app
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
 ERLC   m.erl
 APP    app1.app.src
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
== Checking the modules line in the generated .app file.
 TEST   app
 TEST   app
== Checking that 'gmake clean-app' deletes ebin.
 TEST   app
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
 GEN    clean-app
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
 TEST   app
== Checking that 'gmake app' returns non-zero on compile errors.
 TEST   app
 TEST   app
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
 ERLC   m.erl syntax_error.erl
src/syntax_error.erl:2: syntax error before: lorem_ipsum
erlang.mk:583: recipe for target 'ebin/app1.app' failed
gmake[2]: *** [ebin/app1.app] Error 1
gmake[2]: Leaving directory '/home/kenji/src/erlang.mk/test/app1'
 TEST   app
== Test 'app' passed.
== ct: Testing ct and related targets.
== Setting up test suite.
 TEST   ct
 TEST   ct
 TEST   ct
gmake[2]: Entering directory '/home/kenji/src/erlang.mk/test/app1'
 GEN    clean-app
 GEN    coverdata-clean
 ERLC   m.erl
 APP    app1.app.src
 GEN    test-dir
 GEN    test-build
 GEN    ct
[continuing...]

jj1bdx · 2015-05-15T08:58:58Z

Now I figured it out. I'll make a separate pull request and I will close this.
The key was definition of V in test/Makefile, and it was about &> which wasn't interpreted as a redirection operator for the FreeBSD shell. The line should be >/dev/null 2>%1. Here's diff:

diff --git a/test/Makefile b/test/Makefile
index 1b60371..a77b307 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -14,12 +14,12 @@ V ?= 0
 ifeq ($V,0)
        # Show info messages only
        t = @
-       v = V=0 &>/dev/null
+       v = V=0 >/dev/null 2>&1
        i = @echo
 else ifeq ($V,1)
        # Show test commands
        t =
-       v = V=0 &>/dev/null
+       v = V=0 >/dev/null 2>&1
        i = @echo ==
 else ifeq ($V,2)
        # Show briefly what erlang.mk is doing

essen · 2015-05-15T09:02:33Z

Cool. So everything 'make check' does runs OK? It compiles everything fine? :-)

jj1bdx · 2015-05-15T09:24:05Z

eleveldb doesn't build and stop. I've reported this. See basho/eleveldb#97

This is not an issue of erlang.mk, I guess.

essen · 2015-05-15T09:29:44Z

Yeah nothing I can do about that.

essen merged commit 864ddc0 into ninenines:master May 14, 2015

jj1bdx deleted the jj1bdx-c_src-appbuild branch May 15, 2015 01:28

jj1bdx mentioned this pull request May 15, 2015

Fix test/Makefile for FreeBSD shell redirection compatibility #246

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build NIF C code also on tests #244

Build NIF C code also on tests #244

jj1bdx commented May 14, 2015

essen commented May 14, 2015

essen commented May 14, 2015

essen commented May 14, 2015

essen commented May 14, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

Build NIF C code also on tests #244

Build NIF C code also on tests #244

Conversation

jj1bdx commented May 14, 2015

essen commented May 14, 2015

essen commented May 14, 2015

essen commented May 14, 2015

essen commented May 14, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015

jj1bdx commented May 15, 2015

essen commented May 15, 2015