Fix sporadic failures when using make -j#

[mikejsavage]

Hi,

We've been running into sporadic failures when running a premake5 Makefile with make -j4. We get errors like: ../../external/uuid/clear.c:42:1: fatal error: opening dependency file ../../intermediates/gmake/common/linux64/Debug/clear.d: No such file or directory

Premake generates a target like all: $(TARGETDIR) $(OBJDIR) prebuild prelink $(TARGET), which works fine for single threaded builds because the prerequisites are run left to right, but with -j4 the rule for $(TARGET) can get executed before the rule for $(OBJDIR), causing the error.

Adding $(OBJDIR) as a prerequisite for each object file is not ideal because it causes make; make to build the project twice. The first time make will see that $(OBJDIR) doesn't exist and build everything that depends on it, the second time make will see that $(OBJDIR) has been modified and build everything that depends on it again.

The (slightly unsatisfactory) solution that I know of is to call mkdir -p $(OBJDIR) every time you build an object.

I've attached a patch which implements the unsatisfactory solution for C++ Makefiles. It's possible that there's a similar bug for C# projects too, but I don't have any C# projects and I didn't want to write "fixes" that I can't test. Similarly, I'm not familiar with the premake codebase so this patch isn't as clean as it could be, but hopefully it's a start!

[samsinsane]

You'll need to fix up the failing tests, you can either check the CI build logs or run them locally via bin/release/premake5 test or bin\release\premake5.exe test. This issue is interesting, I've never encountered this at work and we also use -j4, do you know of any other requirements to trigger this issue?

[mikejsavage]

I've only seen it happen in our CI environment, which is running on EC2. I imagine the scheduler isn't so friendly on a heavily loaded machine.

[samsinsane]

I've only seen it happen in our CI environment, which is running on EC2. I imagine the scheduler isn't so friendly on a heavily loaded machine.

Ah ok, thanks for that, very interesting!

[jspohr]

Hi! Thanks for this fix, it's appreciated!
With latest premake, I still have sporadic failures when using PCHs:

++ make config=debug_x64 -C -j8 -k scape_core
==== Building scape_debug (debug_x64) ====
scapedebugpch.h
error: unable to open output file '../obj/macosx/gmake/x64/debug/scape_debug/scapedebugpch.h.gch': 'No such file or directory'
1 error generated.
make[1]: *** [../obj/macosx/gmake/x64/debug/scape_debug/scapedebugpch.h.gch] Error 1
make[1]: Target `all' not remade because of errors.
make: *** [scape_debug] Error 2
make: Target `scape_core' not remade because of errors.

Apparently, the output directory is not created until the first non-PCH target is compiled. I'm a total make noob (sorry), but I guess I could have a look at your change and try to apply it to the PCH target in the same way.
However, I wanted to check with you first, do you think my assumptions are correct?
Thanks!

[tvandijck]

@jspohr I think that would be a correct assumption... it should be relatively easy. the gmake backend already has a simple method make.mkdir('$(OBJDIR)') to add conditional mkdir rules to a target, so it might be as easy as just adding that somewhere in the make.pchRules (line 510+) method.