-
Notifications
You must be signed in to change notification settings - Fork 22
Design of Makefiles and Visual Studio project build
This page is about design, if you just want to know how to build see Building MUSCLE.
-
The Visual Studio project is the primary source, because this is my preferred development environment.
-
Binary files such as object files and the executable must be written to a separate directory for each target platform (Linux, OSX, Windows) to avoid polluting the directory containing the source code and to avoid collisions where different object files have the same pathname (e.g., object files
xyz.o
for Linux and OSX). -
A version control commit identifier must be embedded in the source code so that it can be displayed at runtime. This embedding must be automated such that the identifier is written only if a source code file has been updated. The latter is to ensure that an "up to date" source code directory does not trigger a re-build. It seems that version embedding along these lines should be a simple and common requirement, but I found it very difficult to implement in a way that is compatible both with Visual Studio's proprietary build system and with a Makefile.
-
A Makefile must not require manual updates when the list of source files and headers changes. This could be solved either (a) by generating a Makefile from the Visual Studio project files (as attempted in github.com/rcedgar/vcxproj2makefile), or (b) by using wildcards to include all source and header files. When I first committed MUSCLE to github I was using
vcxproj2makefile.py
, but this turned out to be rather fragile and didn't do a good job of accommodating tweaks needed to compile on different platforms. At the time of writing I am trying to implement strategy (b). My concern about (b) is that it will include "orphaned" source files that are no longer needed in the build. These are likely to arise for a couple of reasons: (1) because Visual Studio does not enforce git integration and ignores files that are not explicitly included in the project, and (2) by merging of different branches by git, though I don't understand the merge process very well at this point. Orphaned source files which break the build will hopefully be identified by a CI workflow which I'm also working on implementing. -
The following platforms must be continuously supported without manual intervention (e.g. editing Makefiles) when the project is updated: (a) the setup I use at home for on-going development, (b) Linux under WSL at home, (c) common Linux variants which a user might have, (d) ditto OSX, (e) github runners for Linux, OSX and Windows to enable CI/CD workflows.
-
The source code and build mechanism must be broadly portable to compiler and operating systems in common use.
For me, it turned out to be very challenging to meet these objectives in practice.
Visual Studio automatically constructs separate sub-directories for each distinct platform (e.g., x64/Release
). In the Makefile
, I use a shell escape to assign the output of uname
to a variable named $(OS)
:
OS := $(shell uname)
A list of object filenames with the platform name directory prefix is constructed by a shell escape to sed
:
OBJS := $(shell echo *.cpp | sed "-es/^/$(OS)\//" | sed "-es/ / $(OS)\//g" | sed "-es/\.cpp/.o/g")
In recent releases of OSX, there are two big compatibility problems: gcc
is an alias for clang
, and OpenMP is not supported by the default installation of clang
. This is solved by requiring that gcc-11
is installed, which can typically be done by brew install gcc
or similar. The Makefile re-assigns $CXX
on OSX (where uname
returns Darwin
) to ensure that genuine gcc
is used:
CXX := g++ ifeq ($(OS),Darwin) CXX := g++-11 endif
This git command is used to generate a string which identifies the commit:
git describe --abbrev=6 --dirty --long --always
This returns something like cada96-dirty
, where dirty
indicates there are uncommitted changes. This string must be surrounded by quotes because I couldn't figure out a way to put quotes around an unquoted string taken from an #include
file. Quotes are added using sed
:
git describe --abbrev=6 --dirty --long --always | sed '-es/^\(.*\)$/"\1"/' > gitver.txt
In myutils.cpp
the string is embedded using #include "gitver.txt"
. Probably the requirement for sed
could be eliminated by using something like
echo \" $(git...) \"
but sed
has worked fine on every platform where I have tested so far.
A pre-build event runs gitver.bat
, which checks whether Cygwin is installed by testing whether the file c:\cygwin64\bin\bash.exe
exists. If Cygwin is not installed, nothing happens and the build will continue without updating gitver.txt
. This ensures that the build will work on Windows platforms where Cygwin is not installed, e.g. a github workflow runner. If Cygwin is installed, a bash script is run which requires that the Cygwin version of git is installed. I prefer Cygwin to requiring a native Windows version of git
because I don't know a robust solution to ensure that git is in the PATH when the pre-build event runs.
The Makefile has the following dependency:
gitver.txt : $(SRCS) bash ./gitver.bash
The $(SRCS)
variable has the source filenames obtained by running ls
in the shell: SRCS := $(shell ls *.cpp *.h)
. Probably there is a way to do this using make wildcards but I'm a make novice and the Makefile syntax is baffling; this seems good enough to me.