Skip to content

Commit

Permalink
Adds "makedep" script to replace mkmf
Browse files Browse the repository at this point in the history
`mkmf` is an external dependency that that uses a multi-stage
approach to building a Makefile with dependencies but fails to yield
an optimal link stage (it links everything, including unused modules).

`makedep` is a bash script that constructs the link stage for as many
programs as found, links only the necessary object files*, and is coded
solely in bash thus removing an external dependency. The addition of this
script is the bulk of this commit.

Code changes:
- when more than one program is encountered the executable is given
  the name following "program". Since all the programs were named
  "MOM_main" I have had to change them to provide unique names:
  - MOM_sum_driver.F90, "program MOM_main" has been changed to
    "MOM_sum_driver"
  - solo_driver/MOM_driver.F90, "program MOM_main" has been
    changed to "MOM6"

Script changes:
- Makefile.in now has a target "depend" to generate dependencies
  using makedep
- configure.ac no longer checks for list_paths and mkmf
- configure.ac now invokes "make depend" in place of mkmf
- Added target "unit" to .testing/Makefile to build all programs
  in config_src/drivers/unit_drivers

Ugliness:
- I had to add a -f option to makedep to handle FMS non-standard macros
  - To compile FMS, the dependencies Makefile is passed CPPDEFS in addition
    to CPPFLAGS.
  - The first version of makedep was consistent with the standard gmake
    rules which were sufficient to build MOM6. Adding -f "rule command"
    allows FMS to be built:
      makemake -f '$(FC) $(FFLAGS) $(CPPFLAGS) $(CPPDEFS) -c $<' -x libFMS.a ../src
- .inc suffix is included when searching for include directories
  - FMS has includes of .inc files which modify the search path passed
    to /lib/cpp .
- Handling of badly formatted comments when searching for modules
  - FMS fm_util.F90, that generates fm_util_mod.mod, has some odd strings
    in a comment on the module declaration line. This was causing wierdness
    in the script.
- Not just Fortran dependencies
  - makedep needs to also generate rules for C files in order to
    build FMS

Todo:
[ ] *A work around is used for TEOS10 (gsw_*) functions that are in separate
    object files even though accessed via a module (WTFortran!!!)
  • Loading branch information
adcroft authored and marshallward committed Jun 17, 2022
1 parent e5580e3 commit 546312a
Show file tree
Hide file tree
Showing 9 changed files with 262 additions and 108 deletions.
19 changes: 4 additions & 15 deletions .testing/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -119,10 +119,6 @@ DIMS ?= t l h z q r
# Dependencies
DEPS = deps

# mkmf, list_paths (GFDL build toolchain)
LIST_PATHS := $(DEPS)/bin/list_paths
MKMF := $(DEPS)/bin/mkmf


#---
# Test configuration
Expand Down Expand Up @@ -295,7 +291,7 @@ build/%/MOM6: build/%/Makefile

# Use autoconf to construct the Makefile for each target
.PRECIOUS: $(foreach b,$(BUILDS),build/$(b)/Makefile)
build/%/Makefile: ../ac/configure ../ac/Makefile.in $(DEPS)/lib/libFMS.a $(MKMF) $(LIST_PATHS)
build/%/Makefile: ../ac/configure ../ac/Makefile.in $(DEPS)/lib/libFMS.a
mkdir -p $(@D)
cd $(@D) \
&& $(MOM_ENV) ../../../ac/configure $(MOM_ACFLAGS) --with-framework=$(FRAMEWORK) \
Expand All @@ -308,7 +304,7 @@ build/%/Makefile: ../ac/configure ../ac/Makefile.in $(DEPS)/lib/libFMS.a $(MKMF)

# Fetch the regression target codebase
build/target/Makefile build/opt_target/Makefile: \
$(TARGET_CODEBASE)/ac/configure $(DEPS)/lib/libFMS.a $(MKMF) $(LIST_PATHS)
$(TARGET_CODEBASE)/ac/configure $(DEPS)/lib/libFMS.a
mkdir -p $(@D)
cd $(@D) \
&& $(MOM_ENV) ../../$(TARGET_CODEBASE)/ac/configure $(MOM_ACFLAGS) \
Expand Down Expand Up @@ -337,7 +333,7 @@ $(DEPS)/lib/libFMS.a: $(DEPS)/fms/build/libFMS.a
$(DEPS)/fms/build/libFMS.a: $(DEPS)/fms/build/Makefile
$(MAKE) -C $(DEPS) fms/build/libFMS.a

$(DEPS)/fms/build/Makefile: $(DEPS)/fms/src/configure $(DEPS)/Makefile.fms.in $(MKMF) $(LIST_PATHS)
$(DEPS)/fms/build/Makefile: $(DEPS)/fms/src/configure $(DEPS)/Makefile.fms.in
$(FMS_ENV) $(MAKE) -C $(DEPS) fms/build/Makefile
$(MAKE) -C $(DEPS) fms/build/Makefile

Expand All @@ -353,13 +349,6 @@ $(DEPS)/fms/src/configure: ../ac/deps/configure.fms.ac $(DEPS)/Makefile $(FMS_SO
$(DEPS)/fms/src: $(DEPS)/Makefile
make -C $(DEPS) fms/src

# mkmf
$(MKMF) $(LIST_PATHS): $(DEPS)/mkmf
$(MAKE) -C $(DEPS) bin/$(@F)

$(DEPS)/mkmf: $(DEPS)/Makefile
$(MAKE) -C $(DEPS) mkmf

# Dependency init
$(DEPS)/Makefile: ../ac/deps/Makefile
mkdir -p $(@D)
Expand Down Expand Up @@ -693,7 +682,7 @@ test.summary:
.PHONY: unit.cov
unit.cov: build/unit/MOM_new_unit_tests.gcov

work/unit/std.out: build/unit/MOM6
work/unit/std.out: build/unit/MOM_unit_tests
if [ $(REPORT_COVERAGE) ]; then \
find build/unit -name *.gcda -exec rm -f '{}' \; ; \
fi
Expand Down
10 changes: 7 additions & 3 deletions ac/Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,16 @@ CPPFLAGS = @CPPFLAGS@
FFLAGS = @FCFLAGS@
LDFLAGS = @LDFLAGS@ @LIBS@

SRC_DIRS = @SRC_DIRS@

# Gather modulefiles
TMPFILES = $(wildcard *.mod)

include Makefile.mkmf
-include Makefile.dep

.PHONY: depend
depend:
../../../ac/makedep -o Makefile.dep $(SRC_DIRS)

# Delete any files associated with configuration (including the Makefile).
.PHONY: distclean
Expand All @@ -64,9 +69,8 @@ distclean: clean
rm -f config.log
rm -f config.status
rm -f Makefile
# mkmf output
rm -f path_names
rm -f Makefile.mkmf
rm -f Makefile.dep


# This deletes all files generated by autoconf, including configure.
Expand Down
43 changes: 5 additions & 38 deletions ac/configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -223,46 +223,13 @@ AC_COMPILE_IFELSE(
]
)


# Search for mkmf build tools
AC_PATH_PROG([LIST_PATHS], [list_paths])
AS_IF([test -z "$LIST_PATHS"], [
AC_PATH_PROG([LIST_PATHS], [list_paths], [], ["$PATH:${srcdir}/ac/deps/bin"])
AS_IF([test -z "$LIST_PATHS"],
[AC_MSG_ERROR([Could not find list_paths.])],
[AC_SUBST(PATH, ["$PATH:${srcdir}/ac/deps/bin"])])
]
)

AC_PATH_PROG([MKMF], [mkmf])
AS_IF([test -z "$MKMF"], [
AC_PATH_PROG([MKMF], [mkmf], [], ["$PATH:${srcdir}/ac/deps/bin"])
AS_IF([test -z "$MKMF"],
[AC_MSG_ERROR([Could not find mkmf.])],
[AC_SUBST(PATH, ["$PATH:${srcdir}/ac/deps/bin"])])
]
)


# NOTE: MEM_LAYOUT unneeded if we shift to MOM_memory.h.in template
AC_CONFIG_COMMANDS([path_names],
[list_paths -l \
${srcdir}/src \
${MODEL_FRAMEWORK} \
${srcdir}/config_src/ext* \
${DRIVER_DIR} \
${MEM_LAYOUT}],
[MODEL_FRAMEWORK=$MODEL_FRAMEWORK
MEM_LAYOUT=$MEM_LAYOUT
DRIVER_DIR=$DRIVER_DIR]
)


AC_CONFIG_COMMANDS([Makefile.mkmf],
[mkmf -p MOM6 -m Makefile.mkmf path_names])

SRC_DIRS="${srcdir}/src ${MODEL_FRAMEWORK} ${srcdir}/config_src/external ${DRIVER_DIR} ${MEM_LAYOUT}"
AC_CONFIG_COMMANDS([Makefile.dep],
[make depend])

# Prepare output
AC_SUBST(CPPFLAGS)
AC_SUBST(SRC_DIRS)
AC_CONFIG_FILES([Makefile:${srcdir}/ac/Makefile.in])
AC_OUTPUT

21 changes: 6 additions & 15 deletions ac/deps/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,6 @@ SHELL = bash
MAKEFLAGS += -R


# mkmf, list_paths (GFDL build toolchain)
MKMF_URL ?= https://github.com/NOAA-GFDL/mkmf.git
MKMF_COMMIT ?= master

# FMS framework
FMS_URL ?= https://github.com/NOAA-GFDL/FMS.git
FMS_COMMIT ?= 2019.01.03
Expand All @@ -30,19 +26,14 @@ FMS_SOURCE = $(call SOURCE,fms/src)
# Rules

.PHONY: all
all: bin/mkmf bin/list_paths lib/libFMS.a
all: bin/makedep lib/libFMS.a

#---
# mkmf checkout
# makedep script

bin/mkmf bin/list_paths: mkmf
bin/makedep: ../../ac/makedep
mkdir -p $(@D)
cp $^/$@ $@

mkmf:
git clone $(MKMF_URL) $@
git -C $@ checkout $(MKMF_COMMIT)

cp $^ $@

#---
# FMS build
Expand All @@ -64,7 +55,7 @@ fms/build/libFMS.a: fms/build/Makefile
make -C fms/build libFMS.a


fms/build/Makefile: Makefile.fms.in fms/src/configure bin/mkmf bin/list_paths
fms/build/Makefile: Makefile.fms.in fms/src/configure bin/makedep
mkdir -p fms/build
cp Makefile.fms.in fms/src/Makefile.in
cd $(@D) && ../src/configure --srcdir=../src
Expand All @@ -87,4 +78,4 @@ clean:

.PHONY: distclean
distclean: clean
rm -rf fms mkmf
rm -rf fms
8 changes: 7 additions & 1 deletion ac/deps/Makefile.fms.in
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,13 @@ FFLAGS = @FCFLAGS@
LDFLAGS = @LDFLAGS@
ARFLAGS = @ARFLAGS@

SRC_DIRS = @SRC_DIRS@

# Gather modulefiles
TMPFILES = $(wildcard *.mod)

include Makefile.mkmf
-include Makefile.dep

.PHONY: depend
depend:
../../bin/makedep -o Makefile.dep -f '$$(FC) $$(CPPFLAGS) $$(CPPDEFS) $$(FFLAGS) -c $$<' -x libFMS.a -d $(SRC_DIRS)
36 changes: 4 additions & 32 deletions ac/deps/configure.fms.ac
Original file line number Diff line number Diff line change
Expand Up @@ -158,38 +158,6 @@ AX_FC_ALLOW_ARG_MISMATCH
FCFLAGS="$FCFLAGS $ALLOW_ARG_MISMATCH_FCFLAGS"


# Search for mkmf build tools
AC_PATH_PROG([LIST_PATHS], [list_paths])
AS_IF([test -z "$LIST_PATHS"], [
AC_PATH_PROG([LIST_PATHS], [list_paths], [], ["$PATH:${srcdir}/../../bin"])
AS_IF([test -z "$LIST_PATHS"],
[AC_MSG_ERROR([Could not find list_paths.])],
[AC_SUBST(PATH, ["$PATH:${srcdir}/../../bin"])])
]
)

AC_PATH_PROG([MKMF], [mkmf])
AS_IF([test -z "$MKMF"], [
AC_PATH_PROG([MKMF], [mkmf], [], ["$PATH:${srcdir}/../../bin"])
AS_IF([test -z "$MKMF"],
[AC_MSG_ERROR([Could not find mkmf.])],
[AC_SUBST(PATH, ["$PATH:${srcdir}/../../bin"])])
]
)


# MKMF commands
AC_CONFIG_COMMANDS([path_names],
[${LIST_PATHS} -l ${srcdir}],
[LIST_PATHS=${LIST_PATHS}]
)


AC_CONFIG_COMMANDS([mkmf],
[${MKMF} -p libFMS.a -m Makefile.mkmf path_names],
[MKMF=${MKMF}]
)


# Autoconf does not configure the archiver (ar), as it is handled by Automake.
# TODO: Properly configure this tool. For now, we hard-set this to `ar`.
Expand All @@ -198,8 +166,12 @@ ARFLAGS=rv
AC_SUBST(AR)
AC_SUBST(ARFLAGS)

SRC_DIRS="../src"
AC_CONFIG_COMMANDS([Makefile.dep],
[make depend])

# Prepare output
AC_SUBST(CPPFLAGS)
AC_SUBST(SRC_DIRS)
AC_CONFIG_FILES(Makefile)
AC_OUTPUT
Loading

0 comments on commit 546312a

Please sign in to comment.