Skip to content

Commit

Permalink
build: Generate AST header deps (re: cce94cf, 4ad72a0, 2e2f374)
Browse files Browse the repository at this point in the history
This commit takes another step towards greater maintainability of
the build system. The cross-directory dependency rules on headers
preinstalled in $INSTALLROOT/include/ast are now generated
automatically and stored in $INSTALLROOT/lib/mam/LIBNAME. This now
properly allows removing all that cruft from the Mamfiles, and
likely corrects multiple errors that have crept in over the years.

src/cmd/INIT/mkdeps.sh:
- Added. This is preinstalled by INIT/Mamfile as 'mkdeps'.
- Grep the include/ast headers for #include <HEADER.h> and
  recursively generate MAM dependency rules that Mamfiles can
  reference with simple 'prev' commands.
- Write a 'setv' that sets INCLUDE_AST to $INSTALLROOT/include/ast.
  (This is renamed from PACKAGE_ast_INCLUDE as set in Mamfiles.)
- Enclose the generated rules in a virtual rule with a name of the
  form _hdrdeps_libNAME_.
- All output is written to standard output; the Mamfiles should
  redirect this to the $INSTALLROOT/lib/mam/NAME file.

src/cmd/INIT/maamke.c:
- Make a 'bind' argument that does not start with '-l' an error.
  There was never any other 'bind' command in the AT&T AST repo
  history or in 93u+m, so there's no reason not to.
- Amend the 'bind -lNAME' command to also read library dependency
  rules from $INSTALLROOT/lib/mam/NAME, making these dependencies
  of the current rule context.
- Do not include dependencies for a library that was built in the
  same Mamfile as the 'bind -lNAME' command. That Mamfile will have
  already defined the dependencies as part of the build process.
  This is checked by looking if a rule by the name of libNAME.a has
  been made. Concretely, this allows mamake to be rebuilt and
  linked to libast in the same Mamfile that builds libast.
- If the _hdrdeps_libNAME_ rule was already made (i.e., the file
  was already included), do the equivalent of a a 'prev' command to
  propagate its timestamp to the current rule.

**/Mamfile:
- Remove manually maintained AST header dependency rules.
- Rename PACKAGE_ast_INCLUDE variable to INCLUDE_AST. Don't bother
  to setv it (except in libast) as this is now done via 'bind'.
- For libraries, add $INSTALLROOT/lib/mam/NAME rules that call
  mkdeps to generate dependencies for the just-created headers.
  The public headers are installed within this rule block so that
  dependencies are regenerated whenever the headers change.
- Some refactoring, especially in libmd/Mamfile.
  • Loading branch information
McDutchie committed Jul 15, 2024
1 parent 685f18d commit 54a3e1c
Show file tree
Hide file tree
Showing 10 changed files with 617 additions and 946 deletions.
2 changes: 1 addition & 1 deletion src/cmd/INIT/Mamfile
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ make install virtual
prev make.probe
exec - cat ${^} > ${@}
done
loop SCRIPT probe iffe mktest regress crossexec mkreq mkreq-maplib mprobe proto dylink
loop SCRIPT probe iffe mktest regress crossexec mkreq mkreq-maplib mprobe proto dylink mkdeps
make ${INSTALLROOT}/bin/${SCRIPT}
make ${SCRIPT}
prev ${@}.sh
Expand Down
31 changes: 27 additions & 4 deletions src/cmd/INIT/README-mamake.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,17 +383,20 @@ At this stage, attributes are ignored.

#### …while building the current directory ####

An argument of `-l`*libraryname*
The `bind` command takes an argument of the form `-l`*libraryname*, which
causes a MAM variable `mam_lib`*libraryname* to be defined (see **MAM variables** above).
The variable will contain either the compiler argument for linking to the library *libraryname*
(either the `-l`*libraryname* flag, or the full path in case of a static library)
or, if the `dontcare` attribute is specified, possibly the empty string.
Any library dependencies are also included (see below).
This can be used both for AST libraries shipped with the distribution and for system libraries.

If the current rule's name starts with `FEATURE/` or is `configure.h`, this
is all that the `bind` command does. Otherwise, it also does the following.

For each corresponding *.a library archive dependency built previously,
its time stamp is checked and the current target is marked as outdated if it is
newer, as if a `prev` had been executed for it.

The variable set by `bind` is global, but the marking of the target as
outdated applies to the current rule only, so it may be necessary to
repeat a `bind` command when statically linking executables that depend
Expand All @@ -409,12 +412,28 @@ and the resulting values are appended to the value of `mam_lib`*libraryname*
as dependencies separated by spaces.
`mamake` does not create these dependency files;
they are expected to be generated by Mamfile shell actions (see **Shell actions** above).

The `INIT` package preinstalls the `mkreq` and `mkreq-maplib` scripts for this purpose.
If no such dependency file exists, and the `dontcare` attribute is added,
then `mamake` compiles a small test program on the fly to check if the library exists;
if this fails, the `mam_lib`*libraryname* variable will be emptied.

Any `bind` command whose argument does not start with `-l` is ignored.
Cross-directory dependencies on AST library headers (preinstalled in
`$INSTALLROOT/include/ast`) are similarly communicated via a file with
the path `${INSTALLROOT}/lib/mam/`*libraryname*.
The `bind` command automatically includes this file as necessary.
Note that this may have side effects on the automatic variables.
The `INIT` package preinstalls the `mkdeps` script that Mamfile shell
actions should use to generate this file while building each library.
The generated file is expected to:
1. set the `INCLUDE_AST` variable to `${INSTALLROOT}/include/ast`;
2. define a single virtual rule by the name of `_hdrdeps_lib`*libraryname*`_`
which contains all the rules that define the library's public `include/ast`
headers and how they depend on each other.
This way, Mamfiles can declare a dependency on a single header and all its
dependencies using a simple `prev ${INCLUDE_AST}/`*headername*`.h` command.
Any `bind -l`*libraryname* command will automatically apply the dependencies
defined in the corresponding file to the context of the current rule.
The non-existence of this file is not an error and is silently ignored.

### Repeatedly iterating through a block ###

Expand Down Expand Up @@ -455,11 +474,15 @@ maintain Mamfiles by hand. The following lists the important changes.
* Fixed a bug that stopped a rule marked `virtual` (not associated with
any file) from being executed if a file by that rule's name exists.
* Unrecognized commands and rule attributes throw an error instead of being silently ignored.
This also applies to the `bind` command with an argument not starting with `-l`.
* It has been made optional to repeat the `make` target after `done`.
* The `notrace` attribute was added to disable xtrace for a rule's shell action.
* The automatic variables `${@}`, `${<}`, `${^}` and `${?}` have been added.
* An iteration block command, `loop``done`, has been added.
* A command to set common code for shell actions, `shim`, has been added.
* The `bind` command now reads library header dependency rules from a central
rules file that is automatically generated for each library by the supplied
`mkdeps` script.
* Attempting to make a rule that has already been made produces a warning.
* Attempting to declare a dependency on a rule currently being made produces a warning.
* **At strict level 1 and up:**
Expand Down
55 changes: 52 additions & 3 deletions src/cmd/INIT/mamake.c
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
* coded for portability
*/

#define RELEASE_DATE "2024-07-10"
#define RELEASE_DATE "2024-07-15"
static char id[] = "\n@(#)$Id: mamake (ksh 93u+m) " RELEASE_DATE " $\0\n";

#if _PACKAGE_ast
Expand Down Expand Up @@ -1849,9 +1849,15 @@ static unsigned long make(Rule_t *r, int inloop, unsigned long modtime, Buf_t **
switch (KEY(u[0], u[1], u[2], u[3]))
{
case KEY('b','i','n','d'):
if (t[0] == '-' && t[1] == 'l' && (s = require(t, !strcmp(v, "dontcare"))) && strncmp(r->name, "FEATURE/", 8) && strcmp(r->name, "configure.h"))
if (!(t[0] == '-' && t[1] == 'l'))
report(3, "bad -lname", t, 0);
s = require(t, !strcmp(v, "dontcare"));
if (s && strncmp(r->name, "FEATURE/", 8) && strcmp(r->name, "configure.h"))
{
/* bind to library file */
char *libname = t + 2;
/*
* bind to the *.a files that require() just derived from $INSTALLROOT/lib/lib/NAME
*/
for (;;)
{
for (t = s; *s && !isspace(*s); s++);
Expand All @@ -1869,11 +1875,54 @@ static unsigned long make(Rule_t *r, int inloop, unsigned long modtime, Buf_t **
modtime = x;
if (q->flags & RULE_error)
r->flags |= RULE_error;
report(-1, q->name, "bind: file", q->time);
}
if (!s)
break;
for (*s++ = ' '; isspace(*s); s++);
}
/*
* read library header dependency rules from $INSTALLROOT/lib/mam/NAME
*
* ...but not for a library that was just made in the same Mamfile; its header
* dependencies will already have been made as part of building that library
*/
append(buf, "lib");
append(buf, libname);
append(buf, ".a");
if ((q = getval(state.rules, use(buf))) && (q->flags & RULE_made))
continue;
/*
* The _hdrdeps_libNAME_ rule is generated by mkdeps; if its
* name is changed below, mkdeps.sh must be changed to match!
* If it has already been made...
*/
append(buf, "_hdrdeps_lib");
append(buf, libname);
add(buf, '_');
if ((q = getval(state.rules, use(buf))) && (q->flags & RULE_made))
{
/* ...then do a 'prev _hdrdeps_libNAME_' */
if (!(q->flags & RULE_ignore) && modtime < q->time)
modtime = q->time;
if (q->flags & RULE_error)
r->flags |= RULE_error;
report(-2, q->name, "bind: prev", q->time);
continue;
}
/* otherwise, include the rules file if it exists */
if (!(s = getval(state.vars, "INSTALLROOT")))
report(3, "variable must be defined", "INSTALLROOT", 0);
append(buf, s);
append(buf, "/lib/mam/");
append(buf, libname);
s = use(buf);
if (push(s, NULL, 0))
{
report(-1, s, "bind: include", 0);
make(rule(""), 0, 0, NULL);
pop();
}
}
continue;

Expand Down
162 changes: 162 additions & 0 deletions src/cmd/INIT/mkdeps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
########################################################################
# #
# This file is part of the ksh 93u+m package #
# Copyright (c) 2024 Contributors to ksh 93u+m #
# and is licensed under the #
# Eclipse Public License, Version 2.0 #
# #
# A copy of the License is available at #
# https://www.eclipse.org/org/documents/epl-2.0/EPL-2.0.html #
# (with md5 checksum 84283fa8859daf213bdda5a9f8d1be1d) #
# #
# Martijn Dekker <martijn@inlv.org> #
# #
########################################################################
#
# MAM dependency rules generator for $INSTALLROOT/include/ast headers
# By Martijn Dekker <martijn@inlv.org>, 2024-07-15
#
# Greps preinstalled AST header files for '#include <foo.h>' to generate header
# dependency rules for a library. The 'bind' command in mamake(1) Mamfiles will
# automatically include the generated dependencies in its current rule context.
#
# Usage: mkdeps -lLIBRARYNAME [ -lDEPENDNECYNAME ... ] [ HEADER.h ... ]
# The first -l option's argument is the short name of the library to be processed.
# The second and further -l options indicate its library header dependencies.
#
# This sh script is POSIX compliant and compatible with shell bugs.
#

# POSIX-ish standard mode where available.
case ${ZSH_VERSION+z} in
z) emulate ksh ;;
*) (command set -o posix) 2>/dev/null && set -o posix ;;
esac

# Safe-ish mode.
CCn='
' # one linefeed
set -o noglob
IFS=$CCn

# === Function definitions ===

note()
{
printf "$0: %s\\n" "$@" >&2
}

err_out()
{
note "$@"
exit 3
}

# Output NAME for each '#include <NAME.h>'
grep_includes()
{
spc=' ' # space followed by tab
sed -n "s|^[$spc]*#[$spc]*include[$spc]*<\([A-Za-z0-9_]*\)\.h>.*|\1|p" "$1"
}

print_indent()
{
tabs=
i=$indent
while test "$i" -gt 0
do tabs=$tabs'\t'
i=$((i - 1))
done
printf "$tabs%s\n" "$1"
}

make_hdrdeps()
{
test -f "$1.h" || return # external header
eval "state=\${state_$1}"
case $state in
making)
print_indent "note * FIXME: circular dependency: $1.h"
;;
made)
# output a 'prev' (not needed at top level; outdatedness propagates upwards)
if test "$indent" -gt 1
then print_indent "prev $prefix$1.h"
fi
;;
*)
# output a 'make' with possible recursive dependent rules
eval "state_$1=making"
print_indent "make $prefix$1.h implicit"
indent=$((indent + 1))
f=$(grep_includes "$1.h") || exit
for f in $f
do make_hdrdeps "$f"
done
indent=$((indent - 1))
print_indent "done $prefix$1.h"
eval "state_$1=made"
;;
esac
}

# === Initialisation ===

# Parse options.
lib= libdeps=
while getopts 'l:' opt
do case $opt in
l) case $lib in
'') lib=$OPTARG ;;
*) libdeps=$libdeps${libdeps:+$CCn}$OPTARG ;;
esac ;;
*) exit 2 ;;
esac
done
shift $((OPTIND - 1))

# Process library header dependencies.
for f in $libdeps
do test -f "$INSTALLROOT/lib/mam/$f" || error_out "$f: header dependencies not found"
# assign state_NAME=made for each dependency header, so only prev commands are generated
f=$(sed -n 's|.*make \${INCLUDE_AST}/\([A-Za-z0-9_]*\)\.h.*|state_\1=made|p' "$INSTALLROOT/lib/mam/$f") || exit
eval "$f"
done

# Init state.
ast=include/ast
root=${INSTALLROOT?AST environment not initialised}/$ast
prevar="INCLUDE_AST"
prefix="\${$prevar}/"
cd "$root" || error_out "cd '$root' failed"
indent=1

# Validate for 'eval' safety: header file names minus $root and .h must be valid variable name components.
for f
do test -f "$f" || error_out "$f: not found"
f=${f#"$root/"}
case ${f%.h} in
'' | *[!A-Za-z0-9_]*)
error_out "$f: invalid header file name" ;;
esac
done

# === Main ===

echo "note * Library header dependency rules for lib${lib:?option -l is required}."
echo "note * Generated by mkdeps(1). Included in Mamfile by 'bind -l$lib'."
echo "setv $prevar \${INSTALLROOT}/$ast"
# The 'bind' command in mamake(1) checks for the _hdrdeps_lib${lib}_ rule to avoid including dependencies more than once
# (it does a 'prev' instead); if the name of this internal rule is changed here, mamake.c must be changed to match!
wrapper_rule=_hdrdeps_lib${lib}_
echo "make $wrapper_rule virtual"
# Get mamake(1) to recursively include this library's header dependencies.
for f in $libdeps
do print_indent "bind -l$f"
done
# Generate header dependencies for the headers given at the command line.
for f
do f=${f#"$root/"}
make_hdrdeps "${f%.h}"
done
echo "done $wrapper_rule"
Loading

0 comments on commit 54a3e1c

Please sign in to comment.