deeplink

Automatic linking of object files using dependency information.

Overview

deeplink was born out of the following problem.

When linking an executable or library in a big project, you need to manually specify all the required .o files. A common practice is to accumulate big parts of the application in .a files. When linking the final executable, you still need to know which .a files are required (even transitively). Furthermore: in the linkage command line, you have to write them in the correct order to avoid errors. Too make things worse, all the .a linking slows down the compilation: even if an executable requires only a small part of a sub-project (say, a single .o file), you will need to provide the "big .a file" of that sub-project to the linker. Although the linker will only pick the required .o files from within the .a archive, you now have several problems:

1. In build system terms, everything that needs something from an .a archive will depend on the full archive (as far as the build system is concerned) even if during linkage only a few .o files are used by the linker from that archive. Changing any of the constituent .o files will cause a rebuild of everything that depends on the .a.

2. The order of .a files in the linker command line is important, further complicating the build rules that you must manually maintain.

3. Your build rules violate DRY: you need to repeat the linkage dependencies at every target.

4. Your build rules require specifying transitive dependencies, violating loose coupling ("law of Demeter").

deeplink is a compilation helper that runs the linker for you, while taking care of including all .o files and other linkage flags (such as -l..) required for the linked target.

In the intended usage of deeplink, the dependencies of a given .o file are determined by which .h files were #included by the code in this object file.

With deeplink, there's no need for .a files. Only the required .o files are linked.

In short, deeplink features:

• Dependency information embedded in the source code; no need to explicitly repeat dependencies in build system rules.

• DRY: Dependency information is only specified once (in the .h of each .o).

• Also dependencies on external libraries can be declared inside the source, saving you the trouble of remembering which .o needs what -l flag.

Example: how deeplink works

Here's an example module, target.c:

#include "foo_api.h"

void target__do_something();

Here's foo_api.h - it invokes a the macro, declaring that for foo_api to be implemented, we're going to need to link with foo.o:

#include <deeplink.h>
DEEPLINK__ADD_OFILE("foo.o");

... other stuff ...

Steps taken by deeplink

2. Run deeplink on target.o
3. When analyzing target.o, deeplink sees that it has a symbol referencing foo.o, and infers the dependency: target.o -> foo.o.
4. deeplink then recursively iterates over newly discovered dependencies: in this case, it now analyzes foo.o. For the example, we assume no more dependencies are found.
5. deeplink executes the linker: ld -o target foo.o target.o

When using buildsome, the awesome build system, this entire process is automated - each time deeplink accesses an .o file, it triggers a build of that file (if it isn't already built). In fact this entire process is automated by pattern rules for tests and mostly for other executables. For example, test_foo is an auto-target that deep links test_foo.o, which in turn auto-depends (due to buildsome build patterns) on test_foo.c and due to deeplink, also auto-depends on all the transitive deps of the test. Thus, deeplink facilitates radically simpler build rules (with just a single rule to declare the existence of a test executable!)

The deeplink section

deeplink requries explicit dependency information to be baked into your .o files. A special section deeplink is expected in each .o file. This section contains consecutive deeplink instructions, each is a static symbol generated by the DEEPLINK macros. Each symbol in this section is expected to be the concatenation of a pair of null-terminated strings:

1. The name of the .h file (dependent)
2. The name of the .o file that is required by the above .h

Note: the only thing used in the .h file is the directory path, to allow relative path name resolution of the dependency .o. The entire path is stored because the C preprocessor can't take just the directory name.

The second string (the .o file) can be arbitrary parameters to the linker. See deeplink.h for an example.

The entire deeplink section is only used during build, and can be stripped from the resulting executable/library.

Credits

See also: Credits file.

@sinelaw (Noam Lewis) - Big thanks for contributing the excellent README, other documentation and fixing build issues.

@da-x (Dan Aloni) - Big thanks for helping to come up with and refine the original idea. Also for the packaging help and better error messages :-)

@nadavshemer (Nadav Shemer) - Thanks for the prune support!