Anyone who has used GNU Make on a nontrivial project surely has at some point wanted to separate inputs from outputs and intermediates. For example, take a C++ shared library using GCC (using -MMD to autogenerate GNU make dependency information):

inputs
  • baz.cxx
  • bar.cxx
intermediates
  • baz.o
  • baz.dep
  • bar.o
  • bar.dep
outputs
  • libfoo.so.1.1234
  • libfoo.so.1 (symlink to libfoo.so.1.1234)
  • libfoo.so (symlink to libfoo.so.1)

First, anyone who hasn’t already should read How Not to Use VPATH. Now, suppose that I want to put intermediates into $(CURDIR)/tmp and outputs into $(CURDIR)/lib, and those directories must be created as a part of the build. That means that at some point in time, the build script needs to execute mkdir tmp and mkdir lib, because GCC will not auto-create them. These are the solutions I’ve come upon (aside from the obvious but useless “don’t use make”):

  1. Order the rules so that the mkdirs occur before the g++s. Of course, this breaks parallel builds, so it’s not really an option. But it’s the most obvious solution, and it doesn’t break in serial builds.
  2. Modify the rule to depend upon the directory (tmp/%.o : %.cxx tmp). There’s only one call to mkdir, but now we get a new sort of error. Whenever the directory gets modified, it triggers recompilation of all of the objects!
  3. Make the compilation rule (tmp/%.o : %.cxx) execute the mkdir before executing the g++. There’ll be a lot of spurious calls to mkdir, but it should never break.
  4. make a local g++ wrapper script that creates the directory before compiling. Invoke the wrapper script from the Makefile instead of invoking the raw compiler.

Personally, I don’t think that any of those solutions is that great. The problem, as I see it, is the combination of how Make handles dependencies (by modified-timestamp) and how directory timestamps work on Linux (if you add or remove files, it “modifies” the directory). I’ve dealt with it using solutions #1-3, but I haven’t tried #4. I suppose for a large project that #4 isn’t such a bad idea… you can wrap up all of your system-wide rules into it, and then the make output gets shorter, as well. Take this as an example:

/usr/local/bin/optim-g++


#!/bin/bash
while read arg ; do
    if [ "$arg" = "-o" ] ; then
        read outdir
        mkdir -p $(dirname "$outdir")
    fi
done < <(echo "$@")

g++ -c -ansi -pedantic -std=c++98 \
    -O3 -m{arch=core2,sse2,fpmath=sse,inline-all-stringops} \
    -W{all,extra,format=2,write-strings,init-self,error} \
    -W{cast-align,cast-qual,pointer-arith,old-style-cast,overloaded-virtual} \
    -f{omit-frame-pointer,strict-aliasing,fast-math,tracer} \
    -I{/usr/local/include,/usr/java/jdk1.5/include,/usr/java/jdk1.5/include/linux} \
    "$@"

Ok, now I’m convinced. I do like #4… It seems like a lot of extra work for small one-off projects, but it’s a marginal cost for a larger environment (where you end up doing more complex tasks like code generation, combining disparate projects into a single build location, automated source-control integration).

Advertisements