You are currently browsing the category archive for the ‘Build Systems’ category.

I’ve seen a few people assert the precompiled headers are a pain in the butt, or not workable for large scale projects.  In fact, it’s incredibly easy to add precompiled headers to a GCC-based project, and it can be quite beneficial.

Since I’m mostly familiar with Makefiles, I’ll present an example here that uses Make. It trivially extends to other similar build systems such as SCons, NMake, or Ant, but I’m not so sure about Visual studio projects. This example builds a single static library and several test applications. I’ve stripped out most of my compiler flags for brevity.

# boilerplate settings...
SHELL = /bin/bash
CXX = g++ -c
CXXFLAGS += -std=c++98 -pedantic -MMD -g -Wall -Wextra
LD = g++
LDFLAGS += -rdynamic -fno-stack-protector
AR = ar

# generic build rules
# $@ is the target, $< is the first source, $^ is all sources
define compile
$(CXX) -o $@ $< $(CXXFLAGS)
endef

define link
$(LD) -o $@ $(filter %.o,$^) $(filter %.a,$^) $(LDFLAGS)
endef

define ar
$(AR) qsc $@ $(filter %.o,$^)
endef

# all library code is in src/
# all test applications are single-source
# e.g. testfoo is produced from from testfoo.cxx and libsseray.a
TEST_SRC = $(wildcard test*.cxx)
LIB_SRC = $(wildcard src/*.cxx)
TESTS = $(basename $(TEST_SRC))
LIB = libsseray.a
all : $(TESTS)

$(TESTS) : $(LIB)
$(TESTS) : % : %.o
	$(link)

%.o : %.cxx
	$(compile)

$(LIB) : $(LIB_SRC:cxx=o)
	$(ar)

# gcc-provided #include dependencies
-include $(TEST_SRC:cxx=d) $(LIB_SRC:cxx=d)

clean :
	rm -f $(LIB) $(TESTS) $$(find . -name '*.o' -o -name '*.d')

In order to use a precompiled header, this is what needs to be added to the Makefile. There are no source code modifications at all. I created a file pre.h that includes all of the system C and C++ headers that I use (in particular <iostream> is a big expense for the compiler).

# PCH is built just like all other source files
# CXXFLAGS must match everything else
pre.h.gch : pre.h
	$(compile)

# all object files depend on the PCH for build ordering
$(TEST_SRC:cxx=o) $(LIB_SRC:cxx=o) : pre.h.gch
# this is equivalent to adding '#include <pre.h>' to the top of every source file
$(TEST_SRC:cxx=o) $(LIB_SRC:cxx=o) : CXXFLAGS += -include pre.h

# pre.h.gch should be cleaned up along with everything else
clean :
	rm -f (...) pre.h.gch

The project itself is fairly small—16 source files totaling 1800 SLOC—but this small change just decreased my total build time from 12 to 8 seconds. This is entirely non-intrusive to the code, so adding it is a one-time cost (as opposed to the Microsoft stdafx.h approach, which is O(N) cost in the number of source files). It is also easy to disable for production builds, if you only want to use the PCH machinery for your day-to-day edit-compile-test cycle.

Advertisements

Anyone who has used GNU Make on a nontrivial project surely has at some point wanted to separate inputs from outputs and intermediates. For example, take a C++ shared library using GCC (using -MMD to autogenerate GNU make dependency information):

inputs
  • baz.cxx
  • bar.cxx
intermediates
  • baz.o
  • baz.dep
  • bar.o
  • bar.dep
outputs
  • libfoo.so.1.1234
  • libfoo.so.1 (symlink to libfoo.so.1.1234)
  • libfoo.so (symlink to libfoo.so.1)

First, anyone who hasn’t already should read How Not to Use VPATH. Now, suppose that I want to put intermediates into $(CURDIR)/tmp and outputs into $(CURDIR)/lib, and those directories must be created as a part of the build. That means that at some point in time, the build script needs to execute mkdir tmp and mkdir lib, because GCC will not auto-create them. These are the solutions I’ve come upon (aside from the obvious but useless “don’t use make”):

  1. Order the rules so that the mkdirs occur before the g++s. Of course, this breaks parallel builds, so it’s not really an option. But it’s the most obvious solution, and it doesn’t break in serial builds.
  2. Modify the rule to depend upon the directory (tmp/%.o : %.cxx tmp). There’s only one call to mkdir, but now we get a new sort of error. Whenever the directory gets modified, it triggers recompilation of all of the objects!
  3. Make the compilation rule (tmp/%.o : %.cxx) execute the mkdir before executing the g++. There’ll be a lot of spurious calls to mkdir, but it should never break.
  4. make a local g++ wrapper script that creates the directory before compiling. Invoke the wrapper script from the Makefile instead of invoking the raw compiler.

Personally, I don’t think that any of those solutions is that great. The problem, as I see it, is the combination of how Make handles dependencies (by modified-timestamp) and how directory timestamps work on Linux (if you add or remove files, it “modifies” the directory). I’ve dealt with it using solutions #1-3, but I haven’t tried #4. I suppose for a large project that #4 isn’t such a bad idea… you can wrap up all of your system-wide rules into it, and then the make output gets shorter, as well. Take this as an example:

/usr/local/bin/optim-g++


#!/bin/bash
while read arg ; do
    if [ "$arg" = "-o" ] ; then
        read outdir
        mkdir -p $(dirname "$outdir")
    fi
done < <(echo "$@")

g++ -c -ansi -pedantic -std=c++98 \
    -O3 -m{arch=core2,sse2,fpmath=sse,inline-all-stringops} \
    -W{all,extra,format=2,write-strings,init-self,error} \
    -W{cast-align,cast-qual,pointer-arith,old-style-cast,overloaded-virtual} \
    -f{omit-frame-pointer,strict-aliasing,fast-math,tracer} \
    -I{/usr/local/include,/usr/java/jdk1.5/include,/usr/java/jdk1.5/include/linux} \
    "$@"

Ok, now I’m convinced. I do like #4… It seems like a lot of extra work for small one-off projects, but it’s a marginal cost for a larger environment (where you end up doing more complex tasks like code generation, combining disparate projects into a single build location, automated source-control integration).