5Working with Makefiles

Last Update: September 26, 2014.

Overview

The overview for this chapter consists of the following topics:

  • Introduction

  • Chapter Outline

Introduction

Building executables and libraries for a large, integrated set of software tools such as the C++ Toolkit, and doing so consistently on different platforms and architectures, is a daunting task. Therefore, the Toolkit developers have expended considerable effort to design a build system based upon the make utility as controlled by makefiles. Although it is, of course, possible to write one's own Toolkit makefile from scratch, it is seldom desirable. To take advantage of the experience, wisdom, and alchemy invested in Toolkit and to help avoid often inscrutable compilation issues:

We strongly advise users to work with the Toolkit's make system.

With minimal manual editing (and after invoking the configure script in your build tree), the build system adapts to your environment, compiler options, defines all relevant makefile macros and targets, allows for recursive builds of the entire Toolkit and targeted builds of single modules, and handles many other details that can confound manual builds.

For information on configuring and building that isn't specific to makefiles, see the chapters on configuring and managing projects.

Major Makefiles

The C++ Toolkit build system was based on the "GNU build system", which employs a configure script and makefiles. Before describing the C++ Toolkit build system in detail, we list the major types of makefiles used by the Toolkit:

  • Meta-makefiles. These files exist for each project and tie the project together in the Toolkit hierarchy; defining those applications and libraries as a project is necessary for (possibly recursively) building.

  • Generic makefile templates (Makefile*.in). The configure script processes these files from the src hierarchy to substitute for the special tags "@some_name@" and make other specializations required for a given project. Note that meta-makefiles are typically derived from such templates.

  • Customized makefiles (Makefile.*.[lib|app]). For each library or application, this file gives specific targets, compiler flags, and other project-specific build instructions. These files appear in the src hierarchy.

  • Configured makefiles (Makefile or Makefile.*_[lib|app]). A makefile generated by configure for each project and sub-project and placed in the appropriate location in the build tree ready for use will be called a "configured makefile". Note that meta-makefiles in the build tree may be considered “configured”.

Makefile Hierarchy

All Toolkit makefiles reside in either the src directory as templates or customized files, or in the appropriate configured form in each of your <builddir> hierarchies as illustrated in Figure 1

Figure 1. Makefile hierarchy.

Figure 1. Makefile hierarchy.

Most of the files listed in Figure 1 are templates from the src directory, with each corresponding configured makefile at the top of the build tree. Of these, <builddir>/Makefile can be considered the master makefile in that it can recursively build the entire Toolkit. The role of each top-level makefile template is summarized as follows:

  • Makefile.in - makefile to perform a recursive build in all project subdirectories.

  • Makefile.meta.in - included by all makefiles that provide both local and recursive builds.

  • Makefile.mk.in - included by all makefiles; sets a lot of configuration variables.

  • Makefile.lib.in - included by all makefiles that perform a "standard" library build, when building only static libraries.

  • Makefile.dll.in - included by all makefiles that perform a "standard" library build, when building only shared libraries.

  • Makefile.both.in - included by all makefiles that perform a "standard" library build, when building both static and shared libraries.

  • Makefile.lib.tmpl.in - serves as a template for the project customized makefiles (Makefile.*.lib[.in]) that perform a "standard" library build.

  • Makefile.app.in - included by all makefiles that perform a "standard" application build.

  • Makefile.app.tmpl.in - serves as a template for the project customized makefiles (Makefile.*.app[.in]) that perform a "standard" application build.

  • Makefile.rules.in, Makefile.rules_with_autodep.in -- instructions for building object files; included by most other makefiles.

The project-specific portion of the makefile hierarchy is represented in the figure by the meta-makefile template c++/src/myProj/Makefile.in, the customized makefile c++/src/myProj/Makefile.myProj.[app|lib] (not shown), and the configured makefile c++/myBuild/build/myProj/Makefile. In fact, every project and sub-project in the Toolkit has analogous files specialized to its project; in most circumstances, every new or user project should emulate this file structure to be compatible with the make system.

Meta-Makefiles

A typical meta-makefile template (e.g. Makefile.in in your foo/c++/src/bar_proj/ dir) looks like this:

# Supply Makefile.bar_u1, Makefile.bar_u2 ...
#
USR_PROJ = bar_u1 bar_u2 ...

# Supply Makefile.bar_l1.lib, Makefile.bar_l2.lib ...
#
LIB_PROJ = bar_l1 bar_l2 ...

# Supply Makefile.bar_a1.app, Makefile.bar_a2.app ...
#
APP_PROJ = bar_a1 bar_a2 ...

# Subprojects
#
SUB_PROJ = app sub_proj1 sub_proj2

srcdir = @srcdir@
include @builddir@/Makefile.meta

This template separately specifies instructions for user, library and application projects, along with a set of three sub-projects that can be made. The mandatory final two lines "srcdir = @srcdir@; include @builddir@/Makefile.meta" define the standard build targets.

Makefile.in Meta Files

The Makefile.in meta-make file in the project's source directory defines a kind of road map that will be used by the configure script to generate a makefile (Makefile) in the corresponding directory of the build tree. Makefile.in does not participate in the actual execution of make, but rather, defines what will happen at that time by directing the configure script in the creation of the Makefile that will be executed (see also the description of standard build targets below).

The meta-makefile myProj/Makefile.in should define at least one of the following macros:

  • USR_PROJ (optional) - a list of names for user-defined makefiles. This macro is provided for the usage of ordinary stand-alone makefiles which do not utilize the make commands contained in additional makefiles in the top-level build directory. Each p_i listed in USR_PROJ = p_1 ... p_N must have a corresponding Makefile.p_i in the project's source directory. When make is executed, the make directives contained in these files will be executed directly to build the targets as specified.

  • LIB_PROJ (optional) - a list of names for library makefiles. For each library l_i listed in LIB_PROJ = l_1 ... l_N, you must have created a corresponding project makefile named Makefile.l_i.lib in the project's source directory. When make is executed, these library project makefiles will be used along with Makefile.lib and Makefile.lib.tmpl (located in the top-level of the build tree) to build the specified libraries.

  • ASN_PROJ (optional) is like LIB_PROJ, with one additional feature: Any projects listed there will be interpreted as the names of ASN.1 module specifications to be processed by datatool.

  • APP_PROJ (optional) - a list of names for application makefiles. Similarly, each application (p1, p2, ..., pN) listed under APP_PROJ must have a corresponding project makefile named Makefile.p*.app in the project's source directory. When make is executed, these application project makefiles will be used along with Makefile.app and Makefile.app.tmpl to build the specified executables.

  • SUB_PROJ (optional) - a list of names for subproject directories (used on recursive makes). The SUB_PROJ macro is used to recursively define make targets; items listed here define the subdirectories rooted in the project's source directory where make should also be executed.

Some additional meta-makefile macros (listed in Table 1) exist to specify various directory paths that make needs to know. The "@"-delimited tokens are substituted during configuration based on your environment and any command-line options passed to configure.

Table 1. Path Specification Makefile Macros

MacroSourceSynopsis
bindir@bindir@,
--bindir
Executables built inside the build tree
build_root@build_root@Path to the whole build tree
builddir@builddir@Top build directory inside the build tree
datadir--datadirRead-only architecture-independent data
incdir@incdir@Top include directory inside the build tree
includedir@includedir@,
--includedir
Top include directory in the source tree
infodir--infodirInfo documentation
libdir@libdir@,
--libdir
Libraries built inside the build tree
libexecdir--libexecdirProgram executables
localstatedir--localstatedirModifiable single-machine data
mandir--mandirMan documentation
oldincludedir--oldincludedirC header files for non-gcc
sbindir--sbindirSystem admin executables
sharedstatedir--sharedstatedirModifiable architecture-independent data
srcdir@srcdir@,
--srcdir
Directory in the source tree that corresponds to the directory (./) in the build tree where the build is currently going on
status_dir@status_dir@Configuration status files
sysconfdir--sysconfdirRead-only single-machine data (default)
top_srcdir@top_srcdir@Path to the whole NCBI C++ package

Expendable Projects

By default, failure of any project will cause make to exit immediately. Although this behavior can save a lot of time, it is not always desirable. One way to avoid it is to run make -k rather than make, but then major problems affecting a large portion of the build will still waste a lot of time.

Consequently, the toolkit's build system supports an alternative approach: meta-makefiles can define expendable projects which should be built if possible but are allowed to fail without interrupting the build. The way to do this is to list such projects in EXPENDABLE_*_PROJ rather than *_PROJ.

Project Makefiles

When beginning a new project, the new_project shell script will generate an initial configured makefile, Makefile.<project_name>_app, that you can modify as needed. In addition, a working sample application can also be checked out to experiment with or as an alternate template.

The import_project script is useful for working on existing Toolkit projects without needing to build the whole Toolkit. In this case things are particularly straightforward as the project will be retrieved complete with its makefile already configured as Makefile.<project_name>_[app|lib]. (Note that there is an underscore in the name, not a period as in the similarly-named customizable makefile from which the configured file is derived.)

If you are working outside of the source tree: In this scenario you are only linking to the Toolkit libraries and will not need to run the configure script, so a Makefile.in template meta-makefile is not required. Some of the typical edits required for the customized makefile are shown in the section on working in a separate directory.

If you are working within the source tree or subtree: Project subdirectories that do not contain any *.in files are ignored by the configure script. Therefore, you will now also need to create a meta-makefile for the newly created project before configuring your build directory to include the new project.

Several examples are detailed on the "Starting New Projects" section.

List of optional packages, features and projects

Table 2 displays the keywords you can list in REQUIRES in a customized application or library makefile, along with the corresponding configure options:

Table 2. Optional Packages, Features, and Projects

KeywordOptional ...Configure options
... packages
C-ToolkitNCBI C Toolkit--without-ncbi-c
Fast-CGIFast-CGI library--without-fastcgi
FLTKthe Fast Light ToolKit--without-fltk,
--with-fltk=DIR
FreeTDSFreeTDS libraries--without-ftds,
--with-ftds=DIR
GEONCBI GEO libraries--without-geo
ORBacusORBacus CORBA--without-orbacus,
--with-orbacus=DIR
PubMedNCBI PubMed libraries--without-pubmed
SPSP libraries--without-sp
SSSDBNCBI SSS DB library--without-sssdb,
--without-sss
SSSUTILSNCBI SSS UTILS library--without-sssutils,
--without-sss
SybaseSybase libraries--without-sybase,
--with-sybase-local(=DIR),
--with-sybase-new
wxWindowswxWindows--without-wxwin,
--with-wxwin=DIR
... features
MTmultithreading is available--with-mt
... projects
appstandalone applications like ID1_FETCH--with-app
ctoolsprojects based on the NCBI C toolkit--without-ctools
guiprojects that use the wxWindows GUI package--without-gui
internalall internal projects--with-internal
objectslibraries to serialize ASN.1/XML objects--with-objects
serialASN.1/XML serialization library and datatool--without-serial
local_lbsmIPC with locally running LBSMD--without-local-lbsm

Standard Build Targets

The following topics are discussed in this section:

Meta-Makefile Targets

The mandatory lines from the meta-makefile example above,

srcdir = @srcdir@
include @builddir@/Makefile.meta

provide the build rules for the following standard meta-makefile targets:

  • all:

    • Run "make -f {Makefile.*} all" for the makefiles with the suffixes listed in macro USR_PROJ:
      make -f Makefile.bar_u1 all
      make -f Makefile.bar_u2 all
      ...

    • Build libraries using attributes defined in the customized makefiles Makefile.*.lib with the suffixes listed in macro LIB_PROJ.

    • Build application(s) using attributes defined in the customized makefiles Makefile.*.app with the suffixes listed in macro APP_PROJ.

  • all_r:

    • First make target all, then run "make all_r" in all subdirectories enlisted in $(SUB_PROJ):
      cd bar_test
      make -f Makefile all_r
      cd bar_sub_proj1
      make -f Makefile all_r
      ...

  • clean, clean_r:

    • Run the same makefiles but with targets clean and clean_r (rather than all and all_r), respectively.

  • purge, purge_r:

    • Run the same makefiles but with targets purge and purge_r.

Makefile Targets

The standard build targets for Toolkit makefiles are all, clean and purge. Recall that recursive versions of these targets exist for meta-makefiles.

  • all:

    • Compile the object modules specified in the "$(OBJ)" macro, and use them to build the library "$(LIB)" or the application "$(APP)"; then copy the resultant [lib|app] to the [libdir|bindir] directory, respectively.

  • clean:

    • Remove all object modules and libs/apps that have been built by all.

  • purge:

    • Do clean, and then remove the copy of the [libs|apps] from the [libdir|bindir] directory.

The customized makefiles do not distinguish between recursive (all_r, clean_r, purge_r) and non-recursive (all, clean, purge) targets -- because the recursion and multiple build is entirely up to the meta-makefiles.

Makefile Macros and Makefile.mk

There is a wide assortment of configured tools, flags, third party packages and paths (see above). They can be specified for the whole build tree with the appropriate entry in Makefile.mk, which is silently included at the very beginning of the customized makefiles used to build libraries and applications.

Many makefile macros are supplied with defaults ORIG_* in Makefile.mk. See the list of ORIG_* macros, and all others currently defined, in the Makefile.mk.in template for details. One should not override these defaults in normal use, but add your own flags to them as needed in the corresponding working macro; e.g., set CXX = $(ORIG_CXX) -DFOO_BAR.

Makefile.mk defines makefile macros obtained during the configuration process for flags (seeTable 3), system and third-party packages (see Table 4) and development tools (see Table 5).

Table 3. Flags

MacroSourceSynopsis
APP_LDFLAGScompiler testCompiler-dependent variaton on LDFLAGS
CFLAGS$CFLAGSC compiler flags
CPPFLAGS$CPPFLAGSC/C++ preprocessor flags
CXXFLAGS$CXXFLAGSC++ compiler flags
DEPFLAGS$DEPFLAGSFlags for file dependency lists
DEPFLAGS_POSTcompiler testRelated to VisualAge (retained for historical reasons)
DLL_LDFLAGScompiler testCompiler-dependent variaton on LDFLAGS
FAST_CFLAGS$FAST_CFLAGS(*) C compiler flags to generate faster code
FAST_CXXFLAGS$FAST_CXXFLAGS(*) C++ compiler flags to generate faster code
LDFLAGS$LDFLAGSLinker flags
LIB_OR_DLL@LIB_OR_DLL@Specify whether to build a library as static or dynamic
STATIC@STATIC@Library suffix to force static linkage (see example)

Table 4. System and third-party packages

MacroSourceSynopsis
FASTCGI_INCLUDE$FASTCGI_INCLUDEFast-CGI headers
FASTCGI_LIBS$FASTCGI_LIBSFast-CGI libraries
KSTAT_LIBS$KSTAT_LIBSKSTAT library (system)
LIBS$LIBSDefault libraries to link with
MATH_LIBS$MATH_LIBSMath library (system)
NETWORK_LIBS$NETWORK_LIBSNetwork library (system)
NCBI_C_INCLUDE$NCBI_C_INCLUDENCBI C toolkit headers
NCBI_C_LIBPATH$NCBI_C_LIBPATHPath to the NCBI C Toolkit libraries
NCBI_C_ncbi$NCBI_C_ncbiNCBI C CoreLib
NCBI_PM_PATH$NCBI_PM_PATHPath to the PubMed package
NCBI_SSS_INCLUDE$NCBI_SSS_INCLUDENCBI SSS headers
NCBI_SSS_LIBPATH$NCBI_SSS_LIBPATHPath to NCBI SSS libraries
ORBACUS_INCLUDE$ORBACUS_LIBPATHPath to the ORBacus CORBA headers
ORBACUS_LIBPATH$ORBACUS_LIBPATHPath to the ORBacus CORBA libraries
PRE_LIBS$PRE_LIBSUse PRE_LIBS to place specific libraries or library directories earlier in the link command line than the standard libraries or directories (i.e. to precede $LIBS). For example, if you wanted to link with your custom library mylib/libmylib.a and also use a locally modified version of an NCBI library saved in a directory called ncbilibs you could use a PRE_LIBS macro similar to:
PRE_LIBS = -lmylib/mylib -Lncbilibs
RPCSVC_LIBS$RPCSVC_LIBSRPCSVC library (system)
SYBASE_INCLUDE$SYBASE_INCLUDESYBASE headers
SYBASE_LIBS$SYBASE_LIBSSYBASE libraries
THREAD_LIBS$THREAD_LIBSThread library (system)

Note: The values of the user-specified environment variables $FAST_CFLAGS and $FAST_CXXFLAGS will substitute the regular optimization flag -O (or -O2, etc.). For example, if in the environment: $FAST_CXXFLAGS=-fast -speedy and $CXXFLAGS=-warn -O3 -std, then in makefile: $(FAST_CXXFLAGS)=-warn -fast -speedy -std.

Cover of The NCBI C++ Toolkit Book
The NCBI C++ Toolkit Book [Internet].
Vakatov D, editor.
  • Table of Contents Page
  • |
  • Cite this Page
Search
  
Source Browsers
SVN Source Repository