# Extending &SCons;: Writing Your Own Builders

Although &SCons; provides many useful methods for building common software products (programs, libraries, documents, etc.), you frequently want to be able to build some other type of file not supported directly by &SCons;. Fortunately, &SCons; makes it very easy to define your own &Builder; objects for any custom file types you want to build. (In fact, the &SCons; interfaces for creating &Builder; objects are flexible enough and easy enough to use that all of the the &SCons; built-in &Builder; objects are created using the mechanisms described in this section.)

## Writing Builders That Execute External Commands

The simplest &Builder; to create is one that executes an external command. For example, if we want to build an output file by running the contents of the input file through a command named `foobuild`, creating that &Builder; might look like:

    bld = Builder(action='foobuild < $SOURCE > $TARGET')

All the above line does is create a free-standing &Builder; object. The next section will show how to actually use it.

## Attaching a Builder to a &ConsEnv;

A &Builder; object isn't useful until it's attached to a &consenv; so that we can call it to arrange for files to be built. This is done through the &cv-link-BUILDERS; &consvar; in an environment. The &cv-link-BUILDERS; variable is a &Python; dictionary that maps the names by which you want to call various &Builder; objects to the objects themselves. For example, if we want to call the &Builder; we just defined by the name `Foo`, our &SConstruct; file might look like:

import os bld=Builder(action = 'foobuild < $SOURCE > $TARGET') env = Environment(BUILDERS={'Foo': bld}) env.AppendENVPath('PATH', os.getcwd()) env.Foo('file.foo', 'file.input')
file.input
cat
bld = Builder(action='foobuild < $SOURCE > $TARGET') env = Environment(BUILDERS={'Foo': bld})

With the &Builder; attached to our &consenv; with the name `Foo`, we can now actually call it like so:

    env.Foo('file.foo', 'file.input')

Then when we run &SCons; it looks like:

scons -Q

Note, however, that the default &cv-BUILDERS; variable in a &consenv; comes with a default set of &Builder; objects already defined: &b-link-Program;, &b-link-Library;, etc. And when we explicitly set the &cv-BUILDERS; variable when we create the &consenv;, the default &Builder;s are no longer part of the environment:

import SCons.Defaults

SCons.Defaults.ConstructionEnvironment['TOOLS'] = {} bld = Builder(action='foobuild < $SOURCE > $TARGET') env = Environment(BUILDERS={'Foo': bld}) env.Foo('file.foo', 'file.input') env.Program('hello.c')
bld = Builder(action='foobuild < $SOURCE > $TARGET') env = Environment(BUILDERS={'Foo': bld}) env.Foo('file.foo', 'file.input') env.Program('hello.c')
file.input
hello.c
scons -Q

To be able to use both our own defined &Builder; objects and the default &Builder; objects in the same &consenv;, you can either add to the &cv-link-BUILDERS; variable using the &Append; function:

import os env = Environment() env.AppendENVPath('PATH', os.getcwd()) bld = Builder(action='foobuild < $SOURCE > $TARGET') env.Append(BUILDERS={'Foo': bld}) env.Foo('file.foo', 'file.input') env.Program('hello.c')
file.input
hello.c
cat
env = Environment() bld = Builder(action='foobuild < $SOURCE > $TARGET') env.Append(BUILDERS={'Foo': bld}) env.Foo('file.foo', 'file.input') env.Program('hello.c')

Or you can explicitly set the appropriately-named key in the &cv-BUILDERS; dictionary:

env = Environment() bld = Builder(action='foobuild < $SOURCE > $TARGET') env['BUILDERS']['Foo'] = bld env.Foo('file.foo', 'file.input') env.Program('hello.c')

Either way, the same &consenv; can then use both the newly-defined `Foo` &Builder; and the default &b-link-Program; &Builder;:

scons -Q

## Letting &SCons; Handle The File Suffixes

By supplying additional information when you create a &Builder;, you can let &SCons; add appropriate file suffixes to the target and/or the source file. For example, rather than having to specify explicitly that you want the `Foo` &Builder; to build the `file.foo` target file from the `file.input` source file, you can give the `.foo` and `.input` suffixes to the &Builder;, making for more compact and readable calls to the `Foo` &Builder;:

import os bld = Builder( action='foobuild < $SOURCE > $TARGET', suffix='.foo', src_suffix='.input', ) env = Environment(BUILDERS={'Foo': bld}) env.AppendENVPath('PATH', os.getcwd()) env.Foo('file1') env.Foo('file2')
file1.input
file2.input
cat
bld = Builder( action='foobuild < $SOURCE > $TARGET', suffix='.foo', src_suffix='.input', ) env = Environment(BUILDERS={'Foo': bld}) env.Foo('file1') env.Foo('file2')
scons -Q

You can also supply a `prefix` keyword argument if it's appropriate to have &SCons; append a prefix to the beginning of target file names.

## Builders That Execute Python Functions

In &SCons;, you don't have to call an external command to build a file. You can, instead, define a &Python; function that a &Builder; object can invoke to build your target file (or files). Such a &buildfunc; definition looks like:

    def build_function(target, source, env):
        # Code to build "target" from "source"
        return None

The arguments of a &buildfunc; are:

`target`

  A list of Node objects representing the target or targets to be built by this function. The file names of these target(s) may be extracted using the &Python; &str; function.

`source`

  A list of Node objects representing the sources to be used by this function to build the targets. The file names of these source(s) may be extracted using the &Python; &str; function.

`env`

  The &consenv; used for building the target(s). The function may use any of the environment's &consvars; in any way to affect how it builds the targets.

The function will be constructed as a SCons FunctionAction and must return a `0` or `None` value if the target(s) are built successfully. The function may raise an exception or return any non-zero value to indicate that the build is unsuccessful.

For more information on Actions see the Action Objects section of the man page.

Once you've defined the &Python; function that will build your target file, defining a &Builder; object for it is as simple as specifying the name of the function, instead of an external command, as the &Builder;'s `action` argument:

def build_function(target, source, env): # Code to build "target" from "source" return None

bld = Builder( action=build_function, suffix='.foo', src_suffix='.input', ) env = Environment(BUILDERS={'Foo': bld}) env.Foo('file')
file.input

And notice that the output changes slightly, reflecting the fact that a &Python; function, not an external command, is now called to build the target file:

scons -Q

## Builders That Create Actions Using a &Generator;

&SCons; Builder objects can create an action "on the fly" by using a function called a &Generator;. (Note: this is not the same thing as a &Python; generator function described in PEP 255) This provides a great deal of flexibility to construct just the right list of commands to build your target. A &generator; looks like:

    def generate_actions(source, target, env, for_signature):
        return 'foobuild < %s > %s' % (target[0], source[0])

The arguments of a &generator; are:

`source`

  A list of Node objects representing the sources to be built by the command or other action generated by this function. The file names of these source(s) may be extracted using the &Python; &str; function.

`target`

  A list of Node objects representing the target or targets to be built by the command or other action generated by this function. The file names of these target(s) may be extracted using the &Python; &str; function.

`env`

  The &consenv; used for building the target(s). The &generator; may use any of the environment's &consvars; in any way to determine what command or other action to return.

`for_signature`

  A flag that specifies whether the &generator; is being called to contribute to a &buildsig;, as opposed to actually executing the command.

The &generator; must return a command string or other action that will be used to build the specified target(s) from the specified source(s).

Once you've defined a &generator;, you create a &Builder; to use it by specifying the `generator` keyword argument instead of `action`.

import os def generate_actions(source, target, env, for_signature): return 'foobuild < %s > %s' % (source[0], target[0])

bld = Builder( generator=generate_actions, suffix='.foo', src_suffix='.input', ) env = Environment(BUILDERS={'Foo': bld}) env.AppendENVPath('PATH', os.getcwd()) env.Foo('file')
file.input
cat
def generate_actions(source, target, env, for_signature): return 'foobuild < %s > %s' % (source[0], target[0])

bld = Builder( generator=generate_actions, suffix='.foo', src_suffix='.input', ) env = Environment(BUILDERS={'Foo': bld}) env.Foo('file')
scons -Q

Note that it's illegal to specify both an `action` and a `generator` for a &Builder;.

## Builders That Modify the Target or Source Lists Using an &Emitter;

&SCons; supports the ability for a Builder to modify the lists of target(s) from the specified source(s). You do this by defining an &emitter; function that takes as its arguments the list of the targets passed to the builder, the list of the sources passed to the builder, and the &consenv;. The emitter function should return the modified lists of targets that should be built and sources from which the targets will be built.

For example, suppose you want to define a Builder that always calls a `foobuild` program, and you want to automatically add a new target file named `new_target` and a new source file named `new_source` whenever it's called. The &SConstruct; file might look like this:

import os def modify_targets(target, source, env): target.append('new_target') source.append('new_source') return target, source

bld = Builder( action='foobuild $TARGETS - $SOURCES', suffix='.foo', src_suffix='.input', emitter=modify_targets, ) env = Environment(BUILDERS={'Foo': bld}) env.AppendENVPath('PATH', os.getcwd()) env.Foo('file')
file.input
new_source
cat
def modify_targets(target, source, env): target.append('new_target') source.append('new_source') return target, source

bld = Builder( action='foobuild $TARGETS - $SOURCES', suffix='.foo', src_suffix='.input', emitter=modify_targets, ) env = Environment(BUILDERS={'Foo': bld}) env.Foo('file')

And would yield the following output:

scons -Q

One very flexible thing that you can do is use a &consvar; to specify different emitter functions for different &consenvs;. To do this, specify a string containing a &consvar; expansion as the emitter when you call the &f-link-Builder; function, and set that &consvar; to the desired emitter function in different &consenvs;:

bld = Builder( action='./my_command $SOURCES > $TARGET', suffix='.foo', src_suffix='.input', emitter='$MY_EMITTER', )

def modify1(target, source, env): return target, source + ['modify1.in']

def modify2(target, source, env): return target, source + ['modify2.in']

env1 = Environment(BUILDERS={'Foo': bld}, MY_EMITTER=modify1) env2 = Environment(BUILDERS={'Foo': bld}, MY_EMITTER=modify2) env1.Foo('file1') env2.Foo('file2')
file1.input
file2.input
modify1.input
modify2.input
cat

In this example, the `modify1.in` and `modify2.in` files get added to the source lists of the different commands:

scons -Q

## Modifying a Builder by adding an Emitter

Defining an emitter to work with a custom Builder is a powerful concept, but sometimes all you really want is to be able to use an existing builder but change its concept of what targets are created. In this case, trying to recreate the logic of an existing Builder to supply a special emitter can be a lot of work. The typical case for this is when you want to use a compiler flag that causes additional files to be generated. For example the GNU linker accepts an option `-Map` which outputs a link map to the file specified by the option's argument. If this option is just supplied to the build, &SCons; will not consider the link map file a tracked target, which has various undesirable efffects.

To help with this, &SCons; provides &consvars; which correspond to a few standard builders: &cv-link-PROGEMITTER; for &b-link-Program;; &cv-link-LIBEMITTER; for &b-link-Library;; &cv-link-SHLIBEMITTER; for &b-link-SharedLibrary; and &cv-link-LDMODULEEMITTER; for &b-link-LoadableModule;;. Adding an emitter to one of these will cause it to be invoked in addition to any existing emitter for the corresponding builder.

This example adds map creation as a linker flag, and modifies the standard &b-link-Program; emitter to know that map generation is a side-effect:

env = Environment() map_filename = "${TARGET.name}.map"

def map_emitter(target, source, env): target.append(map_filename) return target, source

env.Append(LINKFLAGS="-Wl,-Map={},--cref".format(map_filename)) env.Append(PROGEMITTER=map_emitter) env.Program('hello.c')
#include <stdio.h>

int main() { printf("Hello, world!\n"); }

If you run this example, adding an option to tell &SCons; to dump some information about the dependencies it knows, it shows the map file option in use, and that &SCons; indeed knows about the map file, it's not just a silent side effect of the compiler:

scons -Q --tree=prune

## Where To Put Your Custom Builders and Tools

The `site_scons` directories give you a place to put &Python; modules and packages that you can import into your &SConscript; files (at the top level) add-on tools that can integrate into &SCons; (in a `site_tools` subdirectory), and a `site_scons/site_init.py` file that gets read before any &SConstruct; or &SConscript; file, allowing you to change &SCons;'s default behavior.

Each system type (Windows, Mac, Linux, etc.) searches a canonical set of directories for `site_scons`; see the man page for details. The top-level SConstruct's `site_scons` dir (that is, the one in the project) is always searched last, and its dir is placed first in the tool path so it overrides all others.

If you get a tool from somewhere (the &SCons; wiki or a third party, for instance) and you'd like to use it in your project, a `site_scons` dir is the simplest place to put it. Tools come in two flavors; either a &Python; function that operates on an &Environment; or a &Python; module or package containing two functions, `exists()` and `generate()`.

A single-function Tool can just be included in your `site_scons/site_init.py` file where it will be parsed and made available for use. For instance, you could have a `site_scons/site_init.py` file like this:

def TOOL_ADD_HEADER(env): """A Tool to add a header from $HEADER to the source file""" add_header = Builder( action=['echo "$HEADER" > $TARGET', 'cat $SOURCE >> $TARGET'] ) env.Append(BUILDERS={'AddHeader': add_header}) env['HEADER'] = '' # set default value
env=Environment(tools=['default', TOOL_ADD_HEADER], HEADER="=====") env.AddHeader('tgt', 'src')
hi there

and a &SConstruct; like this:

# Use TOOL_ADD_HEADER from site_scons/site_init.py env=Environment(tools=['default', TOOL_ADD_HEADER], HEADER="=====") env.AddHeader('tgt', 'src')

The `TOOL_ADD_HEADER` tool method will be called to add the `AddHeader` tool to the environment.

A more full-fledged tool with `exists()` and `generate()` methods can be installed either as a module in the file `site_scons/site_tools/toolname.py` or as a package in the directory `site_scons/site_tools/toolname`. In the case of using a package, the `exists()` and `generate()` are in the file `site_scons/site_tools/toolname/__init__.py`. (In all the above case `toolname` is replaced by the name of the tool.) Since `site_scons/site_tools` is automatically added to the head of the tool search path, any tool found there will be available to all environments. Furthermore, a tool found there will override a built-in tool of the same name, so if you need to change the behavior of a built-in tool, `site_scons` gives you the hook you need.

Many people have a collection of utility &Python; functions they'd like to include in their &SConscript; files: just put them in `site_scons/my_utils.py` or any valid &Python; module name of your choice. For instance you can do something like this in `site_scons/my_utils.py` to add `build_id` and `MakeWorkDir` functions:

from SCons.Script import * # for Execute and Mkdir

def build_id(): """Return a build ID (stub version)""" return "100"

def MakeWorkDir(workdir): """Create the specified dir immediately""" Execute(Mkdir(workdir))
import my_utils

MakeWorkDir('/tmp/work') print("build_id=" + my_utils.build_id())

And then in your &SConscript; or any sub-&SConscript; anywhere in your build, you can import `my_utils` and use it:

import my_utils print("build_id=" + my_utils.build_id()) my_utils.MakeWorkDir('/tmp/work')

You can put this collection in its own module in a `site_scons` and import it as in the example, or you can include it in `site_scons/site_init.py`, which is automatically imported (unless you disable site directories). Note that in order to refer to objects in the SCons namespace such as &Environment; or &Mkdir; or &Execute; in any file other than a &SConstruct; or &SConscript; you always need to do

from SCons.Script import *

This is true of modules in `site_scons` such as `site_scons/site_init.py` as well.

You can use any of the user- or machine-wide site directories such as `~/.scons/site_scons` instead of `./site_scons`, or use the `--site-dir` option to point to your own directory. `site_init.py` and `site_tools` will be located under that directory. To avoid using a `site_scons` directory at all, even if it exists, use the `--no-site-dir` option.