# Extending &SCons;: Writing Your Own Scanners

&SCons; has built-in scanners that know how to look in C/C++, Fortran, D, IDL, LaTeX, Python and SWIG source files for information about other files that targets built from those files depend on--for example, in the case of files that use the C preprocessor, the `.h` files that are specified using `#include` lines in the source. You can use the same mechanisms that &SCons; uses to create its built-in scanners to write scanners of your own for file types that &SCons; does not know how to scan "out of the box."

## A Simple Scanner Example

Suppose, for example, that we want to create a simple scanner for `.foo` files. A `.foo` file contains some text that will be processed, and can include other files on lines that begin with `include` followed by a file name:

    include filename.foo

Scanning a file will be handled by a Python function that you must supply. Here is a function that will use the Python `re` module to scan for the `include` lines in our example:

    import re

    include_re = re.compile(r'^include\s+(\S+)$', re.M)

    def kfile_scan(node, env, path, arg):
        contents = node.get_text_contents()
        return env.File(include_re.findall(contents))

It is important to note that you have to return a list of File nodes from the scanner function, simple strings for the file names won't do. As in the examples we are showing here, you can use the &File; function of your current &consenv; in order to create nodes on the fly from a sequence of file names with relative paths.

The scanner function must accept the four specified arguments and return a list of implicit dependencies. Presumably, these would be dependencies found from examining the contents of the file, although the function can perform any manipulation at all to generate the list of dependencies.

node

  An &SCons; node object representing the file being scanned. The path name to the file can be used by converting the node to a string using the `str()` function, or an internal &SCons; `get_text_contents()` object method can be used to fetch the contents.

env

  The &consenv; in effect for this scan. The scanner function may choose to use &consvars; from this environment to affect its behavior.

path

  A list of directories that form the search path for included files for this scanner. This is how &SCons; handles the &cv-link-CPPPATH; and &cv-link-LIBPATH; variables.

arg

  An optional argument that you can choose to have passed to this scanner function by various scanner instances.

A Scanner object is created using the &f-link-Scanner; function, which typically takes an `skeys` argument to associate a file suffix with this scanner. The Scanner object must then be associated with the &cv-link-SCANNERS; &consvar; in the current &consenv;, typically by using the &f-link-Append; method:

    kscan = Scanner(function=kfile_scan, skeys=['.k'])
    env.Append(SCANNERS=kscan)

When we put it all together, it looks like:

import re

include_re = re.compile(r'^include\s+(\S+)$', re.M)

def kfile_scan(node, env, path): contents = node.get_text_contents() includes = include_re.findall(contents) return env.File(includes)

kscan = Scanner(function=kfile_scan, skeys=['.k'])

env = Environment(ENV={'PATH': '__ROOT__/usr/local/bin'}) env.Append(SCANNERS=kscan)

env.Command('foo', 'foo.k', 'kprocess < $SOURCES > $TARGET')
include other_file
other_file
cat

## Adding a search path to a scanner: &FindPathDirs;

If the build tool in question will use a path variable to search for included files or other dependencies, then the Scanner will need to take that path variable into account as well - &cv-link-CPPPATH; and &cv-link-LIBPATH; are used this way, for example. The path to search is passed to your scanner as the `path` argument. Path variables may be lists of nodes, semicolon-separated strings, or even contain &consvars; which need to be expanded. &SCons; provides the &f-link-FindPathDirs; function which returns a callable to expand a given path (given as a SCons &consvar; name) to a list of paths at the time the scanner is called. Deferring evaluation until that point allows, for instance, the path to contain &cv-link-TARGET; references which differ for each file scanned.

Using &FindPathDirs; is quite easy. Continuing the above example, using `KPATH` as the &consvar; with the search path (analogous to &cv-link-CPPPATH;), we just modify the call to the &Scanner; factory function to include a path keyword arg:

kscan = Scanner(function=kfile_scan, skeys=['.k'], path_function=FindPathDirs('KPATH'))

&FindPathDirs; returns a callable object that, when called, will essentially expand the elements in `env['KPATH']` and tell the scanner to search in those dirs. It will also properly add related repository and variant dirs to the search list. As a side note, the returned method stores the path in an efficient way so lookups are fast even when variable substitutions may be needed. This is important since many files get scanned in a typical build.

## Using scanners with Builders

One approach for introducing scanners into the build is in conjunction with a Builder. There are two relvant optional parameters we can use when creating a builder: `source_scanner` and `target_scanner`. `source_scanner` is used for scanning source files, and `target_scanner` is used for scanning the target once it is generated.

import re

include_re = re.compile(r'^include\s+(\S+)$', re.M)

def kfile_scan(node, env, path, arg): contents = node.get_text_contents() return env.File(include_re.findall(contents))

kscan = Scanner(function=kfile_scan, skeys=['.k'], path_function=FindPathDirs('KPATH')

def build_function(target, source, env): # Code to build "target" from "source" return None

bld = Builder( action=build_function, suffix='.foo', source_scanner=kscan, src_suffix='.input', ) env = Environment(BUILDERS={'Foo': bld}) env.Foo('file')

An emitter function can modify the list of sources or targets passed to the action function when the builder is triggered.

A scanner function will not affect the list of sources or targets seen by the builder during the build action. The scanner function will however affect if the builder should rebuild (if any of the files sourced by the scanner have changed for example).