# Building From Code Repositories

Often, a software project will have one or more central repositories, directory trees that contain source code, or derived files, or both. You can eliminate additional unnecessary rebuilds of files by having &SCons; use files from one or more code repositories to build files in your local build tree.

## The &Repository; Method

It's often useful to allow multiple programmers working on a project to build software from source files and/or derived files that are stored in a centrally-accessible repository, a directory copy of the source code tree. (Note that this is not the sort of repository maintained by a source code management system like BitKeeper, CVS, or Subversion.) You use the &Repository; method to tell &SCons; to search one or more central code repositories (in order) for any source files and derived files that are not present in the local build tree:

env = Environment() env.Program('hello.c') Repository('__ROOT__/usr/repository1', '__ROOT__/usr/repository2')
int main() { printf("Hello, world!\n"); }

Multiple calls to the &Repository; method will simply add repositories to the global list that &SCons; maintains, with the exception that &SCons; will automatically eliminate the current directory and any non-existent directories from the list.

## Finding source files in repositories

The above example specifies that &SCons; will first search for files under the `/usr/repository1` tree and next under the `/usr/repository2` tree. &SCons; expects that any files it searches for will be found in the same position relative to the top-level directory. In the above example, if the &hello_c; file is not found in the local build tree, &SCons; will search first for a `/usr/repository1/hello.c` file and then for a `/usr/repository2/hello.c` file to use in its place.

So given the &SConstruct; file above, if the &hello_c; file exists in the local build directory, &SCons; will rebuild the &hello; program as normal:

scons -Q

If, however, there is no local &hello_c; file, but one exists in `/usr/repository1`, &SCons; will recompile the &hello; program from the source file it finds in the repository:

env = Environment() env.Program('hello.c') Repository('__ROOT__/usr/repository1', '__ROOT__/usr/repository2')
int main() { printf("Hello, world!\n"); }
scons -Q

And similarly, if there is no local &hello_c; file and no `/usr/repository1/hello.c`, but one exists in `/usr/repository2`:

env = Environment() env.Program('hello.c') Repository('__ROOT__/usr/repository1', '__ROOT__/usr/repository2')
int main() { printf("Hello, world!\n"); }
scons -Q

The &f-link-Glob; function understands about repositories, and will use the same matching algorithm as described for explicitly-listed sources.

## Finding `#include` files in repositories

We've already seen that SCons will scan the contents of a source file for `#include` file names and realize that targets built from that source file also depend on the `#include` file(s). For each directory in the &cv-link-CPPPATH; list, &SCons; will actually search the corresponding directories in any repository trees and establish the correct dependencies on any `#include` files that it finds in repository directory.

Unless the C compiler also knows about these directories in the repository trees, though, it will be unable to find the `#include` files. If, for example, the &hello_c; file in our previous example includes the &hello_h; in its current directory, and the &hello_h; only exists in the repository:

    % scons -Q
    cc -o hello.o -c hello.c
    hello.c:1: hello.h: No such file or directory

In order to inform the C compiler about the repositories, &SCons; will add appropriate `-I` flags to the compilation commands for each directory in the &cv-CPPPATH; list. So if we add the current directory to the construction environment &cv-CPPPATH; like so:

env = Environment(CPPPATH = ['.']) env.Program('hello.c') Repository('__ROOT__/usr/repository1')
int main() { printf("Hello, world!\n"); }

Then re-executing &SCons; yields:

scons -Q

The order of the `-I` options replicates, for the C preprocessor, the same repository-directory search path that &SCons; uses for its own dependency analysis. If there are multiple repositories and multiple &cv-CPPPATH; directories, &SCons; will add the repository directories to the beginning of each &cv-CPPPATH; directory, rapidly multiplying the number of `-I` flags. If, for example, the &cv-CPPPATH; contains three directories (and shorter repository path names!):

env = Environment(CPPPATH = ['dir1', 'dir2', 'dir3']) env.Program('hello.c') Repository('__ROOT__/r1', '__ROOT__/r2')
int main() { printf("Hello, world!\n"); }

Then we'll end up with nine `-I` options on the command line, three (for each of the &cv-CPPPATH; directories) times three (for the local directory plus the two repositories):

scons -Q

### Limitations on `#include` files in repositories

&SCons; relies on the C compiler's `-I` options to control the order in which the preprocessor will search the repository directories for `#include` files. This causes a problem, however, with how the C preprocessor handles `#include` lines with the file name included in double-quotes.

As we've seen, &SCons; will compile the &hello_c; file from the repository if it doesn't exist in the local directory. If, however, the &hello_c; file in the repository contains a `#include` line with the file name in double quotes:

    #include "hello.h"
    int
    main(int argc, char *argv[])
    {
        printf(HELLO_MESSAGE);
        return (0);
    }

Then the C preprocessor will always use a &hello_h; file from the repository directory first, even if there is a &hello_h; file in the local directory, despite the fact that the command line specifies `-I` as the first option:

env = Environment(CPPPATH = ['.']) env.Program('hello.c') Repository('__ROOT__/usr/repository1')
int main() { printf("Hello, world!\n"); }
scons -Q

This behavior of the C preprocessor--always search for a `#include` file in double-quotes first in the same directory as the source file, and only then search the `-I`--can not, in general, be changed. In other words, it's a limitation that must be lived with if you want to use code repositories in this way. There are three ways you can possibly work around this C preprocessor behavior:

1. Some modern versions of C compilers do have an option to disable or control this behavior. If so, add that option to &cv-link-CFLAGS; (or &cv-link-CXXFLAGS; or both) in your construction environment(s). Make sure the option is used for all construction environments that use C preprocessing!

2. Change all occurrences of `#include "file.h"` to `#include <file.h>`. Use of `#include` with angle brackets does not have the same behavior--the `-I` directories are searched first for `#include` files--which gives &SCons; direct control over the list of directories the C preprocessor will search.

3. Require that everyone working with compilation from repositories check out and work on entire directories of files, not individual files. (If you use local wrapper scripts around your source code control system's command, you could add logic to enforce this restriction there.

## Finding the &SConstruct; file in repositories

&SCons; will also search in repositories for the &SConstruct; file and any specified &SConscript; files. This poses a problem, though: how can &SCons; search a repository tree for an &SConstruct; file if the &SConstruct; file itself contains the information about the pathname of the repository? To solve this problem, &SCons; allows you to specify repository directories on the command line using the `-Y` option:

    % scons -Q -Y /usr/repository1 -Y /usr/repository2

When looking for source or derived files, &SCons; will first search the repositories specified on the command line, and then search the repositories specified in the &SConstruct; or &SConscript; files.

## Finding derived files in repositories

If a repository contains not only source files, but also derived files (such as object files, libraries, or executables), &SCons; will perform its normal MD5 signature calculation to decide if a derived file in a repository is up-to-date, or the derived file must be rebuilt in the local build directory. For the &SCons; signature calculation to work correctly, a repository tree must contain the &sconsigndb; files that &SCons; uses to keep track of signature information.

Usually, this would be done by a build integrator who would run &SCons; in the repository to create all of its derived files and &sconsigndb; files, or who would run &SCons; in a separate build directory and copy the resulting tree to the desired repository:

env = Environment() env.Program(['hello.c', 'file1.c', 'file2.c']) Repository('/usr/repository1', '/usr/repository2')
int main() { printf("Hello, world!\n"); }
int f1() { printf("file1\n"); }
int f2() { printf("file2.c\n"); }
cd /usr/repository1
scons -Q

(Note that this is safe even if the &SConstruct; file lists `/usr/repository1` as a repository, because &SCons; will remove the current build directory from its repository list for that invocation.)

Now, with the repository populated, we only need to create the one local source file we're interested in working with at the moment, and use the `-Y` option to tell &SCons; to fetch any other files it needs from the repository:

    % cd $HOME/build
    % edit hello.c
    % scons -Q -Y /usr/repository1
    cc -c -o hello.o hello.c
    cc -o hello hello.o /usr/repository1/file1.o /usr/repository1/file2.o

Notice that &SCons; realizes that it does not need to rebuild local copies `file1.o` and `file2.o` files, but instead uses the already-compiled files from the repository.

## Guaranteeing local copies of files

If the repository tree contains the complete results of a build, and we try to build from the repository without any files in our local tree, something moderately surprising happens:

    % mkdir $HOME/build2
    % cd $HOME/build2
    % scons -Q -Y /usr/all/repository hello
    scons: `hello' is up-to-date.

Why does &SCons; say that the &hello; program is up-to-date when there is no &hello; program in the local build directory? Because the repository (not the local directory) contains the up-to-date &hello; program, and &SCons; correctly determines that nothing needs to be done to rebuild that up-to-date copy of the file.

There are, however, many times when you want to ensure that a local copy of a file always exists. A packaging or testing script, for example, may assume that certain generated files exist locally. To tell &SCons; to make a copy of any up-to-date repository file in the local build directory, use the &Local; function:

env = Environment() hello = env.Program('hello.c') Local(hello)
int main() { printf("Hello, world!\n"); }

If we then run the same command, &SCons; will make a local copy of the program from the repository copy, and tell you that it is doing so:

    % scons -Y /usr/all/repository hello
    Local copy of hello from /usr/all/repository/hello
    scons: `hello' is up-to-date.

(Notice that, because the act of making the local copy is not considered a "build" of the &hello; file, &SCons; still reports that it is up-to-date.)