scons_gd/scons/doc/user/hierarchy.xml

<?xml version='1.0'?>
<!DOCTYPE sconsdoc [
    <!ENTITY % scons SYSTEM "../scons.mod">
    %scons;

    <!ENTITY % builders-mod SYSTEM "../generated/builders.mod">
    %builders-mod;
    <!ENTITY % functions-mod SYSTEM "../generated/functions.mod">
    %functions-mod;
    <!ENTITY % tools-mod SYSTEM "../generated/tools.mod">
    %tools-mod;
    <!ENTITY % variables-mod SYSTEM "../generated/variables.mod">
    %variables-mod;

]>

<chapter id="chap-hierarchical"
         xmlns="http://www.scons.org/dbxsd/v1.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">
<title>Hierarchical Builds</title>

<!--

  __COPYRIGHT__

  Permission is hereby granted, free of charge, to any person obtaining
  a copy of this software and associated documentation files (the
  "Software"), to deal in the Software without restriction, including
  without limitation the rights to use, copy, modify, merge, publish,
  distribute, sublicense, and/or sell copies of the Software, and to
  permit persons to whom the Software is furnished to do so, subject to
  the following conditions:

  The above copyright notice and this permission notice shall be included
  in all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
  KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
  WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
  LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
  WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

-->

<!--


=head2 The Build command

By default, Cons does not change its working directory to the directory
containing a subsidiary F<Conscript> file it is including.  This behavior
can be enabled for a build by specifying, in the top-level F<Construct>
file:

  Conscript_chdir 1;

When enabled, Cons will change to the subsidiary F<Conscript> file's
containing directory while reading in that file, and then change back
to the top-level directory once the file has been processed.

It is expected that this behavior will become the default in some future
version of Cons.  To prepare for this transition, builds that expect
Cons to remain at the top of the build while it reads in a subsidiary
F<Conscript> file should explicitly disable this feature as follows:

  Conscript_chdir 0;

=head2 Relative, top-relative, and absolute file names

(There is another file prefix, ``!'', that is interpreted specially by
Cons.  See discussion of the C<Link> command, below, for details.)


=head2 Using modules in build scripts

You may pull modules into each F<Conscript> file using the normal Perl
C<use> or C<require> statements:

  use English;
  require My::Module;

Each C<use> or C<require> only affects the one F<Conscript> file in which
it appears.  To use a module in multiple F<Conscript> files, you must
put a C<use> or C<require> statement in each one that needs the module.


=head2 Scope of variables

The top-level F<Construct> file and all F<Conscript> files begin life in
a common, separate Perl package.  B<Cons> controls the symbol table for
the package so that, the symbol table for each script is empty, except
for the F<Construct> file, which gets some of the command line arguments.
All of the variables that are set or used, therefore, are set by the
script itself, not by some external script.

Variables can be explicitly B<imported> by a script from its parent
script. To import a variable, it must have been B<exported> by the parent
and initialized (otherwise an error will occur).


=head2 The Export command

The C<Export> command is used as in the following example:

  $env = new cons();
  $INCLUDE = "#export/include";
  $LIB = "#export/lib";
  Export qw( env INCLUDE LIB );
  Build qw( util/Conscript );

The values of the simple variables mentioned in the C<Export> list will be
squirreled away by any subsequent C<Build> commands. The C<Export> command
will only export Perl B<scalar> variables, that is, variables whose name
begins with C<$>. Other variables, objects, etc. can be exported by
reference, but all scripts will refer to the same object, and this object
should be considered to be read-only by the subsidiary scripts and by the
original exporting script. It's acceptable, however, to assign a new value
to the exported scalar variable, that won't change the underlying variable
referenced. This sequence, for example, is OK:

  $env = new cons();
  Export qw( env INCLUDE LIB );
  Build qw( util/Conscript );
  $env = new cons(CFLAGS => '-O');
  Build qw( other/Conscript );

It doesn't matter whether the variable is set before or after the C<Export>
command. The important thing is the value of the variable at the time the
C<Build> command is executed. This is what gets squirreled away. Any
subsequent C<Export> commands, by the way, invalidate the first: you must
mention all the variables you wish to export on each C<Export> command.


=head2 The Import command

Variables exported by the C<Export> command can be imported into subsidiary
scripts by the C<Import> command. The subsidiary script always imports
variables directly from the superior script. Consider this example:

  Import qw( env INCLUDE );

This is only legal if the parent script exported both C<$env> and
C<$INCLUDE>. It also must have given each of these variables values. It is
OK for the subsidiary script to only import a subset of the exported
variables (in this example, C<$LIB>, which was exported by the previous
example, is not imported).

All the imported variables are automatically re-exported, so the sequence:

  Import qw ( env INCLUDE );
  Build qw ( beneath-me/Conscript );

will supply both C<$env> and C<$INCLUDE> to the subsidiary file. If only
C<$env> is to be exported, then the following will suffice:

  Import qw ( env INCLUDE );
  Export qw ( env );
  Build qw ( beneath-me/Conscript );

Needless to say, the variables may be modified locally before invoking
C<Build> on the subsidiary script.

=head2 Build script evaluation order

The only constraint on the ordering of build scripts is that superior
scripts are evaluated before their inferior scripts. The top-level
F<Construct> file, for instance, is evaluated first, followed by any
inferior scripts. This is all you really need to know about the evaluation
order, since order is generally irrelevant. Consider the following C<Build>
command:

  Build qw(
	drivers/display/Conscript
	drivers/mouse/Conscript
	parser/Conscript
	utilities/Conscript
  );

We've chosen to put the script names in alphabetical order, simply because
that's the most convenient for maintenance purposes. Changing the order will
make no difference to the build.

-->

  <para>

  The source code for large software projects
  rarely stays in a single directory,
  but is nearly always divided into a
  hierarchy of directories.
  Organizing a large software build using &SCons;
  involves creating a hierarchy of build scripts
  using the &SConscript; function.

  </para>

  <section>
  <title>&SConscript; Files</title>

    <para>

    As we've already seen,
    the build script at the top of the tree is called &SConstruct;.
    The top-level &SConstruct; file can
    use the &SConscript; function to
    include other subsidiary scripts in the build.
    These subsidiary scripts can, in turn,
    use the &SConscript; function
    to include still other scripts in the build.
    By convention, these subsidiary scripts are usually
    named &SConscript;.
    For example, a top-level &SConstruct; file might
    arrange for four subsidiary scripts to be included
    in the build as follows:

    </para>

    <sconstruct>
SConscript(['drivers/display/SConscript',
            'drivers/mouse/SConscript',
            'parser/SConscript',
            'utilities/SConscript'])
    </sconstruct>

    <para>

    In this case, the &SConstruct; file
    lists all of the &SConscript; files in the build explicitly.
    (Note, however, that not every directory in the tree
    necessarily has an &SConscript; file.)
    Alternatively, the <literal>drivers</literal>
    subdirectory might contain an intermediate
    &SConscript; file,
    in which case the &SConscript; call in
    the top-level &SConstruct; file
    would look like:

    </para>

    <sconstruct>
SConscript(['drivers/SConscript',
            'parser/SConscript',
            'utilities/SConscript'])
    </sconstruct>

    <para>

    And the subsidiary &SConscript; file in the
    <literal>drivers</literal> subdirectory
    would look like:

    </para>

    <sconstruct>
SConscript(['display/SConscript',
            'mouse/SConscript'])
    </sconstruct>

    <para>

    Whether you list all of the &SConscript; files in the
    top-level &SConstruct; file,
    or place a subsidiary &SConscript; file in
    intervening directories,
    or use some mix of the two schemes,
    is up to you and the needs of your software.

    </para>

  </section>

  <section>
  <title>Path Names Are Relative to the &SConscript; Directory</title>

    <para>

    Subsidiary &SConscript; files make it easy to create a build
    hierarchy because all of the file and directory names
    in a subsidiary &SConscript; files are interpreted
    relative to the directory in which the &SConscript; file lives.
    Typically, this allows the &SConscript; file containing the
    instructions to build a target file
    to live in the same directory as the source files
    from which the target will be built,
    making it easy to update how the software is built
    whenever files are added or deleted
    (or other changes are made).

    </para>

    <para>

    For example, suppose we want to build two programs
    &prog1; and &prog2; in two separate directories
    with the same names as the programs.
    One typical way to do this would be
    with a top-level &SConstruct; file like this:

    </para>

    <scons_example name="hierarchy_ex1">
      <file name="SConstruct" printme="1">
SConscript(['prog1/SConscript', 'prog2/SConscript'])
      </file>
      <file name="prog1/SConscript">
env = Environment()
env.Program('prog1', ['main.c', 'foo1.c', 'foo2.c'])
      </file>
      <file name="prog2/SConscript">
env = Environment()
env.Program('prog2', ['main.c', 'bar1.c', 'bar2.c'])
      </file>
      <directory name="prog1"></directory>
      <file name="prog1/main.c">
x
      </file>
      <file name="prog1/foo1.c">
x
      </file>
      <file name="prog1/foo2.c">
x
      </file>
      <directory name="prog2"></directory>
      <file name="prog2/main.c">
x
      </file>
      <file name="prog2/bar1.c">
x
      </file>
      <file name="prog2/bar2.c">
x
      </file>
    </scons_example>

    <para>

    And subsidiary &SConscript; files that look like this:

    </para>

    <scons_example_file example="hierarchy_ex1" name="prog1/SConscript">
    </scons_example_file>

    <para>

    And this:

    </para>

    <scons_example_file example="hierarchy_ex1" name="prog2/SConscript">
    </scons_example_file>

    <para>

    Then, when we run &SCons; in the top-level directory,
    our build looks like:

    </para>

    <scons_output example="hierarchy_ex1" suffix="1">
       <scons_output_command>scons -Q</scons_output_command>
    </scons_output>

    <para>

    Notice the following:

    First, you can have files with the same names
    in multiple directories, like main.c in the above example.

    Second, unlike standard recursive use of &Make;,
    &SCons; stays in the top-level directory
    (where the &SConstruct; file lives)
    and issues commands that use the path names
    from the top-level directory to the
    target and source files within the hierarchy.

    </para>

  </section>

  <section>
  <title>Top-Relative Path Names in Subsidiary &SConscript; Files</title>

    <para>

    If you need to use a file from another directory,
    it's sometimes more convenient to specify
    the path to a file in another directory
    from the top-level &SConstruct; directory,
    even when you're using that file in
    a subsidiary &SConscript; file in a subdirectory.
    You can tell &SCons; to interpret a path name
    as relative to the top-level &SConstruct; directory,
    not the local directory of the &SConscript; file,
    by prepending a &hash; (hash mark) in front of the path name:

    </para>

    <scons_example name="hierarchy_ex2">
       <file name="SConstruct">
SConscript('src/prog/SConscript')
       </file>
       <file name="src/prog/SConscript" printme="1">
env = Environment()
env.Program('prog', ['main.c', '#lib/foo1.c', 'foo2.c'])
       </file>
       <file name="src/prog/main.c">
x
       </file>
       <file name="lib/foo1.c">
x
       </file>
       <file name="src/prog/foo2.c">
x
       </file>
    </scons_example>

    <para>

    In this example,
    the <literal>lib</literal> directory is
    directly underneath the top-level &SConstruct; directory.
    If the above &SConscript; file is in a subdirectory
    named <literal>src/prog</literal>,
    the output would look like:

    </para>

    <scons_output example="hierarchy_ex2" suffix="1">
       <scons_output_command>scons -Q</scons_output_command>
    </scons_output>

    <para>

    (Notice that the <literal>lib/foo1.o</literal> object file
    is built in the same directory as its source file.
    See <xref linkend="chap-separate"></xref>, below,
    for information about
    how to build the object file in a different subdirectory.)

    </para>

    <para>

    A couple of notes on top-relative paths:

      <orderedlist>
        <listitem><para>

        &SCons; doesn't care whether you add a slash after the &hash;.
        Some people consider <literal>'#/lib/foo1.c'</literal>
        more readable than <literal>'#lib/foo1.c'</literal>,
        but they're functionally equivalent.

        </para></listitem>

        <listitem><para>

        The top-relative syntax is <emphasis>only</emphasis>
        evaluated by &SCons;, the &Python; language itself does not
        understand about it.  This becomes immediately obvious
        if you like to use <function>print</function> for debugging,
        or write a Python function that wants to evaluate a path.
        You can force &SCons; to evaluate a top-relative path by
        creating a Node object from it:

        </para>

        </listitem>
      </orderedlist>

      <scons_example name="hierarchy_hash">
        <file name="SConstruct" printme="1">
path = "#/include"

print("path =", path)
print("force-interpreted path =", Entry(path))
        </file>
      </scons_example>

      Which shows:

      <scons_output example="hierarchy_hash" suffix="1">
         <scons_output_command>scons -Q</scons_output_command>
      </scons_output>

    </para>

  </section>

  <section>
  <title>Absolute Path Names</title>

    <para>

    Of course, you can always specify
    an absolute path name for a file--for example:

    </para>

    <scons_example name="hierarchy_ex3">
       <file name="SConstruct">
SConscript('src/prog/SConscript')
       </file>
       <file name="src/prog/SConscript" printme="1">
env = Environment()
env.Program('prog', ['main.c', '__ROOT__/usr/joe/lib/foo1.c', 'foo2.c'])
       </file>
       <file name="src/prog/main.c">
x
       </file>
       <file name="__ROOT__/usr/joe/lib/foo1.c">
x
       </file>
       <file name="src/prog/foo2.c">
x
       </file>
    </scons_example>

    <para>

    Which, when executed, would yield:

    </para>

    <scons_output example="hierarchy_ex3" suffix="1">
       <scons_output_command>scons -Q</scons_output_command>
    </scons_output>

    <para>

    (As was the case with top-relative path names,
    notice that the <literal>/usr/joe/lib/foo1.o</literal> object file
    is built in the same directory as its source file.
    See <xref linkend="chap-separate"></xref>, below,
    for information about
    how to build the object file in a different subdirectory.)

    </para>

  </section>

  <section>
  <title>Sharing Environments (and Other Variables) Between &SConscript; Files</title>

    <para>

    In the previous example,
    each of the subsidiary &SConscript; files
    created its own construction environment
    by calling &f-link-Environment; separately.
    This obviously works fine,
    but if each program must be built
    with the same construction variables,
    it's cumbersome and error-prone to initialize
    separate construction environments
    in the same way over and over in each subsidiary
    &SConscript; file.

    </para>

    <para>

    &SCons; supports the ability to <firstterm>export</firstterm> variables
    from an &SConscript; file
    so they can be <firstterm>imported</firstterm> by other
    &SConscript; files, thus allowing you to share common initialized
    values throughout your build hierarchy.

    </para>

    <section>
    <title>Exporting Variables</title>

      <para>

      There are two ways to export a variable
      from an &SConscript; file.
      The first way is to call the &f-link-Export; function.
      &Export; is pretty flexible - in the simplest form,
      you pass it a string that represents the name of
      the variable, and &Export; stores that with its value:

      </para>

      <sconstruct>
env = Environment()
Export('env')
      </sconstruct>

      <para>

      You may export more than one variable name at a time:

      </para>

      <sconstruct>
env = Environment()
debug = ARGUMENTS['debug']
Export('env', 'debug')
      </sconstruct>

      <para>

      Because a Python identifier cannot contain spaces,
      &Export; assumes a string containing spaces is is a
      shortcut for multiple variable names to export and
      splits it up for you:

      </para>

      <sconstruct>
env = Environment()
debug = ARGUMENTS['debug']
Export('env debug')
      </sconstruct>

      <para>

      You can also pass &Export; a dictionary of values.
      This form allows the opportunity to export a variable
      from the current scope under a different name -
      in this example, the value of <varname>foo</varname>
      is exported under the name <literal>"bar"</literal>:

      <sconstruct>
env = Environment()
foo = "FOO"
args = {"env": env, "bar": foo}
Export(args)
      </sconstruct>

      </para>

      <para>

      &Export; will also accept arguments in keyword style.
      This form adds the ability to create exported variables
      that have not actually been set locally in the SConscript file.
      When used this way, the key is the intended variable name,
      not a string representation as with the other forms:

      </para>

      <sconstruct>
Export(MODE="DEBUG", TARGET="arm")
      </sconstruct>

      <para>

      The styles can be mixed, though Python function calling
      syntax requires all non-keyword arguments to precede any
      keyword arguments in the call.

      </para>

      <para>

      The &Export; function adds the variables to a global
      location from which other &SConscript; files can import.
      Calls to &Export; are cumulative. When you call &Export;
      you are actually updating a Python dictionary, so it
      is fine to export a variable you have already exported,
      but when doing so, the previous value is lost.

      </para>

      <para>

      The other way to export is you can specify a list of
      variables as a second argument
      to the &f-link-SConscript; function call:

      </para>

      <sconstruct>
SConscript('src/SConscript', 'env')
      </sconstruct>

      <para>

      Or (preferably, for readability) using the &exports; keyword argument:

      </para>

      <sconstruct>
SConscript('src/SConscript', exports='env')
      </sconstruct>

      <para>

      These calls export the specified variables
      to only the listed &SConscript; file(s).
      You may specify more than one
      &SConscript; file in a list:

      </para>

      <sconstruct>
SConscript(['src1/SConscript',
            'src2/SConscript'], exports='env')
      </sconstruct>

      <para>

      This is functionally equivalent to
      calling the &SConscript; function
      multiple times with the same &exports; argument,
      one per &SConscript; file.

      </para>

    </section>

    <section>
    <title>Importing Variables</title>

      <para>

      Once a variable has been exported from a calling
      &SConscript; file,
      it may be used in other &SConscript; files
      by calling the &f-link-Import; function:

      </para>

      <sconstruct>
Import('env')
env.Program('prog', ['prog.c'])
      </sconstruct>

      <para>

      The &Import; call makes the previously defined <varname>env</varname>
      variable available to the &SConscript; file.
      Assuming <varname>env</varname> is a &consenv;,
      after import it can be used to build programs, libraries, etc.
      The use case of passing around a &consenv; is extremely common
      in larger &scons; builds.

      </para>

      <para>

      Like the &Export; function,
      the &Import; function can be called
      with multiple variable names:

      </para>

      <sconstruct>
Import('env', 'debug')
env = env.Clone(DEBUG=debug)
env.Program('prog', ['prog.c'])
      </sconstruct>

      <para>

      In this example, we pull in the common &consenv;
      <varname>env</varname>, and
      use the value of the <varname>debug</varname>
      variable to make a modified copy by passing
      that to a &f-link-Clone; call.

      </para>

      <para>

      The &Import; function will (like &Export;)
      split a string containing white-space
      into separate variable names:

      </para>

      <sconstruct>
Import('env debug')
env = env.Clone(DEBUG=debug)
env.Program('prog', ['prog.c'])
      </sconstruct>

      <para>

      &Import; prefers a local definition to a global one,
      so that if there is a global export of <varname>foo</varname>,
      <emphasis>and</emphasis> the calling SConscript has
      exported <varname>foo</varname> to this SConscript,
      the import will find the <varname>foo</varname>
      exported to this SConscript.

      </para>

      <para>

      Lastly, as a special case,
      you may import all of the variables that
      have been exported by supplying an asterisk
      to the &Import; function:

      </para>

      <sconstruct>
Import('*')
env = env.Clone(DEBUG=debug)
env.Program('prog', ['prog.c'])
      </sconstruct>

      <para>

      If you're dealing with a lot of &SConscript; files,
      this can be a lot simpler than keeping
      arbitrary lists of imported variables up to date in each file.

      </para>

    </section>

    <section>
    <title>Returning Values From an &SConscript; File</title>

      <para>

      Sometimes, you would like to be able to
      use information from a subsidiary
      &SConscript; file in some way.
      For example,
      suppose that you want to create one
      library from object files built by
      several subsidiary &SConscript; files.
      You can do this by using the &f-link-Return;
      function to return values
      from the subsidiary &SConscript; files
      to the calling file. Like &Import; and &Export;,
      &Return; takes a string representation of the variable
      name, not the variable name itself.

      </para>

      <para>

      If, for example, we have two subdirectories
      &foo; and &bar;
      that should each contribute an object
      file to a library,
      what we'd like to be able to do is
      collect the object files
      from the subsidiary &SConscript; calls
      like this:

      </para>

      <scons_example name="hierarchy_Return">
        <file name="SConstruct" printme="1">
env = Environment()
Export('env')
objs = []
for subdir in ['foo', 'bar']:
    o = SConscript('%s/SConscript' % subdir)
    objs.append(o)
env.Library('prog', objs)
        </file>
        <directory name="foo"></directory>
        <directory name="bar"></directory>
        <file name="foo/SConscript">
Import('env')
obj = env.Object('foo.c')
Return('obj')
        </file>
        <file name="bar/SConscript">
Import('env')
obj = env.Object('bar.c')
Return('obj')
        </file>
        <file name="foo/foo.c">
void foo(void) { printf("foo/foo.c\n"); }
        </file>
        <file name="bar/bar.c">
void bar(void) { printf("bar/bar.c\n"); }
        </file>
      </scons_example>

      <para>

      We can do this by using the &Return;
      function in the
      <filename>foo/SConscript</filename> file like this:

      </para>

      <scons_example_file example="hierarchy_Return" name="foo/SConscript">
      </scons_example_file>

      <para>

      (The corresponding
      <filename>bar/SConscript</filename>
      file should be pretty obvious.)
      Then when we run &SCons;,
      the object files from the subsidiary subdirectories
      are all correctly archived in the desired library:

      </para>

      <scons_output example="hierarchy_Return" suffix="1">
        <scons_output_command>scons -Q</scons_output_command>
      </scons_output>

      <!--
      XXX Return(stop=False)
      -->

    </section>

  </section>

  <!--

  <section>
  <title>Executing From a Subdirectory:  the -D, -u and -U Options</title>

    <para>

    XXX -D, -u and -U

    </para>

  </section>

  -->

</chapter>