scons_gd/scons/doc/user/troubleshoot.xml
2022-10-15 16:06:26 +02:00

992 lines
27 KiB
XML

<?xml version='1.0'?>
<!DOCTYPE sconsdoc [
<!ENTITY % scons SYSTEM "../scons.mod">
%scons;
<!ENTITY % builders-mod SYSTEM "../generated/builders.mod">
%builders-mod;
<!ENTITY % functions-mod SYSTEM "../generated/functions.mod">
%functions-mod;
<!ENTITY % tools-mod SYSTEM "../generated/tools.mod">
%tools-mod;
<!ENTITY % variables-mod SYSTEM "../generated/variables.mod">
%variables-mod;
]>
<chapter id="chap-troubleshooting"
xmlns="http://www.scons.org/dbxsd/v1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">
<title>Troubleshooting</title>
<!--
__COPYRIGHT__
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-->
<para>
The experience of configuring any
software build tool to build a large code base
usually, at some point,
involves trying to figure out why
the tool is behaving a certain way,
and how to get it to behave the way you want.
&SCons; is no different.
This appendix contains a number of
different ways in which you can
get some additional insight into &SCons;' behavior.
</para>
<para>
Note that we're always interested in trying to
improve how you can troubleshoot configuration problems.
If you run into a problem that has
you scratching your head,
and which there just doesn't seem to be a good way to debug,
odds are pretty good that someone else will run into
the same problem, too.
If so, please let the SCons development team know
using the contact information at
<ulink url="https://scons.org/contact.html"/>
so that we can use your feedback
to try to come up with a better way to help you,
and others, get the necessary insight into &SCons; behavior
to help identify and fix configuration issues.
</para>
<section>
<title>Why is That Target Being Rebuilt? the &debug-explain; Option</title>
<para>
Let's look at a simple example of
a misconfigured build
that causes a target to be rebuilt
every time &SCons; is run:
</para>
<scons_example name="troubleshoot_explain1">
<file name="SConstruct" printme="1">
# Intentionally misspell the output file name in the
# command used to create the file:
Command('file.out', 'file.in', 'cp $SOURCE file.oout')
</file>
<file name="file.in">
file.in
</file>
</scons_example>
<para>
(Note to Windows users: The POSIX &cp; command
copies the first file named on the command line
to the second file.
In our example, it copies the &file_in; file
to the &file_out; file.)
</para>
<para>
Now if we run &SCons; multiple times on this example,
we see that it re-runs the &cp;
command every time:
</para>
<scons_output example="troubleshoot_explain1" os="posix" suffix="1">
<scons_output_command>scons -Q</scons_output_command>
<scons_output_command>scons -Q</scons_output_command>
<scons_output_command>scons -Q</scons_output_command>
</scons_output>
<para>
In this example,
the underlying cause is obvious:
we've intentionally misspelled the output file name
in the &cp; command,
so the command doesn't actually
build the &file_out; file that we've told &SCons; to expect.
But if the problem weren't obvious,
it would be helpful
to specify the &debug-explain; option
on the command line
to have &SCons; tell us very specifically
why it's decided to rebuild the target:
</para>
<scons_output example="troubleshoot_explain1" os="posix" suffix="2">
<scons_output_command>scons -Q --debug=explain</scons_output_command>
</scons_output>
<para>
If this had been a more complicated example
involving a lot of build output,
having &SCons; tell us that
it's trying to rebuild the target file
because it doesn't exist
would be an important clue
that something was wrong with
the command that we invoked to build it.
</para>
<para>
Note that you can also use
<option>--warn=target-not-built</option> which checks
whether or not expected targets exist after a build rule is
executed.
</para>
<scons_output example="troubleshoot_explain1" os="posix" suffix="3">
<scons_output_command>scons -Q --warn=target-not-built</scons_output_command>
</scons_output>
<para>
The &debug-explain; option also comes in handy
to help figure out what input file changed.
Given a simple configuration that builds
a program from three source files,
changing one of the source files
and rebuilding with the &debug-explain;
option shows very specifically
why &SCons; rebuilds the files that it does:
</para>
<scons_example name="troubleshoot_explain2">
<file name="SConstruct">
Program('prog', ['file1.c', 'file2.c', 'file3.c'])
</file>
<file name="file1.c">
file1.c
</file>
<file name="file2.c">
file2.c
</file>
<file name="file3.c">
file3.c
</file>
</scons_example>
<scons_output example="troubleshoot_explain2" os="posix" suffix="1">
<scons_output_command>scons -Q</scons_output_command>
<scons_output_command output=" [CHANGE THE CONTENTS OF file2.c]">edit file2.c</scons_output_command>
<scons_output_command>scons -Q --debug=explain</scons_output_command>
</scons_output>
<para>
This becomes even more helpful
in identifying when a file is rebuilt
due to a change in an implicit dependency,
such as an incuded <filename>.h</filename> file.
If the <filename>file1.c</filename>
and <filename>file3.c</filename> files
in our example
both included a &hello_h; file,
then changing that included file
and re-running &SCons; with the &debug-explain; option
will pinpoint that it's the change to the included file
that starts the chain of rebuilds:
</para>
<scons_example name="troubleshoot_explain3">
<file name="SConstruct">
Program('prog', ['file1.c', 'file2.c', 'file3.c'], CPPPATH='.')
</file>
<file name="file1.c">
#include &lt;hello.h&gt;
file1.c
</file>
<file name="file2.c">
file2.c
</file>
<file name="file3.c">
#include &lt;hello.h&gt;
file3.c
</file>
<file name="hello.h">
#define string "world"
</file>
</scons_example>
<scons_output example="troubleshoot_explain3" os="posix" suffix="1">
<scons_output_command>scons -Q</scons_output_command>
<scons_output_command output=" [CHANGE THE CONTENTS OF hello.h]">edit hello.h</scons_output_command>
<scons_output_command>scons -Q --debug=explain</scons_output_command>
</scons_output>
<para>
(Note that the &debug-explain; option will only tell you
why &SCons; decided to rebuild necessary targets.
It does not tell you what files it examined
when deciding <emphasis>not</emphasis>
to rebuild a target file,
which is often a more valuable question to answer.)
</para>
</section>
<section>
<title>What's in That Construction Environment? the &f-Dump; Method</title>
<para>
When you create a &consenv;,
&SCons; populates it
with &consvars; that are set up
for various compilers, linkers and utilities
that it finds on your system.
Although this is usually helpful and what you want,
it might be frustrating if &SCons;
doesn't set certain variables that you
expect to be set.
In situations like this,
it's sometimes helpful to use the
&consenv; &f-link-Dump; method
to print all or some of
the &consvars;.
Note that the &f-Dump; method
<emphasis>returns</emphasis>
the representation of the variables
in the environment
for you to print (or otherwise manipulate):
</para>
<scons_example name="troubleshoot_Dump">
<file name="SConstruct" printme="1">
env = Environment()
print(env.Dump())
</file>
</scons_example>
<para>
On a POSIX system with gcc installed,
this might generate:
</para>
<scons_output example="troubleshoot_Dump" os="posix" tools="gcc" suffix="1">
<scons_output_command>scons</scons_output_command>
</scons_output>
<para>
On a Windows system with Visual C++
the output might look like:
</para>
<scons_output example="troubleshoot_Dump" os="win32" tools="msvc" suffix="2">
<scons_output_command>scons</scons_output_command>
</scons_output>
<para>
The &consenvs; in these examples have
actually been restricted to just gcc and Visual C++,
respectively.
In a real-life situation,
the &consenvs; will
likely contain a great many more variables.
Also note that we've massaged the example output above
to make the memory address of all objects a constant 0x700000.
In reality, you would see a different hexadecimal
number for each object.
</para>
<para>
To make it easier to see just what you're
interested in,
the &Dump; method allows you to
specify a specific &consvar;
that you want to disply.
For example,
it's not unusual to want to verify
the external environment used to execute build commands,
to make sure that the PATH and other
environment variables are set up the way they should be.
You can do this as follows:
</para>
<scons_example name="troubleshoot_Dump_ENV">
<file name="SConstruct" printme="1">
env = Environment()
print(env.Dump('ENV'))
</file>
</scons_example>
<para>
Which might display the following when executed on a POSIX system:
</para>
<scons_output example="troubleshoot_Dump_ENV" os="posix" suffix="1">
<scons_output_command>scons</scons_output_command>
</scons_output>
<para>
And the following when executed on a Windows system:
</para>
<scons_output example="troubleshoot_Dump_ENV" os="win32" suffix="2">
<scons_output_command>scons</scons_output_command>
</scons_output>
</section>
<section>
<title>What Dependencies Does &SCons; Know About? the &tree; Option</title>
<para>
Sometimes the best way to try to figure out what
&SCons; is doing is simply to take a look at the
dependency graph that it constructs
based on your &SConscript; files.
The <option>--tree</option> option
will display all or part of the
&SCons; dependency graph in an
"ASCII art" graphical format
that shows the dependency hierarchy.
</para>
<para>
For example, given the following input &SConstruct; file:
</para>
<scons_example name="troubleshoot_tree1">
<file name="SConstruct" printme="1">
env = Environment(CPPPATH = ['.'])
env.Program('prog', ['f1.c', 'f2.c', 'f3.c'])
</file>
<file name="f1.c">
#include "inc.h"
</file>
<file name="f2.c">
#include "inc.h"
</file>
<file name="f3.c">
#include "inc.h"
</file>
<file name="inc.h">
inc.h
</file>
</scons_example>
<para>
Running &SCons; with the <option>--tree=all</option>
option yields:
</para>
<scons_output example="troubleshoot_tree1" suffix="1">
<scons_output_command>scons -Q --tree=all</scons_output_command>
</scons_output>
<para>
The tree will also be printed when the
<option>-n</option> (no execute) option is used,
which allows you to examine the dependency graph
for a configuration without actually
rebuilding anything in the tree.
</para>
<para>
By default &SCons; uses "ASCII art" to draw the tree. It is
possible to use line-drawing characters (Unicode calls these
Box Drawing) to make a nicer display. To do this, add the
<option>linedraw</option> qualifier:
</para>
<scons_output example="troubleshoot_tree1" suffix="2">
<scons_output_command>scons -Q --tree=all,linedraw</scons_output_command>
</scons_output>
<para>
The <option>--tree</option> option only prints
the dependency graph for the specified targets
(or the default target(s) if none are specified on the command line).
So if you specify a target like <filename>f2.o</filename>
on the command line,
the <option>--tree</option> option will only
print the dependency graph for that file:
</para>
<scons_output example="troubleshoot_tree1" suffix="3">
<scons_output_command>scons -Q --tree=all f2.o</scons_output_command>
</scons_output>
<para>
This is, of course, useful for
restricting the output from a very large
build configuration to just a
portion in which you're interested.
Multiple targets are fine,
in which case a tree will be printed
for each specified target:
</para>
<scons_output example="troubleshoot_tree1" suffix="4">
<scons_output_command>scons -Q --tree=all f1.o f3.o</scons_output_command>
</scons_output>
<para>
The <parameter class="option">status</parameter> argument may be used
to tell &SCons; to print status information about
each file in the dependency graph:
</para>
<scons_output example="troubleshoot_tree1" suffix="5">
<scons_output_command>scons -Q --tree=status</scons_output_command>
</scons_output>
<para>
Note that <option>--tree=all,status</option> is equivalent;
the <parameter class="option">all</parameter>
is assumed if only <parameter class="option">status</parameter> is present.
As an alternative to <parameter class="option">all</parameter>,
you can specify <option>--tree=derived</option>
to have &SCons; only print derived targets
in the tree output,
skipping source files
(like <filename>.c</filename> and <filename>.h</filename> files):
</para>
<scons_output example="troubleshoot_tree1" suffix="6">
<scons_output_command>scons -Q --tree=derived</scons_output_command>
</scons_output>
<para>
You can use the <parameter class="option">status</parameter>
modifier with <parameter class="option">derived</parameter> as well:
</para>
<scons_output example="troubleshoot_tree1" suffix="7">
<scons_output_command>scons -Q --tree=derived,status</scons_output_command>
</scons_output>
<para>
Note that the order of the <option>--tree=</option>
arguments doesn't matter;
<option>--tree=status,derived</option> is
completely equivalent.
</para>
<para>
The default behavior of the <option>--tree</option> option
is to repeat all of the dependencies each time the library dependency
(or any other dependency file) is encountered in the tree.
If certain target files share other target files,
such as two programs that use the same library:
</para>
<scons_example name="troubleshoot_tree2">
<file name="SConstruct" printme="1">
env = Environment(CPPPATH = ['.'],
LIBS = ['foo'],
LIBPATH = ['.'])
env.Library('foo', ['f1.c', 'f2.c', 'f3.c'])
env.Program('prog1.c')
env.Program('prog2.c')
</file>
<file name="prog1.c">
#include "inc.h"
</file>
<file name="prog2.c">
#include "inc.h"
</file>
<file name="f1.c">
#include "inc.h"
</file>
<file name="f2.c">
#include "inc.h"
</file>
<file name="f3.c">
#include "inc.h"
</file>
<file name="inc.h">
inc.h
</file>
</scons_example>
<para>
Then there can be a <emphasis>lot</emphasis> of repetition in the
<option>--tree=</option> output:
</para>
<scons_output example="troubleshoot_tree2" suffix="1">
<scons_output_command>scons -Q --tree=all</scons_output_command>
</scons_output>
<para>
In a large configuration with many internal libraries
and include files,
this can very quickly lead to huge output trees.
To help make this more manageable,
a <parameter class="option">prune</parameter> modifier may
be added to the option list,
in which case &SCons;
will print the name of a target that has
already been visited during the tree-printing
in square brackets (<literal>[]</literal>)
as an indication that the dependencies
of the target file may be found
by looking farther up the tree:
</para>
<scons_output example="troubleshoot_tree2" suffix="2">
<scons_output_command>scons -Q --tree=prune</scons_output_command>
</scons_output>
<para>
Like the <parameter class="option">status</parameter> keyword,
the <parameter class="option">prune</parameter> argument by itself
is equivalent to <option>--tree=all,prune</option>.
</para>
</section>
<section>
<title>How is &SCons; Constructing the Command Lines It Executes? the &debug-presub; Option</title>
<para>
Sometimes the command lines that &SCons; executes don't
come out looking as you expect. In this case it may be
useful to look at the strings before &SCons; performs
substitution on them.
This can be done with the &debug-presub; option:
</para>
<scons_example name="troubleshoot_presub">
<file name="SConstruct">
env = Environment(CPPPATH = ['.'])
env.Program('prog', 'prog.c')
</file>
<file name="prog.c">
prog.c
</file>
</scons_example>
<!--
Have to capture output here, otherwise the - -debug=presub output
shows the Python functions from the sconsdoc.py execution wrapper
used to generate this manual, not the underlying command-line strings.
<scons_output example="troubleshoot_presub" suffix="1">
<scons_output_command>scons -Q - -debug=presub</scons_output_command>
</scons_output>
-->
<screen>
% <userinput>scons -Q --debug=presub</userinput>
Building prog.o with action:
$CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCOMCOM $SOURCES
cc -o prog.o -c -I. prog.c
Building prog with action:
$SMART_LINKCOM
cc -o prog prog.o
</screen>
</section>
<section>
<title>Where is &SCons; Searching for Libraries? the &debug-findlibs; Option</title>
<para>
To get some insight into what library names
&SCons; is searching for,
and in which directories it is searching,
Use the &debug-findlibs; option.
Given the following input &SConstruct; file:
</para>
<scons_example name="troubleshoot_findlibs">
<file name="SConstruct" printme="1">
env = Environment(LIBPATH = ['libs1', 'libs2'])
env.Program('prog.c', LIBS=['foo', 'bar'])
</file>
<file name="prog.c">
prog.c
</file>
<file name="libs1/libfoo.a">
libs1/libfoo.a
</file>
<file name="libs2/libbar.a">
libs2/libbar.a
</file>
</scons_example>
<para>
And the libraries <filename>libfoo.a</filename>
and <filename>libbar.a</filename>
in <filename>libs1</filename> and <filename>libs2</filename>,
respectively,
use of the &debug-findlibs; option yields:
</para>
<scons_output example="troubleshoot_findlibs" suffix="1">
<scons_output_command>scons -Q --debug=findlibs</scons_output_command>
</scons_output>
</section>
<!--
<section>
<title>What Implicit Dependencies Did the &SCons; Scanner find? the &debug-includes; Option</title>
<para>
XXX explain the - - debug=includes option
</para>
<scons_example name="troubleshoot_includes">
<file name="SConstruct" printme="1">
env = Environment(CPPPATH = ['inc1', 'inc2'])
env.Program('prog.c')
</file>
<file name="prog.c">
#include "file1.h"
#include "file2.h"
prog.c
</file>
<file name="inc1/file1.h">
inc1/file1.h
</file>
<file name="inc2/file2.h">
inc2/file2.h
</file>
</scons_example>
<scons_output example="troubleshoot_includes" suffix="1">
<scons_output_command>scons -Q - - debug=includes prog</scons_output_command>
</scons_output>
</section>
-->
<section>
<title>Where is &SCons; Blowing Up? the &debug-stacktrace; Option</title>
<para>
In general, &SCons; tries to keep its error
messages short and informative.
That means we usually try to avoid showing
the stack traces that are familiar
to experienced Python programmers,
since they usually contain much more
information than is useful to most people.
</para>
<para>
For example, the following &SConstruct; file:
</para>
<scons_example name="troubleshoot_stacktrace">
<file name="SConstruct" printme="1">
Program('prog.c')
</file>
</scons_example>
<para>
Generates the following error if the
<filename>prog.c</filename> file
does not exist:
</para>
<scons_output example="troubleshoot_stacktrace" suffix="1">
<scons_output_command>scons -Q</scons_output_command>
</scons_output>
<para>
In this case,
the error is pretty obvious.
But if it weren't,
and you wanted to try to get more information
about the error,
the &debug-stacktrace; option
would show you exactly where in the &SCons; source code
the problem occurs:
</para>
<scons_output example="troubleshoot_stacktrace" suffix="2">
<scons_output_command>scons -Q --debug=stacktrace</scons_output_command>
</scons_output>
<para>
Of course, if you do need to dive into the &SCons; source code,
we'd like to know if, or how,
the error messages or troubleshooting options
could have been improved to avoid that.
Not everyone has the necessary time or
Python skill to dive into the source code,
and we'd like to improve &SCons;
for those people as well...
</para>
</section>
<section>
<title>How is &SCons; Making Its Decisions? the &taskmastertrace; Option</title>
<para>
The internal &SCons; subsystem that handles walking
the dependency graph
and controls the decision-making about what to rebuild
is the <firstterm>Taskmaster</firstterm>.
&SCons; supports a &taskmastertrace;
option that tells the Taskmaster to print
information about the children (dependencies)
of the various Nodes on its walk down the graph,
which specific dependent Nodes are being evaluated,
and in what order.
</para>
<para>
The &taskmastertrace; option
takes as an argument the name of a file in
which to put the trace output,
with <filename>-</filename> (a single hyphen)
indicating that the trace messages
should be printed to the standard output:
</para>
<scons_example name="troubleshoot_taskmastertrace">
<file name="SConstruct" printme="1">
env = Environment(CPPPATH = ['.'])
env.Program('prog.c')
</file>
<file name="prog.c">
#include "inc.h"
prog.c
</file>
<file name="inc.h">
#define STRING "one"
</file>
</scons_example>
<scons_output example="troubleshoot_taskmastertrace" os="posix" suffix="1">
<scons_output_command>scons -Q --taskmastertrace=- prog</scons_output_command>
</scons_output>
<para>
The &taskmastertrace; option
doesn't provide information about the actual
calculations involved in deciding if a file is up-to-date,
but it does show all of the dependencies
it knows about for each Node,
and the order in which those dependencies are evaluated.
This can be useful as an alternate way to determine
whether or not your &SCons; configuration,
or the implicit dependency scan,
has actually identified all the correct dependencies
you want it to.
</para>
</section>
<section>
<title>Watch &SCons; prepare targets for building: the &debug-prepare; Option</title>
<para>
Sometimes SCons doesn't build the target you want
and it's difficult to figure out why. You can use
the &debug-prepare; option
to see all the targets &SCons; is considering, and whether
they are already up-to-date or not. The message is
printed before &SCons; decides whether to build the target.
</para>
</section>
<section>
<title>Why is a file disappearing? the &debug-duplicate; Option</title>
<para>
When using the &Duplicate; option to create variant dirs,
sometimes you may find files not getting linked or copied to where you
expect (or not at all), or files mysteriously disappearing.
These are usually because of a misconfiguration of some kind in the
SConscript files, but they can be tricky to debug. The
&debug-duplicate; option shows each time a variant file is
unlinked and relinked from its source (or copied, depending on
settings), and also shows a message for removing "stale"
variant-dir files that no longer have a corresponding source file.
It also prints a line for each target that's removed just before
building, since that can also be mistaken for the same thing.
</para>
</section>
<section>
<title>Keep it simple</title>
<para>
Over the years, many developers have chosen to dive in and make vastly
complicated build systems out of &SCons;, which sometimes don't work
quite as expected. As a general rule, make sure you
<emphasis>need</emphasis> to reach for a complex solution before you do so.
&SCons; is mature software and has evolved over time to meet a lot
of feature requests, so there is often an easier way to do something
if you can just find it. The &SCons; community can be helpful here -
the discussion lists and chat channels can be a way to find out if
something can be done an easier way before embarking on an implementation.
</para>
<para>
When something does misbehave, trying to isolate the problem to
a simple test case can really help. The work to create a reproducer
often helps you spot the issue yourself, and a simple example is much
easier for others to look over and possibly spot logical flaws,
misuse of the API, or other ways something could have been done.
In addition, if it turns out there's actually a real &SCons; bug
(we believe it's a high quality piece of software, but all software
has some bugs), it's very likely the bug filing will result in a
request for a simple reproducer anyway.
</para>
</section>
<!--
<section>
<title>Where Are My Build Bottlenecks? the &profile; Option</title>
<para>
XXX explain the - - profile= option
</para>
</section>
-->
<!--
<section>
<title>Troubleshooting Shared Caching: the &cache-debug; Option</title>
<para>
XXX describe the - - cache-debug option
XXX maybe point to the caching.in chapter?
</para>
</section>
-->
</chapter>