You can obtain DejaGnu from the DejaGnu web site at the Free Software Foundation, which is at www.gnu.org/software/dejagnu/
Once you have the DejaGnu source unpacked and available, you must first configure the software to specify where it is to run (and the associated defaults); then you can proceed to installing it.
It is usually best to configure in a directory separate from the
source tree, specifying where to find the source with the optional
--srcdir
option to
configure. DejaGnu uses the GNU
autoconf to configure itself. For more info on using
autoconf, read the GNU autoconf manual. To configure, execute the
configure program, no other options are
required. For an example, to configure in a separate tree for objects,
execute the configure script from the source tree like this:
../dejagnu-1.5.1/configure
DejaGnu doesn't care at config time if it's for testing a native system or a cross system. That is determined at runtime by using the config files.
You may also want to use the configure option
--prefix
to specify where you want DejaGnu and its
supporting code installed. By default, installation is in subdirectories
of /usr/local, but you can select any alternate
directory altdir
by including
--prefix altdir
on the
configure command line. (This value is captured in
the Makefile variables prefix and
exec_prefix.)
Save for a small number of example tests, the DejaGnu distribution itself does not include any testsuites; these are available separately. Testsuites for the GNU development tools are included in those releases. After configuring the top-level DejaGnu directory, unpack and configure the test directories for the tools you want to test; then, in each test directory, run make check to build auxiliary programs required by some of the tests, and run the test suites.
To install DejaGnu in your file system (either in
/usr/local, or as specified by your
--prefix
option to configure),
execute.
eg$ make install
make installdoes these things for DejaGnu:
Look in the path specified for executables
$exec_prefix
) for directories called
lib and bin. If these
directories do not exist, make install creates
them.
Create another directory in the share directory, called dejagnu, and copy all the library files into it.
Create a directory in the dejagnu/share directory, called config, and copy all the configuration files into it.
Copy the runtest shell script into $exec_prefix/bin.
Copy runtest.exp into $exec_prefix/lib/dejagnu. This is the main Tcl code implementing DejaGnu.
DejaGnu provides these Tcl procedures.
Tests for a particular build host environment. If the currently configured host matches the argument string, the result is 1; otherwise the result is 0. host must be a full three-part configure host name; in particular, you may not use the shorter nicknames supported by configure (but you can use wildcard characters, using shell syntax, to specify sets of names). If it is passed a NULL string, then it returns the name of the build canonical configuration.
pattern
Tests for a Canadian cross. This is when the tests will be run on a remotely hosted cross compiler. If it is a Canadian cross, then the result is 1; otherwise the result is 0.
Tests for a particular host environment. If the currently configured host matches the argument string, the result is 1; otherwise the result is 0. host must be a full three-part configure host name; in particular, you may not use the shorter nicknames supported by configure (but you can use wildcard characters, using shell syntax, to specify sets of names).
Tests for a particular target environment. If the currently configured target matches the argument string, the result is 1 ; otherwise the result is 0. target must be a full three-part configure target name; in particular, you may not use the shorter nicknames supported by configure (but you can use wildcard characters, using shell syntax, to specify sets of names). If it is passed a NULL string, then it returns the name of the build canonical configuration.
Tests whether the current configuration has the same host and target. When it runs in a native configuration this procedure returns a 1; otherwise it returns a 0.
Declares that the test is expected to fail on a particular set of configurations. The config argument must be a list of full three-part configure target name; in particular, you may not use the shorter nicknames supported by configure (but you can use the common shell wildcard characters to specify sets of names). The bugid argument is optional, and used only in the logging file output; use it as a link to a bug-tracking system such as GNATS.
Once you use setup_xfail
, the
fail
and pass
procedures
produce the messages XFAIL and
XPASS respectively, allowing you to distinguish
expected failures (and unexpected success!) from other test
outcomes.
Warning you must clear the expected failure after
using setup_xfail in a test case. Any call to |
config
The config triplet to trigger whether this is an unexpected or expect failure.
bugid
The optional bugid, used to tie this test case to a bug tracking system.
Declares a test to have passed. pass
writes in the log files a message beginning with
PASS (or XPASS, if failure
was expected), appending the argument
string
.
string
The string to use for this PASS message.
Declares a test to have failed. fail
writes in the log files a message beginning with
FAIL (or XFAIL, if failure
was expected), appending the argument
string
.
string
The string to use for this FAIL message.
Declares a test to have unexpectedly passed, when it was
expected to be a failure. xpass
writes in the log files a message beginning with
XPASS (or XFAIL, if failure
was expected), appending the argument
string
.
string
The string to use for this output state.
Declares a test to have expectedly
failed. xfail
writes in the log files a message beginning with
XFAIL (or PASS, if success
was expected), appending the argument
string
.
string
The string to use for this output state.
Sets the value of warning_threshold
. A value
of 0 disables it: calls to
warning
will not turn a
PASS or FAIL into an
UNRESOLVED.
threshold
This is the value of the new warning threshold.
Returns the current value of
{warning_threshold
. The default value is 3. This
value controls how many warning
procedures can
be called before becoming UNRESOLVED.
Declares detection of a minor error in the test case
itself. warning
writes in the log files a message
beginning with WARNING, appending the argument
string
. Use warning
rather
than perror
for cases (such as communication
failure to be followed by a retry) where the test case can recover from
the error. If the optional number
is supplied,
then this is used to set the internal count of warnings to that
value.
As a side effect, warning_threshold
or more
calls to warning in a single test case also changes the effect of the
next pass
or fail
command:
the test outcome becomes UNRESOLVED since an
automatic PASS or FAIL may
not be trustworthy after many warnings. If the optional numeric value
is 0, then there are no further side effects to
calling this function, and the following test outcome doesn't become
UNRESOLVED. This can be used for errors with no
known side effects.
string
number
The optional number to set the error counter. This
is only used to fake out the counter when using the
xfail
procedure to control when it flips the
output over to UNRESOLVED
state.
Declares a severe error in the testing framework
itself. perror
writes in the log files a message
beginning with ERROR, appending the argument
string
.
As a side effect, perror also changes the effect of the next
pass
or fail
command: the
test outcome becomes UNRESOLVED, since an
automatic PASS or FAIL cannot
be trusted after a severe error in the test framework. If the optional
numeric value is 0, then there are no further side
effects to calling this function, and the following test outcome
doesn't become UNRESOLVED. This can be used for
errors with no known side effects.
string
number
The optional number to set the error counter. This
is only used to fake out the counter when using the
xfail
procedure to control when it flips the
output over to UNRESOLVED
state.
Appends an informational message to the log
file. note
writes in the log files a message
beginning with NOTE, appending the argument
string
. Use note
sparingly. The verbose
should be used for most
such messages, but in cases where a message is needed in the log file
regardless of the verbosity level use note
.
string
The string to use for this note.
Declares a test was not run. untested
writes
in the log file a message beginning with UNTESTED,
appending the argument string. For example, you
might use this in a dummy test whose only role is to record that a test
does not yet exist for some feature.
string
The string to use for this output state.
Declares a test to have an unresolved
outcome. unresolved
writes in the log file a
message beginning with UNRESOLVED, appending the
argument string. This usually means the test did
not execute as expected, and a human being must go over results to
determine if it passed or failed (and to improve the test case).
string
The string to use for this output state.
Declares that a test case depends on some facility that does not
exist in the testing environment. unsupported
writes in the log file a message beginning with
UNSUPPORTED, appending the argument string.
string
The string to use for this output state.
Generates a string for the name of a tool as it was configured
and installed, given its native name (as the argument
toolname
). This makes the assumption that all
tools are installed using the same naming conventions: For example,
for a cross compiler supporting the m68k-vxworks
configuration, the result of transform gcc is
m68k-vxworks-gcc.
toolname
The name of the cross-development program to transform.
This procedure adds a conditional xfail, based on compiler
options used to create a test case executable. If an include options
is found in the compiler flags, and it's the right architecture,
it'll trigger an XFAIL. Otherwise it'll produce
an ordinary FAIL. You can also specify flags to
exclude. This makes a result be a FAIL, even if
the included options are found. To set the conditional, set
the variable compiler_conditional_xfail_data
to the
fields
"[message string] [targets list] [includes list] [excludes list]"
(descriptions below). This is the checked at pass/fail decision time, so there is no need to call the procedure yourself, unless you wish to know if it gets triggered. After a pass/fail, the variable is reset, so it doesn't effect other tests. It returns 1 if the conditional is true, or 0 if the conditional is false.
message
This is the message to print with the normal test result.
targets
This is a string with the list targets to activate this conditional on.
includes
This is a list of sets of options to search for in the compiler options to activate this conditional. If the list of sets of options is empty or if any set of the options matches, then this conditional is true. (It may be useful to specify an empty list of include sets if the conditional is always true unless one of the exclude sets matches.)
excludes
This is a list of sets of options to search for in the compiler options to activate this conditional. If any set of the options matches, (regardless of whether any of the include sets match) then this conditional is de-activated.
Example 42. Specifying the conditional xfail data
set compiler_conditional_xfail_data { \ "I sure wish I knew why this was hosed" \ "sparc*-sun*-* *-pc-*-*" \ {"-Wall -v" "-O3"} \ {"-O1" "-Map"} \ }
What this does is it matches only for these two targets if "-Wall -v" or "-O3" is set, but neither "-O1" or "-Map" is set. For a set to match, the options specified are searched for independently of each other, so a "-Wall -v" matches either "-Wall -v" or "-v -Wall". A space separates the options in the string. Glob-style regular expressions are also permitted.
Cancel an expected failure (previously declared with
setup_xfail) for a particular set of
configurations. The config
argument is a list
of configuration target names. It is only necessary to call
clear_xfail if a test case ends without calling
either pass or fail, after
calling setup_xfail.
config
The configuration triplets to clear.
Test cases can use this function to issue helpful messages
depending on the number of --verbose
options on the
runtest command line. It prints string if the value of the variable
verbose
is higher than or equal to the optional
number. The default value for number is 1. Use
the optional -log
argument to cause string to always
be added to the log file, even if it won't be printed. Use the
optional -x
argument to log the test results into
a parsable XML file. Use the optional -n
argument
to print string without a trailing newline. Use the optional
--
argument if string begins with "-".
-x
-log
-n
--
string
number
Loads a DejaGnu library file by searching the default fixed paths built into DejaGnu. If DejaGnu has been installed, it looks in a path starting with the installed library directory. If you are running DejaGnu directly from a source directory, without first running make install, this path defaults to the current directory. In either case, it then looks in the current directory for a directory called lib. If there are duplicate definitions, the last one loaded takes precedence over the earlier ones.
filespec
The name of the DejaGnu library file to load.
The global variable libdirs
, handled
as a list, is appended to the default fixed paths built
into DejaGnu.
lib/remote.exp defines these functions, for establishing and managing communications. Each of these procedures tries to establish the connection up to three times before returning. Warnings (if retries will continue) or errors (if the attempt is abandoned) report on communication failures. The result for any of these procedures is either -1, when the connection cannot be established, or the spawn ID returned by the Expect command spawn.
It use the value of the connect
field
in the target_info
array (was
connectmode
as the type of connection to
make. Current supported connection types are tip, kermit,
telnet, rsh, rlogin, and netdata. If the --reboot
option was used on the runtest command line, then the target
is rebooted before the connection is made.
shellid
This is the value returned by a call
to remote_open
. This closes the
connection to the target so resources can be used by
others. This parameter can be left off if the
fileid
field in the
target_info
array is set.
type
This is passed host
or
target
. Host or target refers to
whether it is a connection to a remote target, or a
remote host. This opens the connection to the desired
target or host using the default values in the
configuration system. It returns that
spawn_id
of the process that manages
the connection. This value can be used in
Expect or
exp_send statements, or passed to
other procedures that need the connection process's
id. This also sets the fileid
field in
the target_info
array.
Return value of this function depends on actual implementation
of reboot that will be used, in practice it is expected that
remote_reboot
returns 1
on success and 0 on failure.
host
It looks like that this procedure is never called, instead
${board}_reboot
defined in
base-config.exp will be used because it has
higher priority and base-config.exp is
always imported by runtest.
host
telnet, rsh, tip, kermit
hostname
This refers to the IP address or name
(for example, an entry in
/etc/hosts) for this target. The
procedure names reflect the Unix utility used to
establish a connection. The optional
port
is used to specify the IP
port number. The value of the
netport
field in the
target_info
array is used. (was
$netport
) This value has two parts,
the hostname and the port number, separated by a
:. If host or target is used in
the hostname
field, than the
config array is used for all information.
port
Connect using the Unix utility
tip. Port
must
be a name from the tip
configuration file
/etc/remote. Often, this is called
hardwire
, or something like
ttya
. This file holds all the
configuration data for the serial port. The value of
the serial
field in the
target_info
array is used. (was
$serialport
) If host
or target
is used in the
port
field, than the config
array is used for all information. the
config array is used for all information.
port
Connect using the program
kermit. Port
is the device name,
e.g. /dev/ttyb.
bps
bps
is the line
speed to use (in its per second) for the
connection. The value of the serial
field in the target_info
array is
used. (was $serialport
) If
host
or target
is
used in the port
field, than the
config array is used for all information. the
config array is used for all information.
spawnid
Download file
to the
process spawnid
(the value returned
when the connection was established), using the
~put command under
tip. Most often used for
single board computers that require downloading
programs in ASCII S-records. Returns
1 if an error occurs,
0 otherwise.
file
This is the filename to download.
source
destfile
flags
source
destfile
type
options
source
destfile
type
options
This checks if board_info
array's field
entry has been set already and if not, then
sets it to value.
entry
The name of a board_info
field
to operate on.
value
The value to set the field to.
This treats board_info
array's field
entry as a TCL list and adds
value at the end.
entry
The name of a board_info
field
to operate on.
value
The value to add to the field.
This checks if board_info
array's field
entry has been set and if so, then removes
it.
entry
The name of a board_info
field
to operate on.
This makes the target named name be the
current target connection. The value of name is
an index into the target_info
array and is set in
the global config file.
name
The name of the target to make current connection.
This lists all the supported targets for this architecture.
This makes the host named name be the
current remote host connection. The value of
name is an index into the
target_info
array and is set in the global config
file.
name
This invokes the compiler as set by CC to compile the file file. The default options for many cross compilation targets are guessed by DejaGnu, and these options can be added to by passing in more parameters as arguments to compile. Optionally, this will also use the value of the cflags field in the target config array. If the host is not the same as the build machines, then then compiler is run on the remote host using execute_anywhere.
file
This produces an archive file. Any parameters passed to archive are used in addition to the default flags. Optionally, this will also use the value of the arflags field in the target config array. If the host is not the same as the build machines, then then archiver is run on the remote host using execute_anywhere.
file
This generates an index for the archive file for systems that aren't POSIX yet. Any parameters passed to ranlib are used in for the flags.
file
This executes the cmdline on the proper host. This should be used as a replacement for the Tcl command exec as this version utilizes the target config info to execute this command on the build machine or a remote host. All config information for the remote host must be setup to have this command work. If this is a Canadian cross (where we test a cross compiler that runs on a different host then where DejaGnu is running) then a connection is made to the remote host and the command is executed there. It returns either REMOTERROR (for an error) or the output produced when the command was executed. This is used for running the tool to be tested, not a test case.
cmdline
Each combination of target and tool requires some
target-dependent procedures. The names of these procedures have
a common form: the tool name, followed by an underscore
_, and finally a suffix describing the
procedure's purpose. For example, a procedure to extract the
version from GDB is called
gdb_version
.
runtest itself calls only two of these
procedures, ${tool}_exit
and
${tool}_version
; these procedures use no
arguments.
The other two procedures, ${tool}_start
and ${tool}_load
, are only called by the test
suites themselves (or by testsuite-specific initialization
code); they may take arguments or not, depending on the
conventions used within each testsuite.
The usual convention for return codes from any of these procedures (although it is not required by runtest) is to return 0 if the procedure succeeded, 1 if it failed, and -1 if there was a communication error.
Starts a particular tool. For an interactive tool,
${tool}_start
starts and initializes the
tool, leaving the tool up and running for the test cases; an
example is gdb_start
, the start function
for GDB. For a batch oriented tool,
${tool}_start
is optional; the recommended
convention is to let ${tool}_start
run the
tool, leaving the output in a variable called
comp_output
. Test scripts can then analyze
$comp_output
to determine the test results.
An example of this second kind of start function is
gcc_start
, the start function for GCC.
DejaGnu itself does not call
${tool}_start
. The initialization
module ${tool}_init.exp
must call
${tool}_start
for interactive tools;
for batch-oriented tools, each individual test script calls
${tool}_start
(or makes other
arrangements to run the tool).
Loads something into a tool. For an interactive tool,
this conditions the tool for a particular test case; for
example, gdb_load
loads a new
executable file into the debugger. For batch oriented tools,
${tool}_load
may do nothing---though,
for example, the GCC support uses
gcc_load
to load and run a binary on
the target environment. Conventionally,
${tool}_load
leaves the output of any
program it runs in a variable called
$exec_output
. Writing
${tool}_load
can be the most complex
part of extending DejaGnu to a new tool or a new target, if
it requires much communication coding or file
downloading. Test scripts call
${tool}_load
.
Cleans up (if necessary) before DejaGnu exits. For
interactive tools, this usually ends the interactive
session. You can also use ${tool}_exit
to remove any temporary files left over from the
tests. runtest calls
${tool}_exit
.
Prints the version label and number for
${tool}
. This is called by the DejaGnu
procedure that prints the final summary report. The output
should consist of the full path name used for the tested
tool, and its version number.
Returns a list of all the directories in the single directory a single directory that match an optional pattern.
args
pattern
If you do not specify
pattern
,
Getdirs
assumes a default pattern of
*. You may use the common shell
wildcard characters in the pattern. If no directories
match the pattern, then a NULL string is
returned.
Search for files whose names match pattern (using shell wildcard characters for filename expansion). Search subdirectories recursively, starting at rootdir. The result is the list of files whose names match; if no files match, the result is empty. Filenames in the result include all intervening subdirectory names. If no files match the pattern, then a NULL string is returned.
rootdir
The top level directory to search the search from.
pattern
A csh "glob" style regular expression representing the files to find.
Searches the execution path for an executable file binary, like the BSD which utility. This procedure uses the shell environment variable PATH. It returns 0 if the binary is not in the path, or if there is no PATH environment variable. If binary is in the path, it returns the full path to binary.
binary
The executable program or shell script to look for.
Search the file called filename (a fully specified path) for lines that contain a match for regular expression regexp. The result is a list of all the lines that match. If no lines match, the result is an empty string. Specify regexp using the standard regular expression style used by the Unix utility program grep.
Use the optional third argument line to
start lines in the result with the line number in
filename. (This argument is simply an option
flag; type it just as shown --line
.)
filename
The file to search.
regexp
The Unix style regular expression (as used by the grep Unix utility) to search for.
--line
Prefix the line number to each line where the regexp matches.
This procedure is deprecated and will be removed in the next release of DejaGnu. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite.
This procedure is deprecated and will be removed in the next release of DejaGnu. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite.
This procedure is deprecated and will be removed in the next release of DejaGnu. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite.
This procedure is deprecated and will be removed in the next release of DejaGnu. If a testsuite uses this procedure, a copy of the procedure should be made and placed in the lib directory of the testsuite.
Search runtests for testcase and return 1 if found, 0 if not. runtests is a list of two elements. The first is a copy of what was on the right side of the = if
foo.exp="..."
was specified, or an empty string if no such argument is present. The second is the pathname of the current testcase under consideration. This is used by tools like compilers where each testcase is a file.
runtests
The list of patterns to compare against.
testcase
The test case filename.
Compares the two files and returns a 1 if
they match, or a 0 if they don't. If
verbose
is set, then it'll print the differences to
the screen.
file_1
The first file to compare.
file_2
The second file to compare.
Sets the environment variable var to the value val.
var
The environment variable to set.
val
The value to set the variable to.
Unsets the environment variable var.
var
The environment variable to unset.
Returns the value of var in the environment if it exists, otherwise it returns NULL.
var
The environment variable to get the value of.
For system system, delete text the host or target operating system might issue that will interfere with pattern matching of program output in text. An example is the message that is printed if a shared library is out of date.
system
The system error messages to look for to screen out .
text
The Tcl variable containing the text.
Libgloss is a free BSP (Board Support Package) commonly used with GCC and G++ to produce a fully linked executable image for an embedded systems.
lib/debugger.expdefines these utility procedures:
This takes a csh style regular expression (glob rules) and prints the values of the global variable names that match. It is abbreviated as dv.
vars
The variables to dump.
This takes a csh style regular expression (glob rules) and prints the values of the local variable names that match. It is abbreviated as dl.
args
This takes a csh style regular expression (glob rules) and prints the body of all procs that match. It is abbreviated as dp.
pattern
The csh "glob" style pattern to look for.
This takes a csh style regular expression (glob rules) and prints all the watchpoints. It is abbreviated as dw.
pattern
The csh "glob" style pattern to look for.
This breaks program execution when the variable
var
is unset. It is abbreviated as
wu.
args
This breaks program execution when the variable
var
is written. It is abbreviated as
ww.
var
The variable to watch.
This breaks program execution when the variable
var
is read. It is abbreviated as
wr.
var
The variable to watch.
This deletes a watchpoint from the watch list. It is abbreviated as wd.
args
This is a map of the files in DejaGnu.
runtest
runtest.exp
stub-loader.c
testglue.c
config
baseboards
lib/debugger.exp
lib/dg.exp
lib/framework.exp
lib/ftp.exp
lib/kermit.exp
lib/libgloss.exp
lib/mondfe.exp
lib/remote.exp
lib/rlogin.exp
lib/rsh.exp
lib/standard.exp
lib/target.exp
lib/targetdb.exp
lib/telnet.exp
lib/tip.exp
lib/util-defs.exp
lib/utils.exp
lib/xsh.exp
lib/dejagnu.exp