There are three types of situation where Greg may be used as a test framework -
(greg-test-run)
A testsuite for embedded use does not normally need begin.grg
or end.grg since the application to be tested will be running the
tests on itself - so it doesn't need to control child processes.
(greg-child)
procedure.
A testsuite for a tool is a directory containing one or more test script files and (optionally) begin.grg and end.grg files to handle initialisation and cleanup.
Each script file has a .scm extension and contains Guile (Scheme) code, but you do not need to know much about the Guile programming language to write most tests.
A script file will contain one or more testcases - each of which
constitutes a test of a single well defined feature of the tool
that the script is meant to test. A testcase is always written
using the greg-testcase
procedure.
The greg-testcase
procedure takes three arguments -
#t
if the test is expected to return
#t
(ie. the assertion is expected to be proved correct), and
#f
if the test is expected to return #f
(ie. the assertion
is expected to be proved to be incorrect).
#t
if the assertion is proved correct, #f
if it
is proved incorrect.
The Guile programming language permits the thunk to return in four ways -
#t
greg-testcase
.
#f
greg-testcase
.
#t
#f
(quit)
primitive - an exception is raised.
In this special case, the test is reported as unresolved (UNRESOLVED) and
no further tests are executed. The testrun is terminated.
As there are no other ways in which the thunk may be exited, it is impossible for a testcase to produce a result that doesn't fit into the framework (unless your testcase manages either to crash Guile or enter an infinite loop - in which case you won't get any output).
The value returned by the greg-testcase
procedure is a boolean -
#t
if the test resulted in an expected pass, #f
otherwise.
You can use this return value to make the execution of subsequent testcases
dependent on the success of an earlier testcase.
; ; A testcase to check an instance of numeric addition ; (greg-testcase "One plus One is two" #t (lambda () (eq? (+ 1 1 ) 2) )) ; ; The above testcase will generate output - ; `PASS: One plus One is two' ;
It is normal to have more than one testcase in a file and this produces no problems - the only thing to watch out for is communicating information between testcases -
The scope of variables defined in the thunk in a greg-testcase
procedure call is that thunk - the variable will not be
visible to the next testcase.
So - to pass information from one testcase to the next it is necessary to
define variables that can be seen in each testcase. The way to do this
is normally to define these variables at the start of the file and then
use the set!
procedure within each testcase to set a value for a
variable to be passed to the next testcase.
(define arith-ok #f) ; ; A testcase to check an instance of numeric addition ; (greg-testcase "One plus One is two" #t (lambda () (if (eq? (+ 1 1 ) 2) (begin (set! arith-ok #t) #t) #f) )) ; ; A testcase to check arithmetic - only supported if we have addition. ; (greg-testcase "X multiplied by 2 is X plus X" #t (lambda () (if arith-ok (eq? (+ 1 1) (* 1 2)) (throw 'unsupported)) ))
Of course, if (as above) the only information you want to pass from a testcase
is whether the test succeeded or not, you can use the return value from the
greg-testcase
procedure directly -
(if (greg-testcase "One plus One is two" #t (lambda () (eq? (+ 1 1 ) 2) ) ) (greg-testcase "X multiplied by 2 is X plus X" #t (lambda () (eq? (+ 1 1) (* 1 2)) ) ) (greg-dlog "Arithmetic operations not supported\n") )
When Greg is used to test an external application, you usually want to run that application as a child process on a pseudo-terminal and handle tests sending a sequence of commands to the application and reading anticipated output from the application.
Greg provides the greg-child
procedure to start up a child process
on a pseudo-terminal. You would usually call this procedure in the
begin.grg file in your tool directory, but you could call it
at the start of each script to get a new child process for each script.
The greg-child
procedure expects one argument (the name of
the program to be executed) followed by any number of additional
arguments which are the arguments to be passed to the child process.
If the program name does not begin with a slash, Greg will look in the
directory specified in greg-obj-dir
to find it (by default the
current directory).
If you want your normal PATH to be searched for the program, you should
use -
(greg-child "/bin/sh" "-c" "foo args")
to get the shell to execute program foo
with arguments args
.
The greg-child
procedure will automatically close down the I/O
channels to any process previously started and wait for that process to
die. If the old child process is badly behaved and will not die, this
can result in Greg hanging - in this case you may need to explicitly
kill the old child by another method before starting the new child
process (this is one of the uses of the end.grg script).
As a special case, you can use an empty string as the program name -
if you do this, another copy of the guile process will be created as
a child and the value returned by greg-child
in the child process
will be a list containing the single number 0 (in the parent it will be
a list containing the input port, output port and process id of the child).
You can use this information to get the child copy of the process to
be the program under test. This is useful for embedded testing where you
want to test the I/O capabilities of the program.
NB. The greg-child
procedure is implemented on top of the new
primitive pty-child
. This primitive is used to create a new child
process on the end of a pseudo-terminal. Arguments and return values are
as for greg-child
.
The greg-send
procedure is provided to send instructions to a
child process. This procedure takes one or more string arguments and
sends them to the child process (if one exists).
The greg-recv
macro is used to read data from a child process.
This procedure actually provides a simple front-end to the expect
module. You can use the expect
module facilities directly if you
want more control than is offered by greg-recv
.
The greg-recv
macro expects one or more lists as arguments -
each list containing a string (a pattern to match)
and a result to return on a successful match. The value returned by
greg-recv
is the result for the pattern actually matched.
If no pattern is matched within the timeout period then an empty list
is returned (unless you use (set! expect-timeout-proc xxx)
to
override Gregs timeout handler.
If no pattern is matched before an end-of-file is read, then an empty list
is returned (unless you use (set! expect-eof-proc xxx)
to
override Gregs end-of-file handler.
Go to the first, previous, next, last section, table of contents.