ScalaTest 0.9.5
|
|
org/scalatest/tools/Runner.scala
]
object
Runner
extends
AnyRef
Application that runs a suite of tests.
The application accepts command line arguments that specify optional user-defined properties, an optional
runpath, zero to many Reporter
s, optional lists of test groups to include and/or exclude, zero to many
Suite
class names, zero to many "members-only" Suite
paths, zero to many "wildcard" Suite
paths,
and zero to many TestNG XML config file paths.
All of these arguments are described in more detail below. Here's a summary:
scala [-classpath scalatest-<version>.jar:...] org.scalatest.tools.Runner [-D<key>=<value> [...]] [-p <runpath>] [reporter [...]] [-n <includes>] [-x <excludes>] [-c] [-s <suite class name> [...]] [-m <members-only suite path> [...]] [-w <wildcard suite path> [...]] [-t <TestNG config file path> [...]]
The simplest way to start Runner
is to specify the directory containing your compiled tests as the sole element of the runpath, for example:
scala -classpath scalatest-<version>.jar org.scalatest.tools.Runner -p compiled_tests
Given the previous command, Runner
will discover and execute all Suite
s in the compiled_tests
directory and its subdirectories,
and show results in graphical user interface (GUI).
Specifying user-defined properties
A user-defined property consists of a key and a value. The key may not begin with "org.scalatest.". User-defined properties may be specified on the command line. Each property is denoted with a "-D", followed immediately by the key string, an "=", and the value string. For example:
-Ddbname=testdb -Dserver=192.168.1.188
Specifying a runpath
A runpath is the list of filenames, directory paths, and/or URLs that Runner
uses to load classes for the running test. If runpath is specified, Runner
creates
a custom class loader to load classes available on the runpath.
The graphical user interface reloads the test classes anew for each run
by creating and using a new instance of the custom class loader for each run.
The classes that comprise the test may also be made available on
the classpath, in which case no runpath need be specified.
The runpath is specified with the -p option. The -p must be followed by a space,
a double quote ("
), a white-space-separated list of
paths and URLs, and a double quote. If specifying only one element in the runpath, you can leave off
the double quotes, which only serve to combine a white-space separated list of strings into one
command line argument. Here's an example:
-p "serviceuitest-1.1beta4.jar myjini http://myhost:9998/myfile.jar"
Specifying reporters
Reporters can be specified on the command line in any of the following ways:
-g[configs...]
- causes display of a graphical user interface that allows
tests to be run and results to be investigated
-f[configs...] <filename>
- causes test results to be written to
the named file
-o[configs...]
- causes test results to be written to
the standard output
-e[configs...]
- causes test results to be written to
the standard error
-r[configs...] <reporterclass>
- causes test results to be reported to
an instance of the specified fully qualified Reporter
class name
The [configs...]
parameter, which is used to configure reporters, is described in the next section.
The -r
option causes the reporter specified in
<reporterclass>
to be
instantiated.
Each reporter class specified with a -r option must be public, implement
org.scalatest.Reporter
, and have a public no-arg constructor.
Reporter classes must be specified with fully qualified names.
The specified reporter classes may be
deployed on the classpath. If a runpath is specified with the
-p
option, the specified reporter classes may also be loaded from the runpath.
All specified reporter classes will be loaded and instantiated via their no-arg constructor.
For example, to run a suite named MySuite
from the mydir
directory
using two reporters, the graphical reporter and a file reporter
writing to a file named "test.out"
, you would type:
java -jar scalatest.jar -p mydir -g -f test.out -s MySuite
The -g
, -o
, or -e
options can
appear at most once each in any single command line.
Multiple appearances of -f
and -r
result in multiple reporters
unless the specified <filename>
or <reporterclass>
is
repeated. If any of -g
, -o
, -e
,
<filename>
or <reporterclass>
are repeated on
the command line, the Runner
will print an error message and not run the tests.
Runner
adds the reporters specified on the command line to a dispatch reporter,
which will dispatch each method invocation to each contained reporter. Runner
will pass
the dispatch reporter to executed suites. As a result, every
specified reporter will receive every report generated by the running suite of tests.
If no reporters are specified, a graphical
runner will be displayed that provides a graphical report of
executed suites.
Configuring Reporters
Each reporter specification on the command line can include configuration characters. Configuration
characters
are specified immediately following the -g
, -o
,
-e
, -f
, or -r
. Valid configuration
characters are:
Y
- present runStarting
method invocations
Z
- present testStarting
method invocations
T
- present testSucceeded
method invocations
F
- present testFailed
method invocations
G
- present testIgnored
method invocations
U
- present suiteStarting
method invocations
P
- present suiteCompleted
method invocations
B
- present suiteAborted
method invocations
I
- present infoProvided
method invocations
S
- present runStopped
method invocations
A
- present runAborted
method invocations
R
- present runCompleted
method invocations
Each reporter class has a default configuration. If no configuration
is specified on the command line for a particular reporter, that
reporter uses its default configuration. If a configuration is specified, Runner
will present
to the configured reporter only those report types mentioned in the configuration characters. If the command
line includes argument -oFAB
, for example, only testFailed
,
runAborted
, and suiteAborted
events will be reported to the standard output reporter.
For example, to run a suite using two reporters, the graphical reporter (using its default configuration) and a standard error reporter configured to print only test failures, run aborts, and suite aborts, you would type:
scala -classpath scalatest-<version>.jar -p mydir -g -eFAB -s MySuite
Note that no white space is allowed between the reporter option and the initial configuration
parameters. So "-e FAB"
will not work,
"-eFAB"
will work.
Specifying includes and excludes
You can specify named groups of tests to include or exclude from a run. To specify includes,
use -n
followed by a white-space-separated list of group names to include, surrounded by
double quotes. (The double quotes are not needed if specifying just one group.) Similarly, to specify excludes, use -x
followed by a white-space-separated
list of group names to exclude, surrounded by double quotes. (As before, the double quotes are not needed if specifying just one group.) If includes is not specified, then all tests
except those mentioned in the excludes group (and in the Ignore
group), will be executed.
(In other words, an empty includes list is like a wildcard, indicating all tests be included.)
If includes is specified, then only those tests in groups mentioned in the argument following -n
and not mentioned in the excludes group, will be executed. For more information on test groups, see
the documentation for Suite
. Here are some examples:
-n CheckinTests
-n FunctionalTests -x SlowTests
-n "CheckinTests FunctionalTests"-x "SlowTests NetworkTests"
Executing Suite
s concurrently
With the proliferation of multi-core architectures, and the often parallelizable nature of tests, it is useful to be able to run
tests concurrently. If you include -c
on the command line, Runner
will pass a Distributor
to
the Suite
s you specify with -s
. Runner
will set up a thread pool to execute any Suite
s
passed to the Distributor
's put
method concurrently. Trait Suite
's implementation of
runNestedSuites
will place any nested Suite
s into this Distributor
. Thus, if you have a Suite
of tests that must be executed sequentially, you should override runNestedSuites
as described in the documentation for Distributor
.
Specifying Suite
s
Suites are specified on the command line with a -s followed by the fully qualified
name of a Suite
subclass, as in:
-s com.artima.serviceuitest.ServiceUITestkit
Each specified suite class must be public, a subclass of
org.scalatest.Suite
, and contain a public no-arg constructor.
Suite
classes must be specified with fully qualified names.
The specified Suite
classes may be
loaded from the classpath. If a runpath is specified with the
-p
option, specified Suite
classes may also be loaded from the runpath.
All specified Suite
classes will be loaded and instantiated via their no-arg constructor.
The runner will invoke execute
on each instantiated org.scalatest.Suite
,
passing in the dispatch reporter to each execute
method.
Runner
is intended to be used from the command line. It is included in org.scalatest
package as a convenience for the user. If this package is incorporated into tools, such as IDEs, which take
over the role of runner, object org.scalatest.tools.Runner
may be excluded from that implementation of the package.
All other public types declared in package org.scalatest.tools.Runner
should be included in any such usage, however,
so client software can count on them being available.
Specifying "members-only" and "wildcard" Suite
paths
If you specify Suite
path names with -m
or -w
, Runner
will automatically
discover and execute accessible Suite
s in the runpath that are either a member of (in the case of -m
)
or enclosed by (in the case of -w
) the specified path. As used in this context, a path is a portion of a fully qualified name.
For example, the fully qualifed name com.example.webapp.MySuite
contains paths com
, com.example
, and com.example.webapp
.
The fully qualifed name com.example.webapp.MyObject.NestedSuite
contains paths com
, com.example
,
com.example.webapp
, and com.example.webapp.MyObject
.
An accessible Suite
is a public class that extends org.scalatest.Suite
and defines a public no-arg constructor. Note that Suite
s defined inside classes and traits do not have no-arg constructors,
and therefore won't be discovered. Suite
s defined inside singleton objects, however, do get a no-arg constructor by default, thus
they can be discovered.
For example, if you specify -m com.example.webapp
on the command line, and you've placed com.example.webapp.RedSuite
and com.example.webapp.BlueSuite
on the runpath, then Runner
will instantiate and execute both of those Suite
s. The difference
between -m
and -w
is that for -m
, only Suite
s that are direct members of the named path
will be discovered. For -w
, any Suite
s whose fully qualified
name begins with the specified path will be discovered. Thus, if com.example.webapp.controllers.GreenSuite
exists on the runpath, invoking Runner
with -w com.example.webapp
will cause GreenSuite
to be discovered, because its fully qualifed name begins with "com.example.webapp"
. But if you invoke Runner
with -m com.example.webapp
, GreenSuite
will not be discovered because it is directly
a member of com.example.webapp.controllers
, not com.example.webapp
.
If you specify no -s
, -m
, or -w
arguments on the command line to Runner
, it will discover and execute all accessible Suite
s
in the runpath.
Specifying TestNG XML config file paths
If you specify one or more file paths with -t
, Runner
will create a org.scalatest.testng.TestNGWrapperSuite
,
passing in a List
of the specified paths. When executed, the TestNGWrapperSuite
will create one TestNG
instance
and pass each specified file path to it for running. If you include -t
arguments, you must include TestNG's jar file on the class path or runpath.
The -t
argument will enable you to run existing TestNG
tests, including tests written in Java, as part of a ScalaTest run.
You need not use -t
to run suites written in Scala that extend TestNGSuite
. You can simply run such suites with
-s
, -m
, or -w parameters.
Method Summary | |
def
|
main
(args : scala.Array[java.lang.String]) : Unit
Runs a suite of tests, with optional GUI. See the main documentation for this singleton object for the details.
|
Methods inherited from AnyRef | |
getClass, hashCode, equals, clone, toString, notify, notifyAll, wait, wait, wait, finalize, ==, !=, eq, ne, synchronized |
Methods inherited from Any | |
==, !=, isInstanceOf, asInstanceOf |
Method Details |
def
main(args : scala.Array[java.lang.String]) : Unit
ScalaTest 0.9.5
|
|