ScalaTest 0.9.5
|
|
org/scalatest/Suite.scala
]
trait
Suite
extends
Assertions with
ExecuteAndRun
A suite of tests. A Suite
instance encapsulates a conceptual
suite (i.e., a collection) of tests. This trait defines a default way to create
and execute tests, which involves writing test methods. This approach will likely suffice
in the vast majority of applications, but if desired, subtypes can override certain methods
to define other ways to create and execute tests.
The easiest way to use this trait is to use its default approach: Simply create classes that
extend Suite
and define test methods. Test methods have names of the form testX
,
where X
is some unique, hopefully meaningful, string. A test method must be public and
can have any result type, but the most common result type is Unit
. Here's an example:
import org.scalatest.Suite class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assert(sum === 2) assert(sum + 2 === 4) } def testSubtraction() { val diff = 4 - 1 assert(diff === 3) assert(diff - 2 === 1) } }
You run a Suite
by invoking on it one of three overloaded execute
methods. Two of these execute
methods, which print test results to the
standard output, are intended to serve as a
convenient way to run tests from within the Scala interpreter. For example,
to run MySuite
from within the Scala interpreter, you could write:
scala> (new MySuite).execute()
And you would see:
Test Starting - MySuite.testAddition Test Succeeded - MySuite.testAddition Test Starting - MySuite.testSubtraction Test Succeeded - MySuite.testSubtraction
Or, to run just the testAddition
method, you could write:
scala> (new MySuite).execute("testAddition")
And you would see:
Test Starting - MySuite.testAddition Test Succeeded - MySuite.testAddition
The third overloaded execute
method takes seven parameters, so it is a bit unwieldy to invoke from
within the Scala interpreter. Instead, this execute
method is intended to be invoked indirectly by a test runner, such
as org.scalatest.tools.Runner
or an IDE. See the documentation for Runner
for more detail.
Assertions and ===
Inside test methods, you can write assertions by invoking assert
and passing in a Boolean
expression,
such as:
val left = 2 val right = 1 assert(left == right)
If the passed expression is true
, assert
will return normally. If false
,
assert
will complete abruptly with an AssertionError
. This exception is usually not caught
by the test method, which means the test method itself will complete abruptly by throwing the AssertionError
. Any
test method that completes abruptly with an AssertionError
or any Exception
is considered a failed
test. A test method that returns normally is considered a successful test.
If you pass a Boolean
expression to assert
, a failed assertion will be reported, but without
reporting the left and right values. You can alternatively encode these values in a String
passed as
a second argument to assert
, as in:
val left = 2 val right = 1 assert(left == right, left + " did not equal " + right)
Using this form of assert
, the failure report will include the left and right values, thereby
helping you debug the problem. However, Suite
provides the ===
operator to make this easier.
You use it like this:
val left = 2 val right = 1 assert(left === right)
Because you use ===
here instead of ==
, the failure report will include the left
and right values. For example, the detail message in the thrown AssertionErrorm
from the assert
shown previously will include, "2 did not equal 1".
From this message you will know that the operand on the left had the value 2, and the operand on the right had the value 1.
If you're familiar with JUnit, you would use ===
in a ScalaTest Suite
where you'd use assertEquals
in a JUnit TestCase
.
The ===
operator is made possible by an implicit conversion from Any
to Equalizer
. If you're curious to understand the mechanics, see the documentation for
Equalizer
and Suite
's convertToEqualizer
method.
Expected results
Although===
provides a natural, readable extension to Scala's assert
mechanism,
as the operands become lengthy, the code becomes less readable. In addition, the ===
comparison
doesn't distinguish between actual and expected values. The operands are just called left
and right
,
because if one were named expected
and the other actual
, it would be difficult for people to
remember which was which. To help with these limitations of assertions, Suite
includes a method called expect
that
can be used as an alternative to assert
with ===
. To use expect
, you place
the expected value in parentheses after expect
, and follow that by code contained inside
curly braces that results in a value that you expect should equal the expected value. For example:
val a = 5 val b = 2 expect(2) { a - b }
In this case, the expected value is 2
, and the code being tested is a - b
. This expectation will fail, and
the detail message in the AssertionError
will read, "Expected 2, but got 3."
Intercepted exceptions
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit style, like this:
val s = "hi" try { s.charAt(-1) fail() } catch { case e: IndexOutOfBoundsException => // Expected, so continue }
If charAt
throws IndexOutOfBoundsException
as left, control will transfer
to the catch case, which does nothing. If, however, charAt
fails to throw an exception,
the next statement, fail()
, will be executed. The fail
method always completes abruptly with
an AssertionError
, thereby signaling a failed test.
To make this common use case easier to express and read, Suite
provides an intercept
method. You use it like this:
val s = "hi" intercept(classOf[IndexOutOfBoundsException]) { s.charAt(-1) }
This code behaves much like the previous example. If charAt
throws an instance of IndexOutOfBoundsException
,
intercept
will return that exception. But if charAt
completes normally, or throws a different
exception, intercept
will complete abruptly with an AssertionError
. intercept
returns the
caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside
the exception has the expected values.
Using other assertions
If you are comfortable with assertion mechanisms from other test frameworks, chances
are you can use them with ScalaTest. Any assertion mechanism that indicates a failure with an exception
can be used as is with ScalaTest. For example, to use the assertEquals
methods provided by JUnit or TestNG, simply import them and use them. (You will of course need
to include the relevant JAR file for the framework whose assertions you want to use on either the
classpath or runpath when you run your tests.) Here's an example in which JUnit's assertions are
imported, then used within a ScalaTest suite:
import org.scalatest.Suite import org.junit.Assert._ class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assertEquals(2, sum) assertEquals(4, sum + 2) } def testSubtraction() { val diff = 4 - 1 assertEquals(3, diff) assertEquals(1, diff - 2) } }
Alternatively, you might prefer the more English-like look and more detailed error messages provided by Hamcrest matchers. As with JUnit or TestNG assertions, you can use these in ScalaTest suites simply by importing them and using them. Here's an example:
import org.scalatest.Suite import org.hamcrest.Matchers._ import org.hamcrest.MatcherAssert.assertThat class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assertThat(sum, is(2)) assertThat(sum + 2, is(4)) } def testSubtraction() { val diff = 4 - 1 assertThat(diff, is(3)) assertThat(diff - 2, is(1)) } }
You will, of course, need to include the Hamcrest jar file in your class or run path when you run your ScalaTest suites that use Hamcrest matchers.
You may instead prefer to use the matchers provided by the specs framework, which take greater
advantage of Scala than Hamcrest matchers, since specs is written in Scala. (The Hamcrest library is written in Java.)
To use specs matchers, simply mix in trait org.specs.SpecsMatchers
into your ScalaTest suite. Here's an example:
import org.scalatest.Suite import org.specs.SpecsMatchers class MySuite extends Suite with SpecsMatchers { def testAddition() { val sum = 1 + 1 sum mustBe 2 sum + 2 mustBe 4 } def testSubtraction() { val diff = 4 - 1 diff mustBe 3 diff - 2 mustBe 1 } }
The SpecsMatchers
trait was added to specs version 1.2.8. You will, of course, need to include the specs jar file
in your class or run path when you run your ScalaTest suites that use specs matchers.
Nested suites
A Suite
can refer to a collection of other Suite
s,
which are called nested Suite
s. Those nested Suite
s can in turn have
their own nested Suite
s, and so on. Large test suites can be organized, therefore, as a tree of
nested Suite
s.
This trait's execute
method, in addition to invoking its
test methods, invokes execute
on each of its nested Suite
s.
A List
of a Suite
's nested Suite
s can be obtained by invoking its
nestedSuites
method. If you wish to create a Suite
that serves as a
container for nested Suite
s, whether or not it has test methods of its own, simply override nestedSuites
to return a List
of the nested Suite
s. Because this is a common use case, ScalaTest provides
a convenience SuperSuite
class, which takes a List
of nested Suite
s as a constructor
parameter. Here's an example:
import org.scalatet.Suite class ASuite extends Suite class BSuite extends Suite class CSuite extends Suite class AlphabetSuite extends SuperSuite( List( new ASuite, new BSuite, new CSuite ) )
If you now run AlphabetSuite
, for example from the interpreter:
scala> (new AlphabetSuite).execute()
You will see reports printed to the standard output that indicate nested
suites—ASuite
, BSuite
, and
CSuite
—were run.
Note that Runner
can discover Suite
s automatically, so you need not
necessarily specify SuperSuite
s explicitly. See the documentation
for Runner
for more information.
Test fixtures
A test fixture is objects or other artifacts (such as files, sockets, database
connections, etc.) used by tests to do their work.
If a fixture is used by only one test method, then the definitions of the fixture objects should
be local to the method, such as the objects assigned to sum
and diff
in the
previous MySuite
examples. If multiple methods need to share a fixture, the best approach
is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned
to shared
is used by multiple test methods:
import org.scalatest.Suite class MySuite extends Suite { // Sharing fixture objects via instance variables val shared = 5 def testAddition() { val sum = 2 + 3 assert(sum === shared) } def testSubtraction() { val diff = 7 - 2 assert(diff === shared) } }
In some cases, however, shared mutable fixture objects may be changed by test methods such that
it needs to be recreated or reinitialized before each test. Shared resources such
as files or database connections may also need to
be cleaned up after each test. JUnit offers methods setup
and
tearDown
for this purpose. In ScalaTest, you can use the BeforeAndAfter
trait,
which will be described later, to implement an approach similar to JUnit's setup
and tearDown
, however, this approach often involves reassigning var
s
between tests. Before going that route, you should consider two approaches that
avoid var
s. One approach is to write one or more "create" methods
that return a new instance of a needed object (or a tuple of new instances of
multiple objects) each time it is called. You can then call a create method at the beginning of each
test method that needs the fixture, storing the fixture object or objects in local variables. Here's an example:
import org.scalatest.Suite import scala.collection.mutable.ListBuffer class MySuite extends Suite { // create objects needed by tests and return as a tuple def createFixture = ( new StringBuilder("ScalaTest is "), new ListBuffer[String] ) def testEasy() { val (builder, lbuf) = createFixture builder.append("easy!") assert(builder.toString === "ScalaTest is easy!") assert(lbuf.isEmpty) lbuf += "sweet" } def testFun() { val (builder, lbuf) = createFixture builder.append("fun!") assert(builder.toString === "ScalaTest is fun!") assert(lbuf.isEmpty) } }
Another approach to mutable fixture objects that avoids var
s is to create "with" methods,
which take test code as a function that takes the fixture objects as parameters, and wrap test code in calls to the "with" method. Here's an example:
import org.scalatest.Suite import scala.collection.mutable.ListBuffer class MySuite extends Suite { def withFixture(testFunction: (StringBuilder, ListBuffer[String]) => Unit) { // Create needed mutable objects val sb = new StringBuilder("ScalaTest is ") val lb = new ListBuffer[String] // Invoke the test function, passing in the mutable objects testFunction(sb, lb) } def testEasy() { withFixture { (builder, lbuf) => { builder.append("easy!") assert(builder.toString === "ScalaTest is easy!") assert(lbuf.isEmpty) lbuf += "sweet" } } } def testFun() { withFixture { (builder, lbuf) => { builder.append("fun!") assert(builder.toString === "ScalaTest is fun!") assert(lbuf.isEmpty) } } } }One advantage of this approach compared to the create method approach shown previously is that you can more easily perform cleanup after each test executes. For example, you could create a temporary file before each test, and delete it afterwords, by doing so before and after invoking the test function in a
withTempFile
method. Here's an example:
import org.scalatest.Suite import java.io.FileReader import java.io.FileWriter import java.io.File class MySuite extends Suite { def withTempFile(testFunction: FileReader => Unit) { val FileName = "TempFile.txt" // Set up the temp file needed by the test val writer = new FileWriter(FileName) try { writer.write("Hello, test!") } finally { writer.close() } // Create the reader needed by the test val reader = new FileReader(FileName) try { // Run the test using the temp file testFunction(reader) } finally { // Close and delete the temp file reader.close() val file = new File(FileName) file.delete() } } def testReadingFromTheTempFile() { withTempFile { (reader) => { var builder = new StringBuilder var c = reader.read() while (c != -1) { builder.append(c.toChar) c = reader.read() } assert(builder.toString === "Hello, test!") } } } def testFirstCharOfTheTempFile() { withTempFile { (reader) => { assert(reader.read() === 'H') } } } }
If you are more comfortable with reassigning instance variables, however, you can
instead use the BeforeAndAfter
trait, which provides
methods that will be run before and after each test. BeforeAndAfter
's
beforeEach
method will be run before, and its afterEach
method after, each test (like JUnit's setup
and tearDown
methods, respectively). For example, here's how you'd write the previous
test that uses a temp file with BeforeAndAfter
:
import org.scalatest.BeforeAndAfter import java.io.FileReader import java.io.FileWriter import java.io.File class MySuite extends Suite with BeforeAndAfter { private val FileName = "TempFile.txt" private var reader: FileReader = _ // Set up the temp file needed by the test override def beforeEach() { val writer = new FileWriter(FileName) try { writer.write("Hello, test!") } finally { writer.close() } // Create the reader needed by the test reader = new FileReader(FileName) } // Close and delete the temp file override def afterEach() { reader.close() val file = new File(FileName) file.delete() } def testReadingFromTheTempFile() { var builder = new StringBuilder var c = reader.read() while (c != -1) { builder.append(c.toChar) c = reader.read() } assert(builder.toString === "Hello, test!") } def testFirstCharOfTheTempFile() { assert(reader.read() === 'H') } }
In this example, the instance variable reader
is a var
, so
it can be reinitialized between tests by the beforeEach
method. If you
want to execute code before and after all tests (and nested suites) in a suite, such
as you could do with @BeforeClass
and @AfterClass
annotations in JUnit 4, you can use the beforeAll
and afterAll
methods of BeforeAndAfter
. See the documentation for BeforeAndAfter
for
an example.
Goodies
In some cases you may need to pass information from a suite to its nested suites.
For example, perhaps a main suite needs to open a database connection that is then
used by all of its nested suites. You can accomplish this in ScalaTest by using
goodies, which are passed to execute
as a Map[String, Any]
.
This trait's execute
method calls two other methods, both of which you
can override:
runNestedSuites
- responsible for running this Suite
's nested Suite
srunTests
- responsible for running this Suite
's tests
To pass goodies to nested Suite
s, simply override runNestedSuites
.
Here's an example:
import org.scalatest._ import java.io.FileWriter object Constants { val GoodieKey = "fixture.FileWriter" } class NestedSuite extends Suite { override def execute( testName: Option[String], reporter: Reporter, stopper: Stopper, groupsToInclude: Set[String], groupsToExclude: Set[String], goodies: Map[String, Any], distributor: Option[Distributor] ) { def complain() = fail("Hey, where's my goodie?") if (goodies.contains(Constants.GoodieKey)) { goodies(Constants.GoodieKey) match { case fw: FileWriter => fw.write("hi there\n") // Use the goodie case _ => complain() } } else complain() } } class MainSuite extends SuperSuite(List(new NestedSuite)) { override def runNestedSuites( reporter: Reporter, stopper: Stopper, groupsToInclude: Set[String], groupsToExclude: Set[String], goodies: Map[String, Any], distributor: Option[Distributor] ) { val writer = new FileWriter("fixture.txt") try { val myGoodies = goodies + (Constants.GoodieKey -> writer) super.runNestedSuites(reporter, stopper, groupsToInclude, groupsToExclude, myGoodies, distributor) } finally { writer.close() } } }
In this example, MainSuite
's runNestedSuites method opens a file for writing, then passes
the FileWriter
to its NestedSuite
via the goodies Map
. The NestedSuite
grabs the FileWriter
from the goodies Map
and writes a friendly message to the file.
Test groups
A Suite
's tests may be classified into named groups. When executing
a Suite
, groups of tests can optionally be included and/or excluded. In this
trait's implementation, groups are indicated by annotations attached to the test method. To
create a group, simply define a new Java annotation. (Currently, for annotations to be
visible in Scala programs via Java reflection, the annotations themselves must be written in Java.) For example,
to create a group named SlowAsMolasses
, to use to mark slow tests, you would
write in Java:
import java.lang.annotation.*; @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, ElementType.TYPE}) public @interface SlowAsMolasses {}
Given this new annotation, you could place methods into the SlowAsMolasses
group
like this:
@SlowAsMolasses def testSleeping() = sleep(1000000)
The primary execute method takes two Set[String]
s called groupsToInclude
and
groupsToExclude
. If groupsToInclude
is empty, all tests will be executed
except those those belonging to groups listed in the
groupsToExclude
Set
. If groupsToInclude
is non-empty, only tests
belonging to groups mentioned in groupsToInclude
, and not mentioned in groupsToExclude
,
will be executed.
Ignored tests
Another common use case is that tests must be “temporarily” disabled, with the
good intention of resurrecting the test at a later time. ScalaTest provides an Ignore
annotation for this purpose. You use it like this:
import org.scalatest.Suite import org.scalatest.Ignore class MySuite extends Suite { def testAddition() { val sum = 1 + 1 assert(sum === 2) assert(sum + 2 === 4) } @Ignore def testSubtraction() { val diff = 4 - 1 assert(diff === 3) assert(diff - 2 === 1) } }
If you run this version of MySuite
with:
scala> (new MySuite).execute()
It will run only testAddition
and report that testSubtraction
was ignored. You'll see:
Test Starting - MySuite.testAddition Test Succeeded - MySuite.testAddition Test Ignored - MySuite.testSubtraction
Ignore
is implemented as a group. The execute
method that takes no parameters
adds org.scalatest.Ignore
to the groupsToExclude
Set
it passes to
the primary execute
method, as does Runner
. The only difference between
org.scalatest.Ignore
and the groups you may define and exclude is that ScalaTest reports
ignored tests to the Reporter
. The reason ScalaTest reports ignored tests is as a feeble
attempt to encourage ignored tests to be eventually fixed and added back into the active suite of tests.
Informers
One of the parameters to the primary execute
method is an Informer
, which
will collect and report information about the running suite of tests.
Information about suites and tests that were run, whether tests succeeded or failed,
and tests that were ignored will be passed to the Reporter
as the suite runs.
Most often the reporting done by default by Suite
's methods will be sufficient, but
occasionally you may wish to provide custom information to the Reporter
from a test method.
For this purpose, you can optionally include an Informer
parameter in a test method, and then
pass the extra information to the Informer
via one of its apply
methods. The Informer
will then pass the information to the Reporter
's infoProvided
method.
Here's an example:
import org.scalatest._ class MySuite extends Suite { def testAddition(info: Informer) { assert(1 + 1 === 2) info("Addition seems to work") } }If you run this
Suite
from the interpreter, you will see the message
included in the printed report:
scala> (new MySuite).execute() Test Starting - MySuite.testAddition(Reporter) Info Provided - MySuite.testAddition(Reporter): Addition seems to work Test Succeeded - MySuite.testAddition(Reporter)
Executing suites concurrently
The primary execute
method takes as its last parameter an optional Distributor
. If
a Distributor
is passed in, this trait's implementation of execute
puts its nested
Suite
s into the distributor rather than executing them directly. The caller of execute
is responsible for ensuring that some entity executes the Suite
s placed into the
distributor. The -c
command line parameter to Runner
, for example, will cause
Suite
s put into the Distributor
to be executed concurrently via a pool of threads.
Extensibility
Trait Suite
provides default implementations of its methods that should
be sufficient for most applications, but many methods can be overridden when desired. Here's
a summary of the methods that are intended to be overridden:
execute
- override this method to define custom ways to executes suites of
tests.runTest
- override this method to define custom ways to execute a single named test.testNames
- override this method to specify the Suite
's test names in a custom way.groups
- override this method to specify the Suite
's test groups in a custom way.nestedSuites
- override this method to specify the Suite
's nested Suite
s in a custom way.suiteName
- override this method to specify the Suite
's name in a custom way.expectedTestCount
- override this method to count this Suite
's expected tests in a custom way.
For example, this trait's implementation of testNames
performs reflection to discover methods starting with test
,
and places these in a Set
whose iterator returns the names in alphabetical order. If you wish to run tests in a different
order in a particular Suite
, perhaps because a test named testAlpha
can only succeed after a test named
testBeta
has run, you can override testNames
so that it returns a Set
whose iterator returns
testBeta
before testAlpha
. (This trait's implementation of execute
will invoke tests
in the order they come out of the testNames
Set
iterator.)
Alternatively, you may not like starting your test methods with test
, and prefer using @Test
annotations in
the style of Java's JUnit 4 or TestNG. If so, you can override testNames
to discover tests using either of these two APIs
@Test
annotations, or one of your own invention. (This is in fact
how org.scalatest.junit.JUnit4Suite
and org.scalatest.testng.TestNGSuite
work.)
Moreover, test in ScalaTest does not necessarily mean test method. A test can be anything that can be given a name,
that starts and either succeeds or fails, and can be ignored. In org.scalatest.FunSuite
, for example, tests are represented
as function values. This
approach might look foreign to JUnit users, but may feel more natural to programmers with a functional programming background.
To facilitate this style of writing tests, FunSuite
overrides testNames
, runTest
, and execute
such that you can
define tests as function values.
You can also model existing JUnit 3, JUnit 4, or TestNG tests as suites of tests, thereby incorporating Java tests into a ScalaTest suite.
The "wrapper" classes in packages org.scalatest.junit
and org.scalatest.testng
exist to make this easy. The point here, however, is that
no matter what legacy tests you may have, it is likely you can create or use an existing Suite
subclass that allows you to model those tests
as ScalaTest suites and tests and incorporate them into a ScalaTest suite. You can then write new tests in Scala and continue supporting
older tests in Java.
Method Summary | |
final def
|
execute
(testName : java.lang.String) : Unit
Executes the test specified
testName in this Suite , printing results to the standard output. This method
implementation calls on this Suite the execute method that takes
seven parameters, passing in:
|
final def
|
execute
: Unit
Executes this
Suite , printing results to the standard output. This method
implementation calls on this Suite the execute method that takes
seven parameters, passing in:
|
def
|
execute
(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit
Execute this
Suite . |
def
|
expectedTestCount
(groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String]) : Int
The total number of tests that are expected to run when this
Suite 's execute method is invoked.
This trait's implementation of this method returns the sum of:
|
def
|
groups
: scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
A
Map whose keys are String group names to which tests in this Suite belong, and values
the Set of test names that belong to each group. If this Suite contains no groups, this method returns an empty Map . |
def
|
nestedSuites
: scala.List[Suite]
A
List of this Suite object's nested Suite s. If this Suite contains no nested Suite s,
this method returns an empty List . This trait's implementation of this method returns an empty List . |
protected def
|
runNestedSuites (reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit |
protected def
|
runTest
(testName : java.lang.String, reporter : Reporter, stopper : Stopper, goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Run a test. This trait's implementation uses Java reflection to invoke on this object the test method identified by the passed
testName . |
protected def
|
runTests (testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit |
def
|
suiteName
: java.lang.String
A user-friendly suite name for this
Suite . This trait's
implementation of this method returns the simple name of this object's class. This
trait's implementation of runNestedSuites calls this method to obtain a
name for Report s to pass to the suiteStarting , suiteCompleted ,
and suiteAborted methods of the Reporter . |
def
|
testNames
: scala.collection.immutable.Set[java.lang.String]
An immutable
Set of test names. If this Suite contains no tests, this method returns an empty Set . |
Methods inherited from Assertions | |
assert, assert, assert, assert, convertToEqualizer, intercept, intercept, intercept, expect, expect, fail, fail, fail, fail |
Methods inherited from AnyRef | |
getClass, hashCode, equals, clone, toString, notify, notifyAll, wait, wait, wait, finalize, ==, !=, eq, ne, synchronized |
Methods inherited from Any | |
==, !=, isInstanceOf, asInstanceOf |
Method Details |
def
nestedSuites : scala.List[Suite]
List
of this Suite
object's nested Suite
s. If this Suite
contains no nested Suite
s,
this method returns an empty List
. This trait's implementation of this method returns an empty List
.final
def
execute : Unit
Suite
, printing results to the standard output. This method
implementation calls on this Suite
the execute
method that takes
seven parameters, passing in:
testName
- None
reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose stopRequested
method always returns false
groupsToInclude
- an empty Set[String]
groupsToExclude
- an Set[String]
that contains only one element, "org.scalatest.Ignore"
goodies
- an empty Map[String, Any]
distributor
- None
This method serves as a convenient way to execute a Suite
, especially from within the Scala interpreter.
final
def
execute(testName : java.lang.String) : Unit
testName
in this Suite
, printing results to the standard output. This method
implementation calls on this Suite
the execute
method that takes
seven parameters, passing in:
testName
- Some(testName)
reporter
- a reporter that prints to the standard outputstopper
- a Stopper
whose stopRequested
method always returns false
groupsToInclude
- an empty Set[String]
groupsToExclude
- an empty Set[String]
goodies
- an empty Map[String, Any]
distributor
- None
This method serves as a convenient way to execute a single test, especially from within the Scala interpreter.
def
groups : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
Map
whose keys are String
group names to which tests in this Suite
belong, and values
the Set
of test names that belong to each group. If this Suite
contains no groups, this method returns an empty Map
.
This trait's implementation uses Java reflection to discover any Java annotations attached to its test methods. Each unique
annotation name is considered a group. This trait's implementation, therefore, places one key/value pair into to the
Map
for each unique annotation name discovered through reflection. The value for each group name key will contain
the test method name, as provided via the testNames
method.
Subclasses may override this method to define and/or discover groups in a custom manner, but overriding method implementations
should never return an empty Set
as a value. If a group has no tests, its name should not appear as a key in the
returned Map
.
def
testNames : scala.collection.immutable.Set[java.lang.String]
Set
of test names. If this Suite
contains no tests, this method returns an empty Set
.
This trait's implementation of this method uses Java reflection to discover all public methods whose name starts with "test"
,
which take either nothing or a single Informer
as parameters. For each discovered test method, it assigns a test name
comprised of just the method name if the method takes no parameters, or the method name plus (Informer)
if the
method takes a Informer
. Here are a few method signatures and the names that this trait's implementation assigns them:
def testCat() {} // test name: "testCat" def testCat(Informer) {} // test name: "testCat(Informer)" def testDog() {} // test name: "testDog" def testDog(Informer) {} // test name: "testDog(Informer)" def test() {} // test name: "test" def test(Informer) {} // test name: "test(Informer)"
This trait's implementation of this method returns an immutable Set
of all such names, excluding the name
testName
. The iterator obtained by invoking elements
on this
returned Set
will produce the test names in their natural order, as determined by String
's
compareTo
method.
This trait's implementation of runTests
invokes this method
and calls runTest
for each test name in the order they appear in the returned Set
's iterator.
Although this trait's implementation of this method returns a Set
whose iterator produces String
test names in a well-defined order, the contract of this method does not required a defined order. Subclasses are free to
override this method and return test names in an undefined order, or in a defined order that's different from String
's
natural order.
Subclasses may override this method to produce test names in a custom manner. One potential reason to override testNames
is
to execute tests in a different order, for example, to ensure that tests that depend on other tests are run after those other tests.
Another potential reason to override is to discover test methods annotated with JUnit 4 or TestNG @Test
annotations. Or
a subclass could override this method and return a static, hard-coded Set
of tests, etc.
protected
def
runTest(testName : java.lang.String, reporter : Reporter, stopper : Stopper, goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
testName
.testName -
the name of one test to execute.reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.goodies -
a Map
of key-value pairs that can be used by the executing Suite
of tests.NullPointerException -
if any of testName
, reporter
, stopper
, or goodies
is null
.protected
def
runTests(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Run zero to many of this Suite
's tests.
This method takes a testName
parameter that optionally specifies a test to invoke.
If testName
is Some
, this trait's implementation of this method
invokes runTest
on this object, passing in:
testName
- the String
value of the testName
Option
passed
to this methodreporter
- the Reporter
passed to this method, or one that wraps and delegates to itstopper
- the Stopper
passed to this method, or one that wraps and delegates to itgoodies
- the goodies
Map
passed to this method, or one that wraps and delegates to it
This method takes a Set
of group names that should be included (groupsToInclude
), and a Set
that should be excluded (groupsToExclude
), when deciding which of this Suite
's tests to execute.
If groupsToInclude
is empty, all tests will be executed
except those those belonging to groups listed in the groupsToExclude
Set
. If groupsToInclude
is non-empty, only tests
belonging to groups mentioned in groupsToInclude
, and not mentioned in groupsToExclude
will be executed. However, if testName
is Some
, groupsToInclude
and groupsToExclude
are essentially ignored.
Only if testName
is None
will groupsToInclude
and groupsToExclude
be consulted to
determine which of the tests named in the testNames
Set
should be run. This trait's implementation
behaves this way, and it is part of the general contract of this method, so all overridden forms of this method should behave
this way as well. For more information on trait groups, see the main documentation for this trait.
If testName
is None
, this trait's implementation of this method
invokes testNames
on this Suite
to get a Set
of names of tests to potentially execute.
(A testNames
value of None
essentially acts as a wildcard that means all tests in
this Suite
that are selected by groupsToInclude
and groupsToExclude
should be executed.)
For each test in the testName
Set
, in the order
they appear in the iterator obtained by invoking the elements
method on the Set
, this trait's implementation
of this method checks whether the test should be run based on the groupsToInclude
and groupsToExclude
Set
s.
If so, this implementation invokes runTest
, passing in:
testName
- the String
name of the test to run (which will be one of the names in the testNames
Set
)reporter
- the Reporter
passed to this method, or one that wraps and delegates to itstopper
- the Stopper
passed to this method, or one that wraps and delegates to itgoodies
- the goodies
Map
passed to this method, or one that wraps and delegates to ittestName -
an optional name of one test to execute. If None
, all relevant tests should be executed. I.e., None
acts like a wildcard that means execute all relevant tests in this Suite
.reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.groupsToInclude -
a Set
of String
group names to include in the execution of this Suite
groupsToExclude -
a Set
of String
group names to exclude in the execution of this Suite
goodies -
a Map
of key-value pairs that can be used by the executing Suite
of tests.NullPointerException -
if any of testName
, reporter
, stopper
, groupsToInclude
, groupsToExclude
, or goodies
is null
.
This trait's implementation of this method executes tests
in the manner described in detail in the following paragraphs, but subclasses may override the method to provide different
behavior. The most common reason to override this method is to set up and, if also necessary, to clean up a test fixture
used by all the methods of this Suite
.
def
execute(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit
Suite
.
If testName
is None
, this trait's implementation of this method
calls these two methods on this object in this order:
runNestedSuites(wrappedReporter, stopper, groupsToInclude, groupsToExclude, goodies, distributor)
runTests(testName, wrappedReporter, stopper, groupsToInclude, groupsToExclude, goodies)
If testName
is Some
, then this trait's implementation of this method
calls runTests
, but does not call runNestedSuites
.
testName -
an optional name of one test to execute. If None
, all relevant tests should be executed. I.e., None
acts like a wildcard that means execute all relevant tests in this Suite
.reporter -
the Reporter
to which results will be reportedstopper -
the Stopper
that will be consulted to determine whether to stop execution early.groupsToInclude -
a Set
of String
group names to include in the execution of this Suite
groupsToExclude -
a Set
of String
group names to exclude in the execution of this Suite
goodies -
a Map
of key-value pairs that can be used by the executing Suite
of tests.distributor -
an optional Distributor
, into which to put nested Suite
s to be executed by another entity, such as concurrently by a pool of threads. If None
, nested Suite
s will be executed sequentially.NullPointerException -
if any passed parameter is null
.protected
def
runNestedSuites(reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit
Execute zero to many of this Suite
's nested Suite
s.
If the passed distributor
is None
, this trait's
implementation of this method invokes execute
on each
nested Suite
in the List
obtained by invoking nestedSuites
.
If a nested Suite
's execute
method completes abruptly with an exception, this trait's implementation of this
method reports that the Suite
aborted and attempts to execute the
next nested Suite
.
If the passed distributor
is Some
, this trait's implementation
puts each nested Suite
into the Distributor
contained in the Some
, in the order in which the
Suite
s appear in the List
returned by nestedSuites
.
reporter -
the Reporter
to which results will be reportedNullPointerException -
if reporter
is null
.
def
suiteName : java.lang.String
Suite
. This trait's
implementation of this method returns the simple name of this object's class. This
trait's implementation of runNestedSuites
calls this method to obtain a
name for Report
s to pass to the suiteStarting
, suiteCompleted
,
and suiteAborted
methods of the Reporter
.Suite
object's suite name.
def
expectedTestCount(groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String]) : Int
Suite
's execute
method is invoked.
This trait's implementation of this method returns the sum of:
testNames
List
expecteTestCount
on every nested Suite
contained in
nestedSuites
ScalaTest 0.9.5
|
|