org.scalatest

trait Suite

[source: org/scalatest/Suite.scala]

@scala.serializable

trait Suite
extends Assertions with ExecuteAndRun

A suite of tests. A Suite instance encapsulates a conceptual suite (i.e., a collection) of tests. This trait defines a default way to create and execute tests, which involves writing test methods. This approach will likely suffice in the vast majority of applications, but if desired, subtypes can override certain methods to define other ways to create and execute tests.

The easiest way to use this trait is to use its default approach: Simply create classes that extend Suite and define test methods. Test methods have names of the form testX, where X is some unique, hopefully meaningful, string. A test method must be public and can have any result type, but the most common result type is Unit. Here's an example:

 import org.scalatest.Suite

 class MySuite extends Suite {

   def testAddition() {
     val sum = 1 + 1
     assert(sum === 2)
     assert(sum + 2 === 4)
   }

   def testSubtraction() {
     val diff = 4 - 1
     assert(diff === 3)
     assert(diff - 2 === 1)
   }
 }
 

You run a Suite by invoking on it one of three overloaded execute methods. Two of these execute methods, which print test results to the standard output, are intended to serve as a convenient way to run tests from within the Scala interpreter. For example, to run MySuite from within the Scala interpreter, you could write:

 scala> (new MySuite).execute()
 

And you would see:

 Test Starting - MySuite.testAddition
 Test Succeeded - MySuite.testAddition
 Test Starting - MySuite.testSubtraction
 Test Succeeded - MySuite.testSubtraction
 

Or, to run just the testAddition method, you could write:

 scala> (new MySuite).execute("testAddition")
 

And you would see:

 Test Starting - MySuite.testAddition
 Test Succeeded - MySuite.testAddition
 

The third overloaded execute method takes seven parameters, so it is a bit unwieldy to invoke from within the Scala interpreter. Instead, this execute method is intended to be invoked indirectly by a test runner, such as org.scalatest.tools.Runner or an IDE. See the documentation for Runner for more detail.

Assertions and ===

Inside test methods, you can write assertions by invoking assert and passing in a Boolean expression, such as:

 val left = 2
 val right = 1
 assert(left == right)
 

If the passed expression is true, assert will return normally. If false, assert will complete abruptly with an AssertionError. This exception is usually not caught by the test method, which means the test method itself will complete abruptly by throwing the AssertionError. Any test method that completes abruptly with an AssertionError or any Exception is considered a failed test. A test method that returns normally is considered a successful test.

If you pass a Boolean expression to assert, a failed assertion will be reported, but without reporting the left and right values. You can alternatively encode these values in a String passed as a second argument to assert, as in:

 val left = 2
 val right = 1
 assert(left == right, left + " did not equal " + right)
 

Using this form of assert, the failure report will include the left and right values, thereby helping you debug the problem. However, Suite provides the === operator to make this easier. You use it like this:

 val left = 2
 val right = 1
 assert(left === right)
 

Because you use === here instead of ==, the failure report will include the left and right values. For example, the detail message in the thrown AssertionErrorm from the assert shown previously will include, "2 did not equal 1". From this message you will know that the operand on the left had the value 2, and the operand on the right had the value 1.

If you're familiar with JUnit, you would use === in a ScalaTest Suite where you'd use assertEquals in a JUnit TestCase. The === operator is made possible by an implicit conversion from Any to Equalizer. If you're curious to understand the mechanics, see the documentation for Equalizer and Suite's convertToEqualizer method.

Expected results

Although === provides a natural, readable extension to Scala's assert mechanism, as the operands become lengthy, the code becomes less readable. In addition, the === comparison doesn't distinguish between actual and expected values. The operands are just called left and right, because if one were named expected and the other actual, it would be difficult for people to remember which was which. To help with these limitations of assertions, Suite includes a method called expect that can be used as an alternative to assert with ===. To use expect, you place the expected value in parentheses after expect, and follow that by code contained inside curly braces that results in a value that you expect should equal the expected value. For example:
 val a = 5
 val b = 2
 expect(2) {
   a - b
 }
 

In this case, the expected value is 2, and the code being tested is a - b. This expectation will fail, and the detail message in the AssertionError will read, "Expected 2, but got 3."

Intercepted exceptions

Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit style, like this:

 val s = "hi"
 try {
   s.charAt(-1)
   fail()
 }
 catch {
   case e: IndexOutOfBoundsException => // Expected, so continue
 }
 

If charAt throws IndexOutOfBoundsException as left, control will transfer to the catch case, which does nothing. If, however, charAt fails to throw an exception, the next statement, fail(), will be executed. The fail method always completes abruptly with an AssertionError, thereby signaling a failed test.

To make this common use case easier to express and read, Suite provides an intercept method. You use it like this:

 val s = "hi"
 intercept(classOf[IndexOutOfBoundsException]) {
   s.charAt(-1)
 }
 

This code behaves much like the previous example. If charAt throws an instance of IndexOutOfBoundsException, intercept will return that exception. But if charAt completes normally, or throws a different exception, intercept will complete abruptly with an AssertionError. intercept returns the caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside the exception has the expected values.

Using other assertions

If you are comfortable with assertion mechanisms from other test frameworks, chances are you can use them with ScalaTest. Any assertion mechanism that indicates a failure with an exception can be used as is with ScalaTest. For example, to use the assertEquals methods provided by JUnit or TestNG, simply import them and use them. (You will of course need to include the relevant JAR file for the framework whose assertions you want to use on either the classpath or runpath when you run your tests.) Here's an example in which JUnit's assertions are imported, then used within a ScalaTest suite:

 import org.scalatest.Suite
 import org.junit.Assert._

 class MySuite extends Suite {

   def testAddition() {
     val sum = 1 + 1
     assertEquals(2, sum)
     assertEquals(4, sum + 2)
   }

   def testSubtraction() {
     val diff = 4 - 1
     assertEquals(3, diff)
     assertEquals(1, diff - 2)
   }
 }
 

Alternatively, you might prefer the more English-like look and more detailed error messages provided by Hamcrest matchers. As with JUnit or TestNG assertions, you can use these in ScalaTest suites simply by importing them and using them. Here's an example:

 import org.scalatest.Suite
 import org.hamcrest.Matchers._
 import org.hamcrest.MatcherAssert.assertThat

 class MySuite extends Suite {

   def testAddition() {
     val sum = 1 + 1
     assertThat(sum, is(2))
     assertThat(sum + 2, is(4))
   }

   def testSubtraction() {
     val diff = 4 - 1
     assertThat(diff, is(3))
     assertThat(diff - 2, is(1))
   }
 }
 

You will, of course, need to include the Hamcrest jar file in your class or run path when you run your ScalaTest suites that use Hamcrest matchers.

You may instead prefer to use the matchers provided by the specs framework, which take greater advantage of Scala than Hamcrest matchers, since specs is written in Scala. (The Hamcrest library is written in Java.) To use specs matchers, simply mix in trait org.specs.SpecsMatchers into your ScalaTest suite. Here's an example:

 import org.scalatest.Suite
 import org.specs.SpecsMatchers

 class MySuite extends Suite with SpecsMatchers {

   def testAddition() {
     val sum = 1 + 1
     sum mustBe 2
     sum + 2 mustBe 4
   }

   def testSubtraction() {
     val diff = 4 - 1
     diff mustBe 3
     diff - 2 mustBe 1
   }
 }
 

The SpecsMatchers trait was added to specs version 1.2.8. You will, of course, need to include the specs jar file in your class or run path when you run your ScalaTest suites that use specs matchers.

Nested suites

A Suite can refer to a collection of other Suites, which are called nested Suites. Those nested Suites can in turn have their own nested Suites, and so on. Large test suites can be organized, therefore, as a tree of nested Suites. This trait's execute method, in addition to invoking its test methods, invokes execute on each of its nested Suites.

A List of a Suite's nested Suites can be obtained by invoking its nestedSuites method. If you wish to create a Suite that serves as a container for nested Suites, whether or not it has test methods of its own, simply override nestedSuites to return a List of the nested Suites. Because this is a common use case, ScalaTest provides a convenience SuperSuite class, which takes a List of nested Suites as a constructor parameter. Here's an example:

 import org.scalatet.Suite

 class ASuite extends Suite
 class BSuite extends Suite
 class CSuite extends Suite

 class AlphabetSuite extends SuperSuite(
   List(
     new ASuite,
     new BSuite,
     new CSuite
   )
 )
 

If you now run AlphabetSuite, for example from the interpreter:

 scala> (new AlphabetSuite).execute()
 

You will see reports printed to the standard output that indicate nested suites—ASuite, BSuite, and CSuite—were run.

Note that Runner can discover Suites automatically, so you need not necessarily specify SuperSuites explicitly. See the documentation for Runner for more information.

Test fixtures

A test fixture is objects or other artifacts (such as files, sockets, database connections, etc.) used by tests to do their work. If a fixture is used by only one test method, then the definitions of the fixture objects should be local to the method, such as the objects assigned to sum and diff in the previous MySuite examples. If multiple methods need to share a fixture, the best approach is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned to shared is used by multiple test methods:

 import org.scalatest.Suite

 class MySuite extends Suite {

   // Sharing fixture objects via instance variables
   val shared = 5

   def testAddition() {
     val sum = 2 + 3
     assert(sum === shared)
   }

   def testSubtraction() {
     val diff = 7 - 2
     assert(diff === shared)
   }
 }
 

In some cases, however, shared mutable fixture objects may be changed by test methods such that it needs to be recreated or reinitialized before each test. Shared resources such as files or database connections may also need to be cleaned up after each test. JUnit offers methods setup and tearDown for this purpose. In ScalaTest, you can use the BeforeAndAfter trait, which will be described later, to implement an approach similar to JUnit's setup and tearDown, however, this approach often involves reassigning vars between tests. Before going that route, you should consider two approaches that avoid vars. One approach is to write one or more "create" methods that return a new instance of a needed object (or a tuple of new instances of multiple objects) each time it is called. You can then call a create method at the beginning of each test method that needs the fixture, storing the fixture object or objects in local variables. Here's an example:

 import org.scalatest.Suite
 import scala.collection.mutable.ListBuffer

 class MySuite extends Suite {

   // create objects needed by tests and return as a tuple
   def createFixture = (
     new StringBuilder("ScalaTest is "),
     new ListBuffer[String]
   )

   def testEasy() {
     val (builder, lbuf) = createFixture
     builder.append("easy!")
     assert(builder.toString === "ScalaTest is easy!")
     assert(lbuf.isEmpty)
     lbuf += "sweet"
   }

   def testFun() {
     val (builder, lbuf) = createFixture
     builder.append("fun!")
     assert(builder.toString === "ScalaTest is fun!")
     assert(lbuf.isEmpty)
   }
 }
 

Another approach to mutable fixture objects that avoids vars is to create "with" methods, which take test code as a function that takes the fixture objects as parameters, and wrap test code in calls to the "with" method. Here's an example:

 import org.scalatest.Suite
 import scala.collection.mutable.ListBuffer

 class MySuite extends Suite {

   def withFixture(testFunction: (StringBuilder, ListBuffer[String]) => Unit) {

     // Create needed mutable objects
     val sb = new StringBuilder("ScalaTest is ")
     val lb = new ListBuffer[String]

     // Invoke the test function, passing in the mutable objects
     testFunction(sb, lb)
   }

   def testEasy() {
     withFixture {
       (builder, lbuf) => {
         builder.append("easy!")
         assert(builder.toString === "ScalaTest is easy!")
         assert(lbuf.isEmpty)
         lbuf += "sweet"
       }
     }
   }

   def testFun() {
     withFixture {
       (builder, lbuf) => {
         builder.append("fun!")
         assert(builder.toString === "ScalaTest is fun!")
         assert(lbuf.isEmpty)
       }
     }
   }
 }
 
One advantage of this approach compared to the create method approach shown previously is that you can more easily perform cleanup after each test executes. For example, you could create a temporary file before each test, and delete it afterwords, by doing so before and after invoking the test function in a withTempFile method. Here's an example:
 import org.scalatest.Suite
 import java.io.FileReader
 import java.io.FileWriter
 import java.io.File
 
 class MySuite extends Suite {
 
   def withTempFile(testFunction: FileReader => Unit) {
 
     val FileName = "TempFile.txt"
  
     // Set up the temp file needed by the test
     val writer = new FileWriter(FileName)
     try {
       writer.write("Hello, test!")
     }
     finally {
       writer.close()
     }
  
     // Create the reader needed by the test
     val reader = new FileReader(FileName)
  
     try {
       // Run the test using the temp file
       testFunction(reader)
     }
     finally {
       // Close and delete the temp file
       reader.close()
       val file = new File(FileName)
       file.delete()
     }
   }
 
   def testReadingFromTheTempFile() {
     withTempFile {
       (reader) => {
         var builder = new StringBuilder
         var c = reader.read()
         while (c != -1) {
           builder.append(c.toChar)
           c = reader.read()
         }
         assert(builder.toString === "Hello, test!")
       }
     }
   }
 
   def testFirstCharOfTheTempFile() {
     withTempFile {
       (reader) => {
         assert(reader.read() === 'H')
       }
     }
   }
 }
 

If you are more comfortable with reassigning instance variables, however, you can instead use the BeforeAndAfter trait, which provides methods that will be run before and after each test. BeforeAndAfter's beforeEach method will be run before, and its afterEach method after, each test (like JUnit's setup and tearDown methods, respectively). For example, here's how you'd write the previous test that uses a temp file with BeforeAndAfter:

 import org.scalatest.BeforeAndAfter
 import java.io.FileReader
 import java.io.FileWriter
 import java.io.File

 class MySuite extends Suite with BeforeAndAfter {

   private val FileName = "TempFile.txt"
   private var reader: FileReader = _

   // Set up the temp file needed by the test
   override def beforeEach() {
     val writer = new FileWriter(FileName)
     try {
       writer.write("Hello, test!")
     }
     finally {
       writer.close()
     }

     // Create the reader needed by the test
     reader = new FileReader(FileName)
   }

   // Close and delete the temp file
   override def afterEach() {
     reader.close()
     val file = new File(FileName)
     file.delete()
   }

   def testReadingFromTheTempFile() {
     var builder = new StringBuilder
     var c = reader.read()
     while (c != -1) {
       builder.append(c.toChar)
       c = reader.read()
     }
     assert(builder.toString === "Hello, test!")
   }

   def testFirstCharOfTheTempFile() {
     assert(reader.read() === 'H')
   }
 }
 

In this example, the instance variable reader is a var, so it can be reinitialized between tests by the beforeEach method. If you want to execute code before and after all tests (and nested suites) in a suite, such as you could do with @BeforeClass and @AfterClass annotations in JUnit 4, you can use the beforeAll and afterAll methods of BeforeAndAfter. See the documentation for BeforeAndAfter for an example.

Goodies

In some cases you may need to pass information from a suite to its nested suites. For example, perhaps a main suite needs to open a database connection that is then used by all of its nested suites. You can accomplish this in ScalaTest by using goodies, which are passed to execute as a Map[String, Any]. This trait's execute method calls two other methods, both of which you can override:

To pass goodies to nested Suites, simply override runNestedSuites. Here's an example:

 import org.scalatest._
 import java.io.FileWriter

 object Constants {
   val GoodieKey = "fixture.FileWriter"
 }

 class NestedSuite extends Suite {

   override def execute(
     testName: Option[String],
     reporter: Reporter,
     stopper: Stopper,
     groupsToInclude: Set[String],
     groupsToExclude: Set[String],
     goodies: Map[String, Any],
     distributor: Option[Distributor]
   ) {
     def complain() = fail("Hey, where's my goodie?")

     if (goodies.contains(Constants.GoodieKey)) {
       goodies(Constants.GoodieKey) match {
         case fw: FileWriter => fw.write("hi there\n") // Use the goodie
         case _ => complain()
       }
     }
     else complain()
   }
 }

 class MainSuite extends SuperSuite(List(new NestedSuite)) {

   override def runNestedSuites(
     reporter: Reporter,
     stopper: Stopper,
     groupsToInclude: Set[String],
     groupsToExclude: Set[String],
     goodies: Map[String, Any],
     distributor: Option[Distributor]
   ) {
     val writer = new FileWriter("fixture.txt")
     try {
       val myGoodies = goodies + (Constants.GoodieKey -> writer)
       super.runNestedSuites(reporter, stopper, groupsToInclude, groupsToExclude, myGoodies, distributor)
     }
     finally {
       writer.close()
     }
   }
 }
 

In this example, MainSuite's runNestedSuites method opens a file for writing, then passes the FileWriter to its NestedSuite via the goodies Map. The NestedSuite grabs the FileWriter from the goodies Map and writes a friendly message to the file.

Test groups

A Suite's tests may be classified into named groups. When executing a Suite, groups of tests can optionally be included and/or excluded. In this trait's implementation, groups are indicated by annotations attached to the test method. To create a group, simply define a new Java annotation. (Currently, for annotations to be visible in Scala programs via Java reflection, the annotations themselves must be written in Java.) For example, to create a group named SlowAsMolasses, to use to mark slow tests, you would write in Java:

 import java.lang.annotation.*; 
 
 @Retention(RetentionPolicy.RUNTIME)
 @Target({ElementType.METHOD, ElementType.TYPE})
 public @interface SlowAsMolasses {}
 

Given this new annotation, you could place methods into the SlowAsMolasses group like this:

 @SlowAsMolasses
 def testSleeping() = sleep(1000000)
 

The primary execute method takes two Set[String]s called groupsToInclude and groupsToExclude. If groupsToInclude is empty, all tests will be executed except those those belonging to groups listed in the groupsToExclude Set. If groupsToInclude is non-empty, only tests belonging to groups mentioned in groupsToInclude, and not mentioned in groupsToExclude, will be executed.

Ignored tests

Another common use case is that tests must be “temporarily” disabled, with the good intention of resurrecting the test at a later time. ScalaTest provides an Ignore annotation for this purpose. You use it like this:

 import org.scalatest.Suite
 import org.scalatest.Ignore

 class MySuite extends Suite {

   def testAddition() {
     val sum = 1 + 1
     assert(sum === 2)
     assert(sum + 2 === 4)
   }

   @Ignore
   def testSubtraction() {
     val diff = 4 - 1
     assert(diff === 3)
     assert(diff - 2 === 1)
   }
 }
 

If you run this version of MySuite with:

 scala> (new MySuite).execute()
 

It will run only testAddition and report that testSubtraction was ignored. You'll see:

 Test Starting - MySuite.testAddition
 Test Succeeded - MySuite.testAddition
 Test Ignored - MySuite.testSubtraction
 

Ignore is implemented as a group. The execute method that takes no parameters adds org.scalatest.Ignore to the groupsToExclude Set it passes to the primary execute method, as does Runner. The only difference between org.scalatest.Ignore and the groups you may define and exclude is that ScalaTest reports ignored tests to the Reporter. The reason ScalaTest reports ignored tests is as a feeble attempt to encourage ignored tests to be eventually fixed and added back into the active suite of tests.

Informers

One of the parameters to the primary execute method is an Informer, which will collect and report information about the running suite of tests. Information about suites and tests that were run, whether tests succeeded or failed, and tests that were ignored will be passed to the Reporter as the suite runs. Most often the reporting done by default by Suite's methods will be sufficient, but occasionally you may wish to provide custom information to the Reporter from a test method. For this purpose, you can optionally include an Informer parameter in a test method, and then pass the extra information to the Informer via one of its apply methods. The Informer will then pass the information to the Reporter's infoProvided method. Here's an example:

 import org.scalatest._
 
 class MySuite extends Suite {
   def testAddition(info: Informer) {
     assert(1 + 1 === 2)
     info("Addition seems to work")
   }
 }
 
If you run this Suite from the interpreter, you will see the message included in the printed report:
 scala> (new MySuite).execute()
 Test Starting - MySuite.testAddition(Reporter)
 Info Provided - MySuite.testAddition(Reporter): Addition seems to work
 Test Succeeded - MySuite.testAddition(Reporter)
 

Executing suites concurrently

The primary execute method takes as its last parameter an optional Distributor. If a Distributor is passed in, this trait's implementation of execute puts its nested Suites into the distributor rather than executing them directly. The caller of execute is responsible for ensuring that some entity executes the Suites placed into the distributor. The -c command line parameter to Runner, for example, will cause Suites put into the Distributor to be executed concurrently via a pool of threads.

Extensibility

Trait Suite provides default implementations of its methods that should be sufficient for most applications, but many methods can be overridden when desired. Here's a summary of the methods that are intended to be overridden:

For example, this trait's implementation of testNames performs reflection to discover methods starting with test, and places these in a Set whose iterator returns the names in alphabetical order. If you wish to run tests in a different order in a particular Suite, perhaps because a test named testAlpha can only succeed after a test named testBeta has run, you can override testNames so that it returns a Set whose iterator returns testBeta before testAlpha. (This trait's implementation of execute will invoke tests in the order they come out of the testNames Set iterator.)

Alternatively, you may not like starting your test methods with test, and prefer using @Test annotations in the style of Java's JUnit 4 or TestNG. If so, you can override testNames to discover tests using either of these two APIs @Test annotations, or one of your own invention. (This is in fact how org.scalatest.junit.JUnit4Suite and org.scalatest.testng.TestNGSuite work.)

Moreover, test in ScalaTest does not necessarily mean test method. A test can be anything that can be given a name, that starts and either succeeds or fails, and can be ignored. In org.scalatest.FunSuite, for example, tests are represented as function values. This approach might look foreign to JUnit users, but may feel more natural to programmers with a functional programming background. To facilitate this style of writing tests, FunSuite overrides testNames, runTest, and execute such that you can define tests as function values.

You can also model existing JUnit 3, JUnit 4, or TestNG tests as suites of tests, thereby incorporating Java tests into a ScalaTest suite. The "wrapper" classes in packages org.scalatest.junit and org.scalatest.testng exist to make this easy. The point here, however, is that no matter what legacy tests you may have, it is likely you can create or use an existing Suite subclass that allows you to model those tests as ScalaTest suites and tests and incorporate them into a ScalaTest suite. You can then write new tests in Scala and continue supporting older tests in Java.

Author
Bill Venners
Direct Known Subclasses:
FunSuite, ImpSuite, Spec, SuperSuite, JUnit3Suite, TestNGSuite

Method Summary
final def execute (testName : java.lang.String) : Unit
Executes the test specified testName in this Suite, printing results to the standard output. This method implementation calls on this Suite the execute method that takes seven parameters, passing in:
  • testName - Some(testName)
  • reporter - a reporter that prints to the standard output
  • stopper - a Stopper whose stopRequested method always returns false
  • groupsToInclude - an empty Set[String]
  • groupsToExclude - an empty Set[String]
  • goodies - an empty Map[String, Any]
  • distributor - None
final def execute : Unit
Executes this Suite, printing results to the standard output. This method implementation calls on this Suite the execute method that takes seven parameters, passing in:
  • testName - None
  • reporter - a reporter that prints to the standard output
  • stopper - a Stopper whose stopRequested method always returns false
  • groupsToInclude - an empty Set[String]
  • groupsToExclude - an Set[String] that contains only one element, "org.scalatest.Ignore"
  • goodies - an empty Map[String, Any]
  • distributor - None
def execute (testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit
Execute this Suite.
def expectedTestCount (groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String]) : Int
The total number of tests that are expected to run when this Suite's execute method is invoked. This trait's implementation of this method returns the sum of:
  • the size of the testNames List
  • the sum of the values obtained by invoking expecteTestCount on every nested Suite contained in nestedSuites
def groups : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
A Map whose keys are String group names to which tests in this Suite belong, and values the Set of test names that belong to each group. If this Suite contains no groups, this method returns an empty Map.
def nestedSuites : scala.List[Suite]
A List of this Suite object's nested Suites. If this Suite contains no nested Suites, this method returns an empty List. This trait's implementation of this method returns an empty List.
protected def runNestedSuites (reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit
protected def runTest (testName : java.lang.String, reporter : Reporter, stopper : Stopper, goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Run a test. This trait's implementation uses Java reflection to invoke on this object the test method identified by the passed testName.
protected def runTests (testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
def suiteName : java.lang.String
A user-friendly suite name for this Suite. This trait's implementation of this method returns the simple name of this object's class. This trait's implementation of runNestedSuites calls this method to obtain a name for Reports to pass to the suiteStarting, suiteCompleted, and suiteAborted methods of the Reporter.
def testNames : scala.collection.immutable.Set[java.lang.String]
An immutable Set of test names. If this Suite contains no tests, this method returns an empty Set.
Methods inherited from Assertions
assert, assert, assert, assert, convertToEqualizer, intercept, intercept, intercept, expect, expect, fail, fail, fail, fail
Methods inherited from AnyRef
getClass, hashCode, equals, clone, toString, notify, notifyAll, wait, wait, wait, finalize, ==, !=, eq, ne, synchronized
Methods inherited from Any
==, !=, isInstanceOf, asInstanceOf
Method Details
def nestedSuites : scala.List[Suite]
A List of this Suite object's nested Suites. If this Suite contains no nested Suites, this method returns an empty List. This trait's implementation of this method returns an empty List.

final def execute : Unit
Executes this Suite, printing results to the standard output. This method implementation calls on this Suite the execute method that takes seven parameters, passing in:
  • testName - None
  • reporter - a reporter that prints to the standard output
  • stopper - a Stopper whose stopRequested method always returns false
  • groupsToInclude - an empty Set[String]
  • groupsToExclude - an Set[String] that contains only one element, "org.scalatest.Ignore"
  • goodies - an empty Map[String, Any]
  • distributor - None

This method serves as a convenient way to execute a Suite, especially from within the Scala interpreter.


final def execute(testName : java.lang.String) : Unit
Executes the test specified testName in this Suite, printing results to the standard output. This method implementation calls on this Suite the execute method that takes seven parameters, passing in:
  • testName - Some(testName)
  • reporter - a reporter that prints to the standard output
  • stopper - a Stopper whose stopRequested method always returns false
  • groupsToInclude - an empty Set[String]
  • groupsToExclude - an empty Set[String]
  • goodies - an empty Map[String, Any]
  • distributor - None

This method serves as a convenient way to execute a single test, especially from within the Scala interpreter.


def groups : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
A Map whose keys are String group names to which tests in this Suite belong, and values the Set of test names that belong to each group. If this Suite contains no groups, this method returns an empty Map.

This trait's implementation uses Java reflection to discover any Java annotations attached to its test methods. Each unique annotation name is considered a group. This trait's implementation, therefore, places one key/value pair into to the Map for each unique annotation name discovered through reflection. The value for each group name key will contain the test method name, as provided via the testNames method.

Subclasses may override this method to define and/or discover groups in a custom manner, but overriding method implementations should never return an empty Set as a value. If a group has no tests, its name should not appear as a key in the returned Map.


def testNames : scala.collection.immutable.Set[java.lang.String]
An immutable Set of test names. If this Suite contains no tests, this method returns an empty Set.

This trait's implementation of this method uses Java reflection to discover all public methods whose name starts with "test", which take either nothing or a single Informer as parameters. For each discovered test method, it assigns a test name comprised of just the method name if the method takes no parameters, or the method name plus (Informer) if the method takes a Informer. Here are a few method signatures and the names that this trait's implementation assigns them:

  def testCat() {}         // test name: "testCat"
  def testCat(Informer) {} // test name: "testCat(Informer)"
  def testDog() {}         // test name: "testDog"
  def testDog(Informer) {} // test name: "testDog(Informer)"
  def test() {}            // test name: "test"
  def test(Informer) {}    // test name: "test(Informer)"
  

This trait's implementation of this method returns an immutable Set of all such names, excluding the name testName. The iterator obtained by invoking elements on this returned Set will produce the test names in their natural order, as determined by String's compareTo method.

This trait's implementation of runTests invokes this method and calls runTest for each test name in the order they appear in the returned Set's iterator. Although this trait's implementation of this method returns a Set whose iterator produces String test names in a well-defined order, the contract of this method does not required a defined order. Subclasses are free to override this method and return test names in an undefined order, or in a defined order that's different from String's natural order.

Subclasses may override this method to produce test names in a custom manner. One potential reason to override testNames is to execute tests in a different order, for example, to ensure that tests that depend on other tests are run after those other tests. Another potential reason to override is to discover test methods annotated with JUnit 4 or TestNG @Test annotations. Or a subclass could override this method and return a static, hard-coded Set of tests, etc.


protected def runTest(testName : java.lang.String, reporter : Reporter, stopper : Stopper, goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Run a test. This trait's implementation uses Java reflection to invoke on this object the test method identified by the passed testName.
Parameters
testName - the name of one test to execute.
reporter - the Reporter to which results will be reported
stopper - the Stopper that will be consulted to determine whether to stop execution early.
goodies - a Map of key-value pairs that can be used by the executing Suite of tests.
Throws
NullPointerException - if any of testName, reporter, stopper, or goodies is null.
Overrides
ExecuteAndRun.runTest

protected def runTests(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit

Run zero to many of this Suite's tests.

This method takes a testName parameter that optionally specifies a test to invoke. If testName is Some, this trait's implementation of this method invokes runTest on this object, passing in:

  • testName - the String value of the testName Option passed to this method
  • reporter - the Reporter passed to this method, or one that wraps and delegates to it
  • stopper - the Stopper passed to this method, or one that wraps and delegates to it
  • goodies - the goodies Map passed to this method, or one that wraps and delegates to it

This method takes a Set of group names that should be included (groupsToInclude), and a Set that should be excluded (groupsToExclude), when deciding which of this Suite's tests to execute. If groupsToInclude is empty, all tests will be executed except those those belonging to groups listed in the groupsToExclude Set. If groupsToInclude is non-empty, only tests belonging to groups mentioned in groupsToInclude, and not mentioned in groupsToExclude will be executed. However, if testName is Some, groupsToInclude and groupsToExclude are essentially ignored. Only if testName is None will groupsToInclude and groupsToExclude be consulted to determine which of the tests named in the testNames Set should be run. This trait's implementation behaves this way, and it is part of the general contract of this method, so all overridden forms of this method should behave this way as well. For more information on trait groups, see the main documentation for this trait.

If testName is None, this trait's implementation of this method invokes testNames on this Suite to get a Set of names of tests to potentially execute. (A testNames value of None essentially acts as a wildcard that means all tests in this Suite that are selected by groupsToInclude and groupsToExclude should be executed.) For each test in the testName Set, in the order they appear in the iterator obtained by invoking the elements method on the Set, this trait's implementation of this method checks whether the test should be run based on the groupsToInclude and groupsToExclude Sets. If so, this implementation invokes runTest, passing in:

  • testName - the String name of the test to run (which will be one of the names in the testNames Set)
  • reporter - the Reporter passed to this method, or one that wraps and delegates to it
  • stopper - the Stopper passed to this method, or one that wraps and delegates to it
  • goodies - the goodies Map passed to this method, or one that wraps and delegates to it
Parameters
testName - an optional name of one test to execute. If None, all relevant tests should be executed. I.e., None acts like a wildcard that means execute all relevant tests in this Suite.
reporter - the Reporter to which results will be reported
stopper - the Stopper that will be consulted to determine whether to stop execution early.
groupsToInclude - a Set of String group names to include in the execution of this Suite
groupsToExclude - a Set of String group names to exclude in the execution of this Suite
goodies - a Map of key-value pairs that can be used by the executing Suite of tests.
Throws
NullPointerException - if any of testName, reporter, stopper, groupsToInclude, groupsToExclude, or goodies is null. This trait's implementation of this method executes tests in the manner described in detail in the following paragraphs, but subclasses may override the method to provide different behavior. The most common reason to override this method is to set up and, if also necessary, to clean up a test fixture used by all the methods of this Suite.

def execute(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit
Execute this Suite.

If testName is None, this trait's implementation of this method calls these two methods on this object in this order:

  1. runNestedSuites(wrappedReporter, stopper, groupsToInclude, groupsToExclude, goodies, distributor)
  2. runTests(testName, wrappedReporter, stopper, groupsToInclude, groupsToExclude, goodies)

If testName is Some, then this trait's implementation of this method calls runTests, but does not call runNestedSuites.

Parameters
testName - an optional name of one test to execute. If None, all relevant tests should be executed. I.e., None acts like a wildcard that means execute all relevant tests in this Suite.
reporter - the Reporter to which results will be reported
stopper - the Stopper that will be consulted to determine whether to stop execution early.
groupsToInclude - a Set of String group names to include in the execution of this Suite
groupsToExclude - a Set of String group names to exclude in the execution of this Suite
goodies - a Map of key-value pairs that can be used by the executing Suite of tests.
distributor - an optional Distributor, into which to put nested Suites to be executed by another entity, such as concurrently by a pool of threads. If None, nested Suites will be executed sequentially.
Throws
NullPointerException - if any passed parameter is null.
Overrides
ExecuteAndRun.execute

protected def runNestedSuites(reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any], distributor : scala.Option[Distributor]) : Unit

Execute zero to many of this Suite's nested Suites.

If the passed distributor is None, this trait's implementation of this method invokes execute on each nested Suite in the List obtained by invoking nestedSuites. If a nested Suite's execute method completes abruptly with an exception, this trait's implementation of this method reports that the Suite aborted and attempts to execute the next nested Suite. If the passed distributor is Some, this trait's implementation puts each nested Suite into the Distributor contained in the Some, in the order in which the Suites appear in the List returned by nestedSuites.

Parameters
reporter - the Reporter to which results will be reported
Throws
NullPointerException - if reporter is null.

def suiteName : java.lang.String
A user-friendly suite name for this Suite. This trait's implementation of this method returns the simple name of this object's class. This trait's implementation of runNestedSuites calls this method to obtain a name for Reports to pass to the suiteStarting, suiteCompleted, and suiteAborted methods of the Reporter.
Returns
this Suite object's suite name.

def expectedTestCount(groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String]) : Int
The total number of tests that are expected to run when this Suite's execute method is invoked. This trait's implementation of this method returns the sum of:
  • the size of the testNames List
  • the sum of the values obtained by invoking expecteTestCount on every nested Suite contained in nestedSuites


Copyright (C) 2001-2009 Artima, Inc. All rights reserved.