org.scalatest

trait Spec

[source: org/scalatest/Spec.scala]

trait Spec
extends Suite
Trait that facilitates a “behavior-driven” style of development (BDD), in which tests are combined with text that specifies the behavior the tests verify. Here's an example Spec:
 import org.scalatest.Spec
 import scala.collection.mutable.Stack

 class StackSpec extends Spec {

   describe("A Stack") {

     it("should pop values in last-in-first-out order") {
       val stack = new Stack[Int]
       stack.push(1)
       stack.push(2)
       assert(stack.pop() === 2)
       assert(stack.pop() === 1)
     }

     it("should throw NoSuchElementException if an empty stack is popped") {
       val emptyStack = new Stack[String]
       intercept[NoSuchElementException] {
         emptyStack.pop()
       }
     }
   }
 }
 

A Spec contains describers and examples. You define a describer with describe, and a example with it. Both describe and it are methods, defined in Spec, which will be invoked by the primary constructor of StackSpec. A describer names, or gives more information about, the subject (class or other entity) you are specifying and testing. In the above example, "A Stack" is the subject under specification and test. With each example you provide a string (the spec text) that specifies one bit of behavior of the subject, and a block of code that tests that behavior. You place the spec text between the parentheses, followed by the test code between curly braces. The test code will be wrapped up as a function passed as a by-name parameter to it, which will register the test for later execution.

When you execute a Spec, it will send SpecReports to the Reporter. ScalaTest's built-in reporters will report these SpecReports in such a way that the output is easy to read as an informal specification of the entity under test. For example, if you ran StackSpec from within the Scala interpreter:

 scala> (new StackSpec).execute()
 

You would see:

 A Stack
 - should pop values in last-in-first-out order
 - should throw NoSuchElementException if an empty stack is popped
 

Test fixtures

A test fixture is objects or other artifacts (such as files, sockets, database connections, etc.) used by tests to do their work. You can use fixtures in Specs with the same approaches suggested for Suite in its documentation. The same text that appears in the test fixture section of Suite's documentation is repeated here, with examples changed from Suite to Spec.

If a fixture is used by only one test, then the definitions of the fixture objects should be local to the test function, such as the objects assigned to stack and emptyStack in the previous StackSpec examples. If multiple tests need to share a fixture, the best approach is to assign them to instance variables. Here's a (very contrived) example, in which the object assigned to shared is used by multiple test functions:

 import org.scalatest.Spec

 class ArithmeticSpec extends Spec {

   // Sharing fixture objects via instance variables
   val shared = 5

   it("should add correctly") {
     val sum = 2 + 3
     assert(sum === shared)
   }

   it("should subtract correctly") {
     val diff = 7 - 2
     assert(diff === shared)
   }
 }
 

In some cases, however, shared mutable fixture objects may be changed by test methods such that it needs to be recreated or reinitialized before each test. Shared resources such as files or database connections may also need to be cleaned up after each test. JUnit offers methods setup and tearDown for this purpose. In ScalaTest, you can use the BeforeAndAfter trait, which will be described later, to implement an approach similar to JUnit's setup and tearDown, however, this approach often involves reassigning vars between tests. Before going that route, you should consider two approaches that avoid vars. One approach is to write one or more "create" methods that return a new instance of a needed object (or a tuple of new instances of multiple objects) each time it is called. You can then call a create method at the beginning of each test that needs the fixture, storing the fixture object or objects in local variables. Here's an example:

 import org.scalatest.Spec
 import scala.collection.mutable.ListBuffer

 class MySpec extends Spec {

   // create objects needed by tests and return as a tuple
   def createFixture = (
     new StringBuilder("ScalaTest is "),
     new ListBuffer[String]
   )

   it("should mutate shared fixture objects") {
     val (builder, lbuf) = createFixture
     builder.append("easy!")
     assert(builder.toString === "ScalaTest is easy!")
     assert(lbuf.isEmpty)
     lbuf += "sweet"
   }

   it("should get a fresh set of mutable fixture objects") {
     val (builder, lbuf) = createFixture
     builder.append("fun!")
     assert(builder.toString === "ScalaTest is fun!")
     assert(lbuf.isEmpty)
   }
 }
 

Another approach to mutable fixture objects that avoids vars is to create "with" methods, which take test code as a function that takes the fixture objects as parameters, and wrap test code in calls to the "with" method. Here's an example:

 import org.scalatest.Spec
 import scala.collection.mutable.ListBuffer

 class MySpec extends Spec {

   def withFixture(testFunction: (StringBuilder, ListBuffer[String]) => Unit) {

     // Create needed mutable objects
     val sb = new StringBuilder("ScalaTest is ")
     val lb = new ListBuffer[String]

     // Invoke the test function, passing in the mutable objects
     testFunction(sb, lb)
   }

   it("should mutate shared fixture objects") {
     withFixture {
       (builder, lbuf) => {
         builder.append("easy!")
         assert(builder.toString === "ScalaTest is easy!")
         assert(lbuf.isEmpty)
         lbuf += "sweet"
       }
     }
   }

   it("should get a fresh set of mutable fixture objects") {
     withFixture {
       (builder, lbuf) => {
         builder.append("fun!")
         assert(builder.toString === "ScalaTest is fun!")
         assert(lbuf.isEmpty)
       }
     }
   }
 }
 
One advantage of this approach compared to the create method approach shown previously is that you can more easily perform cleanup after each test executes. For example, you could create a temporary file before each test, and delete it afterwords, by doing so before and after invoking the test function in a withTempFile method. Here's an example:
 import org.scalatest.Spec
 import java.io.FileReader
 import java.io.FileWriter
 import java.io.File
 
 class MySpec extends Spec {
 
   def withTempFile(testFunction: FileReader => Unit) {
 
     val FileName = "TempFile.txt"
  
     // Set up the temp file needed by the test
     val writer = new FileWriter(FileName)
     try {
       writer.write("Hello, test!")
     }
     finally {
       writer.close()
     }
  
     // Create the reader needed by the test
     val reader = new FileReader(FileName)
  
     try {
       // Run the test using the temp file
       testFunction(reader)
     }
     finally {
       // Close and delete the temp file
       reader.close()
       val file = new File(FileName)
       file.delete()
     }
   }
 
   it("should read from a temp file") {
     withTempFile {
       (reader) => {
         var builder = new StringBuilder
         var c = reader.read()
         while (c != -1) {
           builder.append(c.toChar)
           c = reader.read()
         }
         assert(builder.toString === "Hello, test!")
       }
     }
   }
 
   it("should read the first char of a temp file") {
     withTempFile {
       (reader) => {
         assert(reader.read() === 'H')
       }
     }
   }
 }
 

If you are more comfortable with reassigning instance variables, however, you can instead use the BeforeAndafter trait, which provides methods that will be run before and after each test. BeforeAndAfter's beforeEach method will be run before, and its afterEach method after, each test (like JUnit's setup and tearDown methods, respectively). For example, here's how you'd write the previous test that uses a temp file with BeforeAndAfter:

 import org.scalatest.Spec
 import org.scalatest.BeforeAndAfter
 import java.io.FileReader
 import java.io.FileWriter
 import java.io.File

 class MySpec extends Spec with BeforeAndAfter {

   private val FileName = "TempFile.txt"
   private var reader: FileReader = _

   // Set up the temp file needed by the test
   override def beforeEach() {
     val writer = new FileWriter(FileName)
     try {
       writer.write("Hello, test!")
     }
     finally {
       writer.close()
     }

     // Create the reader needed by the test
     reader = new FileReader(FileName)
   }

   // Close and delete the temp file
   override def afterEach() {
     reader.close()
     val file = new File(FileName)
     file.delete()
   }

   it("should read from a temp file") {
     var builder = new StringBuilder
     var c = reader.read()
     while (c != -1) {
       builder.append(c.toChar)
       c = reader.read()
     }
     assert(builder.toString === "Hello, test!")
   }

   it("should read the first char of a temp file") {
     assert(reader.read() === 'H')
   }
 }
 

In this example, the instance variable reader is a var, so it can be reinitialized between tests by the beforeEach method. If you want to execute code before and after all tests (and nested suites) in a suite, such as you could do with @BeforeClass and @AfterClass annotations in JUnit 4, you can use the beforeAll and afterAll methods of BeforeAndAfter. See the documentation for BeforeAndAfter for an example.

Test groups

A Spec's tests may be classified into named groups. As with any suite, when executing a Spec, groups of tests can optionally be included and/or excluded. To place Spec tests into groups, you pass objects that extend abstract class org.scalatest.Group to the methods that register tests, it and ignore. Class Group takes one parameter, a string name. If you have created Java annotation interfaces for use as group names in direct subclasses of org.scalatest.Suite, then you will probably want to use group names on your Specs that match. To do so, simply pass the fully qualified names of the Java interfaces to the Group constructor. For example, if you've defined Java annotation interfaces with fully qualified names, com.mycompany.groups.SlowTest and com.mycompany.groups.DBTest, then you could create matching groups for Specs like this:

 import org.scalatest.Group

 object SlowTest extends Group("com.mycompany.groups.SlowTest")
 object DBTest extends Group("com.mycompany.groups.DBTest")
 

Given these definitions, you could place Spec tests into groups like this:

 import org.scalatest.Spec

 class MySuite extends Spec {

   it("should add correctly", SlowTest) {
     val sum = 1 + 1
     assert(sum === 2)
     assert(sum + 2 === 4)
   }

   it("should subtract correctly", SlowTest, DBTest) {
     val diff = 4 - 1
     assert(diff === 3)
     assert(diff - 2 === 1)
   }
 }
 

This code places both tests into the com.mycompany.groups.SlowTest group, and test "should subtract correctly" into the com.mycompany.groups.DBTest group.

The primary execute method takes two Set[String]s called groupsToInclude and groupsToExclude. If groupsToInclude is empty, all tests will be executed except those those belonging to groups listed in the groupsToExclude Set. If groupsToInclude is non-empty, only tests belonging to groups mentioned in groupsToInclude, and not mentioned in groupsToExclude, will be executed.

Ignored tests

To support the common use case of “temporarily” disabling a test, with the good intention of resurrecting the test at a later time, Spec provides registration methods that start with ignore instead of it. For example, to temporarily disable the test with the name "should pop values in last-in-first-out order", just change “it” into “ignore,” like this:

 import org.scalatest.Spec
 import scala.collection.mutable.Stack

 class StackSpec extends Spec {

   describe("A Stack") {

     ignore("should pop values in last-in-first-out order") {
       val stack = new Stack[Int]
       stack.push(1)
       stack.push(2)
       assert(stack.pop() === 2)
       assert(stack.pop() === 1)
     }

     it("should throw NoSuchElementException if an empty stack is popped") {
       val emptyStack = new Stack[String]
       intercept[NoSuchElementException] {
         emptyStack.pop()
       }
     }
   }
 }
 

If you run this version of StackSpec with:

 scala> (new StackSpec).execute()
 

It will run only the second test and report that the first test was ignored:

 A Stack
 - should pop values in last-in-first-out order !!! IGNORED !!!
 - should throw NoSuchElementException if an empty stack is popped
 

As with org.scalatest.Suite, the ignore feature is implemented as a group. The execute method that takes no parameters adds org.scalatest.Ignore to the groupsToExclude Set it passes to the primary execute method, as does Runner. The only difference between org.scalatest.Ignore and the groups you may define and exclude is that ScalaTest reports ignored tests to the Reporter. The reason ScalaTest reports ignored tests is as a feeble attempt to encourage ignored tests to be eventually fixed and added back into the active suite of tests.

Author
Bill Venners
Method Summary
protected def describe (description : java.lang.String)(f : => Unit) : Unit
Describe a “subject” being specified and tested by the passed function value. The passed function value may contain more describers (defined with describe) and/or examples (defined with it). This trait's implementation of this method will register the description string and immediately invoke the passed function.
override def groups : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
A Map whose keys are String group names to which tests in this Spec belong, and values the Set of test names that belong to each group. If this FunSuite contains no groups, this method returns an empty Map.
protected def ignore (specText : java.lang.String, testGroups : Group*)(testFun : => Unit) : Unit
Register a test to ignore, which has the given spec text, optional groups, and test function value that takes no arguments. This method will register the test for later ignoring via an invocation of one of the execute methods. This method exists to make it easy to ignore an existing test method by changing the call to it to ignore without deleting or commenting out the actual test code. The test will not be executed, but a report will be sent that indicates the test was ignored. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
protected def ignore (specText : java.lang.String)(testFun : => Unit) : Unit
Register a test to ignore, which has the given spec text and test function value that takes no arguments. This method will register the test for later ignoring via an invocation of one of the execute methods. This method exists to make it easy to ignore an existing test method by changing the call to it to ignore without deleting or commenting out the actual test code. The test will not be executed, but a report will be sent that indicates the test was ignored. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
protected def it (specText : java.lang.String)(testFun : => Unit) : Unit
Register a test with the given spec text and test function value that takes no arguments. An invocation of this method is called an “example.” This method will register the test for later execution via an invocation of one of the execute methods. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
protected def it (specText : java.lang.String, testGroups : Group*)(testFun : => Unit) : Unit
Register a test with the given spec text, optional groups, and test function value that takes no arguments. An invocation of this method is called an “example.” This method will register the test for later execution via an invocation of one of the execute methods. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
override def runTest (testName : java.lang.String, reporter : Reporter, stopper : Stopper, goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Run a test. This trait's implementation runs the test registered with the name specified by testName. Each test's name is a concatenation of the text of all describers surrounding an example, from outside in, and the example's spec text, with one space placed between each item. (See the documenation for testNames for an example.)
override def runTests (testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
override def testNames : scala.collection.immutable.Set[java.lang.String]
An immutable Set of test names. If this Spec contains no tests, this method returns an empty Set.
Methods inherited from Suite
nestedSuites, execute, execute, execute, runNestedSuites, suiteName, expectedTestCount
Methods inherited from Assertions
assert, assert, assert, assert, convertToEqualizer, intercept, intercept, intercept, expect, expect, fail, fail, fail, fail
Methods inherited from AnyRef
getClass, hashCode, equals, clone, toString, notify, notifyAll, wait, wait, wait, finalize, ==, !=, eq, ne, synchronized
Methods inherited from Any
==, !=, isInstanceOf, asInstanceOf
Method Details
protected def it(specText : java.lang.String, testGroups : Group*)(testFun : => Unit) : Unit
Register a test with the given spec text, optional groups, and test function value that takes no arguments. An invocation of this method is called an “example.” This method will register the test for later execution via an invocation of one of the execute methods. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
Parameters
specText - the specification text, which will be combined with the descText of any surrounding describers to form the test name
testGroups - the optional list of groups to which this test belongs
testFun - the test function
Throws
TestFailedException - if a test with the same name has been registered previously
NullPointerException - if specText or any passed test group is null

protected def it(specText : java.lang.String)(testFun : => Unit) : Unit
Register a test with the given spec text and test function value that takes no arguments. An invocation of this method is called an “example.” This method will register the test for later execution via an invocation of one of the execute methods. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
Parameters
specText - the specification text, which will be combined with the descText of any surrounding describers to form the test name
testFun - the test function
Throws
TestFailedException - if a test with the same name has been registered previously
NullPointerException - if specText or any passed test group is null

protected def ignore(specText : java.lang.String, testGroups : Group*)(testFun : => Unit) : Unit
Register a test to ignore, which has the given spec text, optional groups, and test function value that takes no arguments. This method will register the test for later ignoring via an invocation of one of the execute methods. This method exists to make it easy to ignore an existing test method by changing the call to it to ignore without deleting or commenting out the actual test code. The test will not be executed, but a report will be sent that indicates the test was ignored. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
Parameters
specText - the specification text, which will be combined with the descText of any surrounding describers to form the test name
testGroups - the optional list of groups to which this test belongs
testFun - the test function
Throws
TestFailedException - if a test with the same name has been registered previously
NullPointerException - if specText or any passed test group is null

protected def ignore(specText : java.lang.String)(testFun : => Unit) : Unit
Register a test to ignore, which has the given spec text and test function value that takes no arguments. This method will register the test for later ignoring via an invocation of one of the execute methods. This method exists to make it easy to ignore an existing test method by changing the call to it to ignore without deleting or commenting out the actual test code. The test will not be executed, but a report will be sent that indicates the test was ignored. The name of the test will be a concatenation of the text of all surrounding describers, from outside in, and the passed spec text, with one space placed between each item. (See the documenation for testNames for an example.) The resulting test name must not have been registered previously on this Spec instance.
Parameters
specText - the specification text, which will be combined with the descText of any surrounding describers to form the test name
testFun - the test function
Throws
TestFailedException - if a test with the same name has been registered previously
NullPointerException - if specText or any passed test group is null

protected def describe(description : java.lang.String)(f : => Unit) : Unit
Describe a “subject” being specified and tested by the passed function value. The passed function value may contain more describers (defined with describe) and/or examples (defined with it). This trait's implementation of this method will register the description string and immediately invoke the passed function.

override def groups : scala.collection.immutable.Map[java.lang.String, scala.collection.immutable.Set[java.lang.String]]
A Map whose keys are String group names to which tests in this Spec belong, and values the Set of test names that belong to each group. If this FunSuite contains no groups, this method returns an empty Map.

This trait's implementation returns groups that were passed as strings contained in Group objects passed to methods test and ignore.

Overrides
Suite.groups

override def runTest(testName : java.lang.String, reporter : Reporter, stopper : Stopper, goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit
Run a test. This trait's implementation runs the test registered with the name specified by testName. Each test's name is a concatenation of the text of all describers surrounding an example, from outside in, and the example's spec text, with one space placed between each item. (See the documenation for testNames for an example.)
Parameters
testName - the name of one test to execute.
reporter - the Reporter to which results will be reported
stopper - the Stopper that will be consulted to determine whether to stop execution early.
goodies - a Map of properties that can be used by this Spec's executing tests.
Throws
NullPointerException - if any of testName, reporter, stopper, or goodies is null.
Overrides
Suite.runTest

override def runTests(testName : scala.Option[java.lang.String], reporter : Reporter, stopper : Stopper, groupsToInclude : scala.collection.immutable.Set[java.lang.String], groupsToExclude : scala.collection.immutable.Set[java.lang.String], goodies : scala.collection.immutable.Map[java.lang.String, Any]) : Unit

Run zero to many of this Spec's tests.

This method takes a testName parameter that optionally specifies a test to invoke. If testName is Some, this trait's implementation of this method invokes runTest on this object, passing in:

  • testName - the String value of the testName Option passed to this method
  • reporter - the Reporter passed to this method, or one that wraps and delegates to it
  • stopper - the Stopper passed to this method, or one that wraps and delegates to it
  • goodies - the goodies Map passed to this method, or one that wraps and delegates to it

This method takes a Set of group names that should be included (groupsToInclude), and a Set that should be excluded (groupsToExclude), when deciding which of this Suite's tests to execute. If groupsToInclude is empty, all tests will be executed except those those belonging to groups listed in the groupsToExclude Set. If groupsToInclude is non-empty, only tests belonging to groups mentioned in groupsToInclude, and not mentioned in groupsToExclude will be executed. However, if testName is Some, groupsToInclude and groupsToExclude are essentially ignored. Only if testName is None will groupsToInclude and groupsToExclude be consulted to determine which of the tests named in the testNames Set should be run. For more information on trait groups, see the main documentation for this trait.

If testName is None, this trait's implementation of this method invokes testNames on this Suite to get a Set of names of tests to potentially execute. (A testNames value of None essentially acts as a wildcard that means all tests in this Suite that are selected by groupsToInclude and groupsToExclude should be executed.) For each test in the testName Set, in the order they appear in the iterator obtained by invoking the elements method on the Set, this trait's implementation of this method checks whether the test should be run based on the groupsToInclude and groupsToExclude Sets. If so, this implementation invokes runTest, passing in:

  • testName - the String name of the test to run (which will be one of the names in the testNames Set)
  • reporter - the Reporter passed to this method, or one that wraps and delegates to it
  • stopper - the Stopper passed to this method, or one that wraps and delegates to it
  • goodies - the goodies Map passed to this method, or one that wraps and delegates to it
Parameters
testName - an optional name of one test to execute. If None, all relevant tests should be executed. I.e., None acts like a wildcard that means execute all relevant tests in this Spec.
reporter - the Reporter to which results will be reported
stopper - the Stopper that will be consulted to determine whether to stop execution early.
groupsToInclude - a Set of String group names to include in the execution of this Spec
groupsToExclude - a Set of String group names to exclude in the execution of this Spec
goodies - a Map of key-value pairs that can be used by this Spec's executing tests.
Throws
NullPointerException - if any of testName, reporter, stopper, groupsToInclude, groupsToExclude, or goodies is null.
Overrides
Suite.runTests

override def testNames : scala.collection.immutable.Set[java.lang.String]
An immutable Set of test names. If this Spec contains no tests, this method returns an empty Set.

This trait's implementation of this method will return a set that contains the names of all registered tests. The set's iterator will return those names in the order in which the tests were registered. Each test's name is composed of the concatenation of the text of each surrounding describer, in order from outside in, and the text of the example itself, with all components separated by a space. For example, consider this Spec:

   class StackSpec {
     describe("A Stack") {
       describe("(when not empty)") {
         it("must allow me to pop") {}
       }
       describe("(when not full)") {
         it("must allow me to push") {}
       }
     }
   }
   

Invoking testNames on this Spec will yield a set that contains the following two test name strings:

   "A Stack (when not empty) must allow me to pop"
   "A Stack (when not full) must allow me to push"
   
Overrides
Suite.testNames


Copyright (C) 2001-2009 Artima, Inc. All rights reserved.