Testing with JSON expected files

• Because we use JSON anyway
• Lots of integration tests

Expected file:

{
  "callback": {
    "callbackid": 1,
    "event": "invoice.create",
    "uri": "http://www.freshbooks.com/",
    "verified": false,
    "verifier": "some random token"
  }
}

Who wants to maintain that if you update something like the verifier?

Especially if you've got dozens of these JSON files.

Here's our lever in setup.cfg

	[overwrite_fixture_on_failure]
	overwrite_fixture_on_failure = False

Prelude

from ConfigParser import SafeConfigParser as ConfigParser
import difflib
from freshbooks.lib.converters import asbool
from os import path
import simplejson
import sys

Nasty hack

config = ConfigParser()
config_path = path.join(path.dirname(path.dirname(__file__)), 'setup.cfg')
config.read(config_path)
overwrite_fixture_on_failure = asbool(config.get(
    'overwrite_fixture_on_failure',
    'overwrite_fixture_on_failure'))

Then we'll provide a helper to load the file

(who cares how it's formatted)

def assert_json_equals(expected_json_file_path, actual_json):
    try:
        with open(expected_json_file_path) as f:
            expected_json_from_file = f.read()
        expected_dict = simplejson.loads(expected_json_from_file)
        expected_json = simplejson.dumps(expected_dict, sort_keys=True, indent=2)

But if it didn't exist yet, fake it

    except IOError:
        if overwrite_fixture_on_failure:
            open(expected_json_file_path, "w").close()
            # We'll end up ignoring this later because we're in
            # overwrite_fixture_on_failure
            # But if we don't set it stringily, difflib cacks
            expected_json = "<empty file>"
        else:
            raise

Same goes for an unparseable file

    except simplejson.JSONDecodeError:
        if overwrite_fixture_on_failure:
            # We'll end up ignoring this later because we're in
            # overwrite_fixture_on_failure
            # But if we don't set it stringily, difflib cacks
            expected_json = expected_json_from_file
        else:
            raise

Get the actual thing

(really not caring how it's formatted)

    actual_dict = simplejson.loads(actual_json)
    actual_json = simplejson.dumps(actual_dict, sort_keys=True, indent=2)

The real magic!

<3 difflib

    try:
        assert expected_json == actual_json
    except AssertionError: # NO DIAPERS
        for line in difflib.unified_diff(
                expected_json.splitlines(True),
                actual_json.splitlines(True),
                "expected", "actual"):
            sys.stdout.write(line)
        if overwrite_fixture_on_failure:
            out_file = open(expected_json_file_path, "w")
            out_file.write(actual_json)
            out_file.close()
        else:
            raise

Don't forget to break the build if you commit the lever!

from json_unit_test_helper import overwrite_fixture_on_failure

def test_overwrite_disabled():
    """Ensure that setup.cfg has overwrite_fixture_on_failure set to False

    This is to make sure that the build must fail when in "update fixture" mode.
    We never actually want to check in a change like that, as most tests would
    stop failing!"""
    assert not overwrite_fixture_on_failure

I dunno, it seemed like a neat idea.

You ARE using source control, right?