mr-edd.co.uk :: horsing around with the C++ programming language

test-o-matic

test-o-matic is a minimal library (around 500 lines) to aid in writing tests for C++ code. It is not a framework; the barrier to entry is very low and it is easy to extend in many directions.

It is portable, unintrusive, easy to use, free for use in commercial and open source software and independent of any 3rd party code.

Recent activity

Code

Clone the repository using mercurial:

> hg clone http://bitbucket.org/edd/test_o_matic

Or get a zip file of the code.

Quick start

If tests are hard to write, they don't get written. So test-o-matic minimizes boilerplate code as far as possible.

For extremely quick-and-dirty tests, you can use the macros provided by test-o-matic in your main() function:

#include <test_o_matic.hpp>
#include <iostream> // std::cout
#include <stdexcept> // std::runtime_error

int main()
{
    CHECK(1 == 1);
    THROWS(throw std::runtime_error("oops"), std::exception);

    // print a summary and return the number of failures
    return test_o_matic::summary(get_local_tom_logger(), std::cout);
}

However, as the number of tests grows, this approach soon becomes unwieldy. It's best to try to keep tests isolated so that they don't step on each other's toes.

Here's a more complete example:

#include <test_o_matic.hpp>
#include "bignum.hpp" // your bignum class

TEST("bignum equality")
{
    bignum one(1), three(3), five(5);
    CHECK(five == five);
    CHECK(!(one == three));
}

TEST("bignum construction")
{
    TRY(bignum("99999999999999999999999999999999"));
}

TEST("bignum division")
{
    THROWS(bignum(1) / bignum(0), divide_by_zero);
}

struct common_bignums // a "fixture" 
{
    bignum zero, one, huge;

    // constructor "sets up", destructor "tears down"
    common_bignums() : zero(0), one(1), huge("9999999999999999999999") { }
};

TESTFIX("bignum multiplication", common_bignums) 
{
    // The member variables of a fixture are made available to the test:
    CHECK(one * one == one);
    CHECK(zero * one == zero);
    TRY(huge * huge);
    //...
}

Here's how to run all the tests we just created:

namespace tom { using namespace test_o_matic };

int main()
{
    const bool verbose = true;
    tom::simple_logger lgr(std::cout, verbose)
    tom::runner rnr;

    for (const tom::test *t = tom::first_test(); t; t = t->next)
        tom::run_test(*t, lgr, rnr);

    return lgr.summary(std::cout);
}

Note how it's possible to iterate over all the tests. Each test object has a bunch of fields:

You can use this information to filter out the tests you're not interested in, or collect tests in to suites of some kind. Perhaps you only want to run all tests that were compiled today, or within the last 10 minutes?

A logger object is used to print information about events and gather basic statistics as tests run. A runner object is used to call each test. Both are entirely customizable.

Organising your tests

I tend to put the tests for a particular class or module in to their own source file and then the main() function in a file on its own, but the organization is really up to you.

Another interesting strategy is to have all tests for a given class in their own file as before, but now compile them in to a shared libraries (.dll/.so/.dylib). Each shared library will export the first_test() and run_test() functions, allowing you to dynamically load and run tests using functions such as

So rather than having lots of little test applications, you instead have lots of little test libraries and a single program to run them. Perhaps the program can descend a directory hierarchy looking for tests and/or test libraries that match a user-specified pattern? It's up to you.

Further reading

Comments

jack

[02/01/2009 at 14:41:00]

I wonder way you test-o-matic does not using the STL iterators (begin, end) but instead uses 'for (const tom::test *t = tom::first_test(); t; t = t->next)'?

Further: is it possible to easily generate XML output? So that it can be used with a continues integration system.

Edd

[05/01/2009 at 00:25:00]

Hi Jack.

I toyed with the idea of implementing STL iterators, but I didn't think it was worth the effort. The tests and their position in the global list are un-changable so their use with STL algorithms would be extremely limited. Even in the case of std::for_each, you'd have to create a somewhat convoluted functor, I suspect, to do anything other than simply run the tests.

With that said, it should be a piece of cake to create an STL iterator wrapper. I guess it would have to model an input iterator.

As for XML output, it's certainly possible, but there's no XML generator provided out the box. To do this yourself (if you're interested), you would have to create your own event handlers. The test_o_matic.cpp file contains the definitions for the default handlers. Perhaps a cookbook entry for XML output might be useful, though?

The goal was really to create a "bare bones" library and make it easy for people to build on top of it, rather than providing an all encompassing framework.

(optional)
(optional)
(required, hint)

Links can be added like [this one -> http://www.mr-edd.co.uk], to my homepage.
Phrases and blocks of code can be enclosed in {{{triple braces}}}.
Any HTML markup will be escaped.