novaprova

Because all the obvious names were already in use.

The name Novaprova comes from the modern Italian word prova, one of whose meanings is a test, a trial, an experiment, and the archaic Italian word nova which like the modern Italian nuova means new. Hence, new test.

Disclaimer: I don't speak Italian, but an actual Italian told me this is a silly but clear reading of the name. Also, it rhymes and it sounds vaguely like a drug.

Sure it is. All you need to do is to be able to read the debugging information that the compiler adds to the executable when you build with the -g option. The debugger reads that information, so it can't be too hard.

It turns out that almost every modern UNIX-like system uses the highly portable DWARF standard for debugging information, so there's little system-dependent code involved. The only tricky bit is reading that information for your own process, rather than for another process. Novaprova uses some runtime linker magic and the bfd library to find the on-disk executable and shared library images for the program it's linked into (including libraries opened with dlopen), and some hand-rolled code to scan the debug information.

There are other libraries that claim to do "reflection" for C++, but they all rely on magic macros or other tricks to explicitly mark classes or functions. By contrast, Novaprova's reflection can see any class, namespace, variable or function which the debugger can see, without the explicit co-operation of the code reflected.

Having real reflection is extremely useful for a test framework. At the very least it means that test functions can be discovered at runtime by trawling through all the functions in an executable and matching function names and signatures.

As far as I know, this is a feature unique to Novaprova.

• calls to exit()
• calls to syslog()
• failed calls to the libc assert()
• memory accesses which trigger SIGSEGV/SIGBUS
• memory leaks (using Valgrind)
• buffer over-runs (using Valgrind)
• use of uninitialised variables (using Valgrind)
• looping or deadlocked tests, using a timeout

Novaprova implements function mocking using a technology similar to debugger breakpoints, but working in the same process rather than in a child process.

There are several test frameworks that support mocking for C, but they all rely on the linker choosing to link against the mock function rather than the mocked one. This has major limitations.

• The mocked function must be extern
• The mocked function must not in the same object file or shared library as the calls to it which need to be intercepted (or special care must be taken to make the mocked function an ELF weak symbol).
• Functions can only be mocked once in a given test executable.
• If a function is mocked, it is mocked for all the tests in that test executable (so the mock function has to be more complex).
• The mock functions are installed at build time.

Novaprova's runtime mocking removes all these limitations!

Novaprova's mocks can be attached to any testnode. They are installed just before the testnode or any of it's descendants is run and uninstalled after the test finishes. This means you can safely mock functions different ways in different tests, or mock functions in only some tests, or mock (almost all) functions which are used by the test framework itself. You can mock functions that are called from other functions in the same object file. You can mock static functions, as long as you know the name of the function or have it's address.

As far as I know, this is a feature unique to Novaprova.

Test parameters are a useful way of re-using test code.

Parameters in Novaprova are static char* variables with a series of pre-determined string values, which the test code or fixture setup code can use to control aspects of its behaviour. Novaprova knows which parameters apply to which test, and will run the test once for each combination of values of the parameters, setting up the parameters before each run. The result from each run is reported separately.

A test executable which uses Novaprova can be built like any other C executable. Tests are written in C, and no special build-time work is necessary (the magic all happens at runtime).

Even though Novaprova is implemented using C++ internally, it has a full C API so that tests can be implemented without a C++ compiler, just a C compiler. This is important when you have older code whose headers may or may not compile cleanly under C++.