If you’ve spent any time in programming circles in the last few1 years, you’ve probably heard people talk about writing tests.
If you’re self-taught, there’s a decent chance you have no idea what they’re talking about.
Why test?
Let’s say you’re writing a pluralizer. You have a main function, pluralize(), which takes a string and returns a plural version of that string.
Here’s a way-too-basic version, in JavaScript ‘cause I heard it’s the most popular language.
// pluralizer.js
function pluralize(string) {
var lastCharacter = string.slice(-1);
switch(lastCharacter) {
case "s":
return string + "es";
default:
return string + "s";
}
}
You’ll probably test it out in the console a bit to see if it works (this is called manual testing):
pluralize("class");
// "classes"
pluralize("chair");
// "chairs"
pluralize("campus");
// "campuses"
Everything checks out! 2
Now, let’s say some scientist on GitHub uses your script, but needs to pluralize the word fungus to fungi. They send you the following pull request:
// pluralizer_broken.js
function pluralize(string) {
var lastCharacter = string.slice(-1);
// handle words like "fungus" and "nucleus"
var lastTwoCharacters = string.slice(-2);
if (lastTwoCharacters === "us") {
return string.replace("us", "i");
} else {
switch(lastCharacter) {
case "s":
return string + "es";
default:
return string + "s";
}
}
}
Now I’m sure you wouldn’t write that code, but you might casually accept it in a pull request.
And now your pluralizer turns campus into campi. Shit.
So you fix your code:
// pluralizer_fixed.js
function pluralize(string) {
var noChange = ["platypus", "fish"];
if (noChange.indexOf(string) > -1) {
return string;
}
var usToI = ["nucleus", "fungus", "virus"];
if (usToI.indexOf(string) > -1) {
return string.replace("us", "i");
}
var lastCharacter = string.slice(-1);
switch(lastCharacter) {
case "s":
return string + "es";
default:
return string + "s";
}
}
Pluralization has so many edge cases that it can be really hard to make sure you don’t break stuff. But you run through all the words you’ve thought of, and it seems to work.
pluralize("fungus");
// "fungi"
pluralize("nucleus");
// "nuclei"
pluralize("octopus");
// "octopuses"
pluralize("chair");
// "chairs"
pluralize("campus");
// "campuses"
Cool!
Maybe it would be handy to build some of these test words into your code to make sure it doesn’t break again. That seems like a good way to keep track of your edge cases, too. That’s called test automation, and it’s what people mean when they say writing tests.
Let’s test.
Maybe you’ll write something like this:
// test_pluralizer.js
function itShould(statement, should) {
if (statement !== true) {
console.error( "it should " + should + " AND IT DOES NOT! :(" );
}
}
itShould(pluralize("chair") === "chairs", "pluralize chair");
itShould(pluralize("fungus") === "fungi", "pluralize fungus");
itShould(pluralize("campus") === "campuses", "pluralize campus");
itShould(pluralize("platypus") === "platypus", "pluralize platypus");
itShould(pluralize("nucleus") === "nuclei", "pluralize nucleus");
itShould(pluralize("octopus") === "octopuses", "pluralize octopus");
Your itShould function takes two arguments. The first one should evaluate to true. The second one describes what you’re testing.
As long as your statement evaluates to true, you’re good. If it doesn’t, you get a nice descriptive error. As long as you run test_pluralizer.js after making any changes, you can be sure you didn’t break
anything.
But wait…
You might eventually figure this out, but the itShould method above already exists in Chrome and Node(/io),
except it’s called console.assert(). The second argument is optional, the error text is a bit more terse, and it throws you into a debugger, but otherwise it does
the same thing.
This style of testing is called assertion testing.
Some other terms
Some people would write a list of assertions before starting work on the pluralizer itself. That’s called test-driven development. It can be a good way to force yourself to think through the edge cases your program will have to deal with.
When designing a program, it’s helpful to separate the functionality of that program into independent, interchangeable modules, such that each contains everything necessary to execute only one aspect of the desired functionality.3 This approach scales very well: functions, classes, and packages are all examples of modules. One reason for designing modular programs is that it makes it easy to recombine the modules to make new programs.
Individually testing the modules that make up your program is called unit testing.
Testing the way modules work when combined together is called integration testing.
Fakes
Often, your modules will depend on interaction with other modules. That makes them hard to unit test. For example, let’s say you’re writing a really fancy alarm clock. If it’s sunny, you want to wake up at 7 and bike. If it’s rainy, you’d rather
sleep in till 7:30 and take the train. You have three modules: Weather checks the weather, Noise makes a loud noise, and Clock calls the first two.
// clock.js
function clock() {
var weather = Weather.get(); // let's say this is the interface for your weather module
// let's further say it returns "rainy" or "sunny"
var currentDate = new Date();
alarmMaybe(currentDate, weather);
}
function alarmMaybe(date, weather) {
var currentHour = date.getHours();
var currentMinute = date.getMinutes();
if (currentHour === 7) {
if (weather === "sunny") {
Noise.play(); // let's say this is the interface for your Noise module.
} else if (currentMinute >= 30) {
Noise.play();
}
}
}
setInterval( clock, 60000 ); // run clock() every minute
Stubs
You want to make sure alarmMaybe() works, but you don’t want to wait until morning. I’d probably start by writing some dummy versions of the clock() method, with hard-coded
weather and date:
// test_clock.js
function test_clock_early_rainy() {
Weather.get() = function { return "rainy" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 7, 29, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
}
function test_clock_ontime_rainy() {
Weather.get() = function { return "rainy" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 7, 30, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
}
function test_clock_early_sunny() {
Weather.get() = function { return "sunny" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 6, 59, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
}
function test_clock_ontime_sunny() {
Weather.get() = function { return "sunny" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 7, 00, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
}
Replacing Weather.get() like that is called creating a method stub, or stubbing out the method. The terms here are a bit cloudy,
but some people say a stubs are a subset of fakes.
Mocks
If you want to automate this testing, you need a way of knowing whether Noise.play() gets called, preferably without actually playing the noise.
Let’s add a fake version of Noise to our Clock test that only keeps track of whether it gets called:
// mock_noise.js
Noise = {called: false};
Noise.play = function() {
Noise.called = true;
}
Noise.reset = function() {
Noise.called = false;
}
That’s called creating a mock, which some people would say is distinct from a stub.
Putting it all together
Now we have everything we need to add some assertions to our test_clock functions from earlier, using itShould, our replacement for console.assert.
// test_clock.js
function test_clock_early_rainy() {
Noise.reset();
Weather.get() = function { return "rainy" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 7, 29, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
itShould(Noise.called === false, "not sound at 7:29 if it's rainy");
}
function test_clock_ontime_rainy() {
Noise.reset();
Weather.get() = function { return "rainy" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 7, 30, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
itShould(Noise.called, "sound at 7:30 if it's rainy");
}
function test_clock_early_sunny() {
Noise.reset();
Weather.get() = function { return "sunny" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 6, 59, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
itShould(Noise.called === false, "not sound at 6:59 if it's sunny");
}
function test_clock_ontime_sunny() {
Noise.reset();
Weather.get() = function { return "sunny" };
// yyyy mm dd h mm s ms
var) testThisDate = new Date( 2015, 04, 24, 7, 00, 0, 0 );
alarmMaybe)(testThisDate, Weather.get());
itShould(Noise.called, "sound at 7:00 if it's rainy");
}
BDD
The practice of starting tests with “should” comes from behaviour driven development, a software developmnet process specified by Dan North. BDD advocates starting with a set of behaviours, written:
- by the client
- in order of business value
- in plain text
- in structured formal language (there should be exactly one way for a programmer to implement a test based on the behaviour).4
Specifically, it recommends that the behaviours (Dan also calls them stories) each specify three things:
As a [X]
I want [Y]
so that [Z]
For example,5
As a math idiot
I want to be told the sum of two numbers
so that I can avoid silly mistakes
A programmer then translates the behaviours into executable tests, tied directly to classes, using full sentences starting with should. For example, if you have a Calculator class, you’d also make a CalculatorBehaviour class, with methods like should_display_70_when_asked_30_plus_40
The should sentences make it easy to understand exactly what a given test is for, make it easy to know whether a behaviour is still correct (a test sounds less mutable
than a behaviour) In some languages, you can omit the descriptive string from your assertions and infer it from the method name.
The most interesting aspect of behaviour driven development is how it tells you what to do. Not only does it say that you should write your tests based on externally visible functionality of the program, but it also ensures that you only write features that actually benefit a person for a reason. I don’t practice BDD, but I think those two guidelines are helpful to think about.
Notes
- Test driven development has been around since 1957.
- This is horribly incomplete. Don’t use it.
- This description is lifted straight from Wikipedia.
- some test frameworks try to allow you to write the actual executable tests in such a business-readable language, to various degrees of success.
- This example is copied directly from the Cucumber homepage.