TDD, my recipe book

This post is based on my talk “TDD, mi cuaderno de recetas” (in Spanish) at Software Crafters Barcelona 2017. You can watch the recording on YouTube.

For several years I helped other companies adopt practices like TDD and continuous integration, and I kept running into the same obstacles. “We tried TDD, but we spent more time fixing tests than building features.” “Every so often we had to throw all the tests away and start again because they were so fragile.” “We took a course, but it never covered how to test the database.”

None of these recipes is new or clever. They’re the things I end up doing day to day when I write tests, and the things I end up suggesting when I see a team struggling. Take whatever is useful and argue with the rest.

Why I do TDD

TDD is a practice, a tool. I use it because I want something out of it, so it’s worth saying what:

• I want to sleep well. I want to go home and not worry about what I shipped today.
• Executable documentation. I’ve worked at companies where the functional spec was treated as the law, and then reality had nothing to do with it. Tests can’t drift from the code the way a document can.
• Facing my design. TDD makes me confront the design I chose. I’m not sure it drives my design anymore: when I code without it, I end up designing the same way. What changed is how I design, and TDD is what changed it.
• A brake. We love solving problems, and as soon as someone hands us one we want to start typing. If you follow the red, green, refactor flow strictly, it makes you slow down and know which phase you’re in: solving the business problem, or making the code decent.

All of that adds up to confidence. The same confidence a continuous integration server gives me by building and running the whole test suite every day. If something has to go to production on a Friday, it can. And it’s confidence in the team’s work, not just mine, because what I write alone is a small part of what the team delivers.

The “Confidence” slide: you only let go when you trust what’s holding you. Photo on Flickr.

Recipe 1: name tests as business scenarios

For a long time my test classes were called FizzBuzzTests, and everything I wanted to test went into that bag. Then I read a 2014 post by Sandro Mancuso about how they structure their tests, and it changed my approach.

If one of my goals is executable documentation, and we’re solving a business problem, that documentation should speak the language of the business. The word test talks about how I’m validating my hypothesis, not about the problem. So now I name my test classes and methods so that the test runner output reads like a list of sentences, each one a business scenario:

public class FizzBuzzShould
{
    [Fact(DisplayName = "FizzBuzz should return the same number when it is not divisible by 3 or 5")]
    public void return_the_same_number_when_it_is_not_divisible_by_3_or_5()
    {
        var fizzBuzz = new FizzBuzz();

        var result = fizzBuzz.Calculate(1);

        result.Should().Be("1");
    }
}

It’s obvious. So obvious that I didn’t do it before. And most codebases I see are still full of tests whose names have nothing to do with the business.

With C# and xUnit I can’t write a plain sentence as a test name, so I pay a price: a DisplayName that duplicates the method name, which is almost as bad as a comment. I pay it. If your language or framework lets you describe tests with real sentences (RSpec, Mamba, Jasmine…), get as close to the business as you can. Every step in that direction makes the domain easier to enter for a new teammate, or for you on Monday after a weekend of conferences.

Recipe 2: don’t let the problem statement design your solution

The FizzBuzz kata says something like: “given a list of numbers, return that list transformed”. Numbers divisible by 3 become Fizz, by 5 become Buzz, by both become FizzBuzz, and the rest stay as they are.

Notice that the test above doesn’t receive a list. I’m not saying the kata must be solved without lists. My point is that the way a problem is worded shapes how we approach the tests and the design. I’ve seen this kata started with a list many, many times, and that leads straight to a loop and a few layers of complexity. Stop and think about the problem, and you realize it’s simply: if I get a 1, return “1”; if I get a 3, return “Fizz”. The list is the easy part.

The same thing happened with the Bank kata. You simulate a bank account: deposits, withdrawals, transfers, statements. A week before the talk I ran it in a training session, and 100% of the people ended up with an Account class with a Balance property doing balance += amount on every deposit.

That’s not how a bank works. Banks record movements, and the balance of your account is an aggregation of everything that has happened to it. Nobody considered it, because the problem statement sounds very data-centric: statements, balances, information to extract. The wording conditioned the design before anybody wrote a test.

Think before you type. I’ve said it many times before, and it’s amazing how often we jump to the keyboard anyway.

Where to start is also part of thinking. Do you build from the bottom up or from the outside in? It depends on how much uncertainty you have about what you want and how to get it, and on your tooling. In a message-driven system, for instance, it’s easy to think upfront about the command coming in and the event I want to go out, and then the direction doesn’t matter much. Do whatever helps your flow.

Recipe 3: think Given, When, Then, and don’t write it down

Most people know the Arrange, Act, Assert structure: set up the conditions for the test, execute the action under test, and check that the system did (or didn’t do) what it should.

What I’ve found many times is tests that could have come from a template, because every single one has // Arrange, // Act and // Assert comments. A comment usually means you’re trying to explain something the code doesn’t.

I prefer to think in Given, When, Then. Not as a different set of words for the comments, please don’t add // Given either. The point is that it’s much easier to give a test meaning when you think “given the system is in this state, when this thing happens (something that has to do with the business, please), then this is the result”.

A three-line test doesn’t need labels. If the parts are mixed up, separate them with blank lines to help the reader. And if the test is 200 lines long, the missing comment isn’t the problem: the test needs another look.

Recipe 4: make assertions speak the domain

The default way to write the then in most testing libraries is something like this:

Assert.Equal(result, "1");

How many times have I found the expected value and the actual value swapped like that. We’re not machines, and this forces us to infer too much: which one is expected, which one is actual, what the failure message will say. Whatever language you use, there are tools that read far better:

result.Should().Be("1");

And if your stack doesn’t have them, which I doubt, write them. If we’ve said tests should speak the language of the business, sometimes the assertion itself belongs to that language too. If what we want to check is that a purchase was completed, don’t check that a status field equals true:

Assert.True(purchase.Status == PurchaseStatus.Completed);

Bring it into the domain:

purchase.Should().BeCompleted();

It reads better, and it’s also one less reason for the test to break when the way you store that state changes.

The same idea applies to production code. In FizzBuzz, whether a number is divisible by another carries real meaning from the business point of view. It can be as silly as:

private static bool IsDivisibleBy(int number, int divisor) => number % divisor == 0;

Divisibility is a toy example, let’s be honest. But we often want to describe a business rule and end up replicating the specific way we happened to solve it. If a concept matters to the business, consider giving it a name in your code.

Recipe 5: Gherkin yes, generated glue code no

Gherkin is the language behind BDD tools like Cucumber or SpecFlow: you describe a scenario with Given, When, Then, in a more literary way. There are success stories, and it’s sold as a way to get even closer to the business, with product people writing scenarios that run as tests.

It doesn’t work for me, because the way you write the scenario conditions the implementation too much. Think of the classic calculator: “Given I enter 50, and I enter 70, when I press add, then the result is 120”. There’s a saying: if you have two teams build a compiler, you’ll get a two-pass compiler. Language does the same. If you write both numbers in a single Given, your implementation will probably receive two parameters. Split them into two steps and you’ll get something else. It’s not easy to decouple the two, and it ends up generating a lot of noise.

My recipe: Gherkin can be fine as a way to talk about scenarios, but I don’t use the plugin that generates the test code from it. That generated code forces me to write glue, boilerplate that exists only for the tests and is shaped by the wording. I’d rather use the patterns in the next recipes.

Recipe 6: group tests by intent, not by input and output

When people see that every FizzBuzz test has the same shape, an input and an output, they reach for a parameterized test:

[Theory]
[InlineData(1, "1")]
[InlineData(3, "Fizz")]
[InlineData(5, "Buzz")]
[InlineData(15, "FizzBuzz")]
public void calculate(int number, string expected)
{
    var result = new FizzBuzz().Calculate(number);

    result.Should().Be(expected);
}

It looks clever, and it has a serious problem: the business rules are gone. The executable documentation is lost completely. A test that says “calculate” and passes doesn’t tell anybody what FizzBuzz does.

What I do instead is group by rule. If you want several examples for the “not divisible by 3 or 5” rule, put them in that test. Add more triangulation cases, or move to property-based testing later if you like:

[Theory(DisplayName = "FizzBuzz should return the same number when it is not divisible by 3 or 5")]
[InlineData(1)]
[InlineData(2)]
[InlineData(98)]
public void return_the_same_number_when_it_is_not_divisible_by_3_or_5(int number)
{
    var result = new FizzBuzz().Calculate(number);

    result.Should().Be(number.ToString());
}

People new to TDD feel bad because the tests look duplicated: “I have this same chunk of code in every test”. What matters isn’t duplicated code, it’s duplicated intent. These tests want different things, so they’re different tests.

Someone in the audience pointed out that two pieces of code can be identical by accident, just because it’s still early in development. Exactly, and that’s why my heuristic is intent and not code: the implementations may coincide today and have nothing to do with each other as soon as the business grows a little. It’s the same difference as between reusable code and reused code. We spend a lot of time designing reusable code that we never reuse.

Recipe 7: don’t use new in your tests

Back to the Bank kata. A typical first test:

[Fact(DisplayName = "An account should have the deposited amount as its balance")]
public void have_the_deposited_amount_as_its_balance()
{
    var account = new Account();

    account.Deposit(100);

    account.Balance.Should().Be(100);
}

This test is tied to how an account is constructed, which is an implementation detail, while what the test cares about is a business concept: an account. As the code evolves, say an account now requires a customer id, you have to go back and fix every test that does new Account(). That’s where a lot of “our tests are so fragile” comes from.

So my recipe is to avoid new in tests and look for semantics instead. I use three mechanisms, all with the same purpose of gaining meaning and flexibility.

Object mothers (or factories, call them what you want). This is test code, not domain code. Every test that needs a new account asks for one:

var account = GivenANewAccount();

When the constructor changes, there’s one place to fix. When a test does care about the customer, use named parameters, whatever your language calls them:

var account = GivenANewAccount(customerId: 2);

Without the name, a bare 2 tells the reader nothing, and depending on argument position is fragile. With it, the test reads well, and it won’t need to change again unless the business changes. And if the business changes, of course the tests change. (Choosing that 2 well is a topic of its own: I wrote about it in Pay attention to your test data.)

Builders, when I want something more fluent:

var account = AnAccount()
    .For(customerId: 2)
    .WithDeposit(100)
    .Build();

One important detail: whatever is mandatory for an account to exist should be required by your mother or builder, and only the rest should be optional. If everything is optional, your tests can no longer express that an account needs a customer, no matter what. In the long run that brings fragility, and the principle of least surprise goes out the window.

Extension methods. In C#, or any language that lets you extend types after the fact, when tests mix different business concepts around the same entity, I use extensions to compose the equivalent of a builder without coupling all the test code into a single huge class.

A word of caution: the same kind of abstraction can help on the When side, for example when working with event-based systems. Watch how far you go. If you have to dive through layers to understand what a test does, you’ve gone too far. There’s a balance.

Recipe 8: keep database tests fast

This one is very common in my corner of the world: people using Entity Framework, or whatever ORM, with migrations. Their strategy for running a single integration test is to drop the test database, recreate it, apply every migration, and then run one big SQL script with sample data shared by all the tests.

That’s hell. It kills fast feedback. And with a single shared SQL file, the data your test depends on lives far away from it. Test data belongs in the test.

Recreating the database is fine while it’s small, but once it grows it takes forever, so look for another strategy. Some that have worked for me:

• Open a transaction and roll it back. If you own the transaction lifecycle, and you should, open a transaction at the start of the test, set up exactly the data it needs (sometimes that means truncating tables and inserting just the right rows), run the test, and roll back. This works very well with legacy code. With really legacy code you may not own the transaction flow at all, and then you’re stuck.
• Respawn. A .NET library that does something similar for you: it takes a checkpoint and resets the database to it. A bit of magic, and quite cheap.

If you really need tests that commit their transactions, run that suite on your continuous integration server. Don’t bring that complexity into your TDD flow or into the tests you run before every commit. Handle the special case in a special way, so it doesn’t hurt your day to day.

Shortly before this talk I was on a project where a single empty test took five minutes. Someone had used aspect-oriented programming to drop the database and create the Entity Framework context whenever the test assembly was loaded. Even after removing that, an integration test that just read one record could not take five minutes. Fast feedback matters too much.

Where I’d take that first test

For the record, I’m not happy with the FizzBuzz test from the first recipe. If I had many tests like it, I’d move the construction to the test class setup, because those details don’t matter to any individual test. I’d write a custom assertion that talks about FizzBuzz. Each test would end up as a single line that states the rule.

That’s my recipe book, or at least the part of it that fit in one talk. Your recipes are probably different, and I’d love to hear them.