Developing from a narrative perspective

This post is based on my talk “Desarrollando desde una perspectiva narrativa” at Commit Conf 2024. You can watch the recording (in Spanish).

In 2016 I gave a talk called “Our code speaks, what is it telling us?”. This one is its sequel, eight years later. If Top Gun could do it, so can I.

I mentioned Copilot in the talk’s abstract only to say the talk wasn’t about Copilot, and it still ended up on a list of AI talks. Narratives get confusing fast. So let’s start with a definition. Of the meanings the Spanish dictionary gives for narrativa, I like the last one: the ability or skill to narrate, or to tell something.

That’s the part I care about. We often say that developing software involves more reading than writing. I’d add that developing the right software, the one that solves real problems, involves more telling than programming. We still build software to do something, so this isn’t a license to only philosophize. But telling things well is a big part of the job.

A story can work in your favor

A narrative isn’t only something you tell. It’s also something that gets created around you, like when we say a political party is pushing a narrative. Sometimes you can ride that narrative and benefit from it.

My son Mateo is good at this. During the town festival a friend offered to take him for a walk. They came back with an enormous blister pack: a bow, arrows, and a few other things. “Weren’t you just going for a walk?” My friend had asked him if he wanted anything, and he did. He saw the context he’d been handed and took it.

For years it was the opposite. Whenever I asked him if he needed money, he did the math and said no. I kept telling him that when someone offers you money and asks for nothing in return, you say yes. The context is already in your favor; use it.

Telling means manipulating

I told that story with a photo of Mateo as a small child, and most of the audience assumed he was still under ten. He’s an adult; he let me use the photo. I arranged the story so it looked that way.

Another one. Friends come over, and Mateo, still a kid, asks one of them to play football in the garden. My friend has zero interest and says he isn’t wearing the right shoes. Fair enough. Next visit, same situation, but this time Mateo opens with a different question: “Are those shoes OK for football?” “Yes.” “Then let’s play.”

Whenever we tell something, or set up the stage for a conversation, we:

• break the timeline,
• jump back to the past,
• stretch the action, or summarize it,
• create an atmosphere,
• arrange the elements so they cause an effect.

None of this is about writing books or doing stand-up. It matters in software because you can put it to work:

• Own the story. Choose what to tell and how to tell it. You never tell everything; life is a sequence of events, and only when you tell it do you pick which parts make it in.
• Build a plot, with a protagonist, a goal, and a conflict.
• Give it a structure, maybe the classic setup, conflict, and resolution.
• Build it from scenes, and combine them to cause an effect.

Saying we manipulate sounds ugly. But every time you stress one detail over another, even without meaning to, you’re steering attention, and that’s a form of manipulation. It’s powerful, so use it for a good cause.

The stories our industry tells itself

I’ve been doing this for a few decades now, and the dominant stories have changed.

Long ago, the story was epic and individualistic. You rode out to fight the client with a couple of colleagues, armed with a lance and a horse, and every project was a great feat of conquest. A toxic story, and not one that served anybody well.

Peter Paul Rubens’s copy of Leonardo da Vinci’s lost Battle of Anghiari: how projects used to be told.

A bit later, when large consultancies grew through the 2000s, the battles got bigger. Armies of consultants won projects at any cost, under the motto “if you can’t convince them, confuse them.” I spent several years at Indra, and I remember the usual ending: the project had been sized badly and scoped badly, and the project manager surrendered to the client after a thousand hours of unpaid work. Unpaid by us, of course.

“The Surrender of Bailén”, by José Casado del Alisal. Picture the project manager surrendering to the client.

Those stories shape software development before anyone writes a line of code. They shape the atmosphere of a company: places where you have to warm the chair, or come to the office so your boss sees you. Luckily there are more and more places where you can build product without falling victim to them.

Then there are stories that are outright misleading. In a South Park episode, some gnomes steal underpants. When they’re caught, they explain their business plan: phase one, collect underpants; phase two, ?; phase three, profit. Nobody ever learns what phase two is.

The underpants gnomes’ business plan, from South Park.

We do this all the time, on purpose or not. Phase one, agile; phase three, profit. Phase one, DDD; phase three, profit. A few years ago, phase one was Kubernetes. Today it’s LLMs. What matters is phase two: what goes in between, and why we do things.

Your context decides the story

We can’t tell the same story every time we build software. Gojko Adzic has a chart I like a lot. On one axis, the consequences of being wrong when you decide to build feature X or take the product in one direction. On the other, your ability to invest: not only money, but time and people. You may have the budget and still not have the teams.

If being wrong costs little and you can iterate as much as you want (a utopia with infinite money, say), iterate. Ship features, don’t even ask the users, and look at which one moves engagement. Failing costs nothing.

If being wrong is serious but you have the means, experiment. Run several experiments in parallel with short feedback cycles from customers and users, and deliver only when you’re confident you’re on the right track. Otherwise you can mess up on a huge scale.

With fewer means, you align on a single path, or discover the user’s needs step by step, going back and forth before delivering.

Ideally you’d live in the top-left corner all the time. You don’t. Recently I worked with a team that badly wanted to work in “iterate” mode. The people leading it were pushing for that, and I understand why: it sounds great. But what they were building sat in the corner where a mistake would blow up in their faces. So my advice was: at least explore the problem space before you start throwing out solutions and keeping whichever moves the metric.

Telling is how knowledge travels

Long ago, knowledge was passed on orally. Fables were tailored to the situation, to keep small children out of trouble. The bogeyman exists so you don’t talk to strangers.

A party at the tavern, by Luis Ricardo Falero: for centuries, this is how stories traveled.

In product development there are many ways to tell the same thing. The dish is the same; the question is which version makes you hungry.

It sounds trivial, but ask yourself how much the way things are told to you affects your motivation to stay where you are. If someone explains the impact your work has on the business, does that motivate you more or less than “product says you have to build this, go code it”? Both are stories.

Then there’s the work itself. Many times a stakeholder or a client doesn’t tell you what they need; they tell you what they want. You have to distill it: separate what they say they want from what they actually need. Then build a context to convince them that what they really want is something else. Then tell the team. We often try to tell things we haven’t fully understood ourselves.

So pick the right tool, depending on what you need to do:

• understand the problem,
• explore solutions,
• build them,
• understand them from the code,
• and from outside the code,
• understand your legacy.

Each of the following tools tells the story in a different way. They’re not for every context. As someone said in the DDD talk before mine, DDD may not apply to your scenario either. Use a tool in the wrong context and it’ll hit you hard.

Impact mapping: start with why

Impact mapping makes you talk about why you want to do something, who is involved, how their behavior should change, and only then what you’ll build. The “what”, the deliverable, comes last.

How many times have we been asked to build software starting from the what? The why gets lost on the way and never even reaches us.

The example on draft.io starts from a goal: grow mobile advertising by 50% over the next 12 months. It splits that goal among three actors: super-fans with mobile devices, live concert organizers, and artist agents and promoters. Then it puts numbers on them. If one actor accounts for 75% of the expected growth, that’s where you invest, and then you look at the impacts you can achieve with that actor, again with numbers.

Half the time those numbers are made up; let’s not fool ourselves. But sometimes you work in a context where you can have them, and then they’re great.

When a team has gone through this process, you work with informed people. You’ve explained the why. And informed people can push back: “why do you want me to build this, if it isn’t aligned with anything above it?” Depending on your management style, that’s annoying. I’d rather have it.

The technique is explained in Gojko Adzic’s short book, Impact Mapping. It’s almost a booklet, nothing like the thick red DDD book.

User story mapping: tell the user’s journey

User story mapping also talks about what you want to achieve, and about how. The why is assumed to be around somewhere, but it isn’t the focus.

The anatomy of a user story map, from Nicky Conti’s “A story about story mapping”.

The technique itself follows a narrative flow:

1. Start with the user journey. Always think of the user first. Sometimes we don’t even know who we’re building for, and if we don’t know that, we can’t question what we build.
2. Identify activities. Picture an online shop: some activities are about logging in and managing users, some are about the product catalog, some about the cart, some about payment. It depends on your business. If you work with sociotechnical architectures, these activities often map to your teams too.
3. Identify tasks, the user stories under each activity.
4. Prioritize. This matters, this doesn’t.
5. Define releases. Draw a horizontal line: out of all the ideas we had, these go into the next release, or the next iteration, or whatever cycle you work in.

We use it with Miro, connected to Jira, so the stories on the board are the actual tickets. One of our real maps covers credit notes, because of the business we’re in: importing documents from a file, creating purchase and sales credit notes, confirming them, with the user stories underneath.

These sessions are great because business people and developers sit at the same table. Developers can talk about how easy or hard each piece is. When everything on the board is a sticky note of the same size, launching a rocket to the moon looks as easy as adding a button.

Event modeling: a story you can follow into the legacy

I had a section on Event Storming and cut it for time. If you don’t know it, this post has some tips for a session, and this one covers how it helps you find the stories your business tells.

Event Modeling works with events too, but it adds two things I find very useful. It puts the user interface in the picture, and it uses swimlanes to make explicit which system each event happens in, or which system a command is sent to.

Remember “own the story”? I don’t need an Event Modeling certification. I can bend the technique to serve me. We use it to describe an 18-year-old legacy system. For each step of the user journey, we link the screen to the HTTP call it makes, to the API that receives it, to the stored procedure that runs, to the view or table it reads. Sometimes there’s a little box with sample columns of the data coming out.

We don’t use this to design something new. We use it so the people who join the team learn not just what happens in the process, but in which pieces of our architecture it happens.

The C4 model: model, don’t draw diagrams

Some people look at the C4 model and see UML that looks a bit nicer. Its motto is what makes it different: model, don’t draw diagrams.

You model your system once, and get views at different levels of zoom. The name comes from the four levels it started with, though it has grown since:

• System context. People, software systems, and the relationships between them. A software system is what you’d call a product, or part of one. In the classic example you see a customer, an internet banking system, an email system, and the mainframe, and what each relationship is for. You don’t need this for something small. For something big, knowing every piece and how they relate is great, and in onboarding it’s a joy.
• Containers. Zoom into one system. A container is anything that runs use cases: a database, an API, a web app, a single-page app, a mobile app. You see how they talk to each other: JSON over HTTPS between the SPA and the API, XML over HTTPS to the mainframe (nobody believes that’s how it works with a mainframe, but they were feeling optimistic), SQL over TCP to the database. At this level you find out what you’re dealing with inside your company, and how many systems have sprung up like mushrooms in the last few months.
• Components. Zoom into a container. In the example they’re the API’s controllers. Think of them as use cases: this one doesn’t touch the database, that one does. You can see how you’re affecting the system when you follow a flow.
• Code. The ugly duckling. The proposal is basically: we already have UML for this, generate it automatically if it helps, and otherwise don’t. I don’t use it. Opening the project, often through its test project, tells me more. The day I need a class diagram, I right-click, generate it, and throw it away.

And then there’s more: deployment diagrams, for instance, which are official C4 too. They tell you the database is MySQL running on Amazon RDS, and so on.

Where the model lives

“Model, don’t draw diagrams” translates into this. All those views come out of a model written in the Structurizr DSL:

workspace {
    model {
        user = person "User" "A user of my software system."
        softwareSystem = softwareSystem "Software System" "My software system."
        user -> softwareSystem "Uses"
    }
    views {
        systemContext softwareSystem "SystemContext" {
            include *
        }
    }
}

The top half is the model: a user, a software system, and a “Uses” relationship. The bottom half asks for a view, a system context of that system that includes everything. With a bigger model, you’d include only some elements to build the view you need, to tell the story you want to tell.

From one model you get the system landscape, the containers, the components, the deployments, each as its own view. You can export them to Mermaid and other formats. That means you can make diagram generation part of your continuous integration, publish them to whatever your wiki supports, and your documentation keeps itself up to date.

“But Modesto, I still have to write that model.” Yes and no, depending on how much detail you want. I’m at the proof-of-concept stage, but it’s working really well, even for discovering artifacts that came out of nowhere. It took me very little to write a small tool that runs in our CI. We work with .NET, which makes this easy; in other ecosystems it may be harder. As part of the build, the tool looks at the projects:

• this one is a website, this one is an API;
• the API has these controllers, so these are the use cases you can invoke;
• these connection strings point to a database and a cache, so there’s a database and a cache.

From that, it generates the DSL. The descriptions of what each system is for, you still write yourself, or ask an AI for them these days. But maintaining this by hand is hell, and it doesn’t work. A new API shows up, and what will you do, add “document this API” to your development cycle? No. Generate it, with the level of detail the story you want to tell needs. Maybe knowing which APIs you have and how they relate is enough, and the names tell you what they’re for.

Structurizr has a product to share all this across teams, and there’s also Structurizr Lite, which you can run locally in a Docker container. Try it. Again: a solution to a specific problem, not something for every context.

Many more tools

Domain Storytelling, context mapping, Wardley mapping, user journey mapping, Event Storming… there are plenty. In the end they’re all tools to help teams navigate the story that gets them, like my son, their bow and arrows.

Just don’t use them to manipulate people so you can do whatever you want afterwards.

What about what we leave behind?

Someone in the audience asked a very good question. You iterate, and each iteration leaves something behind: technical debt, or features nobody ends up using, and they all cost maintenance. Impact mapping helps you decide where to invest, but it doesn’t show that residue. How do you tell that story?

It’s a big problem, and I only have two things that work for me.

The first one: the story that sticks is the one we live day to day, so the residue gets forgotten. When we choose to take on technical debt, or to run an experiment, we treat it like a mortgage. You sign it knowing you’ll have to make payments. So we commit up front: when will we come back to pay off this debt? When does this experiment end, and what conditions decide whether we keep it or drop it? How do we communicate that this feature is temporary and only stays if people use it? You write the date down like you’d write down a birthday. The goal is that remembering doesn’t depend on the one super-motivated person on the team who cares about it. That applies to development, to product, to everything.

The second one, for code: metrics on what gets used and what doesn’t, and which parts of the code we touch the most and hurt the most. Making it visible.

And even so, sometimes it doesn’t work.