Slicing Stories? Don’t Use Horizontal Layers

i’ve had some questions & comments about yesterday’s tweet:

that’s natural and good and thanks to one and all. when one tweets a one-off like that, all pithy and telegraphic, folks who may not “already speak the lingo” can be mystified. i’ll take a few tweets to spell it out a little further.

a little about the context. in the modern synthesis, i speak of four optimizations we want to attend to in writing software for money. we optimize for collaboration, for sustenance, for drive, and for stepping & mis-stepping.

optimizing for sustenance is the most relevant for the story-slicing tweet, so let’s go there. optimizing for sustenance is about building our software in such a way that we are able to feed ourselves sustainably as we go. it is the continuous ongoing reaping of the value that our software adds to the world.

there are good reasons to set the “journey” metaphor aside, but it’s still commonplace, so… optimizing for sustenance means gathering food and fuel *as* *we* *go*. this opposes the older style, where we build software to reap its value only at the “end” of the journey.

user stories are one solid mechanism for getting us and keeping us in this mode or frame. they describe some aspect of what we’re wanting to build, focusing their attention on who it’s for and why they want it. user stories then help us organize what we work on, and — user stories are always applied in time — when we work on it.

this “in time” aspect creates great confusion for noobs, but it’s a critical factor in their usefulness and in the modern synthesis outlook. when one first comes to stories, one is often coming from the world of a requirements document. requirements documents are *not* “in time”, but “once and for all”. they are not built for “feed & fuel as you go”, but for “feed & fuel provided upfront and recouped at the very end.”

so. we use stories to organize what we do and when. but we’re not done nailing down stories just yet. why not? optimizing sustenance is an intricate calculus involving cost & benefit & all the other optimizations. a central number in that calculus: “how big is a story?” to get this theory into actual motion, we need that calculus, and we’re going to have understandings about the size of a story.

how big is a user story? how much work is in the typical one? how much work is in the biggest story we’ll call legit?

twenty years into this, and we’ve tried a ton of different ways to specify the answer to this question. i would like to tell you there is a consensus answer, but i can’t. all i can tell you is that there is a consensus answer among the people whose advice i trust the most.

how big is a story? in my circle, we seek to create stories that take “about a day or two in theory”. i know, i know. there’s a lotta fuzzy pulp in that juice.

it comes down to this, whatever else we’ve learned, we’ve learned that smaller stories are better. they are better for a bunch of reasons i’m not even gonna bother to go into today — read my backline or ask me questions if you need more.

now, armed with all this, we’re finally ready to think about story slicing. the problem is as simple and as hard as this: “we have a thing we think is a story, but no way can it fit into about a day or two in theory, so we need to slice it into smaller stories.”
if a story is to be sliced, split into pieces, will any old pieces do? the answer is no, decidedly not, and my tweet yesterday was a short strike against the most common form of mistake in story-slicing: slicing by horizontal architectural layer.

say our story — in many shops this would apply to 100% of our stories — will require changing 1) the web pages, 2) the endpoints, 3) the access rules, 4) the controllers, 5) the rules engine, 6) the threading, and 7) the database.

easy, the noob says, we’ll slice our story into 7 stories, one for each of those numbers above. but that’s not what an olb says. i’m an olb, and it’s exactly the opposite of what i said yesterday. why not? (that kind of slicing is called horizontal or layered slicing, shorthand i’ll keep using).

what’s wrong with layered slicing?

the problem is that a layered slicing, while keeping the part of our story that has to do with when and what, completely ignores the overarching justification for using stories in the first place: optimizing for sustenance.

when we slice a story, we are not trying to divide (or even optimize, we’ll come to that) the *labor* that goes into it. we are trying to divide the *value* that comes out of it.

on paper, the value we receive at the end of an effort is always the same as the value we planned to get, and it’s always the same as the sum of the efforts we put into it. this is idols of the schema with a vengeance: it is approximately never the case in business practice.

on paper, nothing ever changes during an effort. we encounter neither internal discoveries that make us want to shift our plan, nor external conditions that make us want to shift our priorities in the market. but in actual business practice, both events are ubiquitous.

and that is why we want to split stories in such a way that each story provides value, regardless of whether the as-yet-unstarted stories are ever chosen for implementation.
put another way: if we slice that story above into those seven layers, when can we walk away? when can we harvest whatever sustenance we get and move on to the next thing?

and the answer that kills that kind of slicing is: “when we have done all seven stories.”

“when we have done all seven stories” is too late. we can’t ignore internal discovery or external change for that long. we can’t not harvest value for that long. and thus my tweet. story slicing by horizontal architectural layers is a common mistake. it derives from mis-focused attempts at optimization, from misunderstanding what we’re trying to divide, and from faux-rational arguments on paper that don’t work in reality.

this is a long muse already, and i feel i have two other long ones aspawning: how, then, to slice? and why optimizing for business and optimizing for geek labor are often two different things. so let me wrap this one up.

we seek to optimize for sustenance, the ongoing partial harvest of value. we use stories — descriptions of partial value — to tell us what and when to work. we want them small. when we split them, the thing we’re distributing across the splits is *value*.

I&I: What “Increment And Iterate” Means

i genuinely believe that nearly all the woe in software development, the whole socialtechnical enterprise, derives from the belief that we can sidestep increment & iteration, “a little better now” and “we’ll change it again later”.

this applies to the purest coding geekery, and it applies to the schmooshy marketing, and it applies to the structures and process. it applies everywhere we make changes, and in software, everywhere we make changes is everywhere.

when i make attacks on certainty, i’m saying that i think they’ve sidestepped i&i. when i push against “one best way”, i’m pushing off from i&i. when i advocate always small, always better, always rework, you guessed it, that’s just me giving i&i it’s true value.

the hardest part of i&i to grasp is actually their interconnectedness. everybody gets increment at first blush. “we will split the journey into pieces, and each piece will be one increment closer to the destination”. the problem with that first blush approach is that travelling from NY to SF is so trivial a problem as to be incomparable with the problems we deal with in the software-for-money trade.

marketing is way harder than travelling. coding is way harder than travelling. managing? way harder. process changing? yep, way way harder.

you see, none of these problems *split* well, the way lines on a map split at towns. with a map, the splits add up. every day i get closer. each increment adds to progress, and no increment ever goes truly backwards or even very sideways. but in our problems, the splits are far more challenging to make. it has to do with a variety of factors. three of these stand out large, tho, so i’ll enumerate.

  1. first, the destination moves around. SF does in fact move very slowly over time, but on the scales at which travel takes place, it’s negligible. in software-for-money, destinations move fast and unpredictably.
  2. second, even if our destination holds relatively still, the territory between us right now and that destination is largely unexplored, and thus more or less impossible to draw in simple increments.
  3. third, because there are people everywhere in our problem, we must constantly account in our splitting for the whole staggering complexity of what people can do when, and that is far beyond the dictates of simple reasoning.

so what we’re saying is that the increment part can’t stand by itself. plotting a route and progressing additively along it every day just isn’t enough to get us there. it’s very important that we grasp that this is not about desire or will. yes, of course we wish it were so. yes, of course we are willing to throw considerable effort at making it so.

but we can’t make it so. our desire and our will have to drop in the face of the reality.

enter the iteration. the second i of i&i is the acceptance of the continuous reworking of increments we have already taken. iteration here is the repetition of change: we are changing things we already changed. we are assuming, further, that we will change them *again*, the third, the fourth, as many times as we need to to produce interactive splitting of our journey.

it’s like viewing each increment as tentative, a probe, an exploration, an experiment. we engage in those increments because we believe they’ll result in forward motion, but we accept that our belief and the world are two very different things.

this is not a simple relationship, between the two i’s, and we want to be careful not to underestimate its impact.

  • because i am iterating, i can make my increments small and fast.
  • because i am iterating, i must make those increments suitable for revisiting.
  • because i am incrementing and iterating both, i must resist sitting still while i try to figure out exactly how it’s gonna go.
  • because all of our workers are i&i’ing they need training and experience to give them mastery of the techniques, and they must surely have healthy (iterated) doses of purpose and autonomy.
  • fear is the killer, and we must provide teams doing i&i with the safety to be wrong, and the techniques needed to make “being wrong” relatively cheap to us.
  • and because the work is incremental but with uncertain results, we must expect a high need for rhythm, the frequent building and releasing of compressed tension.

this all adds up to a *lot*. as we might expect with a sociotechnical enterprise, we will be constantly mixing technique, structure, and human interaction. people want so much for there to be an easier way. even some folks within our own tradition market their technique as easier, and some no doubt believe that.

i want an easier way, too. but there isn’t one.

if we are to succeed at software for money, we’ve no better choice at this time than to master the ins & outs of increment and iteration.

“change it better now” and “change it better again later”.

Success Theater, Corruption, And My Own Medicine

the more i work higher up the chain, the more i encounter folks who are openly seeking success theater, w/no interest or concern for actual value.

this observation blinded me for many years. i have an odd-shaped-to-some but very strong sense of personal responsibility, myself, and i am no less prone to moralizing judgments than the next old testament prophet.

my reactions varied, often enough based quite unfairly on the simple metric of how much i liked the person in question. but my basic stance has always been “that’s corrupt”.

is it?

in my little motivational muses, i talked about RAMPS: rhythm, autonomy, mastery, purpose, and safety. and i have repeatedly seen that when individuals aren’t having the right levels for themselves of that five-color spectrum, they will falter.

have you ever been — it’s rhetorical, i know you have — in a situation where you were being held responsible for something over which you have not the slightest influence, let alone direct control?

in situations like that, there are always several things you can try. for instance, u can seek more influence/control. u can re-shape the responsibility. u can undermine the value of that responsibility, so it is understand as only your third or fourth priority. u can combine all these in various proportions. i have seen all these things tried. i have tried many of them. we can attach various theories of moral sentiment to each or to a given compound. and when none of these quite seem to feed the admiral’s cat, what will you do next?

well. if you value your security, you’d be dumb not to consider success theater, wouldn’t you?

i have argued against the software craft movement, because i believe it makes unfair assumptions about the responsibility of a working geek, both implicitly and sometimes openly passing judgment on their souls without actually ever seeing their situations. and now, full of moral superiority over those who openly practice success theater, i’ve little choice but to turn the hose on myself.

i hate it when that happens.

when you ask me why geeks down on the floor aren’t doing what/how you wish they would, i automatically try to investigate. very frequently, the result is to say: because the systems they are in don’t lead to or support doing that.

when you’re down on the floor, and your managers and executives aren’t doing what/how you wish they would, you might investigate. and very frequently, you’ll discover that the systems they live in don’t lead to or support doing that.

managing change upwards is not different from managing it downwards or to the side. if it seems to you that it is, you’re likely still thinking that the power you have to change the world depends on your ability to give orders.

i can assure you that it does not.

i advise you now because it’s often easier to get me to do the thing when i tell other people to do the thing:

be patient. be kind. suspect situations before people. find tiny ways to make those situations more conducive to the change you wish to see.