Slicing Stories? Don’t Use Horizontal Layers

i’ve had some questions & comments about yesterday’s tweet:

that’s natural and good and thanks to one and all. when one tweets a one-off like that, all pithy and telegraphic, folks who may not “already speak the lingo” can be mystified. i’ll take a few tweets to spell it out a little further.

a little about the context. in the modern synthesis, i speak of four optimizations we want to attend to in writing software for money. we optimize for collaboration, for sustenance, for drive, and for stepping & mis-stepping.

optimizing for sustenance is the most relevant for the story-slicing tweet, so let’s go there. optimizing for sustenance is about building our software in such a way that we are able to feed ourselves sustainably as we go. it is the continuous ongoing reaping of the value that our software adds to the world.

there are good reasons to set the “journey” metaphor aside, but it’s still commonplace, so… optimizing for sustenance means gathering food and fuel *as* *we* *go*. this opposes the older style, where we build software to reap its value only at the “end” of the journey.

user stories are one solid mechanism for getting us and keeping us in this mode or frame. they describe some aspect of what we’re wanting to build, focusing their attention on who it’s for and why they want it. user stories then help us organize what we work on, and — user stories are always applied in time — when we work on it.

this “in time” aspect creates great confusion for noobs, but it’s a critical factor in their usefulness and in the modern synthesis outlook. when one first comes to stories, one is often coming from the world of a requirements document. requirements documents are *not* “in time”, but “once and for all”. they are not built for “feed & fuel as you go”, but for “feed & fuel provided upfront and recouped at the very end.”

so. we use stories to organize what we do and when. but we’re not done nailing down stories just yet. why not? optimizing sustenance is an intricate calculus involving cost & benefit & all the other optimizations. a central number in that calculus: “how big is a story?” to get this theory into actual motion, we need that calculus, and we’re going to have understandings about the size of a story.

how big is a user story? how much work is in the typical one? how much work is in the biggest story we’ll call legit?

twenty years into this, and we’ve tried a ton of different ways to specify the answer to this question. i would like to tell you there is a consensus answer, but i can’t. all i can tell you is that there is a consensus answer among the people whose advice i trust the most.

how big is a story? in my circle, we seek to create stories that take “about a day or two in theory”. i know, i know. there’s a lotta fuzzy pulp in that juice.

it comes down to this, whatever else we’ve learned, we’ve learned that smaller stories are better. they are better for a bunch of reasons i’m not even gonna bother to go into today — read my backline or ask me questions if you need more.

now, armed with all this, we’re finally ready to think about story slicing. the problem is as simple and as hard as this: “we have a thing we think is a story, but no way can it fit into about a day or two in theory, so we need to slice it into smaller stories.”
if a story is to be sliced, split into pieces, will any old pieces do? the answer is no, decidedly not, and my tweet yesterday was a short strike against the most common form of mistake in story-slicing: slicing by horizontal architectural layer.

say our story — in many shops this would apply to 100% of our stories — will require changing 1) the web pages, 2) the endpoints, 3) the access rules, 4) the controllers, 5) the rules engine, 6) the threading, and 7) the database.

easy, the noob says, we’ll slice our story into 7 stories, one for each of those numbers above. but that’s not what an olb says. i’m an olb, and it’s exactly the opposite of what i said yesterday. why not? (that kind of slicing is called horizontal or layered slicing, shorthand i’ll keep using).

what’s wrong with layered slicing?

the problem is that a layered slicing, while keeping the part of our story that has to do with when and what, completely ignores the overarching justification for using stories in the first place: optimizing for sustenance.

when we slice a story, we are not trying to divide (or even optimize, we’ll come to that) the *labor* that goes into it. we are trying to divide the *value* that comes out of it.

on paper, the value we receive at the end of an effort is always the same as the value we planned to get, and it’s always the same as the sum of the efforts we put into it. this is idols of the schema with a vengeance: it is approximately never the case in business practice.

on paper, nothing ever changes during an effort. we encounter neither internal discoveries that make us want to shift our plan, nor external conditions that make us want to shift our priorities in the market. but in actual business practice, both events are ubiquitous.

and that is why we want to split stories in such a way that each story provides value, regardless of whether the as-yet-unstarted stories are ever chosen for implementation.
put another way: if we slice that story above into those seven layers, when can we walk away? when can we harvest whatever sustenance we get and move on to the next thing?

and the answer that kills that kind of slicing is: “when we have done all seven stories.”

“when we have done all seven stories” is too late. we can’t ignore internal discovery or external change for that long. we can’t not harvest value for that long. and thus my tweet. story slicing by horizontal architectural layers is a common mistake. it derives from mis-focused attempts at optimization, from misunderstanding what we’re trying to divide, and from faux-rational arguments on paper that don’t work in reality.

this is a long muse already, and i feel i have two other long ones aspawning: how, then, to slice? and why optimizing for business and optimizing for geek labor are often two different things. so let me wrap this one up.

we seek to optimize for sustenance, the ongoing partial harvest of value. we use stories — descriptions of partial value — to tell us what and when to work. we want them small. when we split them, the thing we’re distributing across the splits is *value*.

I&I: What “Increment And Iterate” Means

i genuinely believe that nearly all the woe in software development, the whole socialtechnical enterprise, derives from the belief that we can sidestep increment & iteration, “a little better now” and “we’ll change it again later”.

this applies to the purest coding geekery, and it applies to the schmooshy marketing, and it applies to the structures and process. it applies everywhere we make changes, and in software, everywhere we make changes is everywhere.

when i make attacks on certainty, i’m saying that i think they’ve sidestepped i&i. when i push against “one best way”, i’m pushing off from i&i. when i advocate always small, always better, always rework, you guessed it, that’s just me giving i&i it’s true value.

the hardest part of i&i to grasp is actually their interconnectedness. everybody gets increment at first blush. “we will split the journey into pieces, and each piece will be one increment closer to the destination”. the problem with that first blush approach is that travelling from NY to SF is so trivial a problem as to be incomparable with the problems we deal with in the software-for-money trade.

marketing is way harder than travelling. coding is way harder than travelling. managing? way harder. process changing? yep, way way harder.

you see, none of these problems *split* well, the way lines on a map split at towns. with a map, the splits add up. every day i get closer. each increment adds to progress, and no increment ever goes truly backwards or even very sideways. but in our problems, the splits are far more challenging to make. it has to do with a variety of factors. three of these stand out large, tho, so i’ll enumerate.

  1. first, the destination moves around. SF does in fact move very slowly over time, but on the scales at which travel takes place, it’s negligible. in software-for-money, destinations move fast and unpredictably.
  2. second, even if our destination holds relatively still, the territory between us right now and that destination is largely unexplored, and thus more or less impossible to draw in simple increments.
  3. third, because there are people everywhere in our problem, we must constantly account in our splitting for the whole staggering complexity of what people can do when, and that is far beyond the dictates of simple reasoning.

so what we’re saying is that the increment part can’t stand by itself. plotting a route and progressing additively along it every day just isn’t enough to get us there. it’s very important that we grasp that this is not about desire or will. yes, of course we wish it were so. yes, of course we are willing to throw considerable effort at making it so.

but we can’t make it so. our desire and our will have to drop in the face of the reality.

enter the iteration. the second i of i&i is the acceptance of the continuous reworking of increments we have already taken. iteration here is the repetition of change: we are changing things we already changed. we are assuming, further, that we will change them *again*, the third, the fourth, as many times as we need to to produce interactive splitting of our journey.

it’s like viewing each increment as tentative, a probe, an exploration, an experiment. we engage in those increments because we believe they’ll result in forward motion, but we accept that our belief and the world are two very different things.

this is not a simple relationship, between the two i’s, and we want to be careful not to underestimate its impact.

  • because i am iterating, i can make my increments small and fast.
  • because i am iterating, i must make those increments suitable for revisiting.
  • because i am incrementing and iterating both, i must resist sitting still while i try to figure out exactly how it’s gonna go.
  • because all of our workers are i&i’ing they need training and experience to give them mastery of the techniques, and they must surely have healthy (iterated) doses of purpose and autonomy.
  • fear is the killer, and we must provide teams doing i&i with the safety to be wrong, and the techniques needed to make “being wrong” relatively cheap to us.
  • and because the work is incremental but with uncertain results, we must expect a high need for rhythm, the frequent building and releasing of compressed tension.

this all adds up to a *lot*. as we might expect with a sociotechnical enterprise, we will be constantly mixing technique, structure, and human interaction. people want so much for there to be an easier way. even some folks within our own tradition market their technique as easier, and some no doubt believe that.

i want an easier way, too. but there isn’t one.

if we are to succeed at software for money, we’ve no better choice at this time than to master the ins & outs of increment and iteration.

“change it better now” and “change it better again later”.

Success Theater, Corruption, And My Own Medicine

the more i work higher up the chain, the more i encounter folks who are openly seeking success theater, w/no interest or concern for actual value.

this observation blinded me for many years. i have an odd-shaped-to-some but very strong sense of personal responsibility, myself, and i am no less prone to moralizing judgments than the next old testament prophet.

my reactions varied, often enough based quite unfairly on the simple metric of how much i liked the person in question. but my basic stance has always been “that’s corrupt”.

is it?

in my little motivational muses, i talked about RAMPS: rhythm, autonomy, mastery, purpose, and safety. and i have repeatedly seen that when individuals aren’t having the right levels for themselves of that five-color spectrum, they will falter.

have you ever been — it’s rhetorical, i know you have — in a situation where you were being held responsible for something over which you have not the slightest influence, let alone direct control?

in situations like that, there are always several things you can try. for instance, u can seek more influence/control. u can re-shape the responsibility. u can undermine the value of that responsibility, so it is understand as only your third or fourth priority. u can combine all these in various proportions. i have seen all these things tried. i have tried many of them. we can attach various theories of moral sentiment to each or to a given compound. and when none of these quite seem to feed the admiral’s cat, what will you do next?

well. if you value your security, you’d be dumb not to consider success theater, wouldn’t you?

i have argued against the software craft movement, because i believe it makes unfair assumptions about the responsibility of a working geek, both implicitly and sometimes openly passing judgment on their souls without actually ever seeing their situations. and now, full of moral superiority over those who openly practice success theater, i’ve little choice but to turn the hose on myself.

i hate it when that happens.

when you ask me why geeks down on the floor aren’t doing what/how you wish they would, i automatically try to investigate. very frequently, the result is to say: because the systems they are in don’t lead to or support doing that.

when you’re down on the floor, and your managers and executives aren’t doing what/how you wish they would, you might investigate. and very frequently, you’ll discover that the systems they live in don’t lead to or support doing that.

managing change upwards is not different from managing it downwards or to the side. if it seems to you that it is, you’re likely still thinking that the power you have to change the world depends on your ability to give orders.

i can assure you that it does not.

i advise you now because it’s often easier to get me to do the thing when i tell other people to do the thing:

be patient. be kind. suspect situations before people. find tiny ways to make those situations more conducive to the change you wish to see.

Beating the Human Wave Strategy In The Geek Trades

the seemingly insatiable demand for geekery creates a marketplace with many opportunities, but also many powerful distortions.

among them, the one that seems most puzzling to me is the trade’s notable disregard for the urgency of enculturating the “makers making” upon whom the entire business proposition rests.

successful geekery requires individual humans using individual judgment. that judgment is backed by a number of factors, two of which seem predominant: the range of experience of the judge, and the community in which the judgment takes place.

yet very few software development companies seem to invest heavily in either factor. they don’t build community, and they make few attempts at accelerating and varying the accretion of experience. to the extent some few provide experience, they do it through force-feeding purely technical “training” classes. they strive to replace community w/rule-making and metric-mongering.

at bottom, they seem to believe that the binding curve for software development is centered around the developer’s ability to memorize syntax. this ability — the faculty for absorbing in advance what simple rigidly structured text will do when it’s run on the computer — this is actually an *entry* hurdle, not a predictor or controller for overall success. in logic terms, being able to parse structured text is necessary to successful geekery, but not remotely sufficient to it.

one way i’m fond of seeing professional geekery is as an ongoing incremental and iterative series of acts of translation. geeks translate the fluid, analogical, idiomatic, and ever-shifting language of humans, into the simple, rigid, non-reactive language of the turing machine.

a translator must certainly grasp the syntax of the source and destination languages, without question. that is a *necessary* condition for successful translation. but it’s not enough. actual translation requires far more than this. successful translation depends heavily on exactly the two factors i just mentioned, the range of the translator’s experience, and the community in, of, and around the source and the destination.

the trade spends its extraordinary wealth building an ever-larger population of people who can more or less do syntax, instead of bringing those who pass that entry hurdle what they’d need to become great translators: experience & community. this human wave strategy, sending directly into action as many people as we can find who can get over the entry hurdle, counting on time alone to winnow the field of candidates, and hoping strong individuals will “naturally” rise to excellence, is doing a great deal of damage.

we can beat this approach the same way one usually beats it: by taking a smaller number of people, providing them a far wider range of experience, and supporting the formation of communities in which they can do excellent translation.

i believe we can do better. i am actively seeking organizations that agree with me and want assistance, support, and guidance in their efforts.

i am easy to reach. i’m this twitter handle at this twitter handle dot org.

please get in touch.

 

Agility Perks Up From Below

plumbers make this joke on day one of having a new hire. “the first law of plumbing: shit flows downhill.” that line probably pre-dates western civilization. it’s good to make a noobie kid laugh a little nervously.

agility doesn’t flow downhill.

agility doesn’t flow downhill, it perks up from below. in one sentence, this is because the bottom of the hill is where the people who directly make things make them, and agility is concerned most particularly with them, with how their lives intertwine with the making.

i spend my time in an organization working with the folks at the bottom three floors, only occasionally reaching higher into the org chart. that’s not an accident or a prejudice or the result of my politics or the best i get offered. it’s a conscious choice based in this notion that those first few floors are where the makers live and the making happens. of course, i’m entirely disinterested in floors per se — or even org charts. what i mean is that the further i am away from the “makers making” part of an org, the less value i can offer that org in terms of better outcomes.

over the years, i’ve had many peers in the coaching game whose great desire was to work at the top, not at the bottom. for me, there is only one in-practice reason and lots of spurious in-theory reasons why i would desire that. (in practice, it offers more money and prestige for less actual provided value, a perfectly legitimate reason to choose it, but not one that floats my particular boat — except on jealous days, that is.)

the last two centuries are full of people who famously created and led organizations that make things for money. you certainly know of some, and a few of you even idolize one or more of those folks. so let me ask you a question. how much did your “typical” entrepreneurial hero spend — time, money, thought, experience-building, sheer *energy* — on, with, around, and about, the actual people actually making the actual things being sold?

easy answer: “lots and lots and lots”.

henry ford’s one of the big ones, for all his grotesquerie as a human, who quite literally changed the world. he did this by being intimate, to the point of obsession, with the *makers* *making*. he was a paternalist dickhead, so lots of his decisions we’d find gross, but his *focus* was spot on. “my company depends on makers making, so that is where i’ll work the most.” there was no detail in the world of his makers making that was too small to command his attention.

pigeonholes being what they are, convenient places to stop thinking, coaches are often “technical” or “product” or “process” coaches. they frequently self-identify that way, even. folks often ask me if one of these little tissue-wrapped boxes is the best one. if they’re failing in one of them, they ask me if they should do the other. if they’re succeeding, they ask me why i don’t.

orgs or org-parts that depend on makers making depend on every aspect of that: the technique, the flow, the spirit, the structure, the tools, every part. just as the making — focused on the made — needs lots of different skills that very few individuals possess in their entirety, so does the coaching — focused on the making. these boxes, “process”, “technical”, “coaching”, are useless.

whatever your skillset and your method, if you’re a software development coach you are a “makers making” coach, and the closer you stay to that, the more value you will be add, be it from any of the skills and insights you possess.

shit flows downhill, but agility perks up from below, because agility is focused on the makers making, and that is where they live.

Sticky Change: “Changers Feel Better”

yesterday, this popped out.

today i want to elaborate a little on that second point, that sticky change happens when the changers feel better. all three words of “changes feel better” carry weight, have subtleties, and present possible fail points, so let’s look at them one at a time.

changers are the people who actually have to make the trip from one behavior set to another across time. changers are persons, individual humans, who are doing X now and who we are hoping will do Y soon. as such, they are a source of rich variety that has huge implications for the stickiness of any given X->Y change.

if we did a spectrographic analysis of the changers, we’d find that each one shows a different fingerprint, along measures like “tolerance for change”, “current satisfaction”, “attention available to give”, “persistence when failing”, and on and on the list goes. trends exist, patterns, there are bell curves in each of those measures, but a fail point emerges when we — seeking change for others — hold too tightly our attachment to whatever model we have of the general mass. changers are not a general mass, they are individuals.

“better” is about the relative state of those individuals, when they were doing X before, and when they are trying/doing Y now. the single most common fail point i see in change-seekers is what we call “letting best be the enemy of better”.

wanting the new behavior Y to be ever and always the finished final best way to do things would be fine, *if* that desire were not itself often the very thing preventing its own accomplishment. sadly, it is. choosing best vs better often leads us to failure in two ways. 1) it makes us spend more time arguing about best. 2) it makes us want to make larger changes than the human changers can integrate over time.

and we come to the feel. “changers *feel* better”. this is very hard to get at. not only is feeling even abstractly a very slippery concept, but if you look around yourself right now, you’ll see that even your own feelings are fluid, flickery and flummoxing to you.

i suppose what i’m saying with that word is about a vague general response to the new behavior set Y that’s shaped this way: “i like doing Y more than i liked doing X.”

the obvious fail point in sticky change is simply that that is *not* the vague general response. when asked, the changers say they don’t like doing Y more than X, or worse, they actively dislike it compared to X. but that’s just the obvious one. there are lots of little possible losses here.

  • it happens sometimes that some, even many, like Y better, but that some, a plurality don’t.
  • it happens sometimes that we don’t like Y better for reasons having nothing to do with the value Y was supposed to offer.
  • it happens sometimes that a person likes Y better but actually does something slightly Y-shaped that isn’t Y.
  • it happens sometimes that a single Y-disliker can cast a very long shadow over a group that actually likes it.
  • sometimes “changers feel better”, but things aren’t actually better.
  • sometimes change-seekers feel better but changers don’t.

all of these are chances for us to fail at getting to sticky change. navigating them is exactly the art of coaching. it is why being an expert in agility is not the same as being an expert at coaching.

the secret to sticky change is getting as early and often as possible to this one state: “changers feel better”. the secret to being a coach is sidestepping fail points and aligning the various elements in such a way that you provide value to your clients. it’s all a great mass of nuance and sensitivity, and being a coach means willfully engaging with it. it’s a fascinating field of endeavor, if one has the patience and the good cheer for it.

The Technical Meaning Of Microtest

i write things called “microtests” to do my development. doing this yields me greater success than i’ve had using any other technique in a 40-year career of geekery, so i advocate, wanting to share the technique far & wide.

before we can talk seriously about how or whether this works, we need a strong grasp of what a microtest is and does, from a strictly *technical* perspective, w/o all the trappings of the larger method and its attendant polemic.

a single microtest is a small fast chunk of code that we run, outside of our shipping source but depending on it, to confirm or deny simple statements about how that shipping source works, with a particular, but not exclusive, focus on the branching logic within it.

i want to draw your attention to several points embedded in that very long sentence. it’s easy for folks to let these points slide when they first approach the idea. that’s natural, we always bring our existing context with us when we approach a new idea, but if we underplay their significance, it’s easy to head down both theoretical and practical dead-ends.

Externality

first, we have microtest “externality”, the property that says the microtest code runs outside the shipping code.


this is a snap from a running app. the app’s called contentment, and when it runs it draws this diagram, interpolated over time, as if i were drawing it by hand.


this, on the other hand, is a snap from a microtest run against the contentment app:

why do they not resemble each other at all? because the app is the app and the microtests are the microtests. they are two entirely separate programs.

Source-Only Relationship

second, and this is a little buried, the connection between these two separate apps exists at the *source* level, not at runtime.

the microtest app does not launch the app and then poke it to see what happens. rather, *both* apps rely on some shared code at the source level. the app relies entirely on the shipping source. the microtests rely on the shipping source and the testing source.

this is very important to grasp very early on. in some testing schemes, our tests have a runtime connection to an instance of our app. they literally drive the app and look at the results.

this is not how microtests work. nearly every modern programming language comes with the ability to use a single source file in two different apps. microtest schemes make heavy reliance on this capability.

Branchy Focus

third, microtests focus primarily on branchy logic within the shipping source, with secondary attention to some numerical calculations, and sometimes a small interest in the interface points between the shipping source and the platform on which it runs.

here’s a file, SyncTest.

This is a microtest file that uses the shipping source’s Sync class.

the Sync object has just one job, to have its interpolate()  method called. if the beat is less than the sync’s target, return true (so that interpolate()  will be called again). otherwise return false (so the caller will stop asking).

the Sync object’s behavior *branches* logically. there are two branches, and there are two tests, one that selects for each case.  (this example is real, if against a rather straightforward responsibility. in the full context, tho, correctly syncing the drawing to the video or audio source is actually the whole *point* of the contentment app.)

microtests mostly focus on just this: places where *our* code changes its behavior, doing one thing in one call and another thing in another call. we also use them for testing calculations. contentment does a lot of coordinate math. the part of contentment that does geometry has lots of mostly non-branchy code & plenty of microtests to make sure it does its basic algebra & trig the way it’s supposed to.

sometimes, though not often, microtests will focus on the interface points between the shipping source and its platform. the contentment app uses javafx’s animation framework. there are a couple of classes that serve this functionality to the rest of the app. the microtests here essentially establish that this interface point works correctly.

a more common example out in the wild: a bit of SQL is richer than just ‘select * from table’, and we write some microtests to satisfy ourselves that that bit of SQL gives us the right response in a variety of data cases.

Active Confirmation

the fourth aspect of that long sentence above is caught in the phrase “confirm or deny simple statements”. microtests actively assert about what happens in the shipping source.

microtests don’t just run a piece of the shipping source and let the programmer notice whether or not the computer caught on fire. rather, they run that piece with a particular data context and (partially or fully) validate the results of the call.

in the SyncTest file above, the line that says

is calling the interpolate() AND checking that it answers false.

microtests are active, not passive.

Small and Fast

the fifth and final (for now) aspect of microtests is that they are small and fast.

a typical java microtest is well under 10 lines of code beginning to end. it runs in milliseconds. it tests only a single branch. it doesn’t usually assert everything about the results, but usually only one or two things about them.

the SyncTest above seems trivial, because Sync’s responsibility is, however critical to correct functioning of the app, trivial. *but*. the average microtest in contentment is five lines long, longest is 20, most of which is taken up by 12 lines of try/catch java noise.

Industrial Logic’s great app for serving online instructional material is a good case. its 1200 microtests eat under a minute.

These Are Microtests

so there we go. five important properties of microtests are 1) externality, 2) source-only relationship to app, 3) branchy focus, 4) active confirmation, and 5) small size and high speed.

if we’re talking about microtests, we’re talking about tests that have these properties.

there are many questions waiting in the wings from this starting point. among others, “why on earth would i do this?” and “how can i do this when my code doesn’t want me to?”

we can get there. but we can’t get there without remembering these properties.

TDD: Resist Integration Tests

the expense of an integration test can be extremely high. consider the contentment app. this app makes drawings 1) that  distribute across time, as if they were being drawn live in front of you, 2) that are generated stochastically, 3) with a “pixel-inaccessible” framework.

now, it’s important to understand that none of these problems are insurmountable. before you tell me how you’d surmount them, let me tell you how i could. 1) screw time. rig it so it draws as fast as possible and wait for the end to assert. 2) screw stocastic, rig your prng so that you control the number sequence. 3) screw pixel-inaccessible, treat the medium like h/w and put a tee in front of it, assert against the tee.

all eminently doable, at some expense. so should it be done?

i prattle on about the money premise, and i want to be sure you understand how it applies here. please do attend, as this is absolutely *central* to learning how to make TDD work for you.

the money premise says we’re in this to “ship more value faster”.

reminder: “value” could mean any number of things, including fewer bugs, better runtime, or more cases or features. all that matters about the definition of “value” for this purpose is that it is dependent on us changing even modestly complex logic in code.

suppose you surmounted the three difficulties above. what would you have to assert against at the end? depending on your technique for trapping the output, you’d have either an ASCII log with a bunch of labelled doubles in it, or a literal image snapshot. you could either eyeball a given run and say “that will do, don’t change this caught behavior,” which we call “bless”, or you could figure out approximate values for those doubles in advance and assert against them, which we call “predict”.

either way, you will have spent a great deal of money getting those assertions, yes? the three surmounted challenges are not cheap, and tho blessing is cheaper than predicting — we’re talking about a lot of data here — neither of those is very cheap either.

what do the literal assertions you write actually assert conceptually about the contentment app?

that, say, given a blank screen, a specific sequence from the prng, instructions to human-draw over time a line from one corner to the other, at the end there is indeed a line from approximately one corner to approximately the other.

wow. you haven’t proven very much. a real script involves dozens of such lines, as well as text. a real script uses a real prng. a real script is inherently stochastic beyond your control because it depends on the timing of the not-owned-by-you multi-threading in the rendering. (aside, not to mention the odds are good that your test is quite fragile, and will break when you change things in the code that do not actually matter to the user.) i could prove all the things you proved without any rig at all. i could write the script that does that and run it and look, in a fraction of the time, and know just as much as that automated integrated test tells me.

in order to get very high confidence that my code could be safely changed, i would need *thousands* of these extremely expensive tests, and thousands of hours of predicting or blessing work besides.

now, your app may not be graphical like mine, it may not be performing its main function distributed across time like mine, and it may not be stochastic like mine. or. well. maybe it *is* like mine. if you’re working database to browser in a huge sprawling database with thousands of users and hundreds of screens and complex workflow and the kind of bizarre business logic enterprise apps demand, maybe it is like mine.

writing an integration test for it would be investing a very great deal in proving a fraction of a fraction of a fraction of the program’s viability. selenium, a test server, the slow runtimes of the intranet, the golden master db, and most particularly the effectively infinite cyclomatic complexity of the app as seen entirely from the outside.

so?

don’t do that.

put simply, the money premise says that we do TDD because we want more value faster. integration tests in most complex apps do not provide more value faster. as a direct result, in TDD we write very few integration tests, and suggest them very rarely.

My TDD Isn’t Judging Your Responsibility

An old blog of mine, We’re In This For The Money, recently got some attention, delighting me, of course. A full h/t to Jim Speaker @jspeaker for that! He quoted me thus:


Too busy to #TDD?
Too busy to pair?
Too busy to #refactor?
Too busy to micro-test?

I call bullshit. My buddy Erik Meade calls it Stupid Busy, and I think that nails it pretty well. All you’re really saying is that you’re too busy to go faster.

~@GeePawHill


Most of the responses were RT’s and likes, both lending me fresh energy. A modest number of folks chatted with me, and that gives even more juice. (Readers please note: the sharers of these things you enjoy need to hear that you enjoy them, early and often. It is what gives us the will to share.)

I Believe What I Said…

Since that article, from 2014, I’ve reworked the ideas in it many times. I have developed theory to back it up, experiments by which someone can prove it to themselves, and a great deal of teaching material. I have written and spoken repeatedly on the topic of shipping more value faster. It’s part of almost everything I write about the modern software development synthesis. About the best recent statement, though I am still formulating new takes on it all the time, is in the video (w/transcript) Five Underplayed Premises Of TDD.

If you want to ship more value faster, you might want to consider TDD, as it will very likely help you do that, for nearly any definition of value you wish, provided only that the value involves writing logically branching code. I genuinely believe that the modern synthesis is really the fastest way to ship value that we have at this time.

…But I Don’t Agree With Everyone Who Believes What I Said

And yet. I can easily see how one might infer from that quote — even from that whole blog, or from other parts of my output — that I am suggesting that using the full modern synthesis is a responsibility for a professional developer.  Or to reverse the sense, that I am saying it is irresponsible to be a working programmer who works in another way.

Some who responded took me as saying that and opposed that idea, others took me as saying that and favored that idea. I’ve no idea what the RT’s and likes were inferring.

I am not saying that it is irresponsible of a working programmer to not do TDD et al, because I do not believe that.

Frankly, the idea gives me the willies. So I feel it’s incumbent on me to give you some detail about what I would actually say about this question of responsibility.

(Writers don’t get to decide what readers make of their ideas. You’ll take the money premise and all the rest of my output you decide to bother with, and you’ll make of it what you will. All the same, if can see how you might think I was saying that, and if I disagree strongly with that, I feel my own — normal, individual, human — sense of responsibility to do what I can to make my ideas more transparent.)

Near-Universal, Individual, And Fraught With Internal Conflict

Nearly everyone develops a sense of what they are obliged, by their own description of who and what they are, to do in this world. We normally call this the sense of responsibility. For most of us, the twin pillars of that sense are 1) our deep beliefs about what is morally right or wrong, 2) our deep connections in the many networks of humans in which we find community.

Those senses of responsibility are rich, vague, staggeringly complex, and full of outliers, odd twists, preferences, and, when put in to language,  startling contradictions.

Above all, they are individual. There are aggregate and statistical trends, of course, and if one could find an N-dimensional space big enough to hold the variables, one would see bell-curve like manifolds all through it. I have a sense of responsibility, and the odds are overwhelming that you do, too. Our respective responsibility-senses may appear to be similar, even highly similar. But they are not. It only appears that way because of the limits of language and the locality of the context we use to frame them. Our responsibility-senses are different because we are different.

If you read widely, you will have seen hundreds, maybe thousands, of situations in which one responsibility a person senses is pitted against another responsibility that same person senses. If you don’t read at all but you are old and thoughtful enough to seriously review your past, you will also have seen hundreds, maybe thousands, of situations like this in your own personal history, and that of others you know well.

Cutting To The Chase

Boy oh boy was I ever about to sprawl out and produce a nine-volume treatise on ethics and responsibility. From my simple dog-like goodness, I will spare us all. Instead, I will sum up my beliefs:

  • that the modern synthesis is the best current way we have of shipping more value faster, whenever that value depends on logically branching code with lifetime longer than half a day.
  • that I don’t honor my own sense of responsibility at all times, because I am weak or foolish, or because I carry conflicting senses simultaneously.
  • that it is not for me to say what is responsible or irresponsible — global words — about any local context involving people who are strangers to me.
  • that “doing your job well” is very often not dependent on shipping more value faster.
  • that “doing your job well” can mean many things that are not good for you or the company you work for.
  • that “doing your job well” is in any case not particularly more important to me than any number of other responsibilities I carry.

If you are a working programmer, you are a grown-up.

You are in the difficult position of balancing dozens of competing needs, using only your heart and body and mind, and I am well aware that this is a task far more demanding in real life than it seems, on paper, in a blog, or spoken by an idol. I won’t do that for you because I can’t do that for you. I only sometimes manage to do it for me.

In One (Long) Sentence

Please don’t take me, even in a crabby mood, as someone who assesses your sense of responsibility, commitment, or heaven forbid, morality, on the basis of whether or not you do TDD or any other part of the modern synthesis in your job.

 

Use Supplier, Supplied, Or Both?

a coding pattern; replace supplied data with a supplier or a supplier with the supplied data.

it is very common to create code with an interface like this: do( Data supplied ) . we then use the data somehow to perform our functionality, whatever a do(…) method does.

on the other hand, sometimes we create code with an interface like this: do( DataSource supplier ) , and its body is basically the same as our starting chunk, but with a prolog that asks the DataSource for the Data and *then* performs whatever do(…) does.

so when we sketch a new function, we always have to mull over which of these two is the right thing to do. and when we refactor, similarly, we have to wonder which of these two ways to go. there is not a “right” choice you should always make. sometimes the supplied is the way to go, sometimes the supplier. but there *is* a right thing you should do, and that is to pay attention to the decision, as it has a variety of potential consequences.

here are some notions to mull over that might help you make your decision.

first, consider that you can do both. you can write  do( Data supplied )  and do( DataSource supplier ) . i’ve assumed overloading, but you don’t need it, just change one of the names.) just have the supplier version contain the prologue we discussed and chain into the supplied version. note: please chain. don’t just copy the body of one into the other. doing that is purposefully injecting anti-change in to your codebase.

second, if you don’t do both, make sure that whichever one you do is the least awkward collaboration for you to test with. remember “awkwardness”? this is any property of a collaboration that makes it annoying for us to test whatever branching logic do(…) performs.

in java, for instance, streams can come from anywhere including files, but files can *only* come from a drive. if i have to write and maintain a bunch of files “somewhere else” to do my testing, well, that’s awkward. and when we have awkward tests, we don’t write/run/debug them.

third, regardless of picking one or both, is the object you’re passing in far fatter in interface than what do(…) needs to do its work? you might want to wonder if you need an intermediate object, one that is neither the supplier nor the supply, quite, but is a thin wrapper.

the business of passing around obese interfaces is extremely common in “it rubs the database on its browser” code, it dramatically increases the mental scope required to grasp and change the method. a typical case: passing a database connection (and an account id) when do(…) really just needs an account’s zipcode. you do understand that your database connection aims at a turing complete system, yes? it can do *anything* a computer can do. this issue isn’t just limited to the supplier choice. we do the same thing with supplied’s. again, you just want the zipcode. an account, whether databased or in RAM, has *dozens* of fields and operations.

you might think introducing an intermediate object is dumb, because even tho you *could* perform any of an account’s operations, you’re only really going to ask for zipcode, so why go to the trouble?

(ahhhhhh. here we strike at the very heart of the strictly technical aspects of the modern software development synthesis. change-enabling code is nearly always more valuable than shipping the next change. so begins another muse.)

the short answer is still twofold.

  1. cuz code that only gets the zipcode *must* be dead simple, and dead simple makes us go faster.
  2. when we realize we also need a) a state and b) a method to validate that they’re correct, we have a ready place to put it.

so. when you sketch a method or approach refactoring that method, consider your choices around passing supplier vs supplied vs both. consider both, consider awkwardness, consider interface width. there’s no right answer every time, but it’s almost always worth taking a minute and mulling it over.

go geek out, then.