TDD & The Lump Of Coding Fallacy

Hey, it’s GeePaw, and if you’re just starting to look at TDD, refactoring, the modern technical synthesis, we need to start with a couple of minutes about the Lump Of Coding fallacy.

You’re a working geek: you spend your days coding for money to add value to your company. And one day some random schmoe like me comes up to you and says, hey friend you really ought to try TDD, because that value that you’re adding, you could have added even faster if you used TDD. So, you check it out, because sometimes random schmoes actually have a clue, and you see pretty quickly that TDD means making lots of automated tests against your code. So what do you do?

Well, you back away slowly and you casually slide your hand into your pocket to get that little reassuring grip on your pepper spray, just in case.

Why is that? Well, it’s pretty simple really.

Look, you say to me, I already spent all day rolling code to add value. But if I do TDD, I do automated tests, and try to keep up with me here, doing automated tests is also rolling code.

So, if I adopt TDD, I suppose you think I’ll just spend all night coding the tests that cover the value I spend all day adding. Look random schmo, in the immortal words of Sweet Brown Wilkins,

“Ain’t nobody got time for that.”

OK. OK, my random schmo response to this is to suggest that your analysis is off. And specifically, that it suffers from what we call the lump of coding fallacy. You think of what you do all day as a single behavior, coding. A large undifferentiated lump of work. And you think that TDD means adding to that one lump of coding a whole different equally large lump in order to make the test.

Three Things We Do During A Coding Day

Here’s the thing, I don’t think what you do all day is a single behavior. I think it’s made up of several different behaviors. Let’s take a look.

  • One thing we do, we actually program the computer, and there’s two parts to that. There’s actually changing the source and then there is what we call designing, which is imagining forward how we’re about to change the source. That’s programming the computer. For many of us, it’s the best part of our day.
  • Next, we study. It’s not possible to change the source without knowing something about it. And the studying is how we know something about it. Again, there’s a couple of different parts. We scan source, which is flickering quickly from place to place. And then we read source, which is far more intense. It’s more line by line, a deep analysis of what’s really going on in the code. So that’s study.
  • The third thing we do during a coding day is what we call GAK activity. GAK is an acronym. It means geek at keyboard. And what it means is, running the program in order to accomplish some end or another. In GAK inspection, we’re running the program to see how it works right now. In GAK testing on the other hand, we’re running the program again, but this time to see whether some source change that we just made had the desired effect. And of course, there’s always GAK debugging, where we’re running the program, this time in debug mode, or print statements, or whatever, to see why that source change did not have the desired effect.

Two Points For Later

Now, before we go any further, I want to make sure you know two key aspects of the TDD world. First, when you’ve adopted TDD every source file really becomes a pair of files. One holds the source code we ship, and the other holds the source code for the microtests. And we use both of these files all the time, because they both offer us a great deal of information. That testing code forms a kind of scaffolding around the shipping code and we’re going to take advantage of that.

Second, TDD uses very small, very fast tests called microtests, and it runs them separately from running your entire application. The reason we can get away with testing only parts of the app, is because what matters most about our work is the branching logic that’s in it. And that’s the part we test most heavily using microtests. We run them in a separate app for speed, selectability, and ease of use.

So, take those two points and set them aside. They’re going to become important here in a minute. Hold onto them.

The Proportions Of The Three Activities

OK, let’s go back to our three activities. So, take these three things together, changing codes, studying code, and GAK activity, and you see there isn’t just one solid lump called coding. There’s all these activities. Of course, they’re totally intermingled throughout the day. And that’s why we think of it as a big lump. The truth is, they actually take up very different proportions of our programming day.

Programming the computer, the best part of the day, is often the very smallest part. The GAK activity, much of which is just waiting around for things to run, or clicking through screens and typing in data in order to get to the part where you wanted to see something, that is the largest part of the day by quite a bit. And studying, the scanning and the reading, well, it’s somewhere in the middle. So those are your basic proportions.

When TDD tells you that writing automated tests will make your life better, the lump of coding analysis of the idea is both right and wrong. Lets grow this picture a little bit. We’ll call what we’ve got now before TDD, and then I’m going to disappear, and we’ll put after TDD over on the right.

The After Picture

The lump of coding fallacy is absolutely right about one thing, automated tests are more code that has to be written. Somewhere between half again as much and twice as much as you write now. Let’s say that part of our day doubles. On the other hand, the lump of coding fallacy is totally wrong about the rest of the picture.

First, study time will go down after TDD. It’s not that we have to study any less code in the after picture than in the before. Rather, it’s that studying the same amount of code gets faster. Why? Because those twin files we talked about, one with shipping code and one with testing code, it’s almost like the test code forms a kind of Cliff’s Notes for the shipping code. A scaffolding that makes it easier for us to study, and this makes it far easier to tell what’s going on. This will cut our code study time in about half.

Finally, we come to the GAK time, and this is the big payoff. TDD reduces the amount of time you spend in GAK by 80% or 90%. Because TDD tests run in that special tool kit. They’re fast. They don’t fire up your application. They don’t depend on things like logins, or database permissions, or waiting around for the web to load. They are built to be fast, small, and grouped into convenient suites. Nothing completely eliminates the need for GAK work, but TDD slashes the amount of time you spend GAK-ing during the course of the workday.

So, when we look at the left and the right, from before TDD to after it, you can see it for yourself. We write more code, automated tests. And far from losing our productivity, we actually gain it. All this takes is for you to look past that single lump of coding fallacy.

GeePaw Advises

OK, it’s time for my advice. The first advice is the same advice I always give. Notice things. In your next full working day, notice how much time you spend in the three behaviors, actual programming, code study, and various GAK activities. Once you see it yourself, you might want to consider making some changes.

The second thing, well, TDD doesn’t come in a day. It takes some lessons and some practice. There’s a lot of course material out there, including mine. Watch some videos. Read a little, and really try the various exercises. Start with some toy code. Almost anything will do. Then find a small problem in your day job that has few or no dependencies on other classes. Do this two or three times. And again, notice what happens. If you like the result, well, at that point, you’re ready to get serious about TDD. And then, well, we can take it from there.

So, I’m GeePaw.

Drop that Lump Of Coding fallacy,

and I’m done!

11 thoughts on “TDD & The Lump Of Coding Fallacy

  1. You make many quantitative claims. You make claims about which aspects of programming take up each amount of time. You make claims about how much time TDD will save you if each of the 3 categories. You neglect to make a claim as to what percentage of the original time you are now spending writing and debugging tests.

    Nowhere do you actually back up these (rather precise: 80%-90%… really?) claims with hard data. Or even soft data. You didn’t even offer a “bad” test where you compare your own workflow against the two. You just… Make claims, and then say “see for yourself!”

    This is the irony of most test driven design blog posts. They offer little in the way of actual tests of their claims, but boy do they enjoy making claims.

    I was hoping your post would be different. Clearly I was wrong.

    • Your comment infers that you don’t do TDD, Matt. Perhaps you haven’t tried it with any seriousness. Perhaps you did but just couldn’t grok it. That’s okay. We all have our limits.

      Full disclosure – I’m a TDD nut. My personal experience is not too far off from that of the author. It’s similar with others who I’ve worked with, in various environments in the past, who practice TDD.

      One might make the case for confirmation bias. And perhaps it’s valid. Like does tend to attract like. Birds of a feather, etc.

      Anecdotally though, I’ve seen my own code quality improve through TDD. My code ships with fewer bugs, which means I have more time to focus on delivering more value. I ship more code as a result. Way more. Can I quantify it? Probably. Should I have to quantify it in order to change your already strong opinion? Nope. Either you want to use TDD or you don’t.

      Your primary criticism of the article, it seems, is that there’s no data to substantiate the claim. This is a blog post, not a scientific review. Data isn’t required. It’s conjecture. It’s anecdotal. It’s one person sharing their experience so that others in community might benefit from it.

      You don’t have to do TDD. That’s your choice. The author and people like myself have experienced real benefits from the practice. It’s quite likely that you would as well. No-one is forcing you to adopt the practice. If you don’t, won’t, or can’t see the benefits, that’s all on you.

      Your strong reaction indicates to me that – if you reply at all – it’s likely to be a negative reaction. Thing is, it’s not up to me or the author to change your mind. We’re sharing our experiences. It really is up to you to see for yourself.

    • Thanks for writing, Matt.

      Actually, I did say the amount of time I spend writing tests roughly doubles the activity I’ve called “programming computers”, so that is very much in there. As for debugging, I debug tests quite rarely. This may seem odd, but it’s because of the style the modern synthesis drives one towards. The tests themselves are so small and simple — by intention and effort — they rarely require more debugging effort than to just glance at them to make sure the values passed/asserted are correct. When they do require firing up the debugger, a) it’s usually because shipping code is broken, not the test, and b) it still takes less time because they’re running in their own app.

      I’m sorry my numbers, “doubled”, “halved”, and “80-90%” bothered you, apparently by implying I was going to reveal something called either “hard data” or “soft data”. I can’t know what you mean by those two, but for me, “hard data” would be quantitative statistical data with analysis, and “soft data” would be anecdote with analysis. I position what I said in that latter category.

      Why do I say “see for yourself”? I say that precisely to frame what I’m saying as a goad to action rather than an exercise in either pilpul or math. At any rate, if you are happy with the actual results of your actual practice, regardless of the theory I laid out or the theory you might deploy in response, I think you should keep doing it. Cheers! — GeePaw

  2. I took an online course that required TDD. I appreciate your characterizations but in the end the more time I spend looking at my code, the better it works against any testing and the better I understand what I did. With that said, I do now find myself coding a few key tests up front, because it now just seems so obvious to do. Maybe one day I’ll go all in, but being in my 60s, I doubt it. 😉

  3. In my experience, it’s usually less about devs not wanting to do TDD, and more about companies not wanting to pay for it. In the triangle of good/fast/cheap, tdd is usually the first cut. That’s why I always bloat the hours to include it and refuse to subdivide code time vs test writing time.

    I generally agree with most of what you’re saying, I would add the majority of the benefit occurs when you write the tests before you write the code. Tests as documentation save far less reiterative anaylsis time than tests as objective.

    Also, the biggest time savings isn’t developers in most cases, it’s the elimination of the increasingly popular brute force 30 plus headcount “testing teams” that point and click their way subjectively through the application, and the time it takes to sort their output and eliminate misunderstandings and duplicate bugs.

    • Hi Kitty! Companies choosing not to pay for TDD are confused, in my view. They believe that TDD — as always with me I mean the whole modern synthesis — is about the raising of quality, a mistaken belief. TDD makes working with branch-y code faster, significantly so. The “what do we value” question is pretty irrelevant: whether we want faster systems, faster delivery, higher perceived quality, or really, anything that involves extensive and intimate contact with branchiness, the modern synthesis out-performs.

      And yes. “External QA” is one of the more magnificent of the sins created by “idols of the schema”. I have not seen it work yet. 🙂 — GeePaw

  4. Pingback: Code Dynamics – AppShapes

  5. You see, the thing is, no one has any data of statistical significance to support TDD. Go ahead. All studies to date are inconclusive at best. That fact arises from the difficulty of designing a valid test.

  6. Pingback: Android Weekly #305

  7. Thanks for this article. I’ve been doing TDD for about 15 years now, and I’m now so much used to it that it’s difficult for me to be objective about the productivity … What I can say for sure it’s that it’s an incredible stress reducer. Using it (and the rest of XP), I’ve shipped almost bug free software to picky traders in corporate finance. It saved me from deployment and firefighting nights and week ends. It made my life as a software developer so much easier that I wish that for everyone. I would not promise anything, but I suggest everyone to spend 20 hours doing TDD code kata to learn and see how it works for you. It’s a small fixed cost compared to potential lifelong benefits.

Leave a Reply

Your email address will not be published. Required fields are marked *