What TDD Tests Prove

If u think the semaphore mechanism in your library works, then test to the level of setting the semaphore. If u think it doesn’t work, then u need another library or o/s. Testing across threads is for people who are testing semaphores.

If u think that your JSON transport works — from your UI’s perspective or your backend’s perspective, then test to the level of feeding it JSON. If u think it doesn’t work, then u need another library. Testing across JSON transfer is for people who are testing JSON libraries.

If u think SQL works, then test to the level of emitting SQL or logic’ing recordset values. If u think it doesnt’ work, then idunno even what u should do. Testing across SQL lines is for people who write SQL libraries.

I could belabor this, but I feel i’ve done enough listing of the pattern.

Sometimes my simple string manipulation creates bad output. The odds are 10,000 to 1 at least that that is because the billions of users of that library haven’t found the lurking substring bug. Nope. Far more likely, I just screwed up on my end.

Play the odds.

The money premise says we’re in this to "ship more value faster". We are not in this, unless we’re the authors of said libraries, to prove that semaphores, SQL execution, JSON transmission, or strings, work correctly. Pro-tip: they generally do. Now, let me head the yabbit chorus off at the pass: are there library bugs? Oh, hell yeah. To cite a well-known case, the c++ STL shipped for years with operations that were thought to be thread-safe but weren’t. (look up the "rope" class.) but when my string manipulation goes south, I don’t start with the silicon, the cpu, the language, or the library. I start with what I typed. This is a hard call for juniors. 1) they’re still thinking that the purpose of tests is to prove that all things are meet & just. 2) they don’t really know how their string library works at all.

That’s cool.

  1. Nothing proves that all things are meet & just, nothing. Not one thing. Nothing. Nope nope nope.
  2. There are a variety of techniques for learning how library X works, and you should try whichever ones will help you. I know my string library, but I quite often write tests just to see if i’ve invoked something (a little less common than string) correctly.

I use tests for this cuz it’s a handy way to probe something: u get to invoke a toy situation to understand how the library will behave when u use it for real. But that’s just convenient in context. I usually toss those once I get it. Those tests are useful for me to run. They might be useful for me to keep, if the library is a little unstable or my learning is still wobbly. But they’re not the point of TDD.

I use TDD not to establish that all the things in my program are meet & just, and not to establish that, say, semaphores work. I use it to establish that what I said is what the computer heard is what I meant. And if u think that is a very low bar, then I suggest that you’re either not a working geek, or you’re young, dumb, and full of memory, and convinced that your capacity is infinite. I’ve encountered many folks of both stripes.

Making sure that what u said is what the computer heard is what u meant is, in fact, the hardest part of professional programming for the first X (3? 7?) years. It eventually transforms in to design, where the hardest part is breaking non-toy problems into toy problems.

I am prepared, in the fullness of time, to discover that 90% of my programming failures come from other source than just the trivial breakdown of one of those two links, said -> heard -> meant, from me to the ‘puter. I would like so much for something else to be at fault. I would love it if every complain logged against my code was because a) I didn’t understand what was wanted by the customer, or b) the customer-interpreters I work with are vague and somewhat mentally less capable.

Sadly, that’s not my main problem. My main problem is that the code doesn’t do what i thought it did, regardless of whether it does what I was told it should do or whether what I was told it should do is what the buyer actually wants or needs. I know such problems exist. Of course they do. But they are of a higher order. It doesn’t matter that it isn’t what I was told or that it isn’t any good for the market if it does not do what I want it to do.

TDD proves to my satisfaction that my code does what i think it does. Proving that is just prima facie evidence of awesomeness, but disproving that makes all other links in the chain irrelevant.

Want new posts straight to your inbox once-a-week?
Scroll to Top