Parsing Expressions, TDD, and the Big Why

So, for those following along, the other day I had to decide whether to roll my own parser for boolean expressions involving tags, like "(stinky & doofus) | (ugly)". Very standard stuff.

In the end, I rolled my own, and I have to tell you, I just had a really wonderful time.

A parser for strings is an absolutely awesome place to do TDD. They’re just strings. No clever fakes, no extract & override, nothing. Just strings.

I hadn’t done a recursive descent parser in years, so first I ripped some internet code and mucked around in it until I grasped it all again. Then I tossed that, and TDD’d based on what i’d learned. Rdp’s are nice, cuz you basically start at the bottom and add wrapping function at each step, but the call you make to fire up the parsing is the same every time, the wrapping is all internal.

Anyway, now I have a lovely parser. It’s smart about errors, and hooked into a live editing box, so the user can type the expression. As they type, the text is red if I can’t parse it, green if I can, and I update the applied filter every time it’s green.

There were plenty of screwups along the way, and the TDD was just sitting there doing it’s TDD magic perfectly. I swear to god if i’d had to find all the little regressions I created as I was adding new function, it would have taken at least an order of magnitude more time.

I’ve found three bugs since it went live, and I true-blue TDD’d them: added a new red test before I fixed the problem, got it red, then got it green. It was really a very pure process.

One of the reasons I decided to take up the mantle of teaching TDD: situations as clean & pure & delicious as these are actually quite rare in the workaday world of geekery.

And it seemed to me a lot of folks out there were suggesting that TDD is red-green-refactor all day, an endless spiral of rainbow-farting unicorns and never ever ever a judgment call or a frustration or an impure thought or act.

A working geek using TDD knows that we don’t normally do greenfield work in toy codebases with only leaf dependencies that we wrote ourselves. Hardly ever do we get the chance to just let the tests flow like wine.

So then why do I persist in applying TDD in situations in which it’s not pure and easy? Why do I teach TDD if it’s not normally universally straightforward and learnable in cute little soundbites?

Well here’s the thing. When I do get those simple pure situations, I code like a god among geeks. I am faster, stronger, smarter, and what I ship stays shipped. It’s not just a little better, it’s fantastically better.

And somewhere in my second year of trying TDD everywhere, I began to see situations that were almost simple and pure. One tiny change I could make, and they would be pure TDD situations.

And I kept going, and that trend kept going. Ever larger impure situations I became able to see how to change into pure ones. And it was very often very worth the effort to make those changes.

Two duffers are at the billiard parlor watching the local hero get his ass whipped by some out of town star. The one guy says, "man, that stranger is good." the other one says, "oh, idunno. He never takes a hard shot." this is the heart of the steering premise. We don’t force TDD to take hard shots. In fact, that forcing is what makes TDD duffers decide they hate it, that we’re all mouth-foaming religious fanatics, and that TDD can never work in real life.

Baby TDD is red green refactor. Grownup TDD is taking the shots that look hard at first blush, then rearranging things until they’re not hard, then baby-TDDing them.

And I teach this because it has made me a vastly stronger geek. And I can easily imagine duffers looking on: "oh, idunno, he never takes a hard shot." the steering premise says "tests & testability are first-class participants in design". It means, in the simplest telling, "don’t take hard shots. When you can’t test this design in an easy shot, change it until you can."

Want new posts straight to your inbox once-a-week?
Scroll to Top