Automocking: Behavior Tests Are Not Quite

Part [part not set] of 2 in the series Automocking

So, with the intent firmly in our minds, from yesterday, I feel closer to explaining my automock resistance. There is still some teasing apart to do. Automocking tools bring several possible aspects to the floor.

The first of these is by no means required by automockers, but is commonplace. And that is behavior-based testing. Behavior-based testing is the kind of testing where i validate A in an A->B relationship, by asserting that B’s methods were called. If, in setting X, A is supposed to call B three times w/different method/parameter variants, I can write a test to assert that it did.

This doesn’t work for me.

When I test A, what I want to know is that A works. I am entirely indifferent to how A gets the right answer. I wanna know it gets it. Cheap-shot alert: it’s easy to make up cases to prove that test X doesn’t prove what it’s meant to prove.

Nevertheless, I’m gonna go there. I can readily test that A.X() calls B.Y(…) three times when invoked in situation Z and I feel that I can pass that test without in any way proving that A gives the correct answer.

Now. That’s a feel. And I’m aware it’s a cheap shot. So let’s keep moving.

What am I saying with one of these interaction tests? I am saying that my algorithm is what I said it was. Now that’s close to what I want. But it isn’t quite what I want. And there’s both a basic reason for this and a more complex one.

The basic reason is that there are myriad algorithms available to solve most programming problems. Consider a "getNextPrime()’ operation. Of course there’s the sieve. But in fact there are lots of ways to get the next prime after this one. Any of them will do. As a tester of A, I don’t care which algorithm A uses. I care that A gives the right answer at the right time.

The more complex answer sends us back to the interaction between the money premise and the correlation premise. If my solution is the once-and-for-all solution, if my algorithm is the only possible algorithm, interaction tests are great. But those ifs violate my everyday experience of programming all the time. So often that I deem them relatively useless.

My algorithms change every day, because my needs change every day. They change for lots of different reasons. Sometimes they change cuz the customer changes. Sometimes cuz I misunderstood. Sometimes cuz I need just one more thing. And so on. So locking down my algorithm is a wrong move. I want to lock down my question and answer, not my algorithm.

It’s the geepaw intent that drives this. I want the tests to back the guarantee, but I want the guarantee for more features faster. Things that constrain my freedom-to-change might give me a guarantee, but that guarantee keeps me from moving forward. Interaction testing — focusing my tests of A around its interactions with its collaborators, gives me both more and less than I want.

So this first aspect of what people are often doing when they use an automock, it’s just not something I want to do. Still, that’s just one aspect. Automocking tools bring more than just interaction testing to the table. We’ll have to come back to these.

I hope, by the way, that I’m not coming off rabid here. I have a feeling, I have a case, I have reasons. If what you’re doing is working for you, continue it. If it’s not, change it. This is what I’m doing, and it’s working for me.

Want new posts straight to your inbox once-a-week?
Scroll to Top