The correlation principle says that our productivity is tightly correlated with the internal quality of software.
The two go up together, and they go down together, and you can’t trade away the one to get more of the other. Let’s talk it over.
I remind us that geekery isn’t the main story right now, though you and I can use it as comfort food.
Keep working on changing the world. We can do this. In fact, we’re the only thing that can do this, so stay safe, strong, angry, and kind out there. Black lives matter.
The basic idea of the correlation premise is actually pretty simple, but when we introduce it, we generally get a chorus of yabbits. ("Yeah, but…") There are several confusions common to most of that chorus, so let’s look at them one at a time.
Confusion: mixing up internal software quality and external software quality. The yabbit takes this form: "Yeah, but we can get it done faster if we settle for it working less well."
See, here’s the thing: that’s generally true. We can deliver more quickly by settling for a lower level of value.
The trick is to understand the yabbit as being about external software quality, not internal software quality.
External Software Quality, let’s call it ESQ, is any attribute of our application that a person can experience by using it. Internal Software Quality (ISQ), on the other hand, is any attribute you of our application that you can experience only by seeing the code.
You can trade ESQ for time-to-market. You can make the app slower, or uglier. You can make it do fewer things. You can make it handle fewer cases in the things it does do. All of this will, generally, get you to the market faster.
Well, can we trade ISQ away to get better delivery times, the way we often do with ESQ?
Just as the answer with ESQ is "generally, yes", the answer with ISQ is "generally, no".
It’s because lowering ESQ makes some things easier, but lowering ISQ makes all things harder.
Write down the gigantic production function that will predict how much value gets added by your team today. They’ll be dozens of complicated terms in the expression. Sort them from biggest to smallest, and what do you get? Three of those terms are "the big three", every time.
From most significant to least, the big three terms in our daily production equation are 1) How inherently complex is the problem domain? 2) How good is the team? 3) How changeable is the code?
Think about how you’d alter any of these three things.
Well, dangit.
You can change the problem domain, but such changes are usually paradigm shifts, with years in the making.
You can change the skill level of your team, but that’s not going to get you to market faster right now, it’s more like a capital investment.
But that third thing: "How changeable is the code?" See, that’s just ISQ by any other name.
The third largest term, the easiest one for us to change, is exactly what we mean by ISQ. And that’s why we can’t trade it away for more productivity. They’re directly correlated.
Changing code is the fundamental behavior of software development teams. If we reduce the changeability of the code, we reduce the team’s effectiveness. We get less value or we get value more slowly.
A brief aside: "value". People get distracted by obsessing over what value means in their application’s context. That’s good stuff, urgent stuff, but it’s not an important argument to have here: however you define value, if getting it depends on changing code, ISQ’s the 3rd term.
In particular, I leave it to the value-definers in my team or org to decide about how many bugs it’s okay to ship. Shipped bugs is ESQ. ISQ is about changeability.
Next confusion: The timescale of ISQ impact. The yabbit sounds like "Yeah, but, even granting that ISQ is correlated, the correlation isn’t instantaneous. There’s a window where we can cheat ISQ, get to market faster, and then go back and re-raise the ISQ."
This take has two flaws, one obvious as all hell to anyone who’s ever had a job in the trade, and one — more important — that’s hidden from the view of most non-developers. Let’s do the hidden one first.
The timescale for the impact of ISQ violations is ridiculously short, far shorter than the timescale at which the majority of your time-to-market calculations take place.
Take a weak variable name, for instance. How soon does that weak name’s effect on ISQ take place? Well. The only reason you give a variable a name at all is so you can use it. And you use it immediately. (In some styles of programming, you use it before you even define it.)
This holds true for most of the ISQ violations: weak naming, poor factoring, signal-hiding, testlessness, strong coupling, the list goes on and on. Every one of these starts taking effect immediately.
And their effect isn’t linear, either. One bad name won’t sink a ship. A hundred bad names will. And that’s one of the smaller violations. The big ones scale out of control even more quickly.
So the hidden flaw in the yabbit is that breaking ISQ has serious impact much more rapidly than non-developers think. (And, frankly, than noob developers think, too, and they dominate the mix at most orgs)
What about the obvious flaw? "We’ll fix the ISQ right after we ship."
uhhh-HAHAHAHAHAAHAHA! No we won’t.
There are real reasons why, but you know what, I’m not even going to bother.
We won’t.
Next confusion: conflating ISQ with "clean", its cognates, its synonyms, and its overtones. The yabbit form is a bewildering variety of this shape: "Yeah, but, we don’t have time for you to wash the dishes, we gotta go now."
I am long on record as opposing the use of "clean" as a word to describe ISQ. Not only is it misleading about what ISQ activity looks like, it’s also misleading about the impact of low ISQ. (It’s also commonly used to suggest moral deficits on the part of people we want to beat.)
Maintaining my ISQ doesn’t look like cleaning. It’s not getting on a ladder to brush away cobwebs. It’s not doing the laundry. It’s not washing carefully behind my ears. It’s not emptying the kitchen sink of dirty dishes.
If it resembles anything in the physical world, it’s incrementally remodeling my space. Rerranging walls, putting shower & toilet close together but not on top of each other, and so on. But even that’s a lousy metaphor, because it doesn’t mention why: for optimal performance.
See, my living space is used for lots and lots of different activities, so there isn’t really a single optimal layout. Optimality here is largely a matter of taste, not just in how you prioritize your particular set of activities, but in how you sort the dependencies within them.
But our codespace, unlike our apartment, is used for just one thing: changing code. And optimality for changing code — we call that ISQ — is much more well-defined, outside of taste, understood, and inter-subjectively confirmable.
(Yes, there are still matters of taste. But far fewer of them, with far less variance. Having said that, let me remind you that the left-to-right ordering of arguments in a function definition is largest-scope to smalleset-scope, and Nayan is a doofus for arguing the reverse.)
And what about impact? Look, sure, we lose some efficiency if we start with dirty dishes, or dirty laundry all over everything. But not much. We can still cook dinner, we can still get to work on time, without stopping to clean. We can. People do. All the time.
That’s because the cleanliness of your dishes or your room is simply not one of the largest terms determining your effectiveness as a cook or an employee. It’s just not.
But ISQ is the third largest term in my productivity equation. Trash my ISQ and everything I do is slower.
(I’m not going to talk about the cleanliness-as-morality stick and shtick today. Look for older work if you want that. Suffice to say: Please stop equating software development efficiency with morality, especially in such a pre-adolescent manner.)
Alright, those are three common confusions and yabbits around the correlation premise. Let’s quickly review the case we’re making, this time without all the yabbits.
We want more value faster. The value could be "more features", or "lower MTBF", or "smaller footprint" or "faster runtime" or "better UX" or "fewer bugs". The only fixed aspect of the value definition for the purposes of this case: to get it we have to change code.
Some code is more easily changed than other code. There are a lot of factors involved in this, but they mostly come down to whether it is arranged in such a fashion that humans can quickly grok its meaning.
We call all of the attributes of "easily changed" by another name: Internal Software Quality, or ISQ for short. That’s as opposed to External Software Quality (ESQ). The difference: any user can detect ESQ attributes, but only people with the codebase can detect ISQ attributes.
The correlation principle: Productivity and Internal Software Quality are directly & strongly correlated. They go up together, and they go down together. You can’t trade away ISQ to get more productivity. You can reduce time-to-market by giving up ESQ, but not ISQ.
Supporting The PawCast
If you love the GeePaw Podcast, consider a monthly donation to help keep the content flowing. Support GeePaw Here.