You can put all your configuration in a shared library and eliminate just about every mis-configuration in your multi-process application. It’s not free, but it’s cheap, and it kills a lot of minor pain. Let’s take a gander.
It’s Sunday geek comfort-food time. I hope you enjoy it, and I also hope you remember it’s not the most important story out there. Please keep working for change, once you’re rested.
Stay safe, stay strong, stay kind, stay angry. Black Lives Matter.
A multi-process app is one where there are several programs that are built and provisioned separately, but must collaborate to create the experience of the user. All micro-service architectures are multi-process, for instance, but there are other apps like that, too.
In order to collaborate at runtime, these programs need to share things like endpoints, URLs, feature flags, and sometimes even file paths. Mis-configurations can cause lots of wasted time, in development, testing and release.
By far the most common way to achieve this sharing is by maintaining some variety of "configuration profile" files, one for each program. These contain ASCII — YAML or environment-variable style — depending on and customized for the particular program.
When we have two programs and two profiles and the profiles don’t agree about one or more of the configuration settings, the result is usually, tho by no means always, a kind of easy "wait, what?" problem.
What I mean is, the programs tend to collaborate incorrectly in such an obvious way that we usually notice it pretty quickly. We go double-check the configurations, spot the incorrect ASCII string, hand-edit it, re-up it, bounce it, and hopefully we’re off and running again.
Of course, they’re not always that obvious. I’ve seen multi-service apps mis-connected for days at a time, causing confusing results, intermittents, flickery tests, ‘sploding demo sessions. 95% are easy, but 4% are hard, an 1% are "Sev 1 all hands on deck".
But they are usually cheap, some of the easiest bugs & fixes we encounter. The problem, of course, is that they happen just often enough that they slow us, but not often enough that we take a hard look at what we’re doing.
What are we doing? We’re violating three different pieces of advice:
"Don’t Repeat Yourself (DRY)", "Reduce Mental Bandwidth", and "Easiest Nearest Owwie First (ENOF)". We’re doing a hat-trick of bad habits here.
- "Don’t Repeat Yourself" is a heuristic that tells us to avoid duplication, because if there are two places to keep the same code or data, we will forever after be required to keep them in sync.
- "Reduce Mental Bandwidth" is telling us that the most common answer to these problems, which is "everyone remember to always change or check both of these profiles at the same time", is putting a burden on the human brain that it can’t reliably bear.
- "Easiest Nearest Owwie First" is saying that, even though there are bigger problems than misconfiguration, we still want to fix misconfiguration, because it’s within our reach, it’s easy, and it’s an owwie.
There are several possible ways to kill off misconfigurations like these.
But the one I like the most is one people seem to implement the least: use a shared resource to make config simpler to do, simpler to control, simpler to version, and simpler to test. Here’s how it works.
Create a repo with code contents that are one way (or more, see below) to make filled-out non-primitive Configuration objects based on simple function calls or even just keywords. Write your project to use those instead of ASCII config profiles.
- Detail: The configuration-making is purpose-built to supply only legitimate configurations. In most environments, there are fewer than a dozen of these, tho configuration-by-string lets you create thousands or millions of broken ones.
- Detail: We said "one way (or more)". All of our programs aren’t written in the same language. No worry, your repo can have one in the backend language, one in the frontend, one for that weird old thing written in caveman days, one for those cutting-edge bozos over there.
- Detail: Why non-primitive? This is a very long conversation, and we’re just going to have to work on it another time. The too-long-didn’t-write: so we can give it custom behavior when we want that, which we will.
There are several real benefits to this kind of approach — honestly, to most of the competitor ideas, anything that isn’t "same strings in many files" — so let’s list a few of them.
- It’s cheap. Adding a repo, writing a first pass at a Configuration, these are not rocket-science designs, but very basic stuff, especially in the beginning.
- It’s testable. Since I have all and only the legal Configurations for all my apps, I can roll tests around all of the decision-making process and its resulting values, long before a client app ever gets deployed.
- It’s iterable and incremental. Programs can be stepped over to using this one app or even one value at a time. This is a huge benefit, because we get value not once, at the end, but distributed, every time we take a step.
- It makes the implicit explicit, nearly always a winning move in geekery. Config-by-profile has lots of implicit rules. We can put those rules directly into our Configuration scheme, express them in code, and prove that they aren’t violated.
Two of my recurring themes are triggered by all this: 1) It’s this way because we built it this way. 2) The code works for us, we don’t work for the code. This is a really straightforward case, the problem from theme 1, the solution from theme 2.
We chose untested by-hand ASCII configuration profiles. We didn’t have to, but we did. With a scheme like this one, or any of the other ideas with similar values, we are making the code work for us.
So think it over! I can’t give you up-close details cuz I’m not in your codebase.
But I can tell you, it’s actually pretty easy. The hard part of this, as so often, isn’t implementing an answer, it’s looking for it in the first place.
A shared library for configuration is a way to kill off a bunch of small, annoying, persistent problems, in development, in testing, and in production. It’s not free, but it’s cheap, and it pays off well and rapidly.
Queue up a spike and give it a try!
Do you love the GeePaw Podcast?
If so, consider a monthly donation to help keep the content flowing. You can also subscribe for free to get weekly posts sent straight to your inbox. And to get more involved in the conversation, jump into the Camerata and start talking to other like-minded Change-Harvesters today.
I could use an example. Is our configuration a program that generates the text config files? That seems reasonable, but it doesn’t fit with the “or more” part. Does each app program use its version of the config library (the one that uses its programming language) to specify its own config? That violates DRY for shared config parts. I’m confused.
Sorry for the confusion, Franz. I see no need for text configuration files at all. Imagine we have several java backend services. We can create a project that generates a jar that all those services link to. They can new a configuration just by saying new Configuration(“keyword”), then use that Configuration object however they need. The “or more” part: we sometimes have several systems that collaborate but are written in different languages. We still use one project, but it’s capable of making that jar as well as having, say, a JS importable package. We keep both of those in a single project with its own tests that prove they do the same thing. — GeePaw
So basically you tackle DRY by having all the moving parts (flags, URLs, etc.) in one repository, but potentially repeated in different languages?
You still have to deal with the languages but you do it in a centralised place. Right, that makes sense.
I have seen another attempt to DRY out this problem with configuration profiles: text files can be read by everybody, so share the same one with all apps. Throw a configuration-reader library at it and voilà, you don’t repeat the values and ignore the ones that don’t matter to you.
With that last approach, I don’t like that the configuration values aren’t scoped. Internal configuration values do not need (and arguably should not) be exposed to anybody else. But if you start splitting the files, then you might as well dedicate them per application, and that brings you back to square one, what is described here as the problem.
I am curious as to what kind of test you are talking about to prove that the multi-language approach is consistent.
For example I work with a Python backend serving data for a JS frontend. I think I’d try to come up with one make recipe to extract values from the .py and .js files with e.g. grep/awk and just compare they are the same. (My assumption is that we are talking about simple values, strings, integers, flags.)
I’ve had similar problems, but came up with a different solution.
Now in my context, I’m dealing with products and libraries that have their own configuration files, and I have no good way to make them use a different configuration approach.
So I use “xUnit” tests.
And then I start making and enforcing rules about what a “correct” configuration looks like.
Like I’ve got half a dozen apps deployed to 20 different environments. In each, the apps should talk to each other — *NOT* to different environments. Like for the “QA testing” set of servers, they’re all deployed to “.qa.mycompany.com”. I assert that all their configuration strings use host names conforming to that pattern.
If I can describe what “correct” and “incorrect” look like, I can write rules to enforce.