As many others observe as well, one of the major reasons to write is to firm up ideas in one’s own head. Serializing an idea out into English words is still no guarantee one understands it deeply, but it is great progress over a very fuzzy idea that has never been fleshed out at all.

I’ve known for a long time I wanted to sit down and write this out. But I had no idea how many principles I’ve either drawn out from my times with functional languages, or had sharpened by them forcing me on to certain paths that perhaps I would have walked eventually, in bits and pieces, but functional programming forced upon me in toto.

And it wasn’t until I’d written several of these posts that I noticed the recurring theme of scale, that functional programming principles are best brought in to imperative languages at the medium scale rather than the micro scale. I find this insight to have been almost worth the writing on its own.

I arranged these points roughly in order from least offensive to most offensive, so it’s possible that you arrive here thinking I do not like functional programming. But consider how much I have drawn from it. This is a good thing. I like it.

What I do not like is what I consider the surface level application of the lessons of functional programming. I do not like people obsessing about the micro-scale of functional programming, often to the point they actually harm their own code bases, while at the same time writing higher-level structures indistinguishable from the rest of the imperative world.

If all you’ve done is walk through your program and change all your for loops into maps and reduces, but your high-level architecture is still a confused mess with no discernable boundaries or structure… at best you’re got a marginal gain, and most likely, you’ve taken a step backwards!

If you are lucky enough to work in a domain where you can use high-end functional programming languages, by all means, adopt the relevant idioms wholesale.

But if you are not, I encourage to worry less about whether you should replace a for loop with a map and which library offers the best map call and more about things like:

• How does mutation flow through this program? Can I use mutation flow fences in my architecture to isolate parts effectively?
• How complicated are the parts in my architecture? How can I simplify their requirements?
• Can I harmonize what seem to be disparate types of things into single interfaces that I can then build on?
• Can I prevent strings and ints from flowing around my program naked? How can I derive synergistic benefits from stronger types?
• Even if I’m not using Haskell or a proof language, how can I use the local type system to prevent bugs from existing in the first place? Even in fully dynamic languages there are things that can be done.
• How can I prevent invalid objects from being created in the first place, thus solving all problems created by invalid objects? I know it can be done because functional languages do it.

And the other sorts of things I have described in this essay series. These are far more important then questions like whether these ten lines of code are formatted this way or that way, much more consequential in the long term.

The Pragmatist’s Secret

So often in engineering we have the “80/20” option; 80% of the benefit for 20% of the effort. The temptation of a purist is to sneer at an 80/20 for not trying hard enough when clearly the correct answer is 100/100, and anything less is failing to get all the benefits that are possible.

Most of the time, as one puts more effort in, the benefit obtained rather smoothly increases, usually quickly at the beginning and then asymptotically approaching 100% towards the end. It is unusual for there to be a sudden spike in utility at 100%, which makes the cost/benefits analysis of 100/100 often quite unappealing.

However, I fully agree that functional programming is indeed one of those exceptions! The difference between a programming language in which values are always immutable and one in which values are almost always immutable is quite substantial. In one, you can build on that guarantee of total immutability; in the second, you can’t, because it just isn’t quite reliable, and as you scale up, the odds of that “not quite reliable” biting you approach 1. I consider this a characteristic of type systems in general; a type system that you can rely on is vastly more useful than a type system you can almost rely on, and it doesn’t take much “almost” to greatly diminish the utility of a given type system. Even unsafe functionality is generally fenced in by the local type system, and where that is not possible, by language convention.

So I agree there is value in the 100/100 solution for functional programming.

If that is the case, then the question looms over programming as a whole, why don’t we all, all the time, just use functional programming languages then?

The answer to that is an underappreciated cost of the 100/100 approach, which is that it tends to forstall other possible benefits. You spent all your effort getting to that 100%.

But the pragmatist’s secret, unappreciated by the purist, is that the pragmatist does not have just one 80/20 solution. Having spent only 20%, the pragmatist also has another 80/20 solution, and another, and another, and another, altogether adding up to 400% worth of solutions.

… ok, sure, that’s taking the “80/20” numbers too literally, but the general point holds. For the same “cost” you can often buy a lot more with a bunch of 80/20 solutions than one 100/100 solution, or you can buy 2 of them and a 95/60 for the thing you really need. The diverse problems we face in the real world are often better addressed with such a diverse portfolio.

Not always. There are times and places to go all-in with a certain tool to the exclusion of all else. But those are broadly the exceptions, not the rules. Exceptions a wise engineer does well to be aware of!… but exceptions nonetheless.

And so, when considered against the at-times bewildering constellations of requirements being made upon our programs, it is often the case that a 100% commitment to a pure functional programming language is a bad choice, for reasons ranging the full spectrum of “technical” to “human”.

In this it is not unique; 100% pure commitments to any choice can turn out poorly. Those around during the late 1990s to early 2000s probably still carry scars from the XML all the things! of the era. There have been plenty of other examples, and senior engineers who probably shouldn’t be senior engineers are remarkably creative in their ability to come up with dogmatic “100% solutions” to some problem that you’ve never even heard of that will destroy the local code base in time.

But you don’t have to give up everything you’ve learned. You are not doomed to the outer pit of Imperative Hell just because local engineering considerations preclude a pure functional language. You still have the knowledge and skills you picked up from your foray into the Functional Programming world. You can still apply them. You can still benefit from them.

But those benefits don’t come from forcing micro-styles on to languages that don’t benefit from them. They come from these medium-scale-and-larger lessons.

Closing

If I have infuriated you, I hope I have at least given you something to think about, and I hope that the process of chewing on these comments benefits you and your programming skills. It isn’t even necessary to agree with me to find your own ideas and principles sharpened, and if that happens, even if you disagree, I will still consider this a success.