Types as Assertions

Functional programming languages see types not just as buckets of things that can contain values, but as assertions, e.g., this is not just a string that I vaguely hope is a Username, but this is a validated, certain-to-exist Username for a user, and anything that receives an input parameter to some function that is a Username need not run its own validation code on the Username to ensure it is valid, and thus also doesn’t need an error path for invalid Usernames..

Haskell comes with a zoo of assertions you can make in the type system. Imperative programming languages come with fewer such assertions. No question. Lacking sum types can make some basic things like “I am guaranteed to either have an X and a Y or a Z without an X or Y” difficult to express easily. The power is undeniably less.

However, this lesser power that we have in imperative languages is also frequently neglected. I can’t tell you how sad I am to see a program full of functions with signatures like (string, string, int, int, bool) (bool, int) or (string, string, bool, bool, bool, bool, bool, bool, bool, bool) (string, string, string). Why even use a statically-typed language if you’re just going to write stringly-typed (and intly-typed and boolly-typed) code?

Sure, it’s more convenient in the moment to slap down a string somewhere rather than having a typed value, but in the long term, it tends to bring a lot of pain. You can easily get the “every bit of data is randomly validated in random places depending on what happens to be a bug today” pattern, and naturally, no two bits of code to validate a username will validate the username in the exact same way. And faced with a function taking (string, string, bool, bool, bool) along with the traditional “no documentation whatsoever” often leaves precious little to go off of to understand what a given function is doing.

And if you just want to pound out a test case it may be inconvenient to have to go get some sort of “real” Username for a test, rather than just slapping a string down.

But it does not take long at all for a codebase to descend into total madness when all you do is pass around primitives. I implore you for your own sanity’s sake to spend some time putting actual types in your imperative code. I know it’s inconvenient to have to convert things to strings to run a simple “replace” operation and convert back to the custom type, or whatever other hoops you may have to jump through. I know this pain because I feel it frequently. But the middle-term payoff is well worth it.

That first time you write a few thousand lines of code, only to discover that it turns out you need to casefold all usernames in the system, and you either A: write it in to the creation function for Usernames and know it is already everywhere in the program or B: have the compiler guide you completely correctly to every single place in the code that needs to be adapted is one of those life changing programming moments; if you haven’t had it yet, you really should.

After that you’ll start noticing that you can take a lot of the operations that you used to do with string bashing or various tricks with ints or whatever, and turn them into methods on these strong types. Which nicely organizes them in whatever automatic documentation system your language has. Which regardless of its quirks, you really should be using to its fullest rather than just leaving code lying around undocumented.

Dynamically-typed languages can make this experience more difficult (and is one of the primary reasons that despite working in them for the first ~15 years of my career I eschew them today), but used skillfully they can still do quite a lot, the moreso with all the various optional types that they’ve been growing lately.

Your homework assignment is to write some regular expression or something to extract all the types you take in your various methods and functions, and do a chart showing how many of them are primitive types like string versus types like Username and Domain and InvoiceLineItemRevocation.

The Basics of Types As Assertions

As I was re-reading this prior to first publication I realized I probably ought to include an outline of how to use types as assertions. This is an overview of what has a certain art to it, which is somewhat different from language to language, but this is the broad strokes of how to do it.

Here are the keys:

1. Every type represents some concept of validity within your program. There should be one type per such concept. That’s not a bidirectional one-to-one; you may have many other types that are not necessarily about “valid values” (e.g., “an iterator type” and other such structural types), but there should be one per concept of “valid”.
2. On the way in to the type, the data is checked to be valid. If it is not, the value should not be created and the code that tried to create it should be forced to handle the resulting failure immediately.
3. Methods that manipulate never mutate the data into being invalid.
4. Data within the type should be the “real”, internal values.
5. Rules for encoding and external validity are checked on the way out.

Expanding on the keys:

Every Type Represents Some Concept Of “Valid”

I will again link my post on what constitutes a ‘valid’ value as I am using the specific definition I established in that post.

A type should correspond to a particular definition of “valid value”. Expanding on the discussion in that post, you may have a type Address that represents a “street address” that has been validated by your street address to exist. If you want to add a concept of “an address I can provide service at”, you probably want to have a type ServicableAddress. The reason for the second type in this case is that the ServicableAddress adds additional promises to the address, so they are not the same.

If you have a type per “valid thing”, then the rest of your program can assert the validity of the incoming values by simply declaring it in the incoming type signature.

Languages vary on how much of this the language itself supports. For instance, dynamically-typed languages, used classically with no additional type annotation support, do not permit any type-level enforcement at all. In this case, this approach is so powerful that I recommend that programmers simply adopt the discipline anyhow. That means if a type’s documentation declares the only valid way to get an instance of it is through some particular mechanism, that should be honored. If a type has any sort of constructor or init function, that should be assumed to be the official way to access the value unless otherwise indicated.

All Data Checked On The Way In

Every way of a value of the given type being created in the program must validate the data in the type in accordance to whatever concept of validity the type represents. If the data can not be fit into the type, the construction attempt must fail and the code must deal with the consequences of it.

The advantage is that this forcibly moves all validation as soon as possible in the program. It is often the case that by the time the rest of the program gets invalid data, that it doesn’t really have anything it can possibly do with the invalid data, by which I mean, it’s even already too late to throw reasonable errors. You want to get this stuff checked as quickly as possible.

Again, languages vary in how strongly they allow you to enforce that all data of a given type passed through this validation step, ranging from very strong to effectively none. Regardless of how strong your local language is, you should program in a way that you never bypass this protection layer. Any place you find you have to bypass it in such a way that you must create nominally invalid data suggests that you have a second type here.

It may feel painful to have two types instead of one but I find it is almost always worth it in the medium term. What I find happens is just as the lack of clarity would have polluted the future trajectory of the code base by forcing more and more of the code base to have a lack of clarity, just giving in and creating the second type causes that clarity to pervade the future trajectory of the code base, a vastly preferable alternative. You will often learn things about the problem space by letting it tell you about what different types are.

I’m not even calling for some sort of long-term “it’ll pay back in a few years”, I’m talking more about it’ll pay back in a few hours, maybe a couple of days. Being a super-short-term focused programmer is very expensive over the course of weeks and months and years.

Never Invalidate The Data

Once the previous is established, this should be pretty obvious.

Again, if you find there’s an operation that you feel “must” invalidate the data for some reason, there’s probably a second type trying to emerge there that represents some new concept of validity.

To a large degree, the solution to “these ideas don’t seem to work” is, yes, “go ahead and make more types”. It’s OK to have methods that just convert types, as long as those methods also maintain the validity.

Data Within Should Be The “Real” Data

That is to say, in general once a value is in a typed value, it should be all decoded into the most “real” representation of the data. For example, a “QuerystringValue” should contain Hello World and not Hello+World. Unicode value should be in whatever the local environment considers the “default” encoding. JSON values should be their decoded values, not their raw string values.

If for some reason you need the original values, which is exceptional but can happen, these should be retained but only available through methods that clearly indicate that they are the exceptional case. For example, the QuerystringValue might have a basic .Get that returns the value as a string, but if you needed exactly what was in the original value for some reason, the method should be something like .GetOriginalProtocolLevelValue, yes, clunky name and all. It should be clunky, and the docs for the method should explain what exactly it is and why you should generally not use it.

This fixes up the mishmash of things getting haphazardly decoded here and there and everywhere, sometimes multiple times, sometimes not enough times, etc. You use the type itself to enforce that it is decoded precisely once. This is also very powerful for types whose values may come from multiple places with different decoding rules.

Any Outbound Encoding Rules Enforced Upon Leaving The Type

For example, if you are writing a raw JSON output, you can have a method on some type that returns a JSON-encoded string, but it should not change the internal values. At most it may cache the result or something, but it does not change the internal value for the type.

The combination of the value being the “real” value internally and this rule for only encoding into the final output once the final output is known dodges all the many, many issues with the concept of “sanitization” that make sanitization an actively wrong idea. In this setup, no part of the system is responsible for knowing things it can’t know, in particular, the input layer is not required to guess what output may occur someday. The input layer just handles whatever proper decoding is, which it knows, because it is in the input context. The output layer is only responsible for correctly outputting into whatever the output may be, which it knows, because it is in the output context.

What Does A Bare Type Mean?

I program like this fairly routinely. You will find that I still do have some string and int and such in my code. However, in my code that takes on a particular meaning, which is that the value is truly unconstrained. When you get something from a network, you don’t need a NetworkBytes type. A standard byte array will be fine, because a standard byte array with no further type annotation already means the correct thing, which is, this is an array of bytes which have no further guarantees. A bare string means that the value is not constrained, or depending on your langauge, that it is constrained only by encoding or whatever local considerations your “string” type may have, as long as the definition matches the value.

The principles in this post are absolutely something that you can work out from first principles from years of imperative programming with strong types. If Functional Programming teaches anything here it is mostly that by having immutable values it becomes impossible to create invalid values that will be “fixed up” later, so it really leads you in this direction by virtue of removing that crutch from you, and really drives you harder in the direction of realizing that you often need a second type rather than having a single type with “modes”, or a single type that straddles two concepts of validity at the same time. Strict functional programming is excellent practice and a great way to be taught by fire by being all but forced to learn thees practices, but you can learn it from imperative code alone for sure. However, I observe that in practice most do not and I see the same characteristic mistakes over and over again.