It's 2015. Why do we still write insecure software?

I've read a lot of programming blogs, and if you're reading this, you probably have too. So let me tell you up-front this is not your usual security rant that boils down to "just try harder!" Let's talk about smart, experienced programmers who are trying to write secure code, even if they are not security "experts" per se. This is an important set of people, because there is more security-related software in the world to write than can be written by security experts.

In a perfect world, setting that as the target audience would conclude this essay. As your browser's scrollbar shows in the full view, this essay continues on for quite a while. Alas, decades of experience and a trained reasonably high intelligence are not sufficient to write secure software in the current coding environment.

That's also the highest amount of qualifications that can be feasibly brought to bear at any reasonable scale, so in practice that's equivalent to saying it's impossible to write secure software in the current coding environment.

Let's talk about why it's so hard. My thesis is simple:

We write insecure software because our coding environment makes it easier to write insecure software than secure software.

But exploring what it fully means can lead some surprising places. Please join me on a journey as I try to show you why that is not trivially true, but in fact, profoundly true. We do not occasionally pick up insecure tools, like a broken encryption routine or misusing a web framework; we are fish swimming in an ocean of insecurity, oblivious to how steeped in it we are.

I thought I'd use Prime Music to explore some classical I hadn't gotten around to yet. You know... Mozart, some stuff by Beethoven I haven't heard yet, Bach, MC Hammer... you know, the classics.

Only a poor student of history could fail to notice history's cycles. The future can't be fortold in detail, but asking the question "Where are the cycles taking us?" gives you a better chance of guessing general shapes than anything else I know.

So it's easy for a student of history to look out at the United States and guess that we're approaching a libertine peak, and that over the next couple of decades we should expect to see the pendulum swing away from the wild excesses of the Baby Boomers back in a more "conservative" direction.

But at my age, I've never lived through a shift. So had I guessed how the counter-libertine shift would occur last week, I would have guessed a gradual cultural waning of the libertines and a gradual cultural waxing of those of a more conservative bent, with the advocates not changing their own views but their relative influence changing over time.

The debate about the reproducibility of science bubbles onward, with everyone agreeing that it's a problem but of course nobody with power to fix it doing anything about it. Recently I've been thinking that science as we know it sits in a very unpleasant middle ground. On the one hand, despite the propaganda institutional science is biased against replication. This holes it below the waterline, and any serious scientist (alas) must consider fixing this in their field their top priority or they are consenting to just spin their wheels forever.

Widespread angst about school quality is easy to fix... schools just need to look around and copy what's working out there in the real world. Five Solids that you have to see to believe! 7 Places You Have To See Before You Die This Woman's SHOCKING Actions Will Restore Your Faith In Humanity See why this guy thinks he can make you dance to his tune This one weird chemical will BLOW YOUR MIND!

While Star Trek was ahead of its time in many ways, you could tell they never lived with the technology they hypothesized. For instance, there's no episode in which Wesley Crusher walks around with his PADD unlocked while cradling it on his chest, causing a Major Interstellar Diplomatic Incident when he accidentally ends up emailing pictures of his armpit to the Klingon High Council along with a text message consisting of "

And the cat's in the cradle and the silver spoon, Little boy blue and the man in the moon. "When you coming home, dad?" "I'm home right now, having some fun and how, You know we're having fun and how!"

Go: More UNIX than UNIX

Go comes in part from Rob Pike and Ken Thompson, both influential in early UNIX. Both Rob Pike and Ken Thompson also were influential in working on Plan 9, a followup to UNIX.

UNIX's ideal is that "everything is a file". In Go terminology, this is a declaration that everything should be accessible via a uniform interface, which the OS specially privileges. One of Plan 9's core reasons for existing is that UNIX didn't take this anywhere near as far as it could be taken, and it goes much further in making everything accessible as a file in a directory structure.

I'm skeptical of both of these approaches. Everything isn't a "file".

There's numerous "files" that require ioctls to correctly manipulate, which are arbitrary extensions outside of the file interface. On the flip side, there are all kinds of "files" that can't be seeked, such as sockets, or files that can't be closed, like UDP streams. Pretty much every element of the file interface is one that doesn't apply to some "file", somewhere.

The Procrustean approach to software engineering tends to have the same results as Procrustes himself did, gravely or even fatally wounding the code in question.

Suture - Supervisor Trees for Go

Supervisor trees are one of the core ingredients in Erlang's reliability and let it crash philosophy. A well-structured Erlang program is broken into multiple independent pieces that communicate via messages, and when a piece crashes, the supervisor of that piece automatically restarts it.

This may not sound very impressive if you've never used it. But I have witnessed systems that I have written experience dozens of crashes per minute, but function correctly for 99% of the users. Even as I have been writing suture, I have on occasion been astonished to flip my screen over to the console of Go program I've written with suture, and been surprised to discover that it's actually been merrily crashing away during my manual testing, but soldiering on so well I didn't even know.

(This is, of course, immediately followed by improving my logging so I do know when it happens in the future. Being crash-resistant is good, but one should not "spend" this valuable resource frivolously!)

I've been porting a system out of Erlang into Go for various other reasons, and I've missed having supervisor trees around. I decided to create them in Go. But this is one of those cases where we do not need a transliteration of the Erlang code into Go. For one thing, that's simply impossible as the two are mutually incompatible in some fundamental ways. We want an idiomatic translation of the functionality, which retains as much as possible of the original while perhaps introducing whatever new local capabilities into it make sense.

To correctly do that, step one is to deeply examine not only the what of Erlang supervision trees, but the why, and then figure out how to translate.

The Environment Object Pattern in Go

One of the things I've been really enjoying about Go is how easy testing is. The pervasive use of interfaces and composition-instead-of-inheritance synergize nicely for testing. But as I've expressed this online on reddit and Hacker News a couple of times, I've found that this does not seem to be a universally-shared opinion. Some have even commented on how hard it is to test in Go.

Since we are all obviously using the same language, the difference must lie in coding behavior. I've internalized a lot of testing methodology over the years, and I find some of the things work even better in Go that most other imperative languages. Let me share one of my core tricks today, which I will call the Environment Object pattern, and why Go makes it incrementally easier to use than other similar (imperative) environments.