Complexity and Society
Vernor Vinge is well known as one of the originators of the concept of the technological singularity, which is well-known to inform his sci-fi writings.
One of the less well known concepts which informs his sci-fi writings is one possible fate of societies that do not or can not end in a "singularity", which is the eventual unavoidable collapse of the society in a cascading failure state brought on by excessive, uncontrollable complexity in the ever-more-sophisticated systems that drive the society. In this case, take "system" in the broad sense, including not just software, but business practices, government, and societal mores. A failure occurs somewhere, which brings down something else, which brings down two other something elses, and perhaps quite literally in the blink of an eye, you are faced with a growing complex of problems beyond the ability of any one human to understand or contain.
We've seen small-scale examples of this before; Part 1 of The Hacker Crackdown goes into some detail about the 1990 AT&T phone network collapse.
I've always been a bit dubious of this theory. It's not intrinsically bad, but the truth is all software and systems must have some fault tolerance in them, because in reality, faults happen all the time. As I write this, my office has just experienced 6 straight days of faulty internet connection, and yet, our world has failed to end. We've got problems, but every bit of software we use already knows it has to be able to deal with problems like that. Only a few things were confused by the exact nature of the network failure, and even those were non-fatal. Deliveries will be late, networks will be down, contracts will be violated, only the truly foolhardy fail to make plans for those eventualities... unless....
Switching trains of thought with a near-audible klunk, I've noticed that one of the effects that capitalism has had on the world is to shave the excess out of things, because excess costs money. One of my relative's houses had a beautifully wrought cast-iron grille covering one of their heat exhausts in their bedroom, about one-and-a-half feet in diameter and quite heavy. A tornado could probably have hit that house and it might have suffered no damage. The whole assembly must have weight at least fifty pounds, since it had outlets both into the bedroom and the room below. Very strange, never seen anything like it.
And now I remember it as my canonical example of something that would never be made today. The same metal used for that one outlet, with a bit of carbon added to make it steel, would make up every register I have in my house, and a good bit of the ductwork, too. And my registers are adjustable, and they don't have to be placed in the center of the room because they are so huge. And so on. Fewer resources, superior product.
Yes, this can be taken too far and turn into an actively-cheap product, in the bad sense of "cheap". I don't like Walmart not for the usual talking points, but because the entire chain is one gigantic proof of mankind's ability to remove so much value from products that even at half the price it'd still be a ripoff; despite being cheap, Walmart's position on the value/money curve is very, very bad because the "value" term is so very small. But this is merely one possibility. Walmart is not the inevitable end of this line of development, it is merely an inevitable end.
I use concrete products as an example that you can see, but the effect can be observed across the board in today's society... supply chain management, product development, software development, packaging, anywhere where people actually have to account for money being spent there is an ongoing pressure to optimize the process. This is, largely, a good thing. It makes more products available to us. In the medium-to-long term, it will provide a powerful market-based impetus to reduce environmental impact of product production by continuing to reduce the energy and material costs to produce a given item. (I actually expect our resource-consumption-per-person to peak soon and start going down, and generally it won't mean a reduction in quality-of-life because it will be matched or exceeded by increasing efficiency.)
But it does have one other effect: It reduces the slack in the system. The cast-iron heating vent will not fail. It will fail to fail so thoroughly that one can argue that's actually a bad thing, as in the event of an extreme situation it will have no give. You could shave half the metal off of it and it would still work just fine. My heating vents can be destroyed by stomping on them too hard with some nice boots; it's best to keep them moderately accessible so they can be adjusted, but don't route foot-traffic near them if you can help it.
Again I refer to physical products to help you see what I mean, but again, this manifests in supply chains, product design processes, packaging, etc. Certainly those snazzy little cardboard packets that Amazon ships with take less resources to produce than old-school wooden crates, but they fail much more readily too. (Just look at the packaging around the lamp in A Christmas Story... inconceivable today!) The equivalent supply-chain failure can be equivalently much worse.
The surprising second-order failure here is that the company that shaves everything to the edge will succeed for long enough to beat the company in the market that leaves adequate slack in their system, because slack costs money. Then, the inevitable crisis comes, and the company-with-no-slack gets hammered where the now-defunct company-with-slack would only have been inconvenienced.
And it seems to me that Vernor Vinge's prediction may still come true, but rather than simply pure, failing complexity, the equivalent of a society being brought down by software bugs, the thing that turns the future economy into a powder keg is the fact that ever-less slack exists in the system, and the siren song of taking what slack there is and routing it into more production, which subsequently becomes mission-critical production, will be difficult to avoid.
I've actually been meaning to write this post for months, but there is of course an obvious application much more recent than that, which is the current financial crisis. If you've been following it at all, I shouldn't really need to explain how very much of it can be viewed in terms of having no slack left in the system, but I will call special attention to the increase in permitted leverage ratios, which is virtually a canonical example of falling prey to the siren song of increased "efficiency" and converting slack into mission-critical production. In this case, the "slack" in the system is how far down your assets can drop before your losses exceed your gains, and leverage ratios of 33:1 mean that even a very small drop in the leveraged assets drives you bankrupt. Truly an insane amount of leverage.
The financial crisis shows a much richer example of this phenomenon than the AT&T phone system, because it involves a lot more systems, many of which are more social than software, though it also provides an example of just how hard to draw that line it can be. The sub-prime mortgages were inevitably doomed, but it was truly the worldwide tendency to trade slack for mission-critical production that turned this into a worldwide crisis. I saw today someone say that even if every mortgage in question today defaulted, the sum total is still in the 500-600 billion range, less than the putative bailout, but the reason the bailout "must" be so large is that the crisis cascaded through a system with nowhere near enough slack to handle it, causing disproportionate damage. (Or, perhaps a superior way of thinking of it is as revealing damage that already existed.)
(I don't really want to get into whether the bailout is "truly" necessary or a good idea; that's not germane to my point here. I'm just showing one way to look at it.)
In fact, the international nature of this crisis and how quickly it spread to other countries shows how this wasn't so much a matter of the policies of a particular country; I think it speaks to human nature itself. The longer we go without a crisis or even a downturn, the more likely we are to "spend" our slack. It's debatable whether than can even be called "irrational". Is it rational to hold back some slack when it means you'll be driven bankrupt by your competitors, unless you can hold on to your slack until the next crisis? Whether it's a socialist government or a democracy hardly matters; the people in charge will not be able to resist.
Solutions? None, it's human nature.
... ok, that may be a bit defeatist. I think that as we look at how we regulate these entities going forward, and I mean more than just financial companies, that we need to consider how to regulate a certain minimal level of slack. In the mortgage industry, one obvious solution is to reinstate lower leverage limits, but it would require some creativity with regard to how to do this across other industries. By sharing this slack requirement, you remove the competitive advantages of cheating, plus or minus the inevitable temptation to cheat regulation which always exists regardless.
Slack requirements could be phased in as your company grows, so as not to prevent small companies from growing, and the slack requirements shouldn't be so onerous that it prevents companies from growing at all. (If you want to bust monopolies, great, but this would be the wrong way to do it.) Some tricky rules would have to be written that would describe when it's OK to dip into the slack (since mandating slack that you subsequently are never allowed to use under any circumstances isn't slack at all).
... but that's really quite hopelessly Utopian. The truth is this would require an enormous amount of intelligence to implement on a large scale, something you really can't get out of the many competing interest groups that would be writing these rules, this presents a gigantic opportunity for political gameplaying, and, worst of all, in the end does nothing to prevent the slack-rule committee from loosening the slack requirements after a decade of slack-induced smooth sailing. Like I said... human nature. You can't fight human nature.
Less utopian, though, is doing it in at least a few critical industries, like the debt industry; just because we can't have the perfect doesn't mean we can't shoot for the better. And what really scares me about this bailout is that I continue to hear no word from Congress about anything like this. Maybe I'm just not hearing the news, but nobody's talking about reducing the leverage limits, or how we're going to stop making bad loans. (Normally, the market would take care of that as people suddenly got gun shy about making bad loans, but since many of the bad loans have government mandates involved, the government needs to revoke those mandates too.) Putting a trillion dollars of slack in the system (because who believes it's going to stop at 700 billion?) won't do any good if we're still spending it; it just gives us a trillion dollars' worth of more time to build up a head of steam to slam into the next wall.
And so what concerns me about Professor Vinge's prediction is while I'm not convinced that mere "complexity" will bring about the societal crash in exactly the way he predicted, I am concerned that our ever-increasing ability to remove slack from our ever-larger and ever-more-unified economic system will result in some major crash in the future in a very similar way. Our financial system just spent all of its slack and then some; if we don't stop spending it and start putting it back, we're just doubling down on disaster.
Update, Oct. 28, 11am: A reader writes to tell me that Vinge may have indeed have had this in mind. Because of the fact that I only have Vinge's writings in this crappy "book" form, I will confess that I did not run a search. If this is true, I'll update this essay to remove the idea that Vinge didn't have this in mind sometime soon with appropriate crediting; in the meantime, I wanted to get this disclaimer out there for fairness' sake. (I only got the mail late last night, and then my dog got skunked, which ended up taking priority over a number of things I would rather have been doing... yuck!)