Emergent Weblog Behavior

And now we have reached the point where the science/engineering feedback loop has given engineers the tools and technologies to create the internet, the most recent of my four most important inventions in human history. And just as with the other three (spoken language, writing, movable type printing) it will cause a "knee" in human capabilities and behavior. And because of that, a true superhuman "intelligence" may appear during our lifetime.

But when it does, most of us won't even notice it, because it will be lost amidst the great sea of mediocrity and banality which will always dominate the internet and consume the vast majority of its bandwidth as long as humans exist. - Steven den Beste, Superhuman Intelligence

I was going to update my Weblog Communities piece with something on this topic, but now seems like as good a time as any to bring this up; I still expect to update it later but the writing environment that is in is annoying to use right now.

First, the interesting results I described in my original Weblog Communities piece (which are still largely valid, though some of it is out of date) are results of the fact that the weblog communities are able to self-organize into appropriate hierarchies with no direct human intervention, through exactly the mechanism Steven den Beste describes. Other community structures usually must be explicitly organized, and the way weblog communities re-inforce the positive and filter out the negative is still unique. No matter how much criticism Steven den Beste gets, at least he can still reasonably ignore it, whereas if he was trying to do what he does on a message board, he'd have almost certainly long since have given up. (I can't speak for him, but it's a darn good guess.)

Secondly, this provides an even better hook then I was planning on to answer the perennial "Are Weblogs Journalism?" question. The answer is: Journalism is itself a "hive mind" that has a certain structure and purpose. It consists of several distinct and highly stereotyped levels, roughly hierarchial, and stories flow from the local to the national levels. (Each level also distributes the stories at its own local level, which may also include "regional" or "statewide" depending on the story.)

In order for a story to "make the jump" from local to national, it must pass certain requirements. The fundamental requirement is that "many people nationally will consider it interesting". Unfortunately, this is not possible for the journalistic hive mind to correctly estimate in advance, because it has a structural flaw: Individual humans are forced to decide what will be interesting to large masses of people. "Interesting" is a very, very complicated criterion, so instead of being able to judge "interesting", the journalism hive mind has instead developed very quick and very stereotypical heuristics for whether a story should be taken national. One of the well known ones is "If it bleeds, it leads." Bad news is a "good bet" for being "interesting", so even when it really isn't, it's broadcast on the national news. There are others. To borrow one of the few good terminology contributions of post-modern analysis, these are also often expressable in terms of "narratives"; "Sports Team Wins/Loses", "Bush is Stupid", "War in Iraq Failing", "Your Children Are Going To Die Painfully Unless You...", you can think of others. Stories that fit these narratives/heuristics are more likely to be selected, those that actively contradict them may be actively surpressed by the editors.

The combination of structural defiencies resulting in bad heuristics, and the extreme difficulty for a given story to "jump up a level", results in diseased coverage in a lot of ways. The journalistic hive mind at this point is way too big for its own good, because it is not able to process the feedback from the rest of us any longer, and the resulting disconnect from what it should be doing is now recognized by all of us, with varying levels of awareness. This has many effects that may not initially seem to stem from these issues, but they do; for instance, "reporters never seem to understand what they are covering" is a result of the poor feedback, both to the reporters (their employers don't really care about accuracy) and to the readers (people who do know what they are talking about can not get their information to other readers, so unless the reader knows something they'll never know how inaccurate the coverage is).

The weblog community fixes two structural disparities in journalism. First, it is much easier for a story to jump up a level, because there are many more levels at a finer level of granularity. Since it's so much easier for the stories to self-filter, diseased heuristics no longer need to be in play. Note the incredible difference in tone between the national network news and the largest weblogs. This is not coincidence, it's a structural result of how the "top stories" are chosen.

Secondly, the entire "weblog hive mind" is choosing what is interesting to it as it goes. It should come as no surprise that "a mass of people" can better determine what is interesting to "a mass of people" then any single human (the "editor") or small group. Again, notice the extreme difference in the top stories of the weblog community versus the top stories of the network news. (Also, as of this writing, notice primitive "diseases" as den Beste calls them; the "paris hilton" blogspot is actually referrer spam.) Sure, there's some overlap (the top news selection heuristics aren't completely incorrect), but there's also always stories that are considered "fringe" at the national news level attracting a lot of interest from a lot of people, showing that the "fringe" characterization is not necessarily correct.

(Yes, there is some skew resulting from the fact that the weblog community is somewhat self-selecting and is not necessarily completely representative of the rest of the country, but the sample is now large and the key result, that the current journalistic heuristics are flawed, are still quite likely true.)

Over time, of course, the two hive minds are merging, and weblogs have always fed from the information output of conventional journalism. Eventually it will just be one big "hive mind", probably as soon as a decade or so.

My answer to the question, "Are weblogs journalism?", is a counter-question: "Is journalism journalism?" I see the journalism of the past couple hundred years as a poor approximation of the true structure that should be in place, the structure emerging as the result of pervasive reading and writing of weblogs. Weblogs are the true journalism, "journalism" the pale shadow. The co-opting of the journalism hive mind resources by the weblog hive mind resources, until only the weblog hive mind remains, is effectively inevitable.

Of course, there's still room for a lot of different kinds of participants in the final community. There's still room for massive organizations funding oversea journalism and fact finding. In fact, I think the news organizations and news outlets aren't necessarily going to look much different 10 years from now from the outside. ABC News isn't suddenly going to look like a weblog. What will change is that instead of trying to guess what is interesting, they will be forced to pay more attention to the refined feedback loops that the weblog community can bring to them, and essentially "outsource" the decision of what is interesting to the people themselves. They'll lose the ability to "set the agenda", which they may fight and moan about losing... but as usual, evolving to meet changing times will be to their benefit, because by bringing us more interesting news we will be more likely to care about them then we are now, and we may lose some of our cynicism.

New economic models may even develop; nobody is willing to pay for high-quality news now, at least partially because we can't sample it. But if your favorite blogger says a $1.00 article is worth buying, perhaps we will on the strength of that personal recommendation.

As usual for me, I think this isn't just abstract theory; there's real opportunity to harness this by the already-existing large news sources over the next few years (it's probably still too soon for them right now), and those who do will reap real economic benefits. Those who do not risk getting crushed. Really, it's the same-ole', same-ole' story of technological progress. In the end, we'll all be better off for it.

(It'll be a close thing, but by 2007 Dave's Bet may be simply be moot, rather then "true" or "false". The NTY may itself be considered enough a part of the "weblog community" to be by implication a weblog, or rather a set of weblogs by the reporters.)