Slashdot asks Has the Quality of Consumer Electronics Declined? [Slashdot]

About two years ago, I took Statistics in college. The second most interesting part of the course was the industrial manufacturing focus, specifically quality control.

Even more specifically, reliable ways to determine how much quality you can take out of the product and still meet some specification with some good probability.

Basically, the whole course was a primer on using statistical techniques to make cheaper products by decreasing quality. All anecdotes the prof had to tell about industry experiences was about taking out quality. If this is typical, I think it explains a lot.

Still getting sporadic emails about my Bayes spam predictions. Thanks to those who emailed for staying civil; I expected a bit more flamage.

People are still understandably skeptical (can't blame them, I'd be skeptical too). So, here's another thing I'll do: When the SpamBayes project actually releases files officially, I'll take that and implement the attack I mentioned, to see if I can make it work.

Exactly what I'll do if I can remains to be seen; I sure as *#&$ won't just release the code, though! And if I can't, I'll say so.

[Google Top Stories]

I generally stay away from international jurisdiction stories, considering them relatively uninteresting problems that must be solved through diplomacy, like any other national conflict. This seems wrong; even if the case is heard in Melbourne it should be judged under USA laws, since the article was written in America.

OK, I admit I just wanted to post an article from the Google feed to see how it looks. (Been waiting a while, too.)

Washington Post:

...under authority it already has or is asserting in court cases, the administration, with approval of the special Foreign Intelligence Surveillance Court, could order a clandestine search of a U.S. citizen's home and, based on the information gathered, secretly declare the citizen an enemy combatant, to be held indefinitely at a U.S. military base. Courts would have very limited authority to second-guess the detention, to the extent that they were aware of it.

Federal prosecutors have arrested three men involved in what officials are calling the largest identity fraud case in American history.... Cummings would then use the ruse of "helping" the customers work through software and hardware problems to obtain the customer code that allowed the company to request credit records.

This is like a textbook case on why privacy issues are so importent. There is no such thing as "a company" or "a government" having access to privacy information; only "people in a company" or "people in a government" can have access to privacy-sensitive information. Now the people who had their credit records stolen, who have committed no crime, have to go through the effort of checking their records and potentially changing their Social Security Number and putting a fraud lock on their records, making credit transactions that much more difficult for the duration of the lock.

Matt Haughey: "The [SpamAssassian] arms race has officially begun." [Scripting News]

I'm reading between the lines here based on scanty hints (based on the remarkable uniformity of spammer's arguments that they are doing a good thing), because I'm not intimately familiar with the world of spammers, but the biggest spammers seem to talk to each other fairly regularly. If one of them has figured out how to do it, rest assured that it is not long before they all know how.

Two people have now expressed the opinion that I am underestimating the advantage of personalization, and the power of statistics. It's worth replying to, I suppose, since that's two out of three. You can see why I left this out of an already-long weblog post before.

First, I have built Bayesian filters before, so I'm not completely ignorant about how they work. I'm not an expert, but then, right now I don't know that anyone is, since there's a lot of application-specific tuning that must be done. Still, I rest my argument on general principles, not specifics.

Note: If you're arriving from one of the many links who think I'm underestimating the power of personalization, please see my rebuttal (now with a working link!). Personalization won't work either, and it's nowhere near as powerful as people seem to think.

Recently, a relatively new idea for filtering spam has surfaced: Bayesian classification of e-mail, or at least Bayesian-inspired analysis. This seems to have been recently been brought to the Internet community's attention by Paul Graham in his essay A Plan for Spam, though I know he's not the first to think of it: For instance, here's a programming assignment given at the University of California, Irvine's Information and Computer Science department in Dec. 1999. That the idea was not popular until recently is probably a direct consequence of the fact that since 1999, the war on e-mail spam has been victory Spammers at every turn. The need for bigger guns is now more acutely felt then in 1999.

The stupidity of computers has become a bit of a minor running theme on this weblog over the past few weeks (and I've got another post on that topic on tap, waiting for Monday), so I couldn't resist posting this news from Slashdot. The conversational bot A.L.I.C.E., winner of the Loebner Prize in 2000 and 2001 for most human-like conversation bot, was hooked up to itself and this is the result. Surprise surprise, very stupid conversation results, especially considered on the semantic level.

Addendum to my previous education posts: In general, the best way to learn anything is to simply jump in, do some wild and crazy stuff, make mistakes, get quick, accurate feedback about how well you are doing, and benefit from the previous experience of others in the environment. This goes for both humans and computers, and is essentially true in all environments.

It is in theory possible to learn without direct interaction with the environment, but the learning rate takes a major hit, a minimum of a factor of three to five slow down. Again, this applies to both humans and computers, across all skills. One of the few interesting unifications of AI and human psychology has been a partial empirical theory of learning. This is about the only result it has, but it's worth knowing.