Finding Terrorists and The Problem of the False Positive

If you'd ever like a nice little Orwellian book to make you good and angry (I mean, provided that you read the news as well), I highly recommend Cory Doctorow's Little Brother (which he's put up for free download here). The book describes the misguided nature of attempting to find terrorists and criminals by tracking patterns:

If you ever decide to do something as stupid as build an automatic terrorism detector, here's a math lesson you need to learn first. It's called "the paradox of the false positive," and it's a doozy.

Say you have a new disease, called Super-AIDS. Only one in a million people gets Super-AIDS. You develop a test for Super-AIDS that's 99 percent accurate. I mean, 99 percent of the time, it gives the correct result ­ -- true if the subject is infected, and false if the subject is healthy. You give the test to a million people.

One in a million people have Super-AIDS. One in a hundred people that you test will generate a "false positive" ­ -- the test will say he has Super-AIDS even though he doesn't. That's what "99 percent accurate" means: one percent wrong.

What's one percent of one million?

1,000,000/100 = 10,000

One in a million people has Super-AIDS. If you test a million random people, you'll probably only find one case of real Super-AIDS. But your test won't identify one person as having Super-AIDS. It will identify 10,000 people as having it.

Your 99 percent accurate test will perform with 99.99 percent inaccuracy.

That's the paradox of the false positive. When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for. If you're trying to point at a single pixel on your screen, a sharp pencil is a good pointer: the penciltip is a lot smaller (more accurate) than the pixels. But a penciltip is no good at pointing at a single atom in your screen. For that, you need a pointer ­ a test ­ that's one atom wide or less at the tip.

This is the paradox of the false positive, and here's how it applies to terrorism:

Terrorists are really rare. In a city of twenty million like New York, there might be one or two terrorists. Maybe ten of them at the outside. 10/20,000,000 = 0.00005 percent. One twenty-thousandth of a percent.

That's pretty rare all right. Now, say you've got some software that can sift through all the bank-records, or toll-pass records, or public transit records, or phone-call records in the city and catch terrorists 99 percent of the time.

In a pool of twenty million people, a 99 percent accurate test will identify two hundred thousand people as being terrorists. But only ten of them are terrorists. To catch ten bad guys, you have to haul in and investigate two hundred thousand innocent people.

Guess what? Terrorism tests aren't anywhere close to 99 percent accurate. More like 60 percent accurate. Even 40 percent accurate, sometimes.

What this all meant was that the Department of Homeland Security had set itself up to fail badly. They were trying to spot incredibly rare events ­ -- a person is a terrorist ­ -- with inaccurate systems.

Is it any wonder we were able to make such a mess?

"Fuhrer's Law": The Bush Memos and Treason

A provocative article deserves a
provocative image, dontchathink?

What a doozy today. In perhaps one of the most significant stories of our lifetime (their words and mine), new details have come to light about the memos written by John Yoo while serving as a legal advisor to President Bush and the Executive Branch. Alternet's Naomi Wolf puts it most simply (and alarmingly):

In early March, more shocking details emerged about George W. Bush legal counsel John Yoo's memos outlining the destruction of the republic.

The memos lay the legal groundwork for the president to send the military to wage war against U.S. citizens; take them from their homes to Navy brigs without trial and keep them forever; close down the First Amendment; and invade whatever country he chooses without regard to any treaty or objection by Congress.

...

The memos are a confession. The memos could not be clearer: This was the legal groundwork of an attempted coup. I expected massive front page headlines from the revelation that these memos exited. Almost nothing. I was shocked.

John Yoo (a man who, incidentally, is eligible for war crimes trial) was instrumental in helping to consolidate presidential power and rationalize some of the most extreme breaches of justice and U.S. law perpetrated by the Bush administration over the last eight years, among them the justification of torture and limitation of habeus corpus, presidential power to override the fourth amendment through domestic surveillance, and worse. Constitutional scholar Michael Ratner explains:

What Yoo says is that the president's authority as commander in chief in the so-called war on terror is not bound by any law passed by Congress, any treaty, or the protections of free speech, due process and the right to be free from unreasonable searches and seizures. The First, Fourth and Fifth amendments -- gone.

What this actually means is that the president can order the military to operate in the U.S. and to operate without constitutional restrictions. They -- the military -- can pick you or me up in the U.S. for any reason and without any legal process. They would not have any restrictions on entering your house to search it, or to seize you. They can put you into a brig without any due process or going to court. (That's the Fourth and Fifth amendments.)

The military can disregard the Posse Comitatus law, which restricts the military from acting as police in the the United States. And the president can, in the name of wartime restrictions, limit free speech. There it is in black and white: we are looking at one-person rule without any checks and balances -- a lawless state. Law by fiat.

I won't dignify the allegations of treason made later, mainly because I don't have to - they stand completely by themselves. What was done here was treason, pure and simple - a lawless, irresponsible administration laying the groundwork for the systematic elimination of checks and balances in America. Though the Department of Justice later rejected Yoo's assertions that Congress had no check or balance against the president, history has shown that President Bush proceeded in these areas as if Yoo's memos were valid legal counsel. Ratner puts it best:

"This would be the president making war against the institutions of the United States."

Further Readin':

The memos and summaries on Wikipedia
Salon's Gary Kumiya on the torture memos
Mark Mazzetti, New York Times on interrogations

Obama Administration ditches "enemy combatant," but keeps everything else

Earlier this week, the Obama Administration released a brief announcing that it would drop the term "enemy combatant" in dealing with terrorist suspects, without otherwise making any substantial changes to the previous (read: Bush) policy regarding "terrorism." The "new" rules?

According to the release, the U.S. can still detain prisoners who have provided "substantial support" to Al-Qaeda or the Taliban. An administration who wanted to make more than a token attempt at coming out on the side of civil rights might have bothered to be more specific. The miniscule addition of "substantial" is hardly a dramatic change.

Note that according to principles of due process, people have the right to be informed of the specific charges against them, followed by a speedy trial. Time will tell if these alleged terrorists and terrorist collaborators will receive trials, and not the Queen of Hearts-styled trials ("sentences first!") we've been getting.

Glenn Greenwald of Salon sums up Obama's platform nicely:

Barack Obama has perfected a three-step maneuver that could never even be attempted by a politician lacking his rhetorical skill or cool cynicism.

First: Denounce your presidential predecessor for a given policy, energizing your party’s base and capitalizing on his abiding unpopularity. Second: Pretend to have reversed that policy upon taking office with a symbolic act or high-profile statement. Third: Adopt a version of that same policy, knowing that it’s the only way to govern responsibly or believing that doing otherwise is too difficult.

The new brief makes a token attempt at proving Obama's authority to violate habeas corpus on Constitutional grounds but still falls flat. "Po-TA-to, po-TAH-to. . ." We didn't even hear very much about Bush's violations of due process, and that's when people hated him. Do we really think that that's gonna change?

More evil plans. . .

Sorry for three Orwellian posts in a row, but this scary stuff is all around. This one from Kotaku is just as evil but far more trivial in its application:

Downloadable content as a weapon against second-hand resales is, nothing new, but Epic's Mike Capps has heard other ideas for how it can be used with devastating effect. If you hated the idea of DLC weapons in Bad Company, well, you're really going to hate this.

“I’ve talked to some developers who are saying ‘If you want to fight the final boss you go online and pay USD 20, but if you bought the retail version you got it for free’. We don’t make any money when someone rents it, and we don’t make any money when someone buys it used - way more than twice as many people played Gears than bought it.”

Any such game would ship as an incomplete copy, forcing you to download a crucial part of the experience online, essentially screwing over anybody who doesn't buy a new copy. So why not sell a record that forces you to give a license code and download the last four tracks online? Why not sell a textbook whose glossary is restricted to an approved download?

Rather than reiterate my old arguments I'll just link them here and summarize my main point, not that I know anything:

Whaddya think? It seems to me that the give-and-take of the market takes care of any issues that secondhand sales create. For example, if Halo was only worth $30 to a consumer but retails for $50, then that individual might buy it and later resell it after getting their money's worth of playtime. The buyer understands that the game is a used copy and pays less than they might for one from the factory. It's a central principle of capitalism.And do I have to mention that every used copy that somebody buys creates one less used copy, forcing somebody who might have bought a used copy to buy the game new?

But that's not even the main issue at hand: Mere "lost sales" (of new products) cannot be equivocated with theft or ripping off creators. Whenever an individual buys something other than Halo, they're spending money that might have been given to Bungie's developers. That doesn't mean that anything needs to change. Every time something perfectly natural happens that's good for consumers but less-than-optimal for creators and publishers of material, industry heads bitch and whine about lost profits. Who cares? Why must the advantage fall by default to suppliers and not to consumers? Maximizing profits by butting into the rights of others is inexcusable. I hope that for every higher-up who sweats over every lost opportunity there's somebody who respects consumer rights.

"Location aware" doesn't sound that scary, now does it?

Webmonkey reports:

"Microsoft’s upcoming Windows 7 plans to offer developers location tools [tools that know your geographical location] at the operating system level and the company doesn’t seem to think users care about control or privacy.

Before you freak out at the thought that Redmond will soon be tracking your every move, keep in mind that the new features will be disabled by default. That’s the good news."

Sure, allowing one of the most bloated, patronizing corporations in history to know exactly where you are sounds like a bad thing, but. . . actually, I'm not sure how to finish that sentence.

Yet Another Good Reason Not To Move to the UK

The new hot job for 2008: Level One All-Seeing Eye for the Minister of Traffic. On a positive note, they'll be able to fill a future V For Vendetta sequel on location.

U.S. Representatives Voting on Bailout Bill Privately Threatened With Martial Law

"Vote for our unconstitutional bailout bill or face unconstitutional martial law!" Let's spread this freedom and democracy to the rest of the world!

How We're Destroying Ourselves: A Post of Humor and Mirth

For all that I want to call him a jackass, he actually looks sort of cool.

My college paper ran a cover story three days ago, on the seventh anniversary of the World Trade Center attacks and subsequent destruction, titled "We Will Never Forget." I realized the intended heartwarming nature of the message, but I found it disheartening. (And it's not just the fact that it's a lazy and shiftless excuse for a cover story, a fact we won't even get into here.)

9/11 was a tragedy, yes, as is any incident of human barbarism toward others (not to mention the 3,000 some-odd deaths incurred by the 19 hijackers). But much of the tragedy of 9/11 stems from what we did afterward rather than from the attacks themselves.

After all, deaths aren't the sole measure of tragedy. In the year 2001 in the United States 16,242 died from emphysema, 42,443 from automobile accidents and 157,400 from lung cancer alone (that last number was built from projections but the specific count isn't important).

And to judge from the reactions of our leaders, economic destruction doesn't dictate whether something is tragedy. Nor is destruction of U.S. land and property a shame as long as it doesn't happen in a major economic center.

And the bills that we've signed into law over the last few years have proven that man's inhumanity to man isn't such a big deal, as long as we're being barbaric to the right people. Nor is abduction without charges or due process something to cry over. And the terrorists' "assault on our freedoms?" It's nothing compared to what we're already doing to ourselves (link1) (link2) (link3). And let's not worry about the right of an honest American to feel secure in their daily actions, ok?

The Trade Center attacks have given us license to grant the already powerful among us more power, destroy our checks and balances, hate those who are different from us who we won't even bother to try to understand, bash two countries to the ground, continue to occupy them with soldiers and private firms (and are we doing anything but teaching Iraqis that the big and powerful will always run things?), detain more people without warrants, tread on the Bill of Rights and otherwise muck up our own nation.

This started out as a funny sort of satirical post. I was going to propose that we adopt a new scale of measurement for foreign disasters and deaths, one that measures them in American lives so that we could feel honest in acting the way we do regarding foreign tragedy and domestic tragedy that doesn't involve Muslims. For example, twenty times as many people died in the Sichuan earthquake of China four months ago than on 9/11/01. Seventy-six times as many died in the 2004 Indian Ocean Earthquake and subsequent tsunami than 9/11. The idea was that we could muster up an actual equation to figure out just how much we should care about such destruction. Factoring in proximity, ethnicity, government type of affected country and damage to U.S. interests, for example, we might estimate that the Indonesian tsunami cost the world the equivalent of about 300 American lives. That's American Life Units, or A.L.U.'s for short. If we decided that the Sichuan earthquake only cost the world about 30 A.L.U.'s, for example, we could consider ourselves intellectually honest for still urging people to remember the 3,000 A.L.U.'s lost in the attacks of 9/11 seven years ago.

All of a sudden I decided it wasn't funny anymore. Can you tell why? Look - all of you who think that American lives and interests are more important than anybody else's had better think about what's happened in our country as a result of 9/11. And that's something we'd better not forget.

EDIT: Again, it's not the fact that atrocities have happened in Iraq (isolated individuals will sometimes do terrible things when you give them a gun), but the absolute lack of responsibility being taken in response which causes more of these things to happen. Excuses, lies and trigger-happy mercenaries don't spread freedom.

Printcrime: "Copy this story"

One of the neatest technological innovations of the last few years has been the advent of 3-dimensional printers, machines that can do exactly what the name implies. It's an amazing technology and it's going to be more and more important in the coming years.

And just leave it to Cory Doctorow (copyfight hero, writer, superhero) to write some great dystopian fiction on the possible future ramifications of the technology. This story's a couple of years old, but it's just as relevant now as it deals with topics currently affecting (and afflicting) our lives: expanding, militant copyright law and the (oh noes!) encroaching police state:

The coppers smashed my father’s printer when I was eight. I remember the hot, cling-film-in-a-microwave smell of it, and Da’s look of ferocious concentration as he filled it with fresh goop, and the warm, fresh-baked feel of the objects that came out of it.

The coppers came through the door with truncheons swinging, one of them reciting the terms of the warrant through a bullhorn. One of Da’s customers had shopped him. The ipolice paid in high-grade pharmaceuticals — performance enhancers, memory supplements, metabolic boosters. The kind of things that cost a fortune over the counter; the kind of things you could print at home, if you didn’t mind the risk of having your kitchen filled with a sudden crush of big, beefy bodies, hard truncheons whistling through the air, smashing anyone and anything that got in the way.

Read the Full Story!

The New Rules of the New Aristocrats

It's getting harder and harder for me not to link to BoingBoing, whose near-daily updates on the ongoing corporate war against the rights of ordinary consumers stir up my righteous indignation, making clear the threat against free expression soon to be facing our society as corporate interests begin to dictate the law. This war isn't isolated to a particular country - it's worldwide and it's happening now.

Two consecutive posts prove particularly relevant: first, an update on Michael Geist's writings on the Canadian Digital Millennium Copyright Act, and how its institution would affect Canadians (follow the story's link for the full rundown on how the DMCA criminalizes legal uses of consumer-purchased goods by making it illegal to circumvent copy-protection and adding special provisions to fair use, essentially invalidating it) .

Just as sickening are the new rules from the Associated Press restricting legal quotations of their own articles, in direct contradiction to U.S. law. Those willing to pay for already-legal consumer uses of AP stories may buy overpriced "quotation licenses" designed to limit AP story access, licenses terminable by the AP at will.

The following quote from Patrick Nielsen Hayden, printed in the article, is so important that I'm reprinting it here:

The New York Times, an AP member organization, refers to this as an “attempt to define clear standards as to how much of its articles and broadcasts bloggers and Web sites can excerpt.” I suggest it’s better described as yet another attempt by a big media company to replace the established legal and social order with a system of private law (the very definition of the word “privilege”) in which a few private organizations get to dictate to the rest of society what the rules will be. See also Virgin Media claiming the right to dictate to private citizens in Britain how they’re allowed to configure their home routers, or the new copyright bill being introduced in Canada, under which the international entertainment industry, rather than democratically-accountable representatives of the Canadian people, will get to define what does and doesn’t amount to proscribed “circumvention.” Hey, why have laws? Let’s just ask established businesses what kinds of behaviors they find inconvenient, and then send the police around to shut those behaviors down. Imagine the effort we’ll save.

Welcome to a world in which you won’t be able to effectively criticize the press, because you’ll be required to pay to quote as few as five words from what they publish.

Welcome to a world in which you won’t own any of your technology or your music or your books, because ensuring that someone makes their profit margins will justify depriving you of the even the most basic, commonsensical rights in your personal, hand-level household goods.

The people pushing for this stuff are not well-meaning, and they are not interested in making life better for artists, writers, or any other kind of individual creators. They are would-be aristocrats who fully intend to return us to a society of orders and classes, and they’re using so-called “intellectual property” law as a tool with which to do it. Whether or not you have ever personally taped a TV show or written a blog post, if you think you’re going to wind up on top in the sort of world these people are working to build, you are out of your mind.

This inevitable aristocratization of America is a direct result of inaction on the behalf of every political party in the world, every elected representative and every corporate fat cat who allows these liberty-destroying laws and loopholes through the legal floodgate of modern legislation. The mainstream media is not reporting it because they are part of it, and most people don't even know that it's happening.

Goodbye, democracy. Tomorrow is the dawn of The Corporation.

"Don't Be Evil": Google Fights For Internet Neutrality

Google's fired the latest salvo for the ongoing fight for network neutrality and against the tiered-service model - the idea that Internet Service Providers have the right to "prioritize" certain web sites and domains, giving the short stick to small, underground sites and upstarts that can't pony up the cash to pay the ISP's for top billing.

From TheRegister:

In an effort to identify traffic discrimination by American ISPs, Google is prepping a suite of network analysis tools for everyday broadband users.

"We're trying to develop tools, software tools...that allow people to detect what's happening with their broadband connections, so they can let [ISPs] know that they're not happy with what they're getting - that they think certain services are being tampered with," Google senior policy director Richard Whitt said this morning during a panel discussion at Santa Clara University, an hour south of San Francisco.

If the country doesn't have neutral networks, Whitt contends, innovation stagnates among application developers. And he believes that individual consumers - as well as Washington policy makers - should join the fight for such neutrality.

"The forces aligned against us are real. They've been there for decades. Their pockets are deep. Their connections are strong with those in Washington," he said. "Maybe we can turn this into an arms race on the application software side rather a political game."

The internet is one of the last bastions of free and independent thought left in the world, and it's fantastic to see a giant like Google fighting the good fight, avoiding profit-based temptations as corporate fat cats strive to create a worldwide McInternet.

Unreported Under-the-Table Copyright Meetings

Did you know that militant copyrightists are currently drafting a secret worldwide agreement designed to further crush free communication and eliminate P2P and enforce copy-protection on electronics companies? The new Anti-Counterfeiting Trade Agreement reeks of the smoke-filled rooms in which it was drafted by a host of powerful politicians and Hollywood execs, and none of the major news outlets are reporting it. Not a word.

This new law would allow border officials to search mp3 players and cell phones for copyrighted content, force Internet Service Providers to release consumer information and impose this Orwellian agreement upon developing countries. Does this sound like a threat to democracy, free expression, civil liberties and freedom? NBC doesn't think so. Neither do the flag-wavers at Fox News. Good luck finding anything about the ACTA on CNN.

It's telling that only the blogs are reporting this dystopian expansion of the ball pit of lies, self-righteous elitism and corporate interest that is modern copyright legislation. The blogging revolution came just in time, providing transparency in an age in which powerful forces want to turn us into Winston Smith and Guy Montag all at once - fat, happy and consuming the crap that they feed us. Follow this - it's going to come up again, and you won't hear it from the mainstream media outlets, Barack Obama or John McCain.

Further reading:

Michael Geist

P2Pnet

Wikipedia Article on the ACTA "Agreement"

Link to Download the Actual Document

Ageist Child-Tormenting Device Comes to America

CNN (via BoingBoing) reports that the Mosquito Device has come to America. The device emits a shrill shriek that can only be heard by youngsters. I've heard the sound - I'm at the upper end of the age spectrum for this particular frequency (there are more powerful variations), but it was pretty annoying. It gets in your inner ear like Tinnitus and gets terribly obnoxious after a few minutes.

Naturally, I don't condone vandalism in any form, even against crusty old ignorant ageist shopowners. On an unrelated note, look at the large variety of baseball bats available for online purchase! Some of them look pretty sturdy!

Related link (Feb 22) where I discuss the device

Fascist School Sugar Ban Leads to "Sugar Pushers"

A sugar molecule in its natural habitat.

Governor Schwarzenegger's fascist 2005 bill banning junk food sales in California schools is another grand example of another delightful government push to Make Everything Better. Thankfully, some children are taking justice into their own hands and selling sodas, candy and other junk food on campus.

I'm well aware that the kids are just looking for a quick buck, and making quite a profit to boot. Still, in a state where battling childhood obesity is apparently a greater necessity than preserving freedom, God bless these kids.

EDIT: Happy 200th post, by the way. And for everybody who will balk at my "fascist" label, consider this: The Arnold, on behalf of the California gübernmint, actually signed a law restricting the sale of legal substances in California schools, even between students.