« previous | MUCK HOME | next »
Fuzzy Math: Stats Scrambled in DOD Iraq Reports
"The country is not a one-size-fits-all, a one-description-fits-all. It's much more a mosaic," the U.S. official in charge of training Iraqi security forces, Lieutenant General James Dubik, told military analysts today on a conference call. And he's got a point. So maybe it's fitting that the Pentagon's last two quarterly reports show all sorts of unexplained shifts -- even on the exact same pieces of data.
Brian Katulis, a senior fellow at the liberal Center for American Progress, told reporters at a National Security Network briefing today that the Pentagon reports can't keep their stories straight when it comes to the incidences of sectarian attacks and murders. Take two most recent reports, from March (pdf) and June (pdf).
On page 17 of both reports is a graph entitled "Sectarian Murders and Incidents" that tallies sectarian attacks by month. The March report lists that, for instance, December 2006 hosted over 900 sectarian "incidents" resulting in just under 1300 murders. But in the June report, the numbers shade up: December 2006 hosted over 1000 incidents yielding over 1600 murders.
Similarly, the March report listed a decline of about 150 sectarian murders from September to October 2006. But the June report changes that to an increase of nearly 400 murders during that same time period. Speaking generally, the June report makes 2006 look like a more deadly, sectarian year than did its March predecessor, but there are exceptions: April 2006 had 700 sectarian murders in the March report, but somehow, that figure drops to under 400 murders in the June report.
So what accounts for the difference? Katulis says he doesn't know, and hasn't been able to get an answer from the Pentagon. Two possibilities present themselves: either Multinational Corps-Iraq, from whose database the statistics emerged, changed its definition of "sectarian" incidents and murders; or new information became available after March. Whatever the answer, a reader of the June report doesn't have any way of knowing that the March report gave different statistics on sectarianism.
Similarly, the November 2006 report (pdf) contains seven mentions of the term "death squads." ("Armed groups that conduct extra-judicial killings" is the definition given by the November report.) Yet in March, the term only appears once, with no given definition -- it appears in scare quotes, even -- and in June, the term vanishes from the report's lexicon. Again, there's no stated reason why the term vanishes: it's not as if Iraq is suddenly free from death squads. In many cases, as the draft GAO report leaked to The Washington Post today reveals, they've embedded within the Iraqi security forces we're building.
The question immediately raised by the retroactive changes in statistics on sectarianism and other key measures is whether the next quarterly report, due in September, will similarly juggle the data without notifying readers to the change. "We have to be able to understand the models, and have an apples-to-apples comparison," Katulis says, in order to make sense of what Dubik called the "mosaic" of Iraq. Right now, the quarterly Pentagon report looks less like a mosaic than it does a Rorshach test.

Comments (9)
Anonymous wrote on August 30, 2007 10:32 PM:Notice, though, that although the figures dispute each other, and that these types of differences can lead to a decision as to whether or not attack a nation and/or kill a few hundred thousand people, no one will be held accountable for being entirely wrong here. There will be excuses, and any/all excuses will keep those who mistated the data from any consequences.
Our country is beyond the "more equal" status of the pigs in "Animal Farm"... way beyond. And we have over 10,000 nuclear missile heads to prove it...
fuzz wrote on August 31, 2007 12:26 AM:Although I get the 2000 Bush/Gore debate reference, the "fuzzy" reference is irritating as it perpetuates Bush's misuse of the term. Fuzzy is a reference to fuzzy set theory, which does not mean "unclear" or "inaccurate" set theory--in fact quite the opposite. Fuzz allows us to more precisely describe imperfect sets. For example there may be a set of seven cups on the table. The edge of one of those cups may be hanging off the edge of the table, such that 10% of it is not on the table. Do we say that that cup is member of the set of cups on the table? Or is it a member of the set of cups not on the table? Without fuzz we'd have to compromise and call it one way or the other. With fuzz we can say it is .9 a member of the set of cups on the table.
And yes, I am known as "Fuzz" in real life and, yes, I have attended a Fuzzy Sets & Fuzzy Logic class and, yes, it did keep me jumpy to have the professor saying "fuzz" every other minute.
Joe the Cynic wrote on August 31, 2007 6:49 AM:The reports don't really resemble a "Rorshach"--you mean "Rorsharch"--test at all. They consist of cooked data. Pentagon statisticians are offering us a lesson in the culinary arts.
mhill wrote on August 31, 2007 9:29 AM:You labelled the reports March and June 2006. These reports were from 2007. (The headers are not present in the source pdfs, just a mistake here.)
TheraP wrote on August 31, 2007 9:40 AM:The word is "Rorschach."
But fuzz is right about fuzzy logic. It's like a set that is somewhat indeterminate - as indeed is life. (for example in my field just about every "diagnosis" is based on a fuzzy set)
But yes, cooked data sounds accurate.
br wrote on August 31, 2007 10:01 AM:This happens in SEC filings from time to time. Previous year's numbers are changed with no explanation. In the case I'm familiar with, the company is now under criminal investigation.
Fedup wrote on August 31, 2007 11:53 AM:This Administration creates it's own reality and fabricates the facts to prove their case. When everything is a lie, eventually, you are bound to forget your other lies.
Rodney wrote on August 31, 2007 8:00 PM:Wait maybe the graphs are correctly identified as being from 2006 reports. Maybe it shows that the Department of Defense was planning the sectarian murders.
editor wrote on September 1, 2007 2:24 AM:I think the answer is the very last line of the charts. One says: "Source: MNC-1" and the other says "Source: MNF-1".
MNF, as we know, means Multinational Force.
MNC, according to a brief Google search, means Multinational Corporation.
This fits with the war profiteers motif. Perhaps that was a mistakenly honest reference in the June report?