A Gentle Reminder

The news has been completely dominated with domestic political stories for the last week or so. Is anyone aware that the American military death toll in Iraq is now 2014? We're on quite a quickening pace.

While reading about the entire PlameGate affair, I've seen a number thrown around that is disturbing and confusing. Some are estimating that the Iraqi casualties since the beginning of the war are as low as 10,000. Yet we have the Lancet study from a year ago, released amid a tidal wave of criticism, that showed the Iraqi's have suffered 100,000 dead (as of the study, which is a year old now). Why the confusing counts?

I also felt that a reminder of why PlameGate matters, and why it's so important to get to the bottom of the Niger/uranium document forgery story.

The Lancet Study

Just so happens that today National Public Radio broadcast the show This American Life with Ira Glass on Les Robert's Lancet study on the deaths of Iraqi's. The show devoted 45 mins. to the topic discussing in detail the study, it's methodolgy and the criticisms. There was also a bit of focus on why the study did not get more/better press, and a segment on Marc Garlasco. More on this shortly.

If you'll remember, the Lancet study was released during the 2004 Presidential campaign and caused a bit of a stir. Even so, the media largely ignored the story and has neglected the study's conclusions in estimating the numbers of Iraqi civilians killed during the war.

Some of the confusion on body counts for Iraqi's eminates from the Iraq Body Count study, which shows casualties at 27,000 - 30,000 dead. This study is done by counting those Iraqis killed who are reported in the news media. The originator of the study readily admits that the estimates are off on the low side. Unfortunately, it's these numbers that are often used by the media and war apologists. I find that interesting as this is no small number of dead. But I guess there's some logic to it when compared against the larger number.

So what exactly did the Lancet study find? From the summary:

Mortality before and after the 2003 invasion of Iraq: cluster sample survey (via Lancet, free registration required):

The risk of death was estimated to be 2·5-fold (95% CI 1·6–4·2) higher after the invasion when compared with the preinvasion period. Two-thirds of all violent deaths were reported in one cluster in the city of Falluja. If we exclude the Falluja data, the risk of death is 1·5-fold (1·1–2·3) higher after the invasion. We estimate that 98000 more deaths than expected (8000–194000) happened after the invasion outside of Falluja [emphasis added] and far more if the outlier Falluja cluster is included. The major causes of death before the invasion were myocardial infarction, cerebrovascular accidents, and other chronic disorders whereas after the invasion violence was the primary cause of death. Violent deaths were widespread, reported in clusters, and were in 15 of 33 clusters. Deaths were mainly attributed to coalition forces. Most individuals reportedly killed by coalition forces were women and children. The risk of death from violence in the period after the invasion was 58 times higher (95% CI 8·1–419) than in the period before the war.

Making conservative assumptions, we think that about 100000 excess deaths, or more have happened since the 2003 invasion of Iraq. Violence accounted for most of the excess deaths and air strikes from coalition forces accounted for most violent deaths. We have shown that collection of public-health information is possible even during periods of extreme violence. Our results need further verification and should lead to changes to reduce non-combatant deaths from air strikes.

It's important to remember that this data was released a week prior to the 2004 Presidential elections. The investigator, Les Roberts, felt it crucial that his data be a part of the debate in the election, hoping to elicit committments from both candidates to minimize civilian deaths.

In examining the study the reporter who prepared the This American Life segment, Alec Blumberg, refers to this article:

Researchers Who Rushed Into Print a Study of Iraqi Civilian Deaths Now Wonder Why It Was Ignored, By LILA GUTERMAN

This article examines the Lancet study and attempts to answer the question as to why it's received so little credibility:

The [Lancet study] paper, written by researchers at the Johns Hopkins University, Columbia University, and Baghdad's Al-Mustansiriya University, was based on a door-to-door survey in September of nearly 8,000 people in 33 randomly selected locations in Iraq.

...

In each neighborhood, in at least the first two households where an adult's death had occurred, the interviewers ended by asking for death certificates. They received confirmation of deaths in 63 of the 78 houses where they asked.

The researchers went to extensive pains to insure that the samples were, indeed, random. The interviews also took place under no small amount of danger particularly to Mr. Roberts, the only Westerner, who had to disguise himself.

If you've ever done research, you know that this is a phenomenal response. Not only are the deaths reported, but they are randomly verified in most cases with death certificates. This fact alone lends a high level of validity to the results. Incidently, Mr. Roberts indicates that of all the chosen samples, only five of nearly one thousand interviewees refused to answer the questions, another phenomenal result not often seen in research.

Then we come to Falluja, which had been the scene of very heavy fighting proximate to the study:

The Fallujah data were chilling: 53 deaths had taken place in the study's 30 households there since the invasion commenced, on March 19, 2003. In the other 32 neighborhoods combined, the researchers had counted 89 deaths. While 21 of the deaths elsewhere were attributable to violence, in Fallujah 52 of the 53 deaths were due to violence.

The Falluja numbers were so dramatic that they were eliminated in the final summary as an outlier. If you include the study's results from Falluja, the estimates of Iraq dead are much much higher.

Criticisms

As you can imagine (and may remember) the Pentagon went ballistic, as did Bush supporters. The report was roundly discredited for several reasons. Guterman takes up these criticisms. The biggest and most potent attack was that the study's statistical range was too broad:

They also acknowledged that the true number of deaths could fall anywhere within a range of 8,000 to 194,000, a function of the researchers' having extrapolated their survey to a country of 25 million.

Those who criticized the study did not understand the statistics of it. The internal components of the study determine it's range of possiblity as a predictor of accuracy. Indeed, the study could at best conclude that the true number of Iraq deaths was somewhere in between 8,000 and 194,000. Some argued that this was little better than throwing darts at a dart board.

Those critical did not understand that the high and low numbers represent the "tails" of a normal bell curve. What this means is that the probability of the true number being either 8,000 or 194,000 is statistically very very low, while the probability of the true number being about halfway in between, 100,000 or so, is 95% probable. Thus, as you move away from the high probablity number, 100,000, the probabilities of the number being accurate drop to ultimately zero. In reality, the most important number is to understand that the study found a 95% probability that there have been right around 100,000 Iraqi deaths, a pretty good bet.

Another criticism was that the study sample size was too small for the population involved. But Guterman's article had this to say:

Scientists say the size of the survey was adequate for extrapolation to the entire country. "That's a classical sample size," says Michael J. Toole, head of the Center for International Health at the Burnet Institute, an Australian research organization. Researchers typically conduct surveys in 30 neighborhoods, so the Iraq study's total of 33 strengthens its conclusions. "I just don't see any evidence of significant exaggeration," he says.

...

"Les has used, and consistently uses, the best possible methodology," says Bradley A. Woodruff, a medical epidemiologist at the U.S. Centers for Disease Control and Prevention.

...

The gap between the Lancet estimate and that of Iraq Body Count does not trouble scientists contacted by The Chronicle. John Sloboda, a professor of psychology at Keele University, in England, and a co-founder of Iraq Body Count, says his team's efforts will lead to a count smaller than the true number because not every death is reported in the news media.

Dr. Woodruff says, "Les [Roberts] has the most valid estimate."

Dr. Toole agrees: "If anything, the deaths may have been higher [than the study's estimate] because what they are unable to do is survey families where everyone has died."

Other's claimed that Roberts was not a reliable researcher, that he had an "ax to grind". Guterman:

Indeed, the United Nations and the State Department have cited mortality numbers compiled by Mr. Roberts on previous conflicts as fact -- and have acted on those results.

...

Mr. Roberts has studied mortality caused by war since 1992, having done surveys in locations including Bosnia, Congo, and Rwanda. His three surveys in Congo for the International Rescue Committee, a nongovernmental humanitarian organization, in which he used methods akin to those of his Iraq study, received a great deal of attention. "Tony Blair and Colin Powell have quoted those results time and time again without any question as to the precision or validity," he says.

A final criticism cited by many was that the study was unable to determine the difference between true civilians and insurgents. In other words, there were people that that the Pentagon intended to be dead. How many of these were in the study?

Roberts acknowledges this weakness. But he also notes in the conclusion this:

Most individuals reportedly killed by coalition forces were women and children.

He suggests that if even half the 100,000 are innocent civilians, it's a big big number. Point made.

It appears to me that some prominent members of the scientific community do not dispute Robert's findings. Each of the criticisms that were leveled against this study and Roberts were pretty roundly shot down. This leads me to conclude that the Lancet study was accurate. And again, remember, the study is now a year old. It hasn't gotten any better for Iraqis since the study was conducted.

This is the end of part one. In part two, I'll take a look at Marc Garlasco, a pretty fasinating guy in the humanitarian efforts in Iraq. Also, have the deaths been worth it, why the media blackout on the study, and some of the other war costs.