Lancet roundup and literature review

yes there was and I’ve put it in now, thanks very much. To be honest, the editor of the Lancet leaves a slightly funny taste in my mouth. By being so blatantly political, he gives a lot of succour to the people who are claiming that there were defects in the refereeing process, a point which I haven’t covered here because it would only matter if there were flaws in the actual paper, which there aren’t. (Some people, mainly people with no real idea about science, seem to regard “peer review” as magic oofle dust that has to be sprinkled on a piece of research to turn the words into Science). I agree with the Lancet guy that violent death is a genuine public health problem in Iraq and therefore the factors which cause it are a proper object of study for the Lancet, but, in any case, it was too late in the campaign to have ever made it as a real campaign issue, so why bother? God I feel like Fred Kaplan after all that.

You can count me among the innumerate (no pun intended), so my comment can immediately be discounted. However, I would be interested to know in which class of hackery you would put it. According to the Guardian article, the study was carried out in “33 randomly-chosen neighbourhoods of Iraq representative of the entire population.” How much confidence do they have about this? In a war zone, and in a country closed to the outside world by sanctions for a dozen years, how accurately can one estimate demographic variables such as population density, total population, employment, illness, etc. Employment and illness might seem to be irrelevant, since the study is measuring violent death, but maybe if unemployment is high in one town, there are more people at home when the bombs fall. Maybe in one region, people are more likely to go to the hospital when they are sick. Seems like small variations in these variables could result in significant differences in the confidence interval. But what do I know, I’m not a statistician. Yet let me say this: many people found this study bogus for the simple reason that extrapolating large numbers from small numbers seems to almost always be pure guesswork. The analogy that first occurred to me was crowd size estimates: the people who put on the rally estimate that 100,000 people were there, while the cops put it at closer to 20,000. Presumably they both have some rational basis for their estimate, but the results are so divergent that there seems to be no science involved at all. Anyway, those are my thoughts on why people seem to have such a hard time with this, rather than (in most cases) an unwillingness to admit that the war could have killed people.

George; the procedure for ensuring a random and representative sample is detailed in the paper and it’s not hard to understand – it’s nuts and bolts stuff rather than anything specifically mathematical. It looks good to me and I haven’t seen anyone seriously criticising it. I don’t agree that this is like estimating crowd sizes. As I say, I don’t like the extrapolated number, but the relative risk ratio is solid, and that’s the big thing for me. It would be very difficult to get this result from a random sample if the death rate had not risen.

Hi, I was curious about something. Is the “cluster sampling method” anything like a Monte Carlo technique? I am glad you have taken the time to debunk the hack critiques that many of us have seen. Personally, if a journal like Lancet publishes a study of this nature, I would be very careful in critiquing and rubbishing it.

First let me take a minute or two for some wild enthusiastic cheering. Good job DD. It more than makes up for my disappointment when you attacked Steve Landsburg not on his libertarian sins, but for that paper on quantum game theory. Okay, two comments. First, why can’t major news organizations hire people like you to explain studies like this? (Consider that to be part of the cheering). Second, why can’t they manage to spend more than thirty seconds mentioning them? This story was a one or day wonder in the American press—one story, usually containing some dismissive remarks, and then that’s all the time we had for investigating the utterly irrelevant story about whether we’ve made things worse in Iraq.

Yet let me say this: many people found this study bogus for the simple reason that extrapolating large numbers from small numbers seems to almost always be pure guesswork. George, this is the mere random sample objection, and it’s not well-founded. More generally, I’d agree with Daniel that most of the objections to the study either misunderstand the methods or just assert that because it doesn’t conform to an ideal design it can therefore be dismissed out of hand as garbage. This kind of objection is common in the first year of graduate social science programs, where people who have yet to do any empirical research find it very, very easy to trash the work of others who have. Later they tend to realise that the available methods really are very powerful, all things considered, and that even a modest empirical paper takes a lot of work to pull off. The fact that these guys did work like this under the circumstances they faced is really remarkable. Of course it’s not the last word on the subject, but it’s a real contribution with a striking finding. To write it off with malformed (or stock) objections is sheer ideology.

A reasonable piece about an unreasonable subject, though rather than “flypaper” I tend to see it as just another episode of “when innumerates attack”. Fumento has been fun, though. I shall admit to writing an insulting letter to the editor in response to his column being syndicated by my local paper. One guess what happened when I CCd him.

Add another critique to the study – the model the researchers fit allowed for an increase (and only an increase, based on their write-up – they never explicitly write down their model) in the death rate after the invasion. Thus the lowest estimate for excess death they could come up with would be 0 – no chance for a decrease in deaths. I don’t have enough experience with survival analysis to know about how this will effect the size of confidence bands, but it is suspect. It’s also another example of the authors’ bias.

Mike, that’s simply not true. They did explain what their model was and it did allow for a decrease. They calculated a relative risk ratio associated with the intervention (that’s their model) and if that risk ratio had been less than 1 (it wasn’t) then this would have corresponded to a falling death rate. They in fact did observe a falling death rate in the Kurdish provinces and reported it.

I think George is in for a shock when he finds out how economic indicators here in America (e.g. unemployment) are calculated, since the extrapolations from data are just as dramatic. But it should be obvious this kind of thing is basically sound. We regard it as a news story when opinion polls (which extrapolate from 1000 voters to 100 million) are off by 3 or 4%. If extrapolation was as bad as people are claiming here, getting within 10% would be miraculous, which it pretty clearly isn’t. If this study is off by 3 or 4% in its mean figures, or even 10 or 15%, that means the Iraq war has led (so far at least) to waaaay more deaths than it has prevented.

“…people have been lining up to take a pop at it despite being manifestly not in possession of the baseline level of knowledge needed to understand what they’re talking about.” Gosh, that makes it like most blog comments, alas. It frequently drives me crazy, to be sure. To take a random pop, the stuff early in this post about “mines” makes me wonder the relevance, given that an anti-tank mine is set off by magnetic field, or weight, and “tossing a rock” is irrelevant. So this, for one, seems to be evoking some sort of unreal idea of a “mine,” that then, when an example purporting to represent the real world, uses such a non-existent sense of a “mine,” makes me think whatever proceeds makes no sense, given the premise on a non-existent device in reality. Anti-personnel mines are even trickier in their variability, but given their imaginary use in this theoretical example, I’m uninterested in further pursuing the fantasy of an accurate conclusion. Any argument based upon things that don’t exist, though people popularly imagine them, is suspect, in my book.

Shannon, this objection is dealt with in my piece and (with much greater thoroughness) at Chris Lightfoot’s site. UNICEF did not “report” the 2002 rate as anything. They extrapolated a 2002 rate based on the 1999 research. And from then on in, you are citing Richard Garfield as an authority in an argument against Richard Garfield. You’re also exaggerating the extent to which a mortality rate of 29 is an outlier; given the very wide confidence intervals on the estimation process as a whole (and the substantial bias toward underestimation, which, as a “service” to your own readers you have consistently failed to acknowledge), it strikes me that a 1999 rate of about 100, a guesstimated fall of 40% and a confidence interval of +/- 20 would include the sampled figure of 29 at its lower end. I see that you are no longer defending the clustering critique, for which thanks, but you seem to now be advancing a much less intellectually respectable critique that “Occam’s Razor” tells us that there are strange murky flaws in the methodology which you can’t identify but nevertheless insist are there. For which, not thanks. I see you are also continuing to attribute the worst of motives to people who produce results that you disagree with; you are presumably doing this because you are judging the behaviour of others by your own standards.

But it should be obvious this kind of thing is basically sound. We regard it as a news story when opinion polls (which extrapolate from 1000 voters to 100 million) are off by 3 or 4% Exactly, and you never see people attempting to rubbish polls especially when the numbers give pretexts for other ideological conclusions. Last year’s EU survey about European opinions on the war in Iraq and about Israel was a good example of that. Or the “moral values” election poll. Everyone has draw all sorts of conclusions but I haven’t seen anyone disputing actual numbers. People do think there is a lot of coverage about Iraq because there’s always something in the news about it, but how much of it has been about civilian deaths? So, because they don’t hear or see it on tv, it must not exist. Civilian deaths in Iraq, oh, it must be some foggy concept that can be easily put aside. (Possibly with “anyway, Saddam would have killed more” – now that’s a scientific method the Lancet authors should learn something from!). The combination of attacks on Lancet is also so self-contradictory. I remember the IBC website was considered as reliable as Indymedia and now it’s become more authoritative than a peer-reviewed medical journal. I don’t even understand the argument about UNICEF infant mortality rates presumably proving the Lancet study is all wrong. How can the criticism simultaneously be that Lancet underestimates those rates therefore it must overestimate the war casualties?

For me the big negative about the Lancet survey is they had to do it in the first place – the Geneva Convention insists that occupying forces keeps track of these figures, of course there is little chance of the US doing this – rather they just assume that everyone blow up by a bomb was probably an insurgent anyway.

Oh, one other brief comment. Chris Williams pointed me to a poll done by the International Republican Institute which found (see the “Power Point” presentation linked at the bottom of that page) that 22% of Iraqis answered “yes” to the question, “In the past year and a half, has your household been directly affected by violence in terms of death, handicap, or significant monetary loss? (Close family member, up to 4th degree)” Now, suppose that there are ten people in an average “close” family group; then these figures imply that about 2% of individuals have been “directly affected by violence” (because each person who is killed or injured or whatever is known to about ten others). That gives a raw number of about 600,000 victims; we can’t, of course, tell how many of those were killed, how many injured and so forth (or who did the killing and injuring). But the important point about this is that, whatever the difficulties of conducting an opinion poll in Iraq and the imprecision of the number which comes out, it gives an independent estimate of the number of casualties—and that estimate is of about the same magnitude as Roberts et al.’s estimate.

Chris: average size of a nonextended family in Iraq is six, according to IBC, so your numbers look pretty good.

Gary – anti-tank mines are now set off by magnetic triggers? Last I heard, it was a pressure plate or a tilt rod. I think that you’re thinking of anti-ship mines.

Fred Kaplan of Slate suggested that because the confidence interval was very wide, the Lancet paper was worthless and we should believe something else like the IBC total Does he cite a confidence interval for the Iraq Body Count? Because if not I have a hint for him: just because they don’t give one, it doesn’t mean the confidence interval is small or zero. To their credit, the IBC people emphasise that the best their figure can do is set a minimum, and that it is likely that it is a gross underestimate.

Daniel, I’m not statistically educated, and the Lancet site doesn’t seem to allow a non-subscriber to read the actual study, so excuse me if these questions are addressed in the study or are simply ignorant. 1) Does the study distinguish between combatants and civilians (I can’t tell for sure from the posts and comments here)? During the first Gulf War, the US estimated Iraqi military deaths at 100,000. Is it reasonable to believe that Iraqi combat deaths from the current war would have been comparable or even higher? I.e., is it possible that the figure of 100,000 deaths reported by the Lancet is indeed under-reported but consists largely of combatants? 2) The CIA Factbook reports Iraqi population at 25.3 million and a death rate of 5.66%, which I think suggests that about 150,000 people might die in Iraq in an ordinary year. Does this mean that 100,000 excess deaths represents a 2/3 jump in the death rate (I may be misusing figures, so please correct me)? It strikes me that even a well-functioning city would have trouble finding morgue and grave space to deal with such a number of additional bodies, and so it seems that, under the chaotic wartime conditions in Iraq’s cities, this amount of excess deaths would have resulted in heaps of bodies or mass graves. If so, I would think that America’s opponents (whether the Baathists, Zarqawi, or whomever) would have every incentive to film these corpses and smuggle out the video much as they do with the beheadings. Obviously, this is not a critique of the study but simply uninformed intuition. Am I thoroughly off base, though; is it reasonable to conclude that these excess deaths were more or less absorbed by the Iraqi infrastructure? I’m just asking because these questions occurred to me; I don’t mean to be combative toward the study or your defenses thereof, or to suggest that any number of deaths should be minimized or dismissed.

Well, obvious question is: If you get blown up while waiting in line to join the Iraqi police, or gunned down execution style on your way home from boot camp, why is it OUR fault? Yes, the case can be made that we ought to be willing to accept more casualties on our side, in an effort to reduce civilian casualties. (Good luck selling that idea to the American people, by the way!) But the fact is, we already liberated Iraq, and now we’re fighting a force that’s trying to enslave the country again, and not only are THEY responsible directly for a lot of the deaths Lancet is complaining of, they’re indirectly responsible for all of them, because there wouldn’t be a war if not for their agression.

You know Brett, what a great argument! If only the US capilulated to Osama 3000 people would still be alive. Obviously its the fault of the US! If the Israelis woudl just leave, they wouldn’t suffer terrorist attacks. Every death is their own fault! If only native americans surrendered even faster, they wouldn’t have been as fully wiped out!

Question for Brett, If you hit a hornet’s nest with a stick, are you not at least partially responsible for the increase in stings? We knew going in that there would be an insurgency, there was inadequate planning for it, and obviously the administration considered the losses that would come from an insurgency to be at an acceptable level. So even if a significant number of those deaths were due to the actions of insurgents, it wouldn’t show that the administration bore no responsibility for them. Perhaps more important is that regardless of whether the cause of death was the bullet or bomb of an American or an insurgent, the numbers matter for determining whether this war satisfies the macroproportionality condition on standard versions of the just war theory. It is starting to look like they don’t. Even those of us on the left gracious enough to play along and imagine that there was somewhere in the shifting sands a just cause for this war and to set aside speculation about the purity of the President’s motives can cite the Lancet number as solid evidence that this war quite simply doesn’t pass the tests imposed by traditional just war theory.

That’s a very nice and clear way of putting it; it explains the “Outlier Theory” – this would be the case in which a 2.5% tail event had happened and the study had significantly overestimated.

Dsquared, You are ignoring the dynamic issues, and discussing the study as if it were a cross-sectional (point in time) analysis. Other studies which use this technique don’t push it nearly as hard, and apoligize for it more. For example, Grein et al, conduct a very similar study, but it is superior because: 1. The recall period is shorter (less than half). 2. They are careful to inquire about those entering and leaving the household. Also, their reference period is clearer, and I’d guess that in an urbanized society like Iraq, people enter and leave the household more often (schooling, military service, hospitals, etc.)than in the Angolan refugee camps studied by Grein et al. Nonetheless, Grein et al acknowledge (referring to the same method used by the Lancet study) that “the WHO/EPI sampling technique (originally conceived for immunisation and nutrition surveys) has not been fully validated as a tool to estimate mortality and alternative methods have been suggested.” This is the same point I’m making: the sampling technique is suited for cross-sectional studies, but not so well suited for longitudinal studies. This is especially the case for studies that are sloppy about including people before they entered or after they’ve left the household, as in the Lancet study. To speak in jargon, you should take the issue of informative censoring much more seriously.

Dsquared, The Lancet study does discuss the issue of people entering and leaving the household. As I argued in comments on your previous post, their discussion makes clear that they are doing it wrong. The Lancet authors are basically attempting to exclude household members who leave the household, rather than include them, which is what other studies that use this technique do. This is very important, because there are many reasons to think that leaving the household could be associated with death. The disclaimer in the BMJ article I cited isn’t about cluster sampling, it’s really about the definition of the sample universe. I’m prepared to believe that this was reasonably clear in the BMJ study, but not the Lancet study. It isn’t well-defined who is supposed to be included in the Lancet study’s universe. This is an invitation to overcount the kinds of deaths the Lancet authors were looking for. This isn’t lying. The surveyors in the field may well have thought that by counting violent deaths to people whose membership in the study universe was unclear, they were collecting complete information and doing a good job. And the death certificates don’t validate anything, since they were only collected from a small, non-random, subset of the respondents.

“In other words, many of the 100,000 may have perished anyway when the Ba’ath regime (“naturally”) crumbled, however long down the road.” And if they didn’t die then, they’d have died eventually, so it’s all good, eh? “how many lives is freedom from political repression worth?” Depends if it’s my life or not, don’t it? (Well, it does to me).

This was getting to be a long comment, so I split it up. Both parts are important to what I am trying to say. Part 1: Brian W and John Q use the example of opinion polls to demonstrate how accurate statistical sampling can be. But that comparison only reinforces my skepticism, since in fact opinion polls are almost never right. Look at this table of various polls taken the week before the American election: http://www.realclearpolitics.com/polls.html. All are (I presume) reputable and statistically sound, and all purport to measure the same thing, yet there’s a spread of 4% between the highest and the lowest. In state polls, the spreads were higher. In other words, it’s not news if an opinion poll is 3% to 4% off – it’s news if it’s accurate. And this is in a population that pollsters know abundantly well; how much more difficult must accuracy be in a population we do not know well, and that has in fact recently been through enormous change? Granted, this is not necessarily a critique of the study itself, since I gather that that uncertainty is reflected in the wide confidence interval. But then I fall back on something like the Fred Kaplan argument, which, despite Daniel’s thorough and probably accurate criticism, nevertheless seems to hint at something true: when the actual results might be anywhere in such a wide range, does the study really say anything useful for judging whether the war was a good idea or not?

And if they didn’t die then, they’d have died eventually, so it’s all good, eh? Actually, no. We’re talking about excess deaths. Natural death would occur irrespective of the counter-factual. Depends if it’s my life or not, don’t it? (Well, it does to me). True, I could reframe the question to capture that effect. Suppose the US was overtaken by a nasty tyrant, and X amount of lives from the population, chosen randomly, were required to remove said tyrant via revolution. Suppose you are among the population from which the sample is chosen. How small would X have to be for you to support revolution? Pacifist Response: 0 Pro-war nut response: 300 million Anyway I don’t want to veer this thread off its discussion of the Lancet. My point was that the humanitarian case has taken a serious blow from the Lancet, but it’s not out the window just yet.

D’oh! Despite proofreading, I made a big goof in my arithmetic. The period of time between 1985 and the invasion was 18 years, not 9, so cut all my “status quo” numbers in half. Not that it derails my basic argument, but talk about innumerate…

Thank you. Well done. The Lancet piece is indeed a good piece of science.

Replicate this study 5 times and you’ll likely get close to a good number.

Best,

D

I discuss the confidence interval issue in my ZNet piece: 100,000 Iraqis Dead: Should We Believe It? http://www.zmag.org/content/showarticle.cfm?SectionID=15&ItemID=6565 What I conclude is that a major reason the CIs are so large is that the authors engaged in a conservtive (in a statistical sense) analysis of their data and included the Kurdish region. In this region, there was a reversal of the pattern in the rst of the country: more deaths prior to invasion than after. If this region had been excluded, the CIs would have been narrower, and the point estimate (98,000) higher. The fact that the authors did not exclude the Kurdish region is one piece of evidence that they were NOT seeking to maximize the estimate of casualties. This type of thing is one of the things we researchers look for when judging the quality of another’s study. This study was one of the best I’ve seen. Still, given its discrepancy with the estimates of others (Iraq Body Count, Project for Defense Alternatives), I tend to believe the “true” estimate is somewhat below 98,000, but still in the many tens of thousands.

Ragout, Do you find the results of this survey to any great extent surprising? The IBC numbers plus the number of combatant deaths would give you a low ball at best rather less than an order of magnitude smaller than the Lancet numbers. If they were really only after the sensation value they could have given estimates including the Falluja figures. At best you are going to find holes in what is already known to be a net. Whatever its shortcomings it is the best figure we have to go on and it has been peer reviewed by a reputable journal. It doesn’t say anything very surprising beyond crystalising the effects of a gradually developing situation. I’m not sure why you think its burden of proof should be so high or that it fails to meet it. Why do you think there are no official figures? Surely this number, whatever its true value is an important performance indicator? In any case you will convince more people if you can provide a persuasive case for a significantly different number rather than just trying to find ways this number might not be right.

I’m not positive, but I don’t think this article addressed the strongest arguement agains the Lancet study: the numbers are utterly ridiculous. For 100,000 deaths to have occurred since the end of the war, that would be an average of around 175 a day (I don’t know the exact number, but its been discussed since the study came out). The 100,000 may have included additional infant mortality, and may have included additional ‘natural’ deaths that weren’t occurring under Saddam, but as I recall, it specifically said that ‘most’ deaths were occurring in violence, and ‘most’ of those were due to American (presumably unintentional) weapons. So at most 49,999 were additional infant mortality figures, (leaves ‘most’ due to violence), and at most 24,999 were non-American weapons (leaving ‘most’ due to Americans) So of 180 deahts per day, you’ve got 90 or so deaths per day due to violence, and 45 or so deaths per day due to American violence. 90 (or 45) per day, every day, for the last year and a half. Thus, in the last year and a half, when you have read in the papers about a bombing that killed 70 people, you were actually reading about a slow day! The news media has been writing headlines about the days when violence was actually lower than the norm! 100 people would have had to have been killed for a normal run of the mill day, I guess. So maybe you should address the “these numbers are absurd” statistical test. Steve

In light of this I’m still not sure where I stand on the humanitarian case for war. If Iraq is successful in 10 years, any ex-post justification of the war will still require pondering the question: how many lives is freedom from political repression worth? A fair question, but it would be a better one if the case for war had actually rested on humanitarian grounds. It only really did ex post facto when the WMD and “terrorist links” arguments went into the crapper.

I’m not positive, but I don’t think this article addressed the strongest arguement agains the Lancet study: the numbers are utterly ridiculous. That’s not the stronget argument against the study. If anything, it’s the weakest. As Marc Mulholland has pointed out, 175 deaths/day is about ten times worse than the worst year in Northern Ireland. This doesn’t seem all that unreasonable, given that the British Army was not in the habit of calling in airstrikes on the Falls Road. Forget the media reports. There is saturation coverage of the green zone of Baghdad. There is no good information about the dangerous bits of Iraq. As Tim points out above, journalists aren’t allowed to visit the morgue. I’m not saying the 100K number is gospel; as I hope I’ve been consistent in saying, I don’t like linear extrapolation. But the entire problem is that we have literally no information of sufficient quality to allow us to gainsay it.

‘But the entire problem is that we have literally no information of sufficient quality to allow us to gainsay it.’ …Although we might well have had such information had Allawi not ordered his Ministry of Health to stop publishing the death figures they were collecting from the hospitals.

There are several clear problems with the Lancet report that have not been adaquately addressed by this blogger. I would like to address two. They are the infant mortality rate, and the clear, overwhelming overestimation of the violent deaths in the Fallujah area. First, let me consider the infant mortality rate. Quoting the report: First, the preconflict infant mortality rate (29 deaths per 1000 livebirths) we recorded is similar to estimates from neighbouring countries. The use of neighboring countries appears reasonable…given the tacit assumption that no better comparisons are available from Iraq itself. However, as the initial article here mentioned, there was a study, sponsored by UNICEF. It is available at: http://www.unicef.org/newsline/99pr29.htm Some relevant numbers are: 24,000 households in the South/Center of Iraq (85% of the population, Arabs) 16,000 households in the North of Iraq (15% of the population, Kurds) were randomly sampled. This is far greater than the Lancet survey, which surveyed, roughly, 1000 households. Second, the infant mortality rate was measured at: 108 per 1000 live births in the South/Center not given in the North but said to have fallen like the under 5 mortality rate The under 5 mortality rate was 138 per 1000 live births in the South/Center 72 per 1000 live births in North. So, we have an under-5 mortality rate of 12.8%, and an estimated infant mortality rate of 10% for the entire country in 1999. This contrasts with the remembered estimate of 29 per 1000 live births pre-invasion in the Lancet study. It is considered good technique to address previous literature in the field. This study stands out as important relevant literature. Yet, it was ignored. If these numbers were correct, then one would expect a death rate of 130/5 per year for children under 5. With infant mortality being the majority of this, we can estimate that 1006 under 5 year olds seen correspond to at least 1100 live births in the last 5 years. (This gives a birth rate that is slightly lower than the pre-invasion birth rate 226/year given in the Lancet study. Using these UN figures, we would expect that there would be 32 under 5 deaths during the pre-invasion interval. The Lancet report gives only 12 deaths under 15…and doesn’t break it down by under 5 and over 5. This difference of 20 deaths over 14.6 months is absolutely critical. With that age cohort representing 39% of the total population, it also represents a difference in the under 15 contribution to the mortality rate of 6.4 deaths/1000. With an Iraq population of, approximately, 25 million, this translates to over 125,000 deaths per year…more than the 98,000 given in the Lancet. Given this, the authors of the Lancet study have a duty to mention the UN study and then explain why they think the death toll showed an unprecedented decline between ’99 (Feb to May) and ’02. (Jan 02, to mid-Mar 03) AFAIK, this decline far surpasses any other three-year decline. Since the author of the blog states that the start of the oil for food program is the cause of the improvement, let me address that. There are a couple of problems with this: first the oil for food program started in ’96. With infant mortality, in particular, it is hard to see why it would not have had an effect on infants born two years later. Second, during that time, the oil- for-food program was raided by Hussein for other purposes. One would need to provide evidence that this stopped in ’98 and ’99 in order to argue that the infant mortality rate fell tremendously. The second suspect number is the Fallujah death toll in 8-04 and 9-04. Reading the graph, I obtain 33 deaths in two months in Fallujah. Given that the total sample rate is, roughly, 1 in every 3200 Iraqis, this can be extrapolated to more than 100,000 deaths in this area during those two months. The city only has a population of 250,000. Since civilians were allowed to escape, why would they stay when people were dying at that rate? Further, how would it go unnoticed? As mentioned in the Lancet study, deaths must be taken care of quickly in Arab countries. Wouldn’t 100,000 requests for death certificates, 100,000 funerals, be noticed. Further, since there are usually >1 wounded per death in military actions of this type, wouldn’t nearly everyone else be wounded? Wouldn’t the hospitals and morgues be more than overwhelmed? Wouldn’t someone notice the smell of the rotting bodies, since there wouldn’t be enough able-bodied people to take care of the dead? Yet, the authors simply glide over this point…stating that there was a low possibility that this outlier was improperly sanding Even decent technique would require this type of cross checking between sampling and more direct measurement. Given these two examples of bad technique; given contrary data that appears to be much more solid, I don’t think the Lancet article can be taken seriously. My question is what sort of peer review goes on at the Lancet. It might be the same folks who reviewed Sokal’s paper. :-) Dan M.

I searched for “infant mortality” and “UNICEF”, and obtained one reference to that study: it is below: and the last estimate of under-five mortality was from a UNICEFsponsored demographic survey from 1999.11,12 So, it is true, that it was mentioned. But, it appears that you and I differ on what properly adressing a previous study, with far better statistics is. For example, they also quoted numbers on infant mortality. That wasn’t mentioned. So, it was only briefly mentioned in the introduction, and not considered as relevant data when the infant mortality rate was discussed. Second, the consideration of the Fallujah data was poor. Their data indicates that 100k died in two months where there was minimal US activity in Fallujah. Even if you only take half of that, its still 50k. Cluster sampling is a technique that is fraught with difficulties. Done perfectly well, it works. But, if there are any indications that there are serious problems with the techniques, a serious researcher is oblidged to deal with them directly. What they did, instead, is just say it was an outlier, and gave results with and without it. A scientist is supposed to search for every possible way his data could be wrong before publishing. Cross checking with other data sources should be done, whenever possible. A reasonable scientist, would have cross checked the Fallujah data, and known that there was a serious problem with the sampling. The authors wrote as though it was a minor, or modest problem at most. So, to be accurate, I shouldn’t say they didn’t address these two issues; rather I should say that they didn’t adress them as one would expect in a serious piece of work. If I implied the former, I apologize. One other thing, it is possible that I miscalculated a number, since I don’t perform the same due diligance in posts as I do in professional papers that I write. If you can catch an important numerical error that I made, I’d appreciate you letting me know what my mistake was. Dan M.