Why not subscribe?

Sunday, May 20, 2012

Simple errors scuttle Millenium Village analysis

Because this originally came from “G”, who seems anxious to get broad distribution, I’m going to channel this entire Marginal Revolution post.   http://marginalrevolution.com/marginalrevolution/2012/05/update-on-the-millennium-villages-controversy.html#comment-157588523

 

G., a loyal MR reader, writes to me:

I imagine you may find this interesting…

The blog post: http://blogs.worldbank.org/impactevaluations/the-millennium-villages-project-impacts-on-child-mortality

The retraction on the MVP website: http://www.millenniumvillages.org/field-notes/millennium-villages-project-corrects-lancet-paper

The retraction in the Lancet:

http://press.thelancet.com/MVP.pdf

…from the Lancet editors…http://download.thelancet.com/flatcontentassets/pdfs/S0140673612607879.pdf

Related:

http://www.economist.com/node/21555571

http://www.economist.com/blogs/newsbook/2012/05/jeffrey-sachs-and-millennium-villages?fsrc=gn_ep

http://blog.givewell.org/2012/05/18/millennium-villages-project/

 

The essence of this (from the first blog post) is two basic errors in applied statistical research.

The first involves not being able to count.  If there is a 5 year base period and a 3 year post period, the average time for the evaluation is 4 years (-2.5 to +1.5), not 3 years.  So, the total amount of decline is only 3/4 as high, to oversimplify slightly.

The second involves a false comparison group. The comparison is with trends earlier in the decade, not the current trends (which are improving overall, not just in the Millenium Village test area – perhaps due to antimalerial procedures being implemented overall.)

The net is that the infant mortality is basically equal (maybe a bit higher) in the Millenium Village test areas.

when we correct for a mathematical error and use more recent comparison data, we find that under-5 mortality has fallen at just 5.9% per year at MVP sites, which is slower than the 6.4% average annual decline in under-5 child mortality in the MVP countries nationwide.