I ended up jumping into a Twitter conversation started by international development journalist Tom Murphy about how Rwanda changed the methodology for its Integrated Household Living Conditions survey (EICV), and thus demonstrated that their poverty rate had decreased. The problem is that the new methodology essentially redefines ‘poverty’ to get the numbers to look good; using the previous EICV methodology, it indeed appears that poverty hasn’t decreased but has increased by 6%. While a number of people have already picked apart the methodological problems, is this really a methodological problem or part of a wider indictment of how donor agencies determine success and manage their human resources? Are the people in donor agencies dupes, cynics or both? Neither I reckon. I think they’re just overworked and probably under trained in statistics to get to the root of story, and have little incentive to do so anyway.

Filip Reyntjens does a really nice job of breaking down the problems with Rwanda’s EICV. He makes some good points about the problems with changing the methodology, and in the twitter discussion many other people highlighted technical problems with the new definitions of poverty used in the EICV. While these technical issues are important, the other problem is what the survey means to the stakeholders. This group includes the Rwandan government, donor agencies, and DAC governments. Reyntjens notes that the numbers in the updated EICV make the Rwandan government look good, and by extension make donor agencies look good. Everyone wins (except for the Rwandans who are still in poverty). Setting aside why the Rwandan government would want to modify a survey to make their baseline poverty statistics look better, what do we make of the donor community’s attitude? Are the various aid and development professions that guide policy just cynical bureaucrats happy to tick the box marked “Rwanda got better”?

Some probably are, but in my experience most development professionals take their jobs seriously and want to see the lives of people improved. So what would lead otherwise upstanding development professionals to ignore potentially blatant number cooking by a beneficiary government?  Overwork and a lack of statistical training most likely. The work loads that staff at donor agencies deal with are immense. Combine that with a tendency within agencies to stovepipe the statisticians away from the policy makers and you end up with over burdened staff who may not have the training to quickly digest the vagaries of a survey’s methodology or analyze the reason certain changes happen in data from year to year.

It shouldn’t surprise anyone that Rwanda’s government took the opportunity to redefine the methodology that signals how they’re doing at reducing poverty. Their government stays in the good graces of allied governments and donor agencies by ‘hitting’ their poverty prevention targets. But if we’re going to demand that donor agencies be prepared to call out number cooking, the donor agencies need to bring on more staff to spread the workload and make sure that the statistics capacity isn’t stove piped away from all the policy teams. Unfortunately the trend in donor agency funding right now is to focus on ‘efficiency’ above all else (read: too few people doing too much work), which means frayed policy staff will check the “hit the targets” box and the Rwandan government will continue cooking its data to keep donor money flowing.