Is the Devil in the Details? Estimating Global Poverty

How many poor people are there in the world and how has their number been changing over time? One might think that the intellectual and material resources spent on answering this question would have by now offered credible answers to these questions. The importance of answering them in the context of global goals that give poverty reduction tremendous prominence (the first Millennium Development Goal and the first of the just launched Sustainable Development Goals) is self-evident. Yet, that has strangely not much helped. The existing framework for conceptualizing the problem and for collecting and processing data on this question has been and remains extremely weak, with minor modifications but no fundamental rethinking of the money-metric “$1-per-day” approach used in tracking the goals. Research has long been available that has pointed out problems in the methodology used, as well as the havoc generated by periodic collection of new price data, and proposing alternatives. Despite the criticisms by diverse researchers having triggered, among other things, a multi-million dollar effort to collect new price data relevant to poverty assessment, there has been little public discussion and no substantial change.

In fact, the last notice by the World Bank Chief Economist of the approach that the Bank proposes to use to monitor poverty in the era of the sustainable development goals suggests only a very minor modification of the existing framework, which is to be officially announced imminently. This is the case despite the Bank having invested a great deal of time and money in creating a new Global Poverty Commission, which has not yet given its formal report. The specific variant of the existing “money-metric” method that the Chief Economist has signaled that the Bank appears likely to use involves “updating” the 2005 PPP international poverty line used for the last number of years by an average of inflation rates for less (or least) developed countries. This band-aid cannot save the Bank’s approach from its fundamental incoherence and meaninglessness, as we will discuss in a separate comment after the Bank’s official announcement of its new methodology, as well as in a longer research paper.

The Global Consumption and Income Project:

We have been working (with a third collaborator, Arjun Jayadev) for more than five years on a major project, now giving fruit at long last, which we call the Global Consumption and Income Project (GCIP). One of our most important initial goals was to provide a way to replicate independently the World Bank’s global poverty statistics, and to consider the impact of alternate ways of constructing such statistics, but the database we have created can be used for a wide variety of purposes including to understand the evolution of material living standards generally within and across countries, the dynamics of inequality and of inclusive growth and development — among other questions — over more than 50 years. We have aimed to create a database that is second to none in terms of its transparency, comprehensiveness, and internal consistency, while acknowledging and facing as clearly as possible the intrinsic difficulties that exist in terms of the quantity and quality of the underlying data. Our database is a worldwide one, extending beyond those that cover only the developing world (the sole focus of the World Bank’s Povcalnet) or only the developed world (such as the LIS) and focusing on per-capita data from original surveys. Due to this database, we are now able to ask a range of questions as to what would be the implications of alternate assumptions to those used by the World Bank when constructing global poverty estimates and to do so fairly quickly and easily. Our aim is to make this database available freely to external researchers and the world public as soon as we have more fully ensured its integrity and adequate documentation.

Choices that have Gone Under the Radar:

In this comment, we would like to flag, in addition to the existing rather deep-seated conceptual problems with the Bank’s money metric approach that we have previously identified, which remain very much still present and which we intend to comment on in greater detail in the near future, two additional issues involving the Bank’s recent methodological choices which we have not discussed in previous work, and which ought to be of interest to researchers, activists, public officials, and indeed the world at large. These issues may seem to be ones of detail, but we offer them here as examples of how adding a series of questionable (and often inadequately justified) methodological choices together can lead to cumulatively sizable, and even wild, variation in the estimates of global poverty’s level, regional composition and trend. That this is so must be of deep concern to anyone who claims to possess the objective of reducing severe poverty and deprivations in the world. Assessments of the costs of poverty reduction or of whether specific policies are working or not become meaningless when the underlying estimates of poverty are like quicksand.

We would like to highlight here as examples two detailed choices made by the Bank in recent years and which have gone “under the radar” but should have been more widely known and discussed. The resulting instability of estimates should cause great concern even if one remains within the - in our view in any case inappropriate - money-metric approach of the Bank.

Does a Whole Country have a Single Price Level?

Use of sectorally disaggregated versus national PPP’s: The International Comparison Program (ICP) defines PPP conversion factors for all countries at the national level. The World Bank has gone beyond the ICP to define for use in poverty assessment its own sectorally-specific PPP’s for rural and urban areas for three large countries: India, China and Indonesia. For China in the 2005 ICP exercise data collection was limited to only eleven cities and hence its PPP is considered to be an urban PPP. For India, although data was collected in both rural and urban areas, Bank economists have proposed, although this had not been a view they had put forward in regard to earlier rounds, that the survey was more representative of urban areas. In Indonesia, although Bank economists judged that there was no survey bias, they nevertheless made an adjustment to account for differences in price levels between rural and urban areas.

In principle, if such an approach is appropriate to use in certain countries it is not clear why it could not or has not been used for other large countries, such as Nigeria, or for smaller but still cumulatively populous ones. However, by far the more serious issue is that these sectorally-specific PPPs have been constructed by the Bank on a highly questionable basis involving back-of-the-envelope assumptions. On the surface, the idea of having a single price level for all of rural India is only a little less absurd than the idea of having a single price level for all of India or another such large country, but moreover, the specific methods used to construct that price level are poorly justified and can give rise to very reasonable questions. [1] It is indicative that the several methods have been proposed in the recent literature to estimate differences in rural and urban price levels give widely varying estimates of price differentials which in turn give rise to widely varying poverty estimates (E.g. Deaton and Dupriez (2011), Dikhanov (2010), Majumdar, Ray and Sinha (2014)). For India the World Bank’s estimate of the extent to which rural prices are lower as reflected in the ratio of the rural to urban price level (and poverty headcounts accordingly also lower) is 51% whereas the estimate of the ICP is that the difference is only 3%, with other sources reporting estimates in between. We will not go into the particulars of these further here, as it suffices to say that it matters greatly which method of inter-sectoral adjustment is used as well as whether a sectoral adjustment is used at all. The Bank’s chosen approach leads to the most optimistic portrayal of rural purchasing power and thus of the rural headcount in India. A measure of the impact of using sectorally adjusted PPPs for just three important countries is given by the table below, based upon our own calculations using the GCIP:

As can be seen, estimates of the global poverty level are enormously affected by this single very questionable choice. Using ICP national PPP’s for these three countries would substantially increase the poverty rates in them. Since these are poor and populous countries, it would result in the estimated number of poor persons in the world in 2011 increasing by an alarming 290 million. The trend of global poverty reduction is also affected, with the rate of poverty reduction from 1990 to 2011 appearing more favorable when sectoral PPPs are used, as the Bank has done in the recent period. The Bank has offered no sensitivity analysis nor discussed the impact of this choice, leaving open the question of why it made the particular choices that it did. Although taking note of intra-national specificities, including rural-urban differences, is in principle desirable, doing so in a manner that is both better justified and consistent across countries would be essential. It is not surprising that Bank economists have given some advance indications that it may drop the use of sectoral adjustments for these countries, with the argument being that there was urban bias in the 2005 ICP data collection round that neither existed before nor after! This point of view pays no attention whatever to the reality of systematic price differentials within large countries, but treats the problem as merely one of getting the single “right” national PPP. This is a misunderstanding of the nature of the problem. Ad hoc decisions from on high with inadequate public explanation cannot be the basis of sensible global poverty monitoring.

Count the Apples with the Oranges? Income and Consumption Surveys

Previously, the World Bank had made an effort whenever it was faced with income rather than consumption surveys (which is the case in many countries, most especially in Latin America) to estimate consumption levels by converting the income into consumption estimates by multiplying them by the ratio of consumption to income in the national income accounts. This was previously viewed as necessary because the international poverty line used by the Bank is defined in terms of a quantity of consumption (and not to adjust the data at all seemed to be the equivalent of comparing the heights of people wearing platform shoes with those who are barefoot, since income is typically higher than consumption). In very recent years, the Bank has changed to directly pooling data of both kinds and using them without further adjustment. The Bank claims, based on countries for which it has data of both kinds, that the choice of method makes little difference, although it admits that its new method lowers headcounts. Can the distortions caused by the share of bottom quintiles in income surveys being lower than in consumption surveys be assumed to “cancel out” those caused by the means being lower in consumption than in income surveys? Such a claim, that the use of the unadjusted pooled data come closer to measuring correctly the underlying unknown values, is not wholly plausible since the new method is identifiably inappropriate in two distinct ways, which cannot be assumed to negate each other. The Bank’s “rough and ready” approach is very questionable, and indeed, we find widely varying results depending on whether one consistently uses estimated income surveys, one consistently uses estimated consumption surveys, or one uses the pooled approach. In order better to estimate the underlying and unknown true values, when we estimate consumption poverty on the basis of income surveys we adjust both survey distribution and means in order to enhance comparability (more details on the “standardization” method we use, based on average statistical relationships, are present in a forthcoming paper. A description of an earlier version of our method is available here). This differs both from what the Bank does now and from what the Bank did previously, which was to adjust means alone.

To illustrate the impact on poverty headcount ratios, we calculate them for a few of the countries for which both consumption and income data was available from the same survey [2]. We calculate consumption poverty rates using a single poverty line ($2.50 2005 PPP of consumption) for income surveys with no adjustment to distributions or means (current Bank method), for income surveys with no adjustment to distributions but with adjustment to means (old Bank method) and for the actual consumption surveys. The results are presented in Table 2. They show a difference in poverty estimates across the three methods, which vary by country, as the magnitude of the dependence on the method used depends on various factors (in particular the income and consumption survey distributions and means). It can be seen that in some cases the difference is small and in others more noteworthy. In these cases the new Bank method leads to lower headcounts than the old Bank method, but whether either method leads to lower or higher headcounts than do consumption surveys (which ought to be used wherever available) depends on the country.

These examples are illustrative of a broader and deeper problem. Global poverty estimates are too important to be left to non-transparent, back of the envelope calculations (and their calculators). It is time to move beyond ad hocery not to mention quackery. There are practical alternatives that are possible, and they must be openly discussed. If we can’t do that, then let’s at least drop the pretense of reliability, not to mention that of precision. That will serve the world’s poorest rather better.

[1] As a rough and ready approach, the Bank assumes (see Ravallion (2008) p.35 and footnote 9) that the ratio of rural to urban prices (and thus of sectoral PPPs) can be derived from the ratio of previously defined rural and urban poverty lines, and that the national price level (PPP) is a weighted average of the (unknown) rural and urban price levels, where the number of price points sampled by the ICP in its national PPP determination exercise defines the weights. The first equation specifies the ratio of the sectoral PPPs and the second their absolute level. The resulting two equation system gives rise to the Bank’s estimates, but one could have imagined adopting other approaches such as to construct a rural-vs-urban price index by directly referring to unit-values from household surveys combined with a hypothetical basket or to apply price data used to generate official domestic price indices for different categories of workers (which are available in India for agricultural and industrial workers). The poverty lines used in the first equation represent, insofar as they are well-defined - which is quite questionable, especially given that the Bank uses official poverty lines for India which have come in for severe criticism from many sides of late - the presumed cost of obtaining basic human requirements. The differences between them thus reflect differences both in prices and in assumptions about the commodities that must be purchased in the two sectors to meet human requirements. The weights used in determining the relationship between national price level and sectoral price levels are for specific categories of goods for which price information is collected at all in rural areas (food, clothing and footwear) by the ICP, whereas the national PPP reflects prices of all goods. The net effect is not straightforward to determine.

[2] We choose developing countries and years for which both types of survey and suitable means are available, after 2000, from the WIID database.