Infographics Aside, Are Fragility Indices Useful?

Devising quantitative measures of state weakness is big business in the development industry. As awareness of the importance of institutions to growth and peace has spread, development practitioners and policymakers have been served an ever-expanding smorgasbord of state fragility indices (see here, here, and here). Countries receive a numerical score based on a range of indicators deemed to capture the ability of states to serve their people.

Donors are making increasing use of these indices – to determine country resource allocations, to track progress over time, and to compare progress across countries. Is Nepal becoming less fragile? How does Myanmar compare with its neighbors? If indices include a binary classification – fragile or non-fragile – we can overlay data about aid flows to track trends in assistance to fragile and non-fragile countries. (It turns out that, despite the rhetoric of donors, there is little difference between what goes to poor but stable countries and what fragile countries receive.) The rankings encourage reflection on whether aid is helping, and can be used for advocacy.

To what extent are such indices useful? How can they be improved? And what other types of information and analysis are needed to make development assistance more effective in places where the state is weak and conflict is persistent? The Asia Foundation and the Asian Development Bank recently convened a group of donors, development actors, peace-builders, and researchers to debate different ways of measuring fragility.

The limits of fragility indices

Criticisms of fragility indices tend to focus on what goes into them. Yet this is like shooting fish in a barrel. The report States of Fragility 2015 from the Organization for Economic Cooperation and Development (OECD), and the new Global Peace Index (GPI) released by the Institute for Economics and Peace, do not hide the fact that constructing indices is an art rather than a science. Methodological choices amount to a series of subjective decisions about what we want to measure, what indicators could serve as proxies, and what data sources are available. The taste of what comes out of the various sausage makers will be liked by some and less by others.

Of more interest is a larger question: to what extent are indices useful for planning and implementing development projects? There are three main limitations.

Deciding what to do. A numerical score can communicate simply how great a country’s problems are. Afghanistan, for example, scores lowest of any country on the recent GPI. But a number cannot tell us what we should do to ease state fragility. Index wonks have long recognized this, and recent efforts have disaggregated fragility in order to highlight where assistance is most needed. The OECD now unpacks fragility into five components: levels of violence; access to justice; effective, accountable, and inclusive institutions; economic foundations; and capacity to adapt to social, economic, and environmental shocks and disasters. The GPI now crunches numbers separately for eight pillars of “positive peace,” meaning not just the absence of violence but the presence of attitudes, institutions, and structures that create and sustain peaceful societies. These pillars are government functioning, business environment, equity of resource distribution, acceptance of others’ rights, quality of relations with neighbors, freedom of information flows, human capital, and corruption. This can tell us where a state’s weaknesses are rooted.

Indices are less useful for guiding practical action. The Asia Foundation’s programming can be quite dissimilar in countries with similar “fragility profiles.” There are very different ways, for example, to push for better governance. What approaches are appropriate in a given country?

Deciding how to work. Identifying priority areas for reducing fragility is important, but we still need to understand what entry points exist and what strategies for promoting reform are likely to work. Persistent violence and fragility usually stem from elites who have incentives to keep institutions dysfunctional, so importing technical knowledge from other countries will rarely be effective by itself. We need to grapple with the politics that maintain the status quo. This requires deep, in-country knowledge, something indices cannot provide. In such places, altering the equilibrium of violence can be extremely challenging.

Factoring in subnational fragility. The same institutions that promote positive peace nationally may have very different effects at the periphery, as the GPI recognizes. To date, fragility has been measured solely at the national level, yet we know that large-scale conflicts often exist at the subnational level in otherwise stable states. National indicators are often irrelevant for understanding these pockets of fragility across Asia. We need new measures that can be used to develop understanding and effective approaches below the national level.

Moving beyond fragility indices

While progress in improving indices is welcome, other types of information and assessments are needed if development actors are to be more effective in helping states reduce fragility and violence. First, fragility assessments for specific countries should be broader, and should take into account a wider array of sources than the limited range used in global indices. They need not be based on numerical scores. Second, tools like indices and assessments may matter less than staff mindsets when it comes to developing smart programming to tackle fragility. In short, changing attitudes matters as much as better data.

Bryony Lau is a senior program officer with The Asia Foundation’s Conflict and Development team, and Patrick Barron is the Foundation’s regional director for Conflict and Development. Both are based in Thailand. They can be reached at [email protected] and [email protected]. This essay was commissioned by the Asian Development Bank as part of a joint ADB-Asia Foundation workshop on assessing fragility, held in Bangkok on June 15-16. The views and opinions expressed here are those of the authors and not those of The Asia Foundation.