Using Evidence to Improve Development Assistance

Development assistance is founded on countless theories about how foreign taxpayers’ money can be harnessed to instigate and catalyze economic and social development and provide humanitarian benefits abroad. Basic arguments for how positive change can be achieved have come to be conceptualized as “theories of change” in industry jargon. These theories, commonly presented as short propositions of if and then, are used to underpin aid programming. They include fairly straightforward notions such as the provision of emergency food aid to mitigate hunger and the dispersal of mosquito nets to counter malaria. However, many theories involving social welfare, peacebuilding, and governance are founded on more complex and ambiguous concepts, such as accountability, inclusion, empowerment, and harmony.

There is nothing wrong with this approach; but practitioners do need to make solid arguments for how they intend to use scarce resources to achieve the most benefit, preferably for the most marginalized and needy. While the aid industry has increasingly placed emphasis on improved monitoring and evaluation efforts for individual projects, it has not been particularly adept at assessing whether the basic arguments that underpin development assistance have much validity, particularly as more experience is accrued on the ground. Namely, as the aid industry matures – especially in the fields of governance, social welfare, and peacebuilding – do we progressively have enough evidence from which to base our most popular program rationales and to improve them accordingly? Or, are we continuing to assess ourselves within the comforting confines of a project-by-project understanding of “outputs” and “outcomes” without ever really understanding whether the wider agenda is intellectually solid and based on evidence and achieving the intended results?

It is toward this interest that The Asia Foundation and the London School of Economics’ Justice and Security Research Programme (JSRP) started a research collaboration to examine the evidence base for the Foundation’s theories of change that are applied in its conflict and fragility programming. Begun in 2012, the collaboration provides a rare area of institutional collaboration between social science researchers and development practitioners that intends to contribute to improvements in development practice and policy through a systematic analysis of evidence around common theories of change. The Foundation hosts JSRP researchers at its country offices for roughly three months, allowing them ample field time to critically assess the evidence base for the core arguments underpinning a respective theory of change.

Over the past two-and-a-half years, the collaboration has analyzed a wide variety of the Foundation’s programming in conflict-affected countries in Asia, with the findings presented in a series of 15 “Theories in Practice” research papers. The theories assessed have included those relating to community meditation and dispute resolution in Nepal and Sri Lanka, conflict management and peacebuilding in Mindanao, local governance reforms and community policing in Timor-Leste, and social accountability in Cambodia. Moreover, initial research has motivated further thematic research, including on the gender dynamics of community mediation in Sri Lanka and party politics in Nepal’s land disputes. The findings of this research have helped the Foundation’s programmatic management, with country offices working alongside researchers in critically analyzing their programs and re-evaluating programs to improve impact.

Theories of change may come and go as another conceptual tool, but there will remain an increasing need for the development community to achieve better understandings of just how much industry practice is based on evidence rather than merely normative agendas and what are arguably more superficial project-by-project logics. This collaboration is a relatively small but sincere intellectual effort to help the aid community start achieving a stronger sense of how it can assess the evidence base for its core arguments.

Toward this, the Foundation and JSRP have hosted annual “evidence summits” at the London School of Economics to allow donors, practitioners, and academics to debate the meaning of evidence and how it can be assessed for justifying aid programming. In this week’s In Asia blog series on “theories of change,” DFID’s Iain King examines some assumptions about modern conflict and why they could be wrong, the Foundation’s George Varughese challenges the “evidence-seeking agenda” from Nepal, and ODI’s Craig Valters looks at how theories of change can help us do development differently. The ultimate goal of these efforts is to help stimulate more critical awareness of whether decades of practice are leading to better rationales for program interventions, and whether these are founded on evidence gleaned from testing in the field.

Matthew B. Arnold is The Asia Foundation’s assistant director for Program Strategy, Innovation, and Learning based in Bangkok. He can be reached at [email protected]. The views and opinions expressed here are those of the individual author and not those of The Asia Foundation.